igorbrejc.net

I’ve read recently in some forum post that you couldn’t select all ways in Kosmos and render them, even if they have no tags at all. Well, this evening I implemented support for this in Maperitive:

The white-ish lines represent OSM ways. The green color depicts OSM areas. The rules for this are quite simple:

features
    lines
        all lines : 
    areas
        all areas : 

properties
    map-background-color : #181818

rules
    rulefor : all areas
        define
            fill-color : green
            fill-opacity : 0.1
        draw : fill

    rulefor : all lines
        define
            line-color : lightgray
            line-width : 1
        draw : line

Anyway, I’m still hoping I’ll be able to fix all the “todo” stuff from my list within few weeks, but I cannot promise. It seems that for each item I remove from the list, two new items appear. I guess I’m another victim of the Ninety-ninety rule.

A screenshot from my yesterday’s Maperitive paper prototyping session:

Tools of the trade:

The second part of the rules:

rulefor : railway

    define

        min-zoom : 13

    define

        line-color : gray

        line-width : 2

        curved : false

    draw : line

    define

        line-style : dashlong

        line-color : white

        line-width : 2

        border-style : solid

        border-color : gray

        border-width : 25%

        curved : false

    draw : line

rulefor : road.*

    define

        line-join : round

        line-start-cap : none

        line-end-cap : none

        curved : true

    for : motorway

        define

            min-zoom : 12

            line-color : #849BBD

            border-style : solid

            border-color : #849BBD black 20%

            border-width : 15%

        for : link

            define

                line-width : 12:3;13:4;16:5;18:6

        else

            define

                line-width : 12:5;13:7;16:10;18:12

        draw : line

    elsefor : trunk

        define

            min-zoom : 12

            line-color : #96D296

            border-style : solid

            border-color : #96D296 black 20%

            border-width : 15%

        for : link

            define

                line-width : 12:3;13:4;16:5;18:6

        else

            define

                line-width : 12:4;13:6;16:8;18:10

        draw : line

    elsefor : road primary

        define

            min-zoom : 11

            line-color : #ECA2A3

            line-width : 7:1;9:1.5;10:2;12:5;13:7;16:10;18:12

            border-style : solid

            border-color : #ECA2A3 black 20%

            border-width : 15%

        draw : line

        define

            min-zoom : 7

            max-zoom : 11

            border-style : none

        draw : line

    elsefor : road secondary

        define

            min-zoom : 12

            line-color : #FDCC8B

            line-width : 7:1;9:1.5;10:2;12:4;13:5;16:8;18:10

            border-style : solid

            border-color : #FDCC8B black 20%

            border-width : 15%

        draw : line

        define

            min-zoom : 9

            max-zoom : 12

            border-style : none

        draw : line

    elsefor : road tertiary

        define

            min-zoom : 13

            line-color : #FEFEB2

            line-width : 7:1;9:1.5;10:2;12:4;13:5;16:8;18:10

            border-style : solid

            border-color : #FEFEB2 black 20%

            border-width : 15%

        draw : line

    elsefor : (.*residential)|(.*unclassified)

        define

            min-zoom : 13

            line-color : white

            line-width : 13:2;15:10;18:12

            border-style : solid

            border-color : white black 50%

            border-width : 20%

        draw : line

        define

            min-zoom : 15

            font-family : Arial

            font-size : 15:7;18:10

        draw : text

    elsefor : road track

        define

            min-zoom : 12.5

            line-color : #9D7517

            line-width : 1.5

            line-style : dash

            border-style : solid

            border-color : white

            border-width : 100%

            border-opacity : 0.3

        draw : line

    elsefor : road footway

        define    

            min-zoom : 13

            line-color : #F68474

            line-width : 1.5

            line-style : dot

            border-style : solid

            border-color : white

            border-width : 100%

            border-opacity : 0.3

            curved : false

        draw : line

    else

        stop

rulefor : river

    define

        line-color : #B5D0D0

        line-width : 7:1;8:2

    draw : line

rulefor : national park

    define

        fill-color : #8DC56C

        fill-opacity : 0.2

        line-style : none

    draw : fill

    define

        line-color : #8DC56C black 20%

        line-opacity : 0.5

        line-width : 3

        line-style : dash

        font-size : 3:3;12:12

        text-halo-width : 3

    draw : line

    draw : text

rulefor: parking

    define

        icon-image : http://wiki.openstreetmap.org/images/9/9a/Parking20.png

        min-zoom : 15

        icon-width : 16

    draw : icon

rulefor: contours.*

    define

        line-color : #7f3300

        line-opacity : 0.5

    for : contours major

        define

            line-width : 2

    draw : contour

Since rendering rules parsing and processing has more or less been implemented a few weeks ago, I think it’s time to show how these rules look like.

I’ve tried to mimic some of the styles and colors used for Mapnik OSM map (although this covers only about 10% of Mapnik’s rules). I won’t be explaining these rules right now because I’m interested in what you think of them at the first glance – can you understand the basic concept or not? Are the rules too verbose? Are they legible? The only hint I will give is: indentation is important!

So here it is, the first part (I have to split this into two posts, Wordpress problems) :

features

    parking : amenity=parking

    contours major : contour[elevation=50]

    contours minor : contour[elevation=10]

    areas

        forest : landuse=forest OR natural=wood

        farm : landuse=farm

        fell : natural=fell

        national park : boundary=national_park

        water : natural=water OR waterway=riverbank

        residential area : landuse=residential

    lines

        river : waterway=river

        railway : railway=rail

        road motorway : highway=motorway

        road motorway link : highway=motorway_link

        road trunk : highway=trunk

        road primary : highway=primary

        road secondary : highway=secondary

        road tertiary : highway=tertiary

        road unclassified : highway=unclassified

        road residential : highway=residential

        road track : highway=track

        road footway : highway=footway

        state borders : relation[boundary=administrative AND admin_level=4]

properties

    map-background-color    : #F1EEE8

    map-background-opacity    : 1

rules

    rulefor : areas

        define

            line-style : none

        for : forest

            define

                min-zoom : 9

                fill-color : #8DC56C

        elsefor : farm

            define

                fill-color : #E9D8BD

        elsefor : fell

            define

                fill-color : #C5FF5B black 10%

        elsefor : water

            define

                fill-color : #B5D0D0

        elsefor : residential

            define

                fill-color : #DCDCDC

        else

            stop

        draw : fill

31.01.

    - indicate the command prompt has focus

    - added detection of changed OSM files

    - implemented get-info command

    - started working on the error reporting func.

02.02.

    - implemented MailService

    - implemented AppDiagnostics

    - added error reporting form, but it still needs to be beautified

    - support for polygons/polylines consolidated from relations

04.02.

    - solved the problem: why is footway drawn ABOVE residential road?

        - it's because the footway feature is defined AFTER the residential road

    - fixed: bug with form disposing        

    - if a way is used in a relation, it should be ignored elsewhere

        -written tests for this        

05.02.        

    - TODO: the rules and features should be specified in the reverse order

        - from the most important to the least important

    - TODO: how to solve layering problem?

- TODO: beautify error reporting dialog

- TODO: find an icon for Maperitive

- TODO: implement multipolygon polyline rendering

    - TODO: write tests for multipolygon rendering

This is an excerpt from Maperitive’s Todo.txt file. I use it as a log of the stuff I did for the current day and also as a “database” of the tasks I still have do. On various projects during the years I used Trac, Bugzilla, on Kosmos I used ToDoList for a while, but nothing beats the simplicity of a text file.

The history log is a recent “invention”: since Maperitive is a pet-project, the time allocated to it is very unevenly distributed – sometimes I do a lot of work in a single day (usually on weekends without find weather or hangovers), but then I have to leave it untouched for several days. So often it happens that I forget what I was working on – and this history log helps me to quickly refresh the memory. Also, it’s a good psychological tool: I take a look and the log entries and see that some work has actually been done and I’m (slowly) progressing towards the first release.

The added benefit is that I can copy&paste these log entries into SVN commit comments. And of course, Todo.txt is kept under the source control like the rest of the code.

photo credit: ????

I’ve been twitting about it for a few weeks: I’ve decided to give a new name to the upcoming Kosmos v3.

First to explain my reasons for the renaming. Let me go back to end of 2007 (huh, was it so long ago?) when I started working on a new pet project: a desktop mapping application. Back then I noticed most of the cool stuff related to OpenStreetMap had to contain “osm” in its name, so I searched for words containing “osm” on the net. I’m not mentioning other candidates, but I settled on more exotic version of “cosmos”, to improve searchability.

Well, it turns out some other people thought the same, so now we have all sorts of stuff also called Kosmos: Russian rockets, some new-age journal, a Greek car rental agency, an oil company and I don’t know what else. So when you want to find Kosmos on Google, it typically turns up on the bottom of the first results page. Which is kind of uncool

The second reason for renaming is the fact that Kosmos name had not really have to do anything with the actual purpose of the application.

So a month ago I started toying with the renaming idea. Since I wasn’t too sure if it was a good idea at all, I decided I’d stay with the old name if I couldn’t find more catchy and unique one. I have to say it was a hard search: I have a few dozen name candidates listed in my private Wiki (I’m not revealing any of these to avoid causing distress to readers) – but most of these are either a) already used for other stuff or b) just plain ugly. (As a sidenote: my friend Boris (jokingly I hope) suggested a name that would put my blog squarely in the internet XXX territory. Let’s just say it related the mapping with some human private parts, I won’t go into details on this one).

And The Name Is…

OK, I won’t bore you to death: the name I chose is Maperitive. Let me retype this without any spelling errors: Maperitive. But what does it mean, you might ask? Well, it’s a pun: a map and an apéritif. So to paraphrase a Web dictionary: it’s stimulating the appetite for mapping.

I’m not sure you’ll like the name. As a matter of fact, I’m not sure I like the name that much myself – but it’s better than the other candidates. It’s not revolutionary like iPad or MS Paint. But what I especially like about it is that at the time of me writing this post, it has an absolute zero search results on Google. So Maperitive it is. No going back.

Other News

Other than name-searching, I did a fair amount of work on the actual application, mostly on the parsing of rendering rules and the graphics. I’ll cover this in another post this weekend, but right now I’m finishing this post to go search for a good algorithm for calculating the area of a polygon on Earth surface. Next time I’ll tell you why…

In the previous post I mentioned my work on the command-line interface for the new Kosmos v3 GUI. I’m more and more excited with the possibilities this feature opens. A lot of things which can be tedious to code because of the required GUI widgets can now be provided fairly quickly through the command line (this doesn’t mean there won’t be any GUI, but certain functions are better served through a textual interface).

I managed to implement some basic parsing for what I call spatial specifications which will be used in the new rendering rules system. An example of a spatial specification would be something like…

way[highway=footway].[barrier=gate]

…would select gates on footways.

I wanted to test this parser and so I implemented a simple find command for querying OSM data:

It’s just a prototype – I plan to add some kind of browsing of query results (both on the map and in the log window) – but it shows the possibilities of this interface. The querying could extend to GPS data, too. It could also be used to filter out GPX data (based on the date, for example). These commands could be written in a script file and reused every time you want to process your GPS logs, for example:

1. Download data from the GPS unit
2. Keep only the data from today
3. Simplify tracks
4. Save this to a GPX file
5. Upload the GPS trace to OSM

…could be fully automated.

Although I haven’t been posting much on Kosmos, this doesn’t mean that nothing is happening. In fact, a lot of new stuff is in development. so much so that I’m having trouble concentrating on finishing stuff .

There are main fronts I’ve been battling since I’ve decided to do a big Kosmos rework: the GUI and the rendering engine. Although I started the rework by concentrating on the GUI code, for various reasons the effort moved to rewriting the rendering engine, especially the rules system.

The problem with this is that now there is a lot of “todo” code on all fronts and it left me wondering what the realistic date for releasing Kosmos v3 would be. Even when the rendering engine has been completed, there would still be a huge amount of work needed to finish the GUI framework.

During a hiking trip a few days ago I started thinking how this could be speeded up. I got the idea that instead of waiting for full-fledged GUI to be finished it would be better to provide a simplified user interface as an intermediate / temporary solution (going back to the old v2 GUI code was out of the question).

So my idea was to provide a set of commands which could be executed by the user through a command window. I know, I know, this is a bit geeky and not too friendly, but I see several good points with this approach:

• The time for releasing the first (“alpha”) version of Kosmos v3 would be shortened by at least two to three months.
• Users will be able to test the two main improvements: the new rendering rules system and the improved graphics engine.
• The idea for the commands is to make them automatically available through menus, so it wouldn’t be a so user-unfriendly after all.
• My plan is to release Kosmos more often than before (at least once a week), so the users would experience continuous improvements to it. Later I would try to implement some kind of auto-updater.

Now let’s see what are the new features I’ve been working on.

Rules Engine

I’ve been writing about this before. The new engine has now been more or less implemented, but I still have to write the parser for it. Also, I want to be able to support the old Wiki-tables rules format (obviously), so there’s some work there before the release. All I can say right now is that the new system will be much more powerful regarding selecting elements to be drawn and much easier to control, without unnecessary repetition when writing rules.

Map Rendering

The idea was to rewrite the graphics code to be able to support different outputs (like SVG) and I think I’ve made a good start. There’s still a lot of work (now there are two separate code bases for GDI+ and for Cairo). Also, the rendering will probably not be as quick as in Kosmos v2, but this is because I plan to work on graphics optimization later.

Scripting

This is a result of the decision to provide a command-line interface inside the Kosmos GUI. The natural next step to this is writing a sequence of such commands. This could then become a powerful automation tool and could also replace the existing Kosmos.Console application. I’m still thinking how best to provide such a scripting functionality. One idea is to have an interpreter of some common scripting language (like Ruby, Python or Lua), but I don’t want to go over the top with this. And I don’t want to open a new front I wouldn’t be able to fight

Spatial Databases

I introduced this topic a few months ago. It is very exciting to see Kosmos working with PostGIS and SpatiaLite, although this feature will not be released very soon – I still think Kosmos is currently most useful when working with smaller map areas which can be loaded from an XML file. Once the general scenarios have been covered, I’ll try to take some time to finish the DB support.

Misc

There’s a lot of other stuff I’d like to see in Kosmos, like better GPX support. The rendering rules will support specifying how to draw GPX features, too. Also, there will be functions like calculating the area size (and using this to filter things when rendering). I would also like to have routing. And editing GPS data (and even an OSM editor). But I guess these things will come later, after v3 has been released and stabilized. Too many ideas and too little time, that’s the problem….

Preview

For those of you who endured to the end of this post, I’m attaching a screenshot of the prototype of what I call “Kosmos Commander”:

Some time ago I wrote about experimenting with Cairo drawing engine to render maps. The basic idea was to use Cairo to render SVG map files which can then be manually post-processed using some vector graphics editor like Inkscape.

Unfortunately I worked on lot of other stuff after that, so the experiment was put on hold. But a few weeks ago I started working on a new rendering rules engine for Kosmos and I wanted to write some unit tests to try it out – and the experimental map drawing code seemed a good fit.

The rules engine is far from complete, but I can already reveal a few details of its capabilities. First, you specify a set of features, for example a forest, a motorway, a town. In accordance to some common GIS concepts, there are three types of features: point, line and area feature.

Then you specify how each of these features should be rendered. This consists of a directed graph of commands which supports inheritance of rendering properties (something in the direction of MapCSS, but not as liberal).

I won’t bore you with the details (at least until a proper parser for these rendering rules is available), but I think this new system will be much more powerful than the existing “flat” Wiki table-based rules one. Also, I’ve been working on improving support for relations and multipolygons.

Anyway: back to Cairo: I’ve started implementing more detailed drawing logic and I had quite a few problems figuring out how to render text using Cairo. It turns out the API for drawing text in Cairo is pretty poor (it even crashes when trying to render non-ASCII chars), so the authors of Cairo recommend using Pango, a library for high-quality text rendering. Luckily, it has Mono wrappers and it wasn’t too difficult to make it write out texts on the bitmap.

I’ve written few unit tests for some simple OSM map rendering and here are the results (the left map is rendered using GDI+, the right one using Cairo):

I’ve used some simple rendering rules:

• texts come from OSM tags equipped with “name” tag
• lines come from any non-closed OSM ways
• orange-filled areas come from any areas (closed OSM ways).

As a special treat, I’ve also created a SVG file from the same map, you can download it from here: http://downloads.igorbrejc.net/osm/misc/map1.svg

photo credit: The Pack

Yesterday I started thinking about a new rendering rules system for Kosmos. The existing system (using Wiki tables) was OK for a while, but it’s starting to show its old age: there is no rule inheritance, the rules are cumbersome to edit (because of wikitext), the selectors are not very powerful, rules are not reusable etc.

I started by investigating how similar software is doing these things. Mapnik has a more elaborate XML-based ruling system. It’s powerful, but it seems too verbose for what I had in mind – I want the rules to be simple to write and even simpler to read. And I want to stay away from XML: XML is easy to parse but not that friendly to humans. And writing expressions using XML character entities (example: “<Filter>[CARTO] &gt;= 2 and [CARTO] &lt; 5</Filter>”) is really unfriendly and error-prone.

The next thing to look at was MapCSS, a CSS-like language for map stylesheets and is a brainchild of Richard Fairhurst, the author of Potlatch, the excellent web-based OSM editor. I like the idea, but I’m worried how it could be implemented in the context of a database-driven map engine. The databases can be huge and you want to minimize (and optimize) the queries run on them, but MapCSS seems to allow too many combinations to make it usable for generating maps in (almost) real-time, and that’s what Kosmos is about. When working efficiently with the database, the rendering engine wants to collect all the related objects in one go, and then decide how to render them. So, for example, if it wants all highways to be rendered red, it’ll fetch all the highways from the database (with a single query) and then go and paint them all red. MapCSS doesn’t really allow this: you need to process each OSM object on its own and go through the list/tree of MapCSS rules in order to know whether (and how) to render it. Maybe I’m wrong and there is a better way to approach this, I’ll have to discuss this further with Richard and see what he thinks.

So now I’m thinking about defining my own domain-specific language (DSL) which would be used to specify rendering rules. I don’t have any specifics yet, but there are some things worth thinking about:

• The DSL should not be XML-based: angular brackets are not for humans.
• Separating definition of features from the definition of styles: features tell you what something represents. A forest, for example, is a feature that corresponds to an area tagged with natural=wood or landuse=forest. How it is rendered depends on the type of the map: on topo maps you would render it in green color. On driving maps maybe you wouldn’t render it at all. So the idea is to have a features definitions that could be stored in one place (one wiki page) and then reused all across the styling rules. Right now this is not possible in Kosmos.
• Expressions: the selector expressions should be more powerful than just simple logical operators. The same goes for expressions for style attributes.

Anyway, this all means I have to implement some kind of a parser for the DSL. So I started looking into possible C# libraries/tools for writing parsers: ANTLR, Coco/R, GOLD, Irony.NET, … There is also a good article called Grammars and parsing with C# 2.0. Time to do some refreshing of knowledge lost way back from my university days…