igorbrejc.net

UPDATE: A bug has already been discovered thanks to StefanB… a new download link for the fixed version.

There were some comments from this blog’s readers about me not delivering new Kosmos versions as promised :). So I decided to wrap up everything and produce a new version as soon as possible. Some things I planned for this release are postponed to the next version(s).

Last two to three weeks my main effort was in implementing the slippymap (web map) integrated directly into the main map. You can now view the whole Earth using one of OSM’s web maps (Mapnik, Osmarender, CycleMap). I find this to be a much better solution than opening an extra Web map browser window and then fiddling with map buttons like it was in previous versions. That’s why I plan to remove web map browser from Kosmos in future versions (unless some users find it useful, please let me know).

You can set the transparency of web map tiles so you can combine Kosmos rendering of OSM data together with the tiles. The slippymap tiles are also rendered on exported bitmaps and Kosmos’ web map tiles. I haven’t been able to show them on printed pages though, I have to investigate this further.

Icons can now be placed on areas:

Kosmos GUI now has a Full Map Mode switch which, when turned on, maximizes the map display and hides all other windows (see the screenshot at the top of the page). There is also a new Reload Rendering Rules menu item (and hotkey) to ease experimenting with rendering rules.

Another improvement is in Kosmos’ performance (both speed and memory-wise). I already wrote something about it in September. Anyway, you should be able to load larger chunks of OSM data now.

Below is a list of other new features and bugfixes for this release. As always, please let me know if you encounter any new bugs. The code base has changed a lot since the last version because of all the new features added to Kosmos, so there is always a chance of catching a bug or two…

- BUGFIX: multiple rule selectors did not work with {{tag|key}} form
- BUGFIX: density parameter for symbols wasn’t handled properly
- generation of SRTM index can now be canceled by the user
- BUGFIX: the generation of SRTM index sometimes hangs, probably due to FTP client command executing too quickly
- when determining map boundaries, data files which are hidden are ignored
- BUGFIX: when showing large bitmaps there are visible white horizontal stripes
- changed the way how the OSM tags are stored in memory (array instead of Dictionary)
- OSM reading code now ignores “created_by” tag - this results in a significant memory savings, since all of the nodes
have this tag
- BUGFIX: the map-related menus remained visible when the project was closed
- implemented custom reading and writing of OSM XML files
- various performance optimizations
- BUGFIX: direct URLs for user pages containing rendering rules did not work
- TextLineOffset-ing is now available for Node, AreaCenter and WayCenter text modes
- added “Max zoom factor” setting to limit the allowed zoom level. This is in order to avoid application freezing up on high zooms.
- Kosmos now uses arrays instead of Lists for storing way’s nodes list
- added support for “bounds” osm tag, “bound” is now deprecated
- BUGFIX: Color alpha value in map shading code is now checked not to exceed the allowed range
- added User-Agent header when accessing Web

This weekend I was playing around with Kosmos by trying to load large maps (by “large” I mean country-size large). I have never really done this before, since I was more concentrated on smaller areas when I developed the software.

The initial results weren’t really encouraging - Kosmos managed to load the OSM extract of Austria (25 MB of .osm.bz2 file), but the GUI was burning around 1 GB or RAM and it took some time to load. Then I got into investigating how this could be improved.

I managed to do a few things which greatly improved the performance:

• OSM tags were stored as C# generic Dictionaries (for each OSM element). This is a problem, since typically there are no more than few tags for an element. It turns out the ListDictionary class is better suited for small dictionaries, since it uses linked lists instead of hash tables.
• A lot of memory was being taken by “created_by” tags for each OSM node. Since this tag isn’t really much used when rendering maps, I decided to ignore it when loading OSM files (OSMXAPI ignores it too, by the way). This can save a lot of space, since most nodes do not have any other tags - which means the tags dictionary doesn’t even have to be created for such a node.
• Immediately after loading the map, the garbage collector is now called to dispose of any temporary objects. I know, GC isn’t supposed to be called manually by applications, but I guess in these extreme circumstances it can be forgiven - especially since it frees up several MB of RAM immediately.
• The last thing to do was to write a custom OSM XML loader. Until now I used XmlSerializer classes autogenerated from OSM XSD. This has several disadvantages: you have to load the whole OSM file into memory in order to (de)serialize it and also produces a large quantity of intermediate objects which then have to be translated into “proper” OSM objects. But by writing a custom loader which reads XML sequentially (using XmlReader) both of these obstacles can be overcome. Still, writing XML parsing using XmlReader isn’t easy (in fact is real pain in the you know where).

Anyway, after these modifications Kosmos loads Austria data more quickly and with much less memory footprint (around 400 MB). It’s still a slow experience on my machine (E6400, 4 GB RAM), but at least it’s improvement.

I tried to load progressively larger country extracts - the largest one that did load so far without throwing the OutOfMemoryException was France (54 MB). The screenshot above shows Czech Republic (43 MB).

NOTE: by doing this I wasn’t really aiming to achieve some kind of “planet coverage” with Kosmos - it’s virtually impossible for Kosmos GUI to draw the whole planet interactively. But I would like for non-interactive Kosmos Console at least to be able to generate tiles for larger areas (UK or Germany, for example, or even the whole of Europe) in some reasonable time - but there’s a lot of things to do before this is possible. I guess the first steps have been made…

I’ll be upgrading my blog to the latest WordPress version this weekend, so there will probably be some glitches (the download site should work though). Sorry for any inconvenience.

UPDATE: the migration was easier than I expected - hopefully no hidden problems will appear later. If you encounter anything strange, please send me a note. Thanks!

Yes, Kosmos has reached a new version (download the latest package from here). It contains some cool new features, but I have spent most of the work on improving the user experience. But let’s start with new features:

Mobile GMaps support

I’ve already blogged about this. In short, you can now use Kosmos-generated maps on with a Java mobile application called Moble GMaps (MGMaps). The idea is to generate map tiles from your Kosmos project, copy them to your phone and tell MGMaps to use them. Since the map is stored on the phone, you don’t need to use the mobile network or pay anything to view it. I’ll soon write instructions on how to set up the whole thing on the Kosmos OSM Wiki pages. UPDATE: the instructions are here.

Error list window

When loading projects, parsing rendering rules etc. Kosmos GUI will now display all the errors and warnings in a special “error list” window. This is combined with much improved handling of various errors in rendering rules - instead of causing Kosmos to crash, you should now be able to see all the detected errors in this window. In case some a parameter has an invalid value, the rendering rules engine will now use a default value - typical example is entering an invalid color value.

Handling of rendering rules errors has been a longstanding issue with many users - hopefully this has been improved now, but there will probably be some bugs, so please report if you encounter any.

Changes in the Kosmos console

Kosmos console has a new command called “zappy” which allows you to download OSM data from the OSMXAPI server. This has been already available in Kosmos GUI, but I wanted to extend it to command line support since it can be used for automating certain tasks. A simple example: you can now periodically download new OSM data for a certain area and generate fresh tiles from this data - so your tile server will offer up-to-date maps.

By the way, the command line options for generating and serving tiles have been changed. Instead of having to manually edit parameters in the Kosmos project file, you can now specify everything you need from the command line. Also, these parameters will be stored in a special file together with generated tiles, so when serving tiles, you only need to specify the location of the tiles cache. I’ll write more about this on the OSM Wiki later. UPDATE: the instructions have been updated.

Exporting georeferenced bitmaps with world files

All bitmaps generated in Kosmos (except the tiles) are now equipped with a world file, which should allow users to use these bitmaps in other GIS tools. However, this is only a partial solution - you still cannot import bitmaps into Kosmos and also GeoTIFFs are not supported. I will implement importing of bitmaps with world files in later versions, but only if they use Transverse Mercator projection. Supporting other types of projections (by using image rectification) is not planned in the near future since it’s beyond the scope of this application (read: I just don’t have enough time to do it).

NOTE/WARNING: you will not be able to open bitmaps from projects created with previous versions of Kosmos, since the new version does not use the georef information embedded in the project file - it now uses world files for this. Unfortunately backward compatibility is one of the things I chose to sacrifice due to lack of time - mostly because I think all the bitmaps generated in Kosmos can be regenerated without much problems.

Other new stuff in brief

• Generating relief contours in GUI is now customizable - you can specify the elevation interval. I also wanted to add customization of how the contours are rendered, but this will have to wait for the next release.
• Kosmos can now read GPX v1.0 files
• Entering of direct OSM wiki URLs for rendering rules (like http://wiki.openstreetmap.org/index.php/Kosmos_General_Purpose_Rules) is now allowed.
• Data files that cannot be loaded in the GUI project are now greyed-out.
• The same way can now appear in multiple multipolygons without causing Kosmos to crash.
• Various bug fixes.

That’s it - the next task is to update the Kosmos documentation in the OSM Wiki. After that, I’ll try to get back on Kosmos 3D stuff which I neglected for the past few months.

I hope you enjoy the new version and please report any bugs you discover (there’s bound to be a few of them ).

The work on the new version of Kosmos is in full speed. I planned to release it in August, but I had too little spare time for it - most of my weekends were spent hiking and mapping and was also pretty busy on my “real” work.

Anyway, I updated the “Kosmos Future” page on OSM wiki pages with the latest status. What I am really excited about is the new feature - showing Kosmos-generated maps on mobile phones using MGMaps, MGMaps is a free Java ME application for showing Web maps from various sources (OpenStreetMap, Yahoo, MSN and more). The default behavior is to download map tiles on demand, but the user can also download them manually and store them in the phone’s memory.

This “stored maps” feature of MGMaps gave me the idea of providing a way to show Kosmos maps with MGMaps. I added support for generating MGMaps-style map tiles in Kosmos console - you simply generate the tiles for a specified Kosmos project and then copy them to your phone. The results can be seen here (NOTE: the “Osmarender” text on the bottom of the screen is shown because I had to mark my tiles with one of the map types supported by MGMaps, and I chose Osmarender).

Apart from the fact that using this method you do not overload OSM servers by mass-downloading tiles, it also provides a way to have maps customized to your needs on your phone to help you with the navigation (MGMaps can connect to a GPS receiver). I’m already thinking about using this solution when hiking instead of Garmin + mkgmaps combination. Note however that MGMaps work for Java ME-enabled phones only.

I decided to take my time and study the literature on 3D rendering, especially on maths (vertices, matrices, transformations…), which I forgot all about since my student days. I can highly recommend the 3D Math Primer for Graphics and Game Development book which (contrary to many math books for developers) covers not just mathematical equations and theory, but also tries to explain this in a way more familiar to developers.

I abandoned the approach of continuing working on the cannibalized code from Mini 3D Terrain Viewer simply because there were too many problems which I didn’t know how to solve easily. Anyway, I think the skipping over basic stuff like 3D maths and DirectX is not very wise - sooner or later the things you neglected come to haunt you.

I started posting current results of my effort on flickr - this will be a sort of a snapshot history of Kosmos WorldFlier from the day one, step by step. Not much to see yet - I managed to render all OSM ways as 3D lines, you can see the Isle of Wight there. This is just a temporary solution - road should be rendered as mashes, but that will come later. Right now I’m working on the camera navigation stuff.

I’m still using DirectX managed SDK for this. I decided to implement the inital version with this SDK simply because I already have some limited experience with it - I didn’t want to make this harder by introducing some other SDK which I haven’t worked with yet. Later, I’ll probably migrate the code to use some other library (there are several options: OpenTK, Irrlicht or something else).

I uploaded some new screenshots of WorldFlier to flickr.com, this time of Mount Etna. The work on WorldFlier is progressing, but there are still a lot of things to fix - relief still doesn’t exactly match the contours (well it is a prototype after all).

The screenshots show WorldFlier and Kosmos GUI - what you set in Kosmos GUI to be visible will be displayed in the 3D. So first I generated a elevation coloring map and then I used Landsat’s satellite images.

No OpenStreetMap data this time - there were some changes in OSMXAPI that prevented me from downloading the latest data.

Kosmos is today featured on the German IT-news portal golem.de. The article presents the Kosmos application and also features screenshots and a video I recorded few days ago of the upcoming WorldFlier 3D rendering addition to the Kosmos.

This is the second article about Kosmos on golem.de. The first one was published in January.

I uploaded a short video of the OSM 3D rendering in action:

Now I go to bed

New version of Kosmos is here. Since I’ve neglected the rendering engine during the implementation of 2.0 release, I decided to do most of the work for this release on improving the rendering.

You can download the 2.1 version from the standard place: http://downloads.igorbrejc.net/osm/kosmos/

UPDATE (July 19th): I discovered a small bug in the new release - generated contours were not shown on the map. This has been fixed now, please download a new package.

So what has been improved?

Relations

Kosmos can now render relations. I’ve added a sample rule in general purpose rules Wiki page for rendering cycle routes:

Rule Name	Targets	Selector	Template	Options	Comment
RouteCycle		type=route route=bicycle	Polyline (MinZoom=7, Color=#50FF0718, Width=7:1;17:18, Curved=true)

will produce a pinkish half-transparent cycle route:

Note that rules ignore roles of members in a relation - currently I don’t see how the roles could be incorporated into the rendering rules.

Text Rendering

There is a new TextMode called WayCenter which draws the text (roughly) in the center of a way:

There are also two new parameters which enable rendering of outlined text: OutlineColor and OutlineWidth, example (sorry for the odd formatting):

PlaceTown place=town Text (MinZoom=10, MaxZoom=15, Color=black, TagToUse=name, FontName=Trebuchet MS, FontStyle=bold, FontSize=10:10;17:25, OutlineColor=white, OutlineWidth=20%)

will produce something like this:

Kosmos now knows how to wrap labels:

The wrapping is done using the “Text label wrap factor” preferences setting. It is a ratio of (roughly) maximum line length compared to the line height. Normal values are somewhere from 4 to 10, I guess.

Map Scale Indicator

… is now shown on the map:

You can turn it off. You can also switch from metric to U.S. Customary units in Preferences dialog (”Use metric units“).

Rendering Engine Speed-Up

I’ve refactored quite a large piece of rendering engine code to increase the rendering speed and lower the memory consumption. Basically, in previous versions each rendered map element had all its settings (color name, width, …) stored by itself. Now the settings shared by multiple elements are stored in one place (a “style”). This also allows caching of GDI+ objects for each style without the need to recreate them for each map element.

There are also a number of bugfixes in this release (basically every bug that was reported has been fixed).

Source Code

I’ve also published the sources here: http://downloads.igorbrejc.net/osm/kosmos/source/ . I’ll prepare a special Wiki page with some information about the source code and what one can and what one cannot expect from it. So please be patient before starting to send me hate notes about not being able to compile the source code