igorbrejc.net

These are my new delicious links for October 30th:

• Capstone projects and time management – Joel on Software
  
• OpenGeoData » TomTom down another 10% today
  
• Developer Codpieces
  
• Less Css for .NET
  
• Google adds free turn-by-turn navigation, car dock UI to Android 2.0 (video)
  
• Manual Mocking: Resisting the Invasion of Dots and Parentheses
  
• Rick Byers : AnyCPU Exes are usually more trouble than they’re worth
  

On a current project we’re working on in our company, we are developing a web application which accesses the back-end through some web services. Nothing special really, except that certain web services provide pretty static information like lookup tables, which don’t change very often, so it’s not really necessary to refetch them all the time.

Since we are heavily relying on the dependency injection and all the back-end services are exposed through C# interfaces, it wasn’t really hard to develop a new implementation of such an interface which calls the back-end service and then caches the results in the HTTP cache to be used by any subsequent calls.

Here’s a simple example of a service interface:

public interface IBackendService
{
    List<string> ListSomeStrings(string parameter1, int parameter2);
}

I won’t bore you with an implementation of this, since it’s really a dumb one. What’s interesting is how the cached implementation looks like:

public class BackendServiceCachingProxy : CachingProxyBase<IBackendService>, IBackendService
{
    public BackendServiceCachingProxy(
        IBackendService wrappedService, 
        Cache cache,
        TimeSpan slidingCacheExpiration)
        : base(wrappedService, cache, slidingCacheExpiration)
    {
    }

    public List<string> ListSomeStrings(string parameter1, int parameter2)
    {
        return CallServiceMethod<string>(
            () => ConstructCacheKey("SomeStrings", parameter1, parameter2),
            s => s.ListSomeStrings(parameter1, parameter2));
    }
}

Our caching wrapper inherits from a base class CachingProxyBase. Before showing you the code for CachingProxyBase, let me just quickly explain what the BackendServiceCachingProxy code does.

We’re using HTTP cache, so we have to supply it to the class in the constructor. We also specify how long the cache should be valid (slidingCacheExpiration parameter).

Each service method in the caching wrapper now uses CallServiceMethod method to implement the cached service method calls. You specify two parameters:

1. A function delegate for constructing the cache key. This key should be unique for each unique combination of method’s input parameters. The CachingProxyBase offers a helper method ConstructCacheKey to help you with this task.
2. A function delegate for calling the method on the service. This delegate will be used for calling the actual service implementation which will contact the back-end.

CachingProxyBase

And now the implementation of the CachingProxyBase:

public abstract class CachingProxyBase<TService>
{
    public void ExpireAllCachedData()
    {
        foreach (System.Collections.DictionaryEntry entry in cache)
            cache.Remove(entry.Key.ToString());
    }

    protected CachingProxyBase(
        TService wrappedService,
        Cache cache,
        TimeSpan slidingCacheExpiration)
    {
        this.wrappedService = wrappedService;
        this.cache = cache;
        this.slidingCacheExpiration = slidingCacheExpiration;
    }

    protected Cache Cache
    {
        get { return cache; }
    }

    protected TimeSpan SlidingCacheExpiration
    {
        get { return slidingCacheExpiration; }
    }

    protected TService WrappedService
    {
        get { return wrappedService; }
    }

    protected TValue CallServiceMethod<TValue>(
        Func<string> constructCacheKeyFunc,
        Func<TService, TValue> serviceMethodFunc)
        where TValue : class
    {
        string cacheKey = constructCacheKeyFunc();

        TValue cachedValue = GetCachedValue<TValue>(cacheKey);

        if (cachedValue == null)
        {
            cachedValue = serviceMethodFunc(WrappedService);
            CacheValue(cacheKey, cachedValue);
        }

        return cachedValue;
    }

    protected string ConstructCacheKey(
        string function, 
        string environmentId, 
        params object[] args)
    {
        StringBuilder cacheKeyBuilder = new StringBuilder();
        cacheKeyBuilder.AppendFormat(
            CultureInfo.InvariantCulture,
            "Proxy-{0}-{1}",
            function,
            environmentId);

        foreach (object arg in args)
            cacheKeyBuilder.AppendFormat(CultureInfo.InvariantCulture, "-{0}", arg);

        return cacheKeyBuilder.ToString();
    }

    private void CacheValue(string cacheKey, object value)
    {
        Cache.Insert(
            cacheKey,
            value,
            null,
            Cache.NoAbsoluteExpiration,
            SlidingCacheExpiration);
    }

    [SuppressMessage("Microsoft.Design", "CA1004:GenericMethodsShouldProvideTypeParameter")]
    private T GetCachedValue<T>(string cacheKey) 
    {
        return (T)cache.Get(cacheKey);
    }

    private readonly TService wrappedService;
    private readonly Cache cache;
    private readonly TimeSpan slidingCacheExpiration;
}

TService is a generic parameter representing the service interface the proxy is enhancing with the caching. I think the code is pretty self-explanatory. The ExpireAllCachedData method is exposed to be used in unit tests to make sure the cache is empty before doing any work.

Using It With Windsor Castle

I’ve made a helper method for configuring in Windsor Castle a specific back-end service with or without caching:

[SuppressMessage("Microsoft.Design", "CA1004:GenericMethodsShouldProvideTypeParameter")]
protected void RegisterServiceWithOptionalCache<TService, TImplementation, TCachedImplementation>(
    IWindsorContainer container,
    TimeSpan slidingCacheExpiration)
    where TImplementation : TService
    where TCachedImplementation : TService
{
    string serviceName = typeof(TService).Name;

    if (IsCachingUsed)
    {
        container.Register(
            Component.For<TService>().ImplementedBy<TCachedImplementation>()
                .ServiceOverrides(ServiceOverride.ForKey("wrappedService").Eq(serviceName))
                .Parameters(Parameter.ForKey("slidingCacheExpiration").Eq(slidingCacheExpiration.ToString())));

        if (log.IsDebugEnabled)
            log.DebugFormat("Registered {0}", typeof(TCachedImplementation).Name);
    }

    container.Register(
        Component.For<TService>().ImplementedBy<TImplementation>()
            .Named(serviceName));            
}

Conclusion

There are probably more elegant ways of doing this (probably using Windsor’s interceptors and reflection), but I think this code does its job well if you have a small number of service methods to cover.

It’s OCZ 120 GB Agility. I had to fiddle a bit with the power cables since my power supply only provides two SATA power cables, and I already had two disks using them. Unfortunately OCZ SSD doesn’t come with a 3.5” mount bracket to fix it firmly into my housing, so I have to buy this separately. Or I’ll just use DIY suspension mounting.

One other nuisance: the SATA power cable doesn’t want to “click” into the SSD’s slot, so it pretty loose right now. I have to find a different way to make it stick.

Now I’m waiting for my copy of Windows 7 to arrive to set up a new SSD-based system.

These are my new delicious links for October 26th:

• Outsourcing Continuous Integration | The Build Doctor
  
• Signing 3rd Party Assemblies without Recompiling
  
• Starter Licenses – Atlassian
  
• Arcetis, the project management company for Subversion
  
• Openstreetmap auf der Offerta on Twitpic
  

These are my new delicious links for October 24th:

• MonoTouch Comparison with Apple Tools – One More Red Nightmare – Los Techies : Blogs about software and anything tech!
  
• Building an iPhone app – Aaron Jensen – CodeBetter.Com – Stuff you need to Code Better!
  
• Two new tools for making Garmin custom maps | GPS Tracklog
  
• Map control confusion – Peter’s Gekko – CodeBetter.Com – Stuff you need to Code Better!
  
• We must ship now and deal with consequences
  
• The Most Challenging Part of Becoming a Freelance Developer | Software by Rob
  

Last couple of months I’ve been working on database support for Kosmos. This work started as a request to produce bitmaps of the whole UK in the British national grid reference system from the OSM data.

The current version of Kosmos loads all the OSM data into memory. This obviously proves to be a problem for larger areas. Some time ago I worked on optimizing Kosmos memory usage and it proved to be quite successful, but there is still a physical limit of how large an area can be rendered.

The latest UK OSM data takes about 200 MB of zipped XML (2.5 GB unzipped). This is way too much for the existing in-memory system, so I had to find another way to render UK without reading the XML file directly.

After a bit of investigation, the obvious candidate for OSM data storage proved to be PostgreSQL / PostGIS. By “obvious” I mean that there were already some tools which could import OSM data into the PostgreSQL database, namely Osmosis. I was totally new to PostgreSQL, but the actual installation of the database engine was pretty easy.

The next step was importing of OSM data using Osmosis. Although this can be achieved with a few simple steps, there were quite a few “tricks” I had to learn the hard way before the performance of the database was satisfactory enough to produce map renderings in a realistic time:

• The PostgreSQL database engine uses non-optimal default settings, so you need to do a bit of investigation to set certain things up. One tip: if you have several disks, make sure your data is stored on a disk separate from the system.
• Osmosis OSM DB import command has a few settings on its own which can have a dramatic effect on the DB query performance. First of all, you need to use the extended OSM DB schema which contains bbox column for ways, otherwise the spatial queries would always have to contain multiple table joins which will terribly slow things down.
• Kosmos rendering rule engine was designed for instant-access in-memory data source. The way the data was fetched wasn’t really compatible with slow DB sources, so I had to do a lot of refactoring on the engine before the performance was good enough for any serious work.

After a few weeks of work, I finally managed to get some decent results. The rendering code generated UK for approximate zoom levels 7, 8 and 9 in about two hours on my machine, with the level of detail similar to Mapnik’s layer of the OSM main map. This may seem a lot of time, but considering there were about 11 million nodes and 1.5 million ways to process, I’m quite pleased. And there are further improvements still possible in the rendering engine which could reduce this time at least by half.

British National Grid

One of the biggest worries I had was how the rendering engine will behave when rendering using a map projection and reference system different from the “standard” Mercator and WGS-84. This actually proved to be the easiest thing to solve: the rendering engine internally uses the National grid coordinates and the spatial DB queries transform those transparently to WGS-84 (and back). I also needed to implement a new map projection, but this was just a matter of writing a few lines of math code.

Other Improvements

Since the map was supposed to be “professional” looking, there were a few things that needed to be implemented or improved on the existing rendering engine. These features will probably be included in the next generation of Kosmos, which will be released some time next year.

Text Placement

One of the first (and hardest) things needed for a good looking map is to make sure the text labels do not overlap each other. This is called (automatic) label placement. Good algorithms for label placement are quite complex and I didn’t really have time to implement a full-blown algorithm, so I chose to do a simple point-selection: the algorithm removes labels for smaller places (towns) until there are no more overlaps (see the sample map at the beginning of the post). I’ll write more about this feature some other time.

Better Relations Support

Some of the features needed to be rendered are now defined in OSM using relations. One example of this is national parks. So a new algorithm for consolidating relation’s ways into a single polygon was implemented.

Better Sea Filling Support

The coastline processing algorithm is now more resilient: it will ignore poorly connected coastlines and will still render all of those which are properly defined. This improvement is already in use in the new GroundTruth version released a week ago.

What Next?

During the work on PostgreSQL support I learned about SpatiaLite, a spatial extension to sqlite. As you may know, sqlite is a popular self-contained database engine which stores the data into a single file. So I decided to do a bit of playing around with it and see if it can also be used in Kosmos. I was a bit skeptical whether SpatiaLite will be fast enough for map rendering, but I can already say that it managed to achieve some very good results. I’ll write about SpatiaLite in the next blog post.

These are my new delicious links for October 20th:

• Never work with thick people.
  
• Continuous Integration, Plugins and Going Too Far
  
• Time for a New Computer – Karl Seguin – CodeBetter.Com – Stuff you need to Code Better!
  
• Help Google model 3D buildings with a simple new tool | Google Earth Blog
  
• 417 – As It Might Have Been: Hexagonal London « Strange Maps
  
• Incredible Satellite Shot Of Mars | Free Geography Tools
  

The new version of GroundTruth has just been released (download link). As I mentioned yesterday, the main work was on filling sea polygons based on the natural=coastline OSM tag. One additional feature is the ability to set the background color of the map (even if there are no coasts in your area). All of this is configured using the Options section of the rendering rules:

Sometimes it is not possible for the GT’s coastline processing algorithm to close up all of the coastlines and the resulting map can get ‘flooded’. In such a case I suggest you should use the new ‘–nosea’ option, which turns off sea polygon filling. Unfortunately a lot of the OSM planet extracts are lousy and they don’t include enough of the coastline to be able to connect it to the extract boundary, so I guess ‘-nosea’ option will have to be used quite frequently.

Some Background Information

This version of GroundTruth contains a lot of new (and improved) code which I was working on for the past several months. The work is primarily targeted towards a new generation of Kosmos tools, but since GT shares common geometry and OSM libraries with Kosmos, it can benefit from these libraries too.

These are my new delicious links for October 18th:

• Coding Horror: The State of Solid State Hard Drives
  
• Castle Windsor and the LoggingFacility
  
• Do you sign the default contract?
  
• My thoughts on software piracy
  
• Review: GoGrid vs.Amazon EC2
  
• Windows 7 HD and SSD Performance Analyzed – HotHardware
  
• Guide Vista 32/64 SSD Windows Registry tweaks – OCZ Forum
  
• 3D Map of Bournemouth with shockingly high detail
  

Before going to bed I’m posting first screenshots of newly implemented coastline rendering in the next version of GroundTruth (due to be released tomorrow, if all goes well). The previous versions just rendered coastlines as lines, now the sea actually gets colored.

The maps shows parts of Isle of Wight, the screenshots were taken from Garmin Oregon 300.