igorbrejc.net

A few months ago I was looking for some way of tracking activity on our new software project. What I wanted is a free and simple tool which could analyze Subversion logs and generate some statistics and graphs on lines of code, commits and similar stuff.

So I did a search on Google, SourceForge and some other search engines and I found two tools: StatSVN and (oddly enough) SvnStat. The latter generates a single HTML page with a collection of statistics graphs on daily commits (here is a sample report), while the former is more sophisticated and does analysis on lines of code, file and directory sizes, commit activity analyzed per hours of day and days of week and other things (see sample report). It even shows a repository heatmap - a clickable hierarchical map of your SVN repository with indicated increases or decreases of lines of code per directory or file.

The biggest difference between them is that StatSVN not only analyzes the SVN log file, but also goes through the latest version of your SVN repository. That’s how it calculates the lines of code in your project, for example.

I decided to use both of them, since they complement each other (SvnStat generates some graphs which the other one does not). I created a batch script and set it to run as a scheduled task on our build server. Here is a generalized version of the script, you have to fill in your specifics (see below):

<1> = a file path to your checked out repository

<2> = a place where you want to put the SVN XML log file (the whole file path including the name)

<3> = a place where the first (SvnStat) report will be generated

<4> = a place where the second (StateSVN) report will be generated

One warning: the reports also analyze activity for each SVN user, so watch out if you have members in your team that feel that this is a little too Big Brother-ish.

I work on a lot of different projects, both in the company I work for and in my spare time. A lot of code has been produced during that time and I constantly find myself searching for that “I remember I once wrote something like this” code. Google helps, but its search engine is not specialized for C# code and I sometimes have to do a lot of searching before finding a snippet that is perfect for my needs. Snippets in VisualStudio are helpful too, but I use them mostly for really repetitive code which merits taking time to write the snippet.

So I started to look for a search engine that would be able to go through my C# source code (and possibly source code repositories like SVN) and offer more developer-oriented search capabilities (like, for example, searching for all classes that implement a certain interface).

Desktop Search

The first thing popping up in my mind was using general desktop search engines (like Google Desktop or similar). In fact I used this approach few years ago when I played around with desktop search engines on my laptop. The problem with general search engines is the fact that they are too general: when you search for code using general search, all kinds of unrelated results pop up. And besides, installing Google Desktop just to search for my source code is a bit of an overkill for me.

Specialized Search Engines

My first stop was Koders Pro. This is a professional solution provided by Koders, an Internet source code search engine which indexes open source projects. I tried Koders Pro beta version just before it went to market and I liked it. It offers most of the features I was looking for. I liked the fact that it knows how to connect to various code versioning systems like SVN. What I didn’t like is the fact that you have to sign up online on Koders website in order to access your company’s internal search site. I like to have those things separated: what I do on my (or my company’s) private network is my own business, thank you (looks like someone else agrees with me).

Open Source Solutions

But since I wanted to have the search capability available not only on projects I work on professionally, but also for my own personal stuff, I was looking for a free (or not too expensive) preferably open source solution. So I started investigating (more or less googling).

I found a very interesting article called Using Lucene to Search Java Source Code. As the name says, it describes how to implement a search engine for Java source code using Lucene, an open source search engine library. The article describes the approach in quite a detail, I won’t go into it since this is not the purpose of this post. But anyway, it was a good point from which to start searching in a more focused way. I also found a related article, Source Code Search with Syntax-Based Heuristics, which builds upon the first article’s idea, but is more ambitious.

CS2 Project

Finally, I stumbled upon CS2 Project. This is an open source academic project created by Simone Busoli. It too uses Lucene (Lucene.Net actually) as a search engine. What I like about CS2 is its extreme simplicity. You just download the zipped binaries package, extract it somewhere on your disk, create an IIS web application for it and that’s it! No MSI installations, no extra Windows services.

When you browse to the CS2 application page, you get two text boxes: the first one is used for entering search keywords, the second one for registering paths on your disk which contain source code and should be indexed.

CS2 even offers some basic query parameters, like searching for method names, comments etc. It also supports search wildcards.

Okey, it’s not feature rich as Koders, but on the other hand the basic functionality it offers is a step in the right direction.

OpenGrok

While I was writing this post I ran into OpenGrok, an open source search engine written in Java. It supports multiple languages search, but I couldn’t find any information on whether it supports indexing of C# sources.

I will post about any new findings in the future.