This is a brainstorming post about an idea I’m toying with since morning.

The Problem

First let me describe the problem. Our team is into continuous integration (CI) and we are usually using multiple servers in the CI process: one for building the code and one for integration or acceptance tests. We usually use CruiseControl.NET project(s) to run the initial build and then push the build package onto an integration server (which also uses CruiseControl) to be tested thoroughly. This works, but the problem is writing and setting up scripts to wire all these things together. It boils down to several steps:

  • Packaging multiple files into a ZIP package.
  • Transferring the package to a remote location.
  • Unzipping the package there.
  • Executing a script or executable from the package.
  • Pushing the results somewhere else.

These steps apply for various procedures, from deploying software to running integration or GUI tests. So basically we have a package we want to run on a remote computer. My usual approach was to have a custom batch files on the remote computer which unzip the package and then run executables inside. The problem is that:

  • This is a custom solution for each individual situation. Every time we encounter another such scenario, new batch files need to be prepared.
  • It requires pre-installation (i.e. manually copying of these batch scripts to the right place) before the actual process can be started. And again, each scenario requires its own pre-installation.
  • CruiseControl.NET: although it is flexibile enough to allow this kind of usage, it is too cumbersome to set up.
  • Since the scripting code runs on physically separate locations, they it is also separated into multiple source code files, which means an additional effort is needed to maintain them (keep them in sync).

The Solution

What I was thinking about is some sort of a service (let’s call it Agent) that would accept requests for publishing certain files to a remote location and then running client-specified file(s). The Agent would have a .NET client library which would look like this:

using (IPublisher publisher = PublisherService.GetPublisher("remote_machine_name"))
{
    publisher.Publish(
        PublishingMode.ZippedFile,
        @"..\..\IntegrationTests.zip",
        @"C:\temp\IntegrationTests",
        @"RunTests.exe");
}

The Agent would of course have to be running on the remote machine. The client library would take care of zipping files into a single package (if necessary) and sending them to the remote publisher service. The remote publisher service would unzip the package to the requested location (in this case "C:\temp\IntegrationTests") and run the requested executable ("RunTests.exe"). Some things to note:

  • The call to Publish() would return after the files have been transferred to the remote computer. The actual execution on the remote machine would then proceed asynchronously.
  • There would have to be some kind of way to specify which user accounts have access to the Agent. Only those users would be allowed to publish stuff.
  • The executable would run under the Agent’s Windows service account.
  • I still haven’t figured out what kind of protocol to use to actually transfer the files (HTTP?).
  • The Agent should be able to run either from a command-line or as a Windows service (much like CruiseControl.NET does).
  • It should require the minimal effort to install, configure and maintain once it has been set up.

I should also note that Agents should only run on internal build/test servers, not production ones, since this kind of approach poses a considerable security risk – so I’m not saying it’s secure.

Final Thoughts

Agents would be installed only once and access configured for users that actually use them. After that the service would be available at all times to process various publishing requests of the CI process(es).

As I said from the start, this is just an idea. We’ll see if anything comes out of it ;)