My Blahg

April 28, 2008

CruiseControl.NET and multiple build configurations: Part 4 – Velocity Templates

Filed under: Continuous Integration/CruiseControl.NET, NAnt, c#, dotnet — treyhutcheson @ 3:16 am

[More Posts On This Topic]

This is post 4 in a series devoted to extending CruiseControl.NET to support multiple build configurations for a single project. In this post, I will cover the use of the Velocity engine and outputting responses from the plugin.

ICruiseAction.Execute method
The second post in this series addressed the interfaces that your plugin is required to implement. The main method to implement is the ICruiseAction.Execute method, which accepts an ICruiseRequest object and returns an IResponse object. What exactly is an IResponse, and how can your plugin create it?

As mentioned in other posts, if you ever have questions about how something works, I suggest that you consult the ccnet source code. The main reference that I’ve been using is the ProjectReportProjectPlugin class. Examination of this class shows that it uses an object called IVelocityViewGenerator to generate responses. It acquires the IVelocityViewGenerator reference from its constructor(injected by the runtime).

IVelocityViewGenerator – How is it used?
The following code has been borrowed from the ProjectReportProjectPlugin’s Execute method implementation:

Hashtable velocityContext = new Hashtable();

velocityContext["projectName"] = projectSpecifier.ProjectName;
velocityContext["externalLinks"] = farmService.GetExternalLinks(projectSpecifier);
velocityContext["noLogsAvailable"] = (buildSpecifiers.Length == 0);

return viewGenerator.GenerateView(@"ProjectReport.vm", velocityContext);

Take a moment and let the code sync in. The object is only used once, in the method’s return statement. The interface is rather simple, and has but a single method. The interface follows:

public interface IVelocityViewGenerator
{
  HtmlFragmentResponse GenerateView(string templateName, Hashtable velocityContext);
}

If you examine the HtmlFragmentResponse object, it implements the IResponse interface, which is the return type for the ICruiseAction.Execute method.

The GenerateView method accepts a template name, and a velocityContext; a basic hashtable. The return statement from the code listing shows a string literal, “ProjectReport.vm”, and a hashtable.

The hashtable that is passed to the GenerateView method has been populated with a number of items, such as the project name. The hash table is keyed on string values, and the values are objects.

Now that you know what the GenerateView method accepts, what do you think the arguments actually are? What is “ProjectReport.vm”, and what is that hashtable?

Velocity & Templates
The literal “ProjectReport.vm” is obviously a filename, so I looked through the ccnet distro for the file. The file is located in the dashboard’s templates directory, along with a bunch of other .vm files. It’s a simple text file, that appears to be some sort of html template. The full source for the file follows:

<h2>Project Report for $projectName</h2>

#if ($externalLinks.Length > 0)
<table class="ExternalLinks">
	<tr>
		<th>External Links</th>
	</tr>
	#foreach ($externalLink in $externalLinks)
	<tr bgcolor="Ivory">
		<td align="Left">
			<a href="$externalLink.Url">$externalLink.Name</a>
		</td>
	</tr>
	#end
</table>
#end

<p>
#if ($noLogsAvailable)
There are currently no build logs available for this project - make sure your Build Server configuration is correct and that this Project's build is not throwing any exceptions
#else
	#if ($pluginInfo)
	<h3>Most Recent Build Results</h3>
	$pluginInfo

	Click <a href="$mostRecentBuildUrl">here</a> for the full build report.
	#else
	Click <a href="$mostRecentBuildUrl">here</a> for the most recent build report.
	#end
#end
</p>

Like I said, it appears to be html. However, scattered throughout the source are various directives. Statements or commands appear to begin with the pound sign, while property/variable references appear to begin with a dollar sign. In fact, the very first line of the template refers to a property/variable named $projectName. Does that ring a bell? If you refer to the first code listing, the velocityContext hashmap has a key named “projectName”.

Putting two and two together, its apparent that the IVelocityViewGenerator.GenerateView method takes the name of a template, and the second argument, the velocityContext hashmap, contains references that can be expanded when the template is processed.

This templating system piqued my interest, and I discovered that Velocity is the name of a templating project hosted by Apache. The official home page is available here. The reference is here, and the user’s guide is here. These resources will prove to be very valuable as you develop your own velocity templates.

And one more thing. This is a dotnet project, so the original java version of Velocity isn’t being used. One of the project references for the dashboard is an assembly named NVelocity. A quick google search of that term yield the home page for the project, which is a dotnet port of Velocity. I haven’t fully researched the project, so I don’t know how complete it is, nor do I know the differences. This definitely deserves further investigation.

Conclusion
You should now know how the ICruiseAction.Execute method behaves, and how to generate responses via the velocity runtime. I’ve also provided the docs and reference for Velocity, which will aid you in composing your own custom templates.

The next post will cover the template that I have developed for my build configuration plugin.

April 23, 2008

CruiseControl.NET and multiple build configurations: Part 3 – Dependency Injection

Filed under: Continuous Integration/CruiseControl.NET, NAnt, c#, dotnet — treyhutcheson @ 3:09 pm

[More Posts On This Topic]

This post is part 3 in a series devoted to extending CruiseControl.NET to support multiple build configurations for a single project. In this post, I will cover a single topic: ccnet’s method of dependency injection.

Constructor Dependency Injection
The ccnet docs for developing dashboard plugins contains a section titled “Defining dependencies”, which states:

The CruiseControl.NET Web Dashboard uses a Constructor Dependency Injection (CDI) pattern to enable classes to define what types they are dependent on. The Dashboard API has a number of types you can depend upon which can do things like return you the currently viewed build log. You can also specify dependencies to your own types. It is recommended that you use interfaces to define responsibilities.
The Dashboard's CDI implementation does not currently allow for runtime configuration. It will use sensible defaults where available. We plan on adding more configuration later.

That’s all great, but what does it mean? From my first pass of the docs, I understood that somehow my constructor would be analyzed and the correct object references would be provided to my plugin. A fine theory, but how does it work in practice? Specifically, *which* objects would I receive?

Again I consulted the ccnet’s source code. I referred to the handy ProjectReportProjectPlugin class, and found the following constructor:

public ProjectReportProjectPlugin(IFarmService farmService, IVelocityViewGenerator viewGenerator, ILinkFactory linkFactory)
{
}

This class is receiving a reference to the IFarmService interface, IVelociyViewGenerator interface, and the ILinkFactory interface. I copied these same arguments into the constructor for my own plugin, and attached to the asp.net process. I refreshed the dashboard page, caught the break point in the debugger, and sure enough, my constructor received a valid object reference for each argument. It must be magic.

Constructor Argument Type Availability
I’ve now established that at the minimum I can receive valid object references for three interfaces. But what else is available? At this point, I didn’t know what any of these interfaces actually *did*, so I was worried about obtaining dashboard services that might or might not be available through these interfaces.

Still attached to the debugger, I used the Call Stack window to go backwards and find out what was actually instantiating my plugin. This technique is pivotal; you will use this technique many times during the development of your plugin, depending on the services your plugin requires.

Following the call stack backwards shows a bunch of methods being executed within the NetReflector assembly, and the first method one can find that is debuggable is ThoughtWorks.CruiseControl.WebDashboard.DashboardConfigurationLoader.Load method. The last line of the method looks like this:

return NetReflector.Read(node, typeTable);

Evidently, the dashboard is relying upon the NetReflector runtime to deserialize all of the plugins from the dashboard config file. The read method accepts an xml node, and a NetReflectorTypeTable. This guy proves very interesting.

The NetReflectorTypeTable has a member named “instantiator”, of type ObjectionNetReflectorInstantiator. That object has a member named “objectStore”, of type ObjectionStore. All of these objects and members are viewable from the runtime type inspector.

The ObjectionStore apparently contains all of the reflection information and available object instances. The ObjectionStore has a hashtable named “implementationTypes”, and it is this hashtable that I used to determine which objects were available to be injected into my constructor.

My implementationTypes map has 17 types available. I have no idea if these types are universal; I imagine the contents of the implementationTypes table will vary depending on the deployment. Inspection of my copy of this table yields the following types:

ThoughtWorks.CruiseControl.WebDashboard.IO.ICruiseRequest
ThoughtWorks.CruiseControl.Core.Reporting.Dashboard.Navigation.IPhysicalApplicationPathProvider
ThoughtWorks.CruiseControl.Remote.ICruiseManagerFactory
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.ILinkFactory
ThoughtWorks.CruiseControl.Core.Reporting.Dashboard.Navigation.ICruiseUrlBuilder
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.IAllBuildsViewBuilder
ThoughtWorks.CruiseControl.WebDashboard.MVC.View.IVelocityViewTransformer
ThoughtWorks.CruiseControl.WebDashboard.ServerConnection.ICruiseManagerWrapper
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.IProjectGrid
ThoughtWorks.CruiseControl.WebDashboard.ServerConnection.IFarmService
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.ILinkListFactory
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.IProjectGridAction
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.IActionInstantiator
ThoughtWorks.CruiseControl.WebDashboard.Configuration.IDashboardConfiguration
ThoughtWorks.CruiseControl.Core.Util.IMultiTransformer
ThoughtWorks.CruiseControl.WebDashboard.MVC.View.IVelocityViewGenerator
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.IBuildNameFormatter

I really don’t know what most of these objects do, but it’s not too hard to figure out when you have the source code.

Now you know how to determine what types are available for your constructor at runtime. From here, you can consult the source code to determine what services these interfaces provide, and parameterize your constructor accordingly.

The next post will cover the use of the Velocity subsystem.

April 22, 2008

CruiseControl.NET and multiple build configurations: Part 2 – Starting the dashboard plugin

Filed under: Continuous Integration/CruiseControl.NET, NAnt, c#, dotnet — treyhutcheson @ 10:11 am

[More Posts On This Topic]

This post continues a thread about how to support multiple build configurations from CruiseControl.NET and NAnt.

My goal was to expose some method through the web dashboard by which the user could select a build configuration and kick off a build. To accomplish that goal, some modification or extension of the dashboard is required. I decided against modifying the actual ccnet source, and instead chose to develop a dashboard plugin.

I started with the official ccnet docs regarding the subject, available here. I suggest readers acquaint themselves with the material.

Plugins Won’t Load
Before I get to the meat of the post, I must warn you that as it stands *today*, custom dashboard plugins will not load. The documentation states that plugin assemblies should be named “ccnet.*.plugin.dll”, and be placed in the dashboard’s bin folder. Presently, the code for loading plugins for the dashboard looks for assemblies via the path returned from Assembly.GetExecutingAssembly().Location(the Assembly.GetExecutingAssembly() call refers to the actual dashboard assembly). This line of code is logical, but it returns the path of the assembly as it is hosted by asp.net, which is some deep-dark directory buried within Windows because of asp.net’s shadow copy.

The dashboard’s source code must be changed. The Assembly.CodeBase property should be used instead, which returns a string uri to the assembly. This uri then needs to be parsed to obtain the actual physical location of the assembly so that satellite assembly lookup can work. I started a thread in the ccnet-devl groups. A ticket was opened, and as far as I know, the issue has been corrected. I suspect the correct code will be present for version 1.4, whenever that is released. Please note, however, that that is my assumption only, and has not been promised in any way by the ccnet team. Details on the defect, and how to fix it, are available here. Also please note that until the correction for the defect is available, you must pull down a copy of the source code, make the fix yourself, and deploy the updated dashboard binary.

Goal
I’ve already established my goal. Now let me describe how I will achieve that goal. Building upon my previous post about using NAnt properties to build debug or release configurations via the csc NAnt task, I have decided that I would like to allow the user to modify the initial values of NAnt properties for each configured ccnet project. The modified values for these properties must ultimately be recognized by ccnet so that these values can be passed to the root NAnt build script for the project when it is launched. For these reasons I have named my dashboard plugin “PropertiesPlugin”.

Source Code & Debugging
It’s possible to develop a dashboard plugin without benefit of the ccnet source code, but I strongly encourage you to download the source as it will make things much easier. For my development, I used version 1.2.1 of the source.

To debug your plugin, you will need to make sure you have the ccnet server running(either as a service or from the console), and you will need to attach to the aspnet_wp.exe process. Make sure that the build output of the project is the dashboard’s bin directory, otherwise the debugger won’t hit your breakpoints because the assembly will be out of date. Another issue that I sometimes face is that objects provided to your plugin from the dashboard will not evaluate and cannot be browsed. In such cases, I use the IIS control panel applet to stop/start the ccnet www app, which usually corrects the problem.

Starting the plugin, interfaces, and NetReflector
To start the plugin, create a public class that implements the ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.ICruiseAction interface, and the ThoughtWorks.CruiseControl.WebDashboard.Dashboard.IPlugin interface. Additionally, the class must be decorated with the ReflectoryType attribute(exposed via the Exortech.NetReflector interface).

At a minimum, your plugin assembly will require references to NetReflector.dll, ThoughtWorks.CruiseControl.Core.dll, ThoughtWorks.CruiseControl.Remote.dll, and ThoughtWorks.CruiseControl.WebDashboard.dll (each is present in the dashboard’s bin folder).

Example:

[ReflectorType( "propertiesPlugin" )]
public class PropertiesPlugin : ICruiseAction , IPlugin
{
  private const string PLUGIN_DESCRIPTION = "NAnt Project Properties";

  private const string ACTION_NAME = "ViewProjectProperties";

  #region IPlugin Members
  public IResponse Execute( ICruiseRequest cruiseRequest )
  {
    return null;
  } //END Execute method
  #endregion

  #region IPlugin Members

  /// <summary>
  /// Returns the description for the link
  /// </summary>
  public string LinkDescription
  {
    get
    {
      return PLUGIN_DESCRIPTION;
    }
  } //END LinkDescription property

  public INamedAction[] NamedActions
  {
    get
    {
      return new INamedAction[] { new ImmutableNamedAction( ACTION_NAME , this ) };
    }
  } //END NamedActions property
}

The above code is the bare minimum for creating a plugin. As it stands, it will not execute correctly, because the ICruiseAction.Execute implementation returns null. Ultimately we will need to return a valid IResponse by using the Velocity view generator, but I’ll touch on that subject later. Regardless, at this point the plugin should compile, and you should be able to debug it.

ReflectorType Attribute
As noted earlier, the class is decorated with the ReflectorType attribute. This attribute is significant. When the dashboard loads the configuration(dashboard.config), it will attempt to load named plugins from all assemblies(internal and external) via NetReflector. This plugin is named “propertiesPlugin”, and NetReflector will look for any public types decorated with this value.

ICruiseAction interface
This class implements the ICruiseAction interface. This interface is used to perform any number of actions from the dashboard user interface. Almost every link you click within the dashboard is represented by an accompanying action implementation.

This interface defines but a single method: IResponse Execute( ICruiseRequest req ). The basic idea is that the action accepts the current request and generates a response. It’s pretty easy really. The request identifies all kinds of things, including the current ccnet server and project. The response is expected to be consumable html that will ultimately be sent to the browser. More on that later.

IPlugin interface
The IPlugin interface is what allows the dashboard to expose your plugin via its user interface. The IPlugin interface is rather simple:

interface IPlugin
{
  string LinkDescription
  {
    get;
  }

  INamedAction[] NamedActions
  {
    get;
  }
}

The LinkDescription property is the text for the link as it will be displayed to the user. The INamedAction[] property returns an array of named actions that will be executed when the link is clicked by the user.

I have not yet experimented with what happens when more than one named action is returned in the array, but I assume the actions are executed in succession. I do wonder however what happens to the IResponse from each action’s Execute method. Are they just appended to the stream? Unknown.

In the previous example, the plugin’s LinkDescription implementation simply returns a constant value. However, examine the NamedActions implementation:

public INamedAction[] NamedActions
{
  get
  {
    return new INamedAction[] { new ImmutableNamedAction( ACTION_NAME , this ) };
  }
}

This code defines an INamedAction array inline, with only a single element. That element is of type ImmutableNamedAction, with an action name and a reference to the actual action implementation(this – the plugin).

At this point I would like to explain that I didn’t inherently know to use the ImmutableNamedAction class. In fact, I didn’t know how to implement this property. To figure that part out, I consulted the dashboard source code, and found the ProjectReportProjectPlugin class. I saw how it implemented this property, and borrowed its behavior.

Regardless, it is important to note the two entities being used by the dashboard here, and the interaction of these entities.

The dashboard will load the configuration, see that a plugin is defined, and load the plugin. This plugin must implement the IPlugin interface. This interface allows the dashboard to expose the plugin to the user interface. It also instructs the dashboard as to what actions are available from the plugin.

The actions for the plugin are altogether different from the plugin. An action is what happens when the link for the plugin is clicked. Notice the distinction?

In this plugin, the action and the plugin are the same object, only because this plugin is very simple.

Plugin Configuration
The documentation states that dashboard plugins can take one of four forms:

  • Farm Plugin
  • Server Plugin
  • Project Plugin
  • Build Plugin
  • More information about what type of plugin does what is available in the documentation. For my purposes, I chose to implement a project plugin because I wanted to be able to modify the build properties for each project independently.

    After you have implemented the basic plugin shell and deployed the assembly, you must configure the dashboard to recognize the plugin. This topic is also covered in the documentation. But for the purposes of this plugin, the configuration was very simple. I simply added a node named “propertiesPlugin” as a child element to the “projectPlugins” node(in the file dashboard.config). Remember, this node name – projectPlugins – must match the value of the ReflectorType attribute.

    An example dashboard config follows(boilerplate stuff exempted):

    <dashboard>
      <plugins>
        <farmPlugins>
          ...
        </farmPlugins>
        <serverPlugins>
          ...
        </serverPlugins>
        <projectPlugins>
    
          <!-- this declaration instructs the dashboard to include the project plugin -->
          <propertiesPlugin />
    
        </projectPlugins>
      </plugins>
    </dashboard>
    

    That’s it for now. The above code(along with the dashboard source code fix) should get any one started with a very basic project plugin. I will detail the Velocity stuff and ccnet’s use of constructor dependency injection in the next post.

    April 21, 2008

    CruiseControl.NET and multiple build configurations: Part 1 – Introduction

    Filed under: Continuous Integration/CruiseControl.NET, NAnt, c#, dotnet — treyhutcheson @ 3:28 pm

    [More Posts On This Topic]

    I recently created a post regarding handling debug or release configurations within nant scripts. This post will extend the problem to CruiseControl.NET.

    Question: how does one manage automated builds within CruiseControl.NET where multiple configurations are present?

    I’ve been considering this problem for a couple of years now, and a solution that I liked has evaded me, until now.

    My first thought was to have separate projects configured, each pointing to the same project, but with its own configuration. I even implemented this approach, once, and it lasted until my first test pass. One project performed a DEBUG build, while another configuration performed a RELEASE build. When I committed a change to the repository, ccnet dutifully observed the change and kicked off both builds. But as both builds shared the same physical location on disk, both build scripts attempted to create and delete temporary build folders, resulting in file locking errors.

    I considered having completely separate folders on disk for each project configuration, both pointing to the same module in the CVS repository, but soon dismissed the idea. It seemed like a waste of both disk space and cycles. Disk space is cheap, and so are processor cycles(my builds are for the most part very quick), but I didn’t want to follow this same duplicated-folder pattern for each of my projects. After all, my build environment is presently handling about 10 separate projects. Duplicating each project for DEBUG or RELEASE configurations would be a shameful waste.

    I’ve played around with the idea of creating a custom ccnet plugin, but the documentation is quite poor and I didn’t have the luxury of devoting time or concentration to the issue. That is, until very recently.

    I reviewed the public ccnet docs again, researching plugins. I could create a plugin hosted by the server process, or I could create a plugin for the dashboard. I chose to implement a dashboard plugin because ultimately I wanted the ability for the user to choose a configuration from the dashboard itself, and to kick off a build after selecting the desired configuration.

    My first thought of a dashboard plugin was to modify the main farm page of the dashboard; the page with the project grid that allows users to force builds, or stop active builds. I wanted to add a new column to the table with a drop-down box that contained the available configurations. The user would choose the configuration, and click the Force button to perform a specific-configuration build.

    It took me a while to come to grips with creating a dashboard plugin. The documentation is sparse, and the object model difficult to understand, specifically the model of obtaining object references at runtime.

    My first attempt ultimately proved be a failure, but it wasn’t completely unfruitful. I learned a bit of how to use the Velocity system to generate views, and how object references are injected into your plugin’s constructor at runtime. I also uncovered a bug within the ccnet dashboard itself.

    Fortunately, this bit of code-spelunking and discovery didn’t take very long. Overall I believe I spent maybe 10 hours of clock-time to end up with a compiled and deployed solution that meets my needs. I will document my discoveries, as well as the final solution, over the coming days. I believe blogging about this process will prove interesting, as I predict it will take more time to sufficiently cover all of the topics than it took to create a final solution in the first place.

    NAnt build scripts for C#: Debug or Release configurations

    Filed under: Continuous Integration/CruiseControl.NET, NAnt, c#, dotnet — treyhutcheson @ 1:19 pm

    When one begins to delve into the use of NAnt for automating C# builds, one must choose which NAnt task to use to perform the compile.

    One option is the csc task, which invokes the c# compiler via the command line. Another option is the solution task, which uses Visual Studio/devenv to compile a solution.

    The solution task at first glance appears to be the more useful; after all, it respects all project properties such as optimization flags and compilation symbols. However, it has two critical flaws.

    The first flaw becomes an issue in multi-user development environments; the settings of the solution must be synchronized in the repository across each development workstation/user. On the surface, this issue doesn’t seem to be very large. However, in practice, its all too common for users to check in changes to the solution that eventually cause the builds to fail. I want my builds to fail because of compile errors or failed unit tests; not because some one accidentally changed an assembly reference.

    The other issue is that the solution task requires visual studio to be present(and properly configured) on the build environment. Although the size of my company’s software budget does not directly effect my salary, I see no reason to purchase a separate license just so I can automate my builds.

    It is chiefly because of these two issues(and other, much more minor issues) that I choose to use the csc task. The main tradeoff is the fact that I must explicitly script the inclusion of source code, resources, and project references. That’s a tradeoff I eagerly welcome. However, there remains another tradeoff: project configurations.

    Suppose you have a simple target defined as below:

    <target name="build">
      <csc target="library" output="assembly.dll">
         ...
         ...
      </csc>
    </target>
    

    How does one go about supporting separate configurations, or defining conditional compilation symbols or to perform optimization?

    I’ve settled on a technique that has served me well for a couple of years now. At the root of my nant script, I define a property named build.release, like so:

    <property name="build.release" value="false" unless="${property::exists('build.release')}" />
    

    This property is present to indicate that a release build should be performed. The property defaults to the value “false”, unless it has already been defined(such as in a calling script, or via the command line).

    I then use this property within the call to the csc task:

    <csc target="library" output="assembly.dll" debug="${not build.release}"
      optimize="${build.release}">
      ...
      ...
    </csc>
    

    The use of this property allows me to control both the optimize argument and the debug argument. If debug.release is true, then the assembly will be optimized, and the debug argument will be set to false. If debug.release is instead false, then the assembly will not be optimized, and the debug argument will be set to true. In the latter case, according to the NAnt docs, both the DEBUG and TRACE symbols will be defined.

    If your project requires alternate compile symbols, one can always use a similar method to conditionally define them and pass them to the csc task(via it’s define argument).

    Unfortunately, there’s one glaring flaw; fortunately, this flaw has not affected me yet in my 2+ years of this approach. The flaw is the fact that only two configurations are supported. It’s either a debug build, or a release build.

    Great – so you now have one set of nant code that uses the csc task that can emit either debug or release builds. How does one take advantage?

    One method is to change the initialized value of the debug.release property, either directly in the script or its calling script. Another method is to pass the value via the command line, via the line:
    NAnt [buildfile] -D:debug.release=true

    But what happens if you have the build triggered by a continuous integration environment, like CruiseControl.NET? That’s the topic of another post altogether.

    Blog at WordPress.com.