Martin Woodward

Over the years, I have tried many different approaches to software estimation.  From the Ceiling and Weller method (look up to the ceiling, scratch your chin and say "Well-errr"), FITA analysis (Finger In The Air) through various variations (both formal and informal) of metric based estimation techniques to things like IFPUG etc.

Today, I had something happen that reminded me why software estimation is so hard. I've just wasted about 8 hours trying to figure out an issue with what I thought was some bizarre firewall problem with Windows Vista SP1 x64 when it turns out that some code that I wrote was actually working correctly and was picking up a HTTP proxy preference that was pointing to a server that no longer exists.  I'd just forgotten I'd fixed this bug, and that I had the proxy set in my preferences.

I took two lessons from this frustrating day.

1. We need a better error message when your proxy is no-longer available
2. Software estimation is hard

In discussions with my wife she frequently struggles when some nights I finish work complaining that I am 3 days behind and the very next afternoon I can be caught up or even a little ahead.

The best way I have come up with to explain this, is to get her to imagine that I had given her 60 Suduko puzzles all rated as 10 minute puzzles.  Should take you 10 hours right?  Now see how long it takes you to do each puzzle.

Suduko is the closest analogy to computer programming that I can find for "normals" - i.e. people that don't code.  This only really works if the person does Suduko puzzles, but my wife does so it works in our house.  Suduko is a numeric analytical problem solving activity.  While there are tricks and techniques to solving some puzzles, there is a significant challenge and difference to each one. Looking at the puzzle, it is hard to know if it is going to be hard or easy.  You can get stuck down blind alleys and have to start all over. Also when you "get into the zone" you can often make surprising intuitive leaps that often defy verbal explanation afterwards. Finally, solving a hard Suduko puzzle quickly involves a fair degree of luck and depends on your state of mind at the time of trying the puzzle.  There is a great amount of satisfaction to be gained from solving a Suduko puzzle along with a high degree of frustration when you cannot solve one - you know that it must be possible after all.

So.  Each task (solving a Suduko puzzle) should take about 10 minutes, with-in a certain error range.  In software estimation a really good developer doing a well known and well defined task can only hope to get their estimate to around 25% accuracy (a "hard estimate") - and then there are always the odd random occurrences that throw you way off.

I've been in interviews where the candidate swears blind that they always finish a task on time.  This tells me two things about that candidate:

1. They are a liar
2. They either never estimate how long something is going to take up-front or they over-subscribe to the Scotty principle of estimation. (Or as Microsoft are fond of saying, under promise - over deliver)

Does this mean that we should give up on estimation?  Of course not.  Planning a project is going to be pretty hard if you have no idea of roughly when you are going to be finished, what the end result is going to do or how much it is going to cost.  Not to forget that when estimating a large number of tasks you can rely on the fact that some of the work you finish early will offset the work that takes longer than expected.  However, understanding the nature of software development and how it differs from, say, laying bricks, helps you be more likely to succeed.

The problems inherent in software estimation also help me to understand why Agile software development methodologies work. 

• You are forced to break problems down into small bits that can be managed, tracked and measured.
• Small iterations means that you can only get so far behind.
• You re-estimate and re-prioritise work at each iteration when a hard estimate on that task is possible (and therefore your estimates are more likely to succeed)
• You frequently listen to the person who will be using the software
• You can load your iterations so that there is always work to do of the correct priority to the customer when you have finished a task early and things of the appropriate priority that you can drop off the list when something takes longer

There are also many other techniques that people can adopt from Agile methodologies or just from common sense to reduce the amount of time taken when a task is taking longer than expected, such as

• Limit distractions. You need time to concentrate on a single problem otherwise you will never finish it.
• Pair programming. Identify when a second pair of eyes is needed and quickly as for help in your team (can be hard when your team are distributed across multiple time-zones)
• Daily progress meetings. The guy that has been stuck for 24 hours on what he thinks is a firewall issue is quickly identified)
• Lack of ego.  I'm the "Windows guy" in our company, if I say something is a firewall issue with Windows Vista SP1 x64 and a probably collision with the new Eclipse 3.4 launcher executable - then the new intern who only knows about the Mac he used at college should feel like he can ask a "dumb" question and say "Have you checked your proxy settings after that bug you were fixing yesterday".

Anyway.  All common sense stuff and writing this post has been a nice way to get rid of the frustration of finding why I was stupidly stuck for the past day.  Now to stop distracting myself further and get on with some more work :-)

As we all know by now - the build system in TFS2008 was substantially improved.  Along with the many improvements came an official API for talking to the build system.  This is the same API that the Team Foundation Build UI in Visual Studio uses, however there are many additional methods that were added that were not for the UI at all but for potential consumers of the Build API.

In talking with folks at community events, and on the MSDN forums I have realized that there isn't a huge awareness of this API.  Having written a parallel implementation of the build API, but in Java for the Teamprise 3.0 release I have spent a great deal of time with the .NET API and have a few examples lying around of how to accomplish certain common tasks - so I thought I would run through a some of them.  If you have an example of something you would like to see with the build API then please leave a comment for this post or drop me a line.

So - let's start with a basic one.  How to queue a build.  This will introduce us to a few concepts with-in the build API that are common across all of the methods.

The Easy Way.

A quick look at the class diagram above will show you that the main interfaces you'll be dealing with in the Build API are the IBuildServer and IBuildDefinition interfaces.  To get started with these you'll need to add references to the Microsoft.TeamFoundation.dll, Microsoft.TeamFoundation.Client.dll and Microsoft.TeamFoundation.Build.Client.dll.

C#

TeamFoundationServer tfs = TeamFoundationServerFactory.GetServer("http://tfsserver:8080”);
IBuildServer buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));
IBuildDefinition buildDef = buildServer.GetBuildDefinition("TeamProject", "Build Name");
buildServer.QueueBuild(buildDef);

VB.NET

Dim tfs As TeamFoundationServer = TeamFoundationServerFactory.GetServer("http://tfsserver:8080")
Dim buildServer As IBuildServer = DirectCast(tfs.GetService(GetType(IBuildServer)), IBuildServer)
Dim buildDef As IBuildDefinition = buildServer.GetBuildDefinition("TeamProject", “Build Name")
buildServer.QueueBuild(buildDef)

I'm afraid that's the last example you'll be seeing in VB.NET.  Being a Java developer by day I tend to like my semi-colons.  However if you are a VB developer then hopefully you'll be able to follow along as the rest of the examples are more or less just method calls with the occasional bit of casting.

So - what's going on here.  Well we first get hold of a TeamFoundationServer object.  If this was server side code and we wanted to specify some credentials then we would use a slightly difference mechanism but in this case the constructor works well and this will automatically connect using the credentials of the current thread.

Next we get hold of the build service that implements the IBuildServer interface.  Finally we get a hold of the build definition by specifying a team project and build definition name (build definitions are unique per team project in TFS).  We then queue a build using the defaults for that build definition.  This API call is actually shorthand for the following...

The Hard Way

Supposed you want to queue a build, but you don't want to use the default build agent, priority, drop location etc.  Well in that case you need to look at the IBuildRequest interface.  Here you will find all the options to customize the build request, you can specify a build agent, drop location, priority etc like you can do from the Visual Studio UI.  You will also find other options such as being able to queue the build with a maximum acceptable queue position, pass a custom get version for the build or even queue the build in a postponed status.

In the following example, I am going to find a non-default build agent from the server and then queue a build using it.

        TeamFoundationServer tfs = new TeamFoundationServer("http://tfsserver:8080");
        IBuildServer buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));

        String teamProject = "TeamProject";

        String buildDefinitionName = "MyBuildDefinition";

        String buildAgentName = "MyBuildAgent";

        

        // Find our (non default) build agent.

        IBuildAgentQueryResult queryResult = buildServer.QueryBuildAgents(buildServer.CreateBuildAgentSpec(teamProject, buildAgentName));

        if (queryResult.Failures.Length > 0 || queryResult.Agents.Length != 1)

        {

          throw new Exception("Invalid Build Agent");

        }

        IBuildAgent buildAgent = queryResult.Agents[0];

        IBuildDefinition buildDefinition = buildServer.GetBuildDefinition(teamProject,buildDefinitionName);

        // Create a build request based on our chosen definition.

        IBuildRequest buildRequest = buildDefinition.CreateBuildRequest();

        // Optionally set command line args, drop location, priority, custom source version etc here

        // in this case just overriding default build agent.

        buildRequest.BuildAgent = buildAgent;

        buildServer.QueueBuild(buildRequest);

Pretty simple really and very powerful.  I encourage you to go take a look at the IBuildServer interface to see some of the stuff that you can do.  If you have any suggestions as to what you would like to see as a build API example then leave a comment to this post.

Jeffrey Fredrick just announced that CruiseControl 2.7.2 is now available for download: http://tinyurl.com/2zm9mz.

There are lots of bug fixes, lots of changes to the Dashboard and some new plug-ins, but the bit that is of most interest to me was (from the release notes)

TeamFoundationServer source control
----------------------
* Fix compatibility with Microsoft Visual Studio Team Foundation Server 2008 (CC-735). Submitted by Martin Woodward.

This was to work around an issue that came up when using CruiseControl (java version) to talk to a TFS2008 server (TFS2005 worked fine and still does).  If you are attempting to use CruiseControl with TFS 2008 then you should go with CruiseControl 2.7.2.  For that matter - if you are using CruiseControl.NET with TFS then you should also take a look at the latest release of the integration to TFS - as that contains the same fix allowing you to happily talk to a 2008 version of Team Foundation Server (also using the TFS 2008 client API's).

Anyway, congratulations to the CruiseControl team on the 2.7.2 release!

At EclipseCon 2008 this morning, we just announced that Teamprise 3.0 has been released!  If you've been wondering why I have been quiet on the blog lately - but also why anything I have been talking about is Team Foundation Build related, then you are about to find out why :-)  First of all, I'd encourage you to go visit the shiny new website at http://www.teamprise.com.  Our marketing team had too much fun putting that together, including getting a real, live, massive Teamprise power button made up and shipped in a huge crate from New York to be photographed and used as the new site/icon image.

The full release notes are available here, but as has been the tradition for the past few Teamprise releases, I thought I would give you a run down of my favourite new features in the 3.0 release.

At a high level, the features in 3.0 can be summarised as:-

• Full Team Foundation Build integration (including ability to execute Ant based builds)
• Check-in policy support
• Recursive folder compare
• Single sign-on (from Microsoft Windows machines)
• "Destroy" command for version control
• Show deleted items and undelete from Source Control Explorer UI
• much much more (see release notes)

While it is not my area, I should also mention that we've taken this opportunity to make our licensing more affordable for smaller teams.  We have been very pleasantly surprised by the number of people buying 1 to 20 licenses at a time.  Originally, Teamprise pricing was skewed to the Enterprise customers (i.e. simple, all inclusive and with steep volume discounts).  So we have done a couple of things to help out the smaller companies:-

• You can now purchase the various components (Teamprise Plug-in for Eclipse, Teamprise Explorer, Teamprise Command Line Client) individually as well as the Teamprise Client Suite which gives you the lot.
• We have lowered the initial prices for a single seat, meaning that people buying one or two licenses can now get the same discounts that used to only be available to folks purchasing 100.

If you have any licensing issues / queries then feel free to contact me, or you can talk to the sales team direct at sales@teamprise.com.  Anyway - back to the part of this release that I do know about - the technology. 

The first feature I want to talk about is one that I had no involvement with.  It's one of those features that many people will not notice because it just works but anyone who has done any Java to .NET web service interop work will instantly recognise as being a little bit clever.

Single Sign-On

The initial log-in screen has undergone a big overhaul.  On Windows machines you are given the option to use "default credentials", i.e. the username and password that you are logged onto windows with.  It obviously doesn't know your password, but does some JNI magic to get the native Windows API's to handle the authentication logic with Team Foundation Server.  While you are also on the login screen, you may notice the Profile feature.  This is an area that many people probably won't use, but we added for our power users and for ourselves.  Basically, the profiles feature allows you to store sets of servers/credentials that you commonly use to connect to Team Foundation Server and then you can bring up the details using a simple drop down.  Makes it much easier to switch between your production TFS instance and your CodePlex project for example - or switch credentials if you are a TFS administrator.

Check-in Policy Support

In Visual Studio, check-in policies are implemented as a .NET assembly runs every time a policy is evaluated or configured.  The policy also has full access to the .NET API's, the Visual Studio API's as well as anything it might want to pinvoke out to on the Win32 API side.  As you can imagine, this presented us some problems when we wanted to have check-in policies that ran the same in Eclipse on Windows Vista as Teamprise Explorer on the Mac or Aptana on Ubuntu - therefore we have had to develop a parallel Teamprise check-in policy framework.

As we were doing this, we took the opportunity to learn from some of the feedback folks have been having with the Visual Studio check-in policies.  While our framework and SDK will look very familiar to anyone that has developed a custom check-in policy for Visual Studio, you will notice some differences.

Firstly, we supply different policies out of the box.  The vast majority of custom check-in polices that people deploy are things like "Check for Comments" etc, so we just shipped the common ones our customers wanted to prevent them from having to write their own.

Secondly, we make use of the Eclipse plug-in framework to implement our policies as extension points.  This means that they are easy to deploy (using the Eclipse update site mechanisms built in to the IDE).  We have also separated the configuration (stored as a blob of XML data in our framework) from the implementation - represented by the plug-in deployed.  The again makes it easier to deploy, especially when it comes to version 2 of a policy...

Thirdly, all of our policies can be scoped by the path in version control to which they correspond - you are not limited to per Team Project scoping and you do not have to wrap your policies in a custom policy to get more detailed scoping like you do with the current Visual Studio framework.

Team Foundation Build Integration

Anyone that has been following this blog for a while, or who attended the Team Build talk I did at TechEd with Brian Randell, will notice that I have been increasingly involved in the inner workings of Team Foundation Build.  Now you can see the fruits of that labour.

In Teamprise we now have full integration with the shiny new build functionality in TFS 2008 as well as support for TFS 2005.  Backwards compatibility with the TFS 2005 server is very similar to if you were using a Visual Studio 2008 client, accept that ours is slightly more backwards compatible (you can create new builds on a TFS 2005 server as well as manage build qualities etc).  However it is with TFS 2008 that you get to see the majority of the features.  I could go on about this aspect all day as their are so small things that I am proud of, but at a high level you can:

• View existing build definitions
• Manage builds in Build Explorer
• Queue new builds
• View build report
• Edit Build Quality
• Delete build
• Manage Build Qualities
• Open Drop Folder
• New/Edit Build Definition

• Edit Retention Policies
• Keep Build
• Set Queue Priority
• Postpone Build
• Stop/Cancel Build
• Delete Build Definition

One of the smaller features I will call out is that from the build definition in the Team Explorer, you can right click and do a "View Build Configuration" that will open the Source Control Explorer at the place in which the TFSBuild.proj file is stored so that you can check it out and edit it.  A feature that I added solely for my own sanity during dogfooding :-). 

All this would be fairly academic, if you didn't have some way to do a cross-platform build using Team Foundation Build.  In the current release, we provide a the Teamprise Extensions for Team Foundation Build which basically Ant enables the Team Foundation build server.  The Teamprise extensions are a set of MSBuild targets that insert the Ant build process into the standard Team Build mechanism as well as a custom MSBuild We hope to extend this to support in the near future to some of the other common build/test tool-chains in the cross-platform world.  However, the Ant integration case will help a lot number of people out there.

Best yet, the Teamprise Extensions for Team Foundation Build are available free of charge for everyone - wether or not you are a Teamprise customer.  Also, if you want to see how they work and customize them to meet your own non-standard build system then the source is available under the permissive open source Microsoft Public License (MS-PL).

I would personally like to thank the Team Foundation Build Team (especially Buck Hodges and Aaron Hallberg) who have been incredibly helpful through the development of the build functionality in Teamprise 3.0 while they were also busy working on TFS 2008. 

Hopefully that gives you a quick flavour of Teamprise 3.0 and where we are going with this release.  If you head over to the new site now and take a look at the many improvements we've made, we'd love to hear what you think.

The fantastic team at CodePlex have just rolled out yet another great feature - this time it is statistics for your CodePlex project.  I dropped by my TFS Plug-in for CruiseControl.NET to take a look...

The project is currently averaging over 12 downloads a day and has had nearly 5000 downloads since I moved it to CodePlex in August 2006.  Not bad for a chunk of code I originally wrote on a plane..

This is incredibly motivating as a maintainer of an Open Source project.  I've been knowing that I need to give the CC.NET integration some love as there are a few issues and things that I need to address from the excellent feedback I have been receiving.  However - seeing how many people are still interested in the plug-in despite the excellent CI functionality in the 2008 release of TFS has made me realise that I need to get on this ASAP.  Thanks CodePlex!

Developer Developer Developer is coming to the beautiful west of Ireland in May with GAMTUG hosting the event on Saturday May 3rd.  If you are in or around Ireland at that time then I would encourage you sign up.  My good friend, and fellow Team System MVP, Mike Azocar will be doing a session on Scrum which should be well worth sitting in on.

There will be no Microsoft speakers presenting, just speakers from the .NET developer community, although I hear our local friendly Platform Evangelists (DPEs) will be on hand to help out and chat to everyone and to take in the atmosphere.

The day promises to be a great learning experience, but also a fantastic way to meet like-minded people - if anything else, you get a trip to the lovely Galway which is never a wasted journey.

The new "Gutsy Gibbon" version of Ubuntu was officially released this week.  While some of the developers at Teamprise use Ubuntu as their primary OS, I personally prefer Windows Vista, however I keep an Ubuntu image around so that I can quickly test any new UI code I write in an alternate OS.  Getting Ubuntu to install in VMWare is pretty trivial - the standard installation seems to work just fine.  However getting it to work in Microsoft Virtual PC is a little more complicated - the various tricks are located in various forum and bug posts around the web that I can never find when I need them so I am collating them here for my own use later - hopefully you may also find them useful.  Please bear in mind that I know enough about Linux to be dangerous rather than useful, so this may not be the "best" way - however it is my way ;-)

There are a few basic issues with getting Ubuntu 7.10 working inside Virtual PC.

1. The default display color settings use a higher color depth than Virtual PC supports
2. Ubuntu has a hard time recognizing the emulated Virtual PC mouse
3. Networking does not work initially
4. Time syncronization with the host can be a little off.
5. Soundcard does not work initially
6. The ubuntu splash screen "usplash" causes problems when shutting down/restarting.

Anyway, here we go.  Obviously first things first, you need Virtual PC 2007 and Ubuntu 7.10 (x86 desktop).

Installation:

• In Virtual PC, create a new virtual machine by pressing "New..." in the console.  I use the following settings:
  • In the OS selection select "Other"
  • Ram 512Mb
  • New Disk of 8192 Mb
• One created, go back into settings and Enable Hardware Virtualization if available (only available if you have a modern processor, and even then you probably had to enable this in your host machines BIOS first).  This can make a serious difference if your hardware supports it, however having the flag enabled makes the image less portable as some machines cannot support this option so they have to switch it off before using the image for the first time. 
• Start the VM.  Then go to Edit, Capture ISO Image... and select the downloaded Ubuntu install image.  You may need to restart your Virtual PC to get it to boot from the mounted DVD
• When the ubuntu install menu comes up, move down to "Start Ubuntu in safe graphics mode" then press "F6" for more options. 
• In the boot options, add the text "i8042.noloop" without the quotes.  This is a parameter we are passing to the kernel to get the emulated Virtual PC mouse to work correctly.  You entire boot options line should look like this:
  file=/cdrom/pressed/ubuntu.seed boot=casper xforcevesa initrd=/casper/initrd.gz quiet splash  -- i8042.noloop 
• Press enter to start the boot into the installation live cd.  Sometime when I do this, when the UI comes into view and the mouse is captured but it is like the virtual PC is only showing the top left hand corner of the screen.  Not sure why, but to fix it hit CTRL-ALT-F1 to go into a console and then CTRL-ALT-F7 to restart the window manager.  You should be able to see the whole screen now.
• Install by double-clicking on the Install icon on the desktop.  I use the guided partition manager to assign the whole virtual disk.
• Once the install has completed, unmount the DVD from Virtual PC by going to "CD" and Release "ubuntu-7.10-desktop-x386.iso" (remembering that in Virtual PC, the right ALT button releases your mouse from the guest machine).  Then go to Action, Reset to restart the VPC.

Initial Configuration:

When you boot for the first time, there are a few things that you need to do to get everything working properly.  We need to get all the things (including the mouse etc) working again.  These changes will then be stored on the virtual machine disk so you won't need to do them again for this image.

• Log in (remember the mouse isn't working yet), so press CTRL-ALT-F1 to drop into a console.
• Let's now get the mouse working again.  type sudo nano /boot/grub/menu.lst.  Page down to the bottom of the file and find the first kernel line after the phrase " ## ## End Default Options ##".
• Add i8042.noloop clock=pit to the kernel parameters.  You'll recognize the i8042.noloop thing from before, the clock=pit is a fix for the time drift issues (KB918461)
• Now we want to ensure the modules are loaded for the Virtual PC soundcard and Network card.  type sudo nano /etc/modules.  At the end of the file add the lines
tulip
snd-sb16
  Ensuring that each is on a line of it's own, followed by a carriage return.
• Then we want to remove the fancy Ubuntu splash screen that has some issues under Virtual PC.  Type sudo apt-get remove usplash -y
• Restart by typing sudo reboot
• When you restart your soundcard will be working - you might want to get rid of the login/logout sounds...  System, Preferences, Sound, Sounds, setting Log out and Log in to "None".
• As this is a virtual pc, depending on the usuage you may also want it to automatically log-in with the user account created on install.  In which case go to System, Administration, Login Window, Security and check the "Automatic login" box - selecting your user.  Also, because this is a virtual PC you may also want to disable the display power-management (System, Preferences, Power Management, Display) and disable the screen saver (System, Preferences, Screen Saver, uncheck Activate Screensaver when computer is idle).
• To get networking fully enabled, I've found that I have problems with the "Roaming Mode" introduced in 7.10.  Therefore I go to System, Adminitstation, Network.  Click on the Wired Network and select properties.  Disable the "enable roaming mode" check box and then select DHCP configuration then press ok.  Finally, check the checkbox next to the "Wired connection" box to make this the default connection used on startup.

Configuring to run Teamprise

Now I have a booting Ubuntu instance, I usually want to configure it with Java, Eclipse and Teamprise.  This part may be of interest to less folks, but I need to do it so I'll write down the steps but feel free to ignore.

• Installing Sun Java.  Teamprise needs some encryption stuff to support NTLM authentication that is not available in the CGJ - therefore another JVM needs to be installed - in this case I'll use Sun's.  Go to Applications, Add/Remove programs, change the drop down to "All available applications" and then in the search box type "Sun Java".  In my case I'm going to pick Sun Java 6 Console then I press "Apply".
• Next is Eclipse.  Easy way is from Add/Remove programs, search for "Eclipse" and Apply - however that was Eclipse 3.2 at the time of writing.  Nothing wrong with Eclipse 3.2 of course, I just like to be on the Early Adopter curve so I go to the Eclipse.org site and download the latest for linux.
• Then finally, I install Teamprise.  That part is easy for me (copy paste the update site URL into Eclipse and unzip Teamprise Explorer)

Hope that helps if you are going on a similar journey.

Recently I had the pleasure of chatting with Carl Franklin and Richard Campbell about Source Control and Continuous Integration - this was recorded for DotNetRocks and the show is now available for you to enjoy.

As Eric has mentioned, I've been a fan of DNR for a while.  In fact, back in November 2004 when I used to burn the shows onto CD (in fact it was 2-CD's per episode back then), Carl mentioned that he was going to start making the show available as a "podcast" and it was because of this that I ended up purchasing my first iPod.

Having been a long time listener it was really interesting to see how well organised the recording sessions are, and also how well edited it is afterwards.  I've always thought that the PWOP podcasts have great sound quality, but only after being present during the actual recording do I understand all the tweaking that they do to take out fluffs / people overlapping when they talk etc (interupting people as they talk is a really annoying habit of mine).  Thanks again to Carl and Richard for having me on the show and to Eric for getting me invited - I hope that people find it moderately interesting.

At the recent MVP summit, I attended a Mash-up lounge event.  It was a bit Web 2.0-ey, but basically they had a bunch of bean bags, fake palm trees, and members of the Windows Live team and some Virtual Earth MVP's around to help folks get up to speed with the Windows Live API's.  (Thanks to Tyler and Robert, Virtual Earth MVP's from InFusion Development for helping me with some sample code to grok).  This is a whole area that I've not really been paying attention to in the past year, so I dropped by to see what the deal was and have a play.

I have to say, I'm impressed.  It is super easy to control the Virtual Earth map control from your JavaScript, and the documentation is pretty decent.  Now, as I freely admit, I am no expert on the differences between the various mapping providers around and their web interfaces - but I will say this; The local.live.com data for Ireland and Northern Ireland is by far the best of the mapping sites that I've seen.  I also love the driving instructions and the custom print capabilities.  I get the feeling that if local.live.com was not made by Microsoft then there would probably be more fuss about it - but hey ho.  I've just come back from a trip to Redmond so I must have had my borg implant secretly added if I'm feeling Microsoft are being unjustly treated :-)

Anyway, on my return, I was having a chat with my good friend Rob Burke about some of the exciting possibilities that the mapping API gives you.  As an example, I did this very quick mash-up of traffic cameras from the Dublin City Council website and a Virtual Earth Map.  It is very crude as it only took me 5 minutes to write the HTML (in Notepad2) and then a further 15 figuring out the lat/long positions of the cameras - but hopefully you'll get the point.  If anyone wants the GeoRSS data I used to create the map then here it is.

Dublin City Center Traffic Cameras - data from Dublin City Council.

Every source control system I have worked with until recently supported keyword expansion.  My code comment blocks have always tended to be something like (in Java):-

/**
 * Description.
 * @author $Author: martin $
 * @version $Revision: 12 $ $Date: 2007-01-02 16:15 $ 
 */

The source control system updates the block in-between the $tag: $ on check-in and that's just how it works.  Over time, I standardized on author, version, date because that is what was common between all the major SCM tool vendors and things like the history tag just got in the way.  Keyword expansion did have it's downsides thought, the main one is that it always gave you problems when diffing and merging files and had the habit of giving you un-necessary conflicts to resolve.

I say that every source control system "until recently" supported keyword expansion.  Over the past couple of years there has been a notable exception - the version control provided by Team Foundation Server does not have this capability.  I found this very surprising at first - just because I expected to have them at my disposal, I never stopped to ask why I wanted them.

The reason why I used to want them was because I used to review code by printing it out on candy stripe paper and taking it along to a code review meeting.  Also, my source control tool was always a separate (frequently hard to use) application that meant me swapping tools and then finding the file I was working on in that separate application when all I wanted to know was to who to talk to about the line of code I was looking at.  I'll admit that was in the days when I did all my coding in PL/I with the odd sprinkle of JCL - but still.  It was awfully handy for the source control system to help you keep your comment blocks up to date, to tell you who was the last person who touched the code, therefore the first person you spoke to if your suspected the code was broke.

Now-a-days, it is rare that a single person "owns" a file.  If they do, you get a much better idea of what has happened by looking at the history.  As all this is nicely integrated inside the IDE, checking the history is a matter of right-clicking on the file in the IDE and selecting "History", I don't have to jump tools, re-navigate to find files etc.  With diff between versions and annotate functionality I can easily see who did what and when, what other changes they made at the same time and what work items they linked all this stuff too.  Maybe keyword expansion was just a solution for a problem I no longer have (except possibly when dealing with Stored Procedures, but IDE integration has come a long way there recently).

Yet I still miss it.  Does anyone out there have a reason why they *need* keyword substitution as part of their software development process, or is it just one of those things that you've always done that has long lost the value?

Visual Studio Team Foundation Server gives a fantastic ability to track and report on various facts about your software development process.  It frees managers (and team members) from the chore of recording and collating data about the development process.  Managers can now get on with managing the project - removing obstacles before they seriously impact the team, reporting on the progress of the project to its owners and organizing deliverables to maximize the productivity of the teams.  With all new powers, there comes some danger and the need for responsibility (just ask King Midas or the scriptwriters of countless Hollywood films). 

The Uncertainty Principle

The uncertainty principle was first enunciated by Werner Heisenberg in 1927.  Broadly stated is describes that we can know the path an electron takes as it moves through space or we can know its position as a given point in time - but we can not know both.  Any attempt to measure one will unavoidably disturb the other.  What Heisenberg proved is sometimes improving the instruments won't help - a measurement affects the thing being measured and that it is an immutable property of the universe.

The reporting aspects of Team Foundation Server are a new, more accurate, instrument to take measurements inside your software development process but you need to be wary about the things you do measure.  The metrics need to mean something useful rather than just be interesting.  The affect of taking the metric should be carefully considered before taking it.  This is not a new problem that Team Foundation Server has introduced, just that because it makes it so easy to get data out of the system the temptations are greater.

Lets take an example of measurements affecting the process in the world of business rather than in the world of quantum mechanics.  In the UK, we recently had our telephone directory enquiries system de-regulated.  This meant that instead of just one number to dial when you wanted to get the telephone number of the local pizza shop, you now have a multitude of providers offering you that service.  After a few months, a study was carried out by a consumer organization to rate the different telephone directory information companies.  When a person calls directory enquiries, their goal is that they get the correct number for the person or service that they require - therefore when the consumer organization did the study they looked at accuracy and cost of the call.  Costs were more or less equivalent across the new companies - a one off charge of 50p per number.  The thing that surprised everyone is how in-accurate many of these new companies were.  All the companies talked to the same central number database so there must have been something else going on.  It turned out that many of these call-centers bonused their staff on the quantity of calls they got through a day - a reasonable measure for the company as there is a direct relationship between numbers of calls and revenue. The problem with this measurement is that it is in the interests of the staff not to bother being accurate, but to give the person a number (any number) to get them off the phone and move onto the next.  It only takes a couple of times for a customer to call one particular directory service and be given the wrong number for them to stop using that company and go with a more reliable one...

Similar things happen in the software development process.  For example, I think it is a bad thing to look back at an iteration and ask the question "who did what".  The more useful measure is which requirements did we solve and how many hours did we collectively spend doing this.  There are a few reasons for this. 

In any software team there are different characters that are needed.  Take the software developers for example (because this is the world I tend to live in).  You have the "hardened pros" who can just sit down, concentrate on their task and churn out the code, you have the "junior" people learning more and you have "floaters" - people that tend to move around a bit helping others out while getting on with their own deliverables when nobody is blocked.  In any large development, you need this mix of people.  Too many floaters and you spend your time discussing clever frameworks to reduce the amount of code needed to be churned out, Not enough floaters and you find that tasks can get blocked for weeks on end.  The same is true of all the characters in the team.  You need a mix of them, and you need them to work as a team for the benefit of the software development overall.

Measuring an individuals productivity is incredibly hard, because you can measure certain facts (such as how many lines of code did they write a day, how many bugs and tasks assigned to an individual have been resolved) but do these facts tell you anything about how good a person is at their job or how much they individually contributed to the success of a project?  As Heisenberg showed us, the downside of measuring is that this affects the process.  If individuals within a team are rated based on how many tasks they have completed then this inevitably leads to people being more selfish.  They are more likely to ignore somebody who is stuck on something, and concentrate on making their own score as high as possible.

In a large development process, it is frequently healthy to have a certain amount of competition between teams.  This bonds a team together and makes them collectively strive to out-do the other teams increasing productivity.  The downside of competition between teams is that it increases the feelings of "them-and-us" between groups, which can often get in the way of successful delivery.

I'm afraid that I have no easy answers, and no silver bullets.  There is nothing I can say "you must do this" and "you mustn't do that".  This is because software development is a hard and complicated problem that depends on the individual circumstances you face and the goals you are trying to achieve.

Team Foundation Server is a fantastic new instrument that allows a manager to take accurate measurements of their software development process, but you should keep Werner Heisenberg in mind, think about what the affect of taking the measurement will be and resist the temptation to abuse your new found powers.

Tonight, I finally sat down and recompiled the CruiseControl.NET plugin to work with the RC version of the Team Foundation Server API. 

Turns out there was only one change – the TeamFoundationServer object now has a public constructor to pass the credentials to rather than using the factory as I has previously been doing.  I also put a quick fix in there, but I really want to re-write the whole thing.  Looking back at the use of the client API it is very naive, there would be a ton of improvements I would love to do to it to make it faster by actually using some of the great features that Version Control in Team Foundation Server gives you.  I also want to talk to the PublishTestResults service.  Hey ho – so little time so many projects…

Anyway, cruise on over to the VSTSPlugins project to find out more.

I’ve been coding for around 25 years, since my parents bought me a VIC-20. I think it was probably a similar advert to this one by William Shatner that convinced them to buy it after seeing how much time my borther and I spent playing with our Atari 2600.  It was a sizable investment in new technology for them at the time, but one that has helped me to earn my living for over 10 years now. In that time I’ve written many multi-threaded applications and considered myself reasonably “thread-savvy”.  Most of the stuff has been web based applications but I’ve done my fair share of client side work before and I’m currently doing a lot of it.

This morning I encountered a Deadlocking problem for the very first time outside of a question in a technical interview, in fact if it had not been preparing for technical interviews then I may never have even come across the term before.  As I did my degree in Physics and not Computer Science, I’ve always had a bit of a weak spot around my use of formal computing terms – sadly my knowledge of Physics isn’t much better as I spent most of my time in the college bar and the JCR.  The reason I’ve probably only come across a real deadlock now is that most of my professional life has been spent inside J2EE application servers or on the server side of .NET where you tend to be deliberately insulated from such things.

Anyway, thought it was worth a post to commemorate this day – it took me a while to track down (would have been longer if it wasn’t for the debugger in Eclipse).  Had I not spent the odd day here and there preparing for technical interviews it probably would have took me longer.  You never know – one of these days I might actually write a real binary search algorithm or use some of the other interview favourites… 

I’m currently writing a diagnostics tool to perform all the same connectivity diagnostic checks that I do manually when I am having trouble connecting to something.  The following code snippet is how I am checking to see if a web or proxy server is listening on a particular port.  I thought I’d post a similar example here in case it was useful to you.

Note that the OPTIONS http method is very useful as a ping or noop type call on a web server.  Some web server administrators (such as Google) deny the OPTIONS request as it isn’t something that you’ll see a browser doing and they want to minimise their attack surface for the naughty folks out there in internet land.  Even if they do, you will still get some http response back (i.e. a String beginning with “HTTP”) which you can test for to tell if a web server exists on that port.

I was just trying to get some backups configured and I wanted a way of deleting files from a directory stucture that were older than a certain date. I couldn't figure out how to do it easier from a Windows 2003 command script, so I wrote a quick C# console app to do the job. I've included the code and a copy in case you find this useful. WARNING: Use this at your own risk, as I wrote this for myself I haven't spent any time putting in "Are You Sure" prompts or anything. If you were to do something crazy like DeleteOldFiles 5 c:\ it will delete any file on your C:\ drive that has not been written to in 5 days - including things in the Windows directory..

Update: The second I posted this, I noticed the ForFiles command in the Windows Server 2003 resource kit that works a bit like find on unix. Hey ho, at least I wrote some C# code today.

The code is below, as you can see there is not a lot to it - but a nice example of recursion.

Download file and source

A nice post to start the day from Bill Caputo. I know he doesn't think of himself as an XP evangelist anymore, but I like to think of him as the Pragmatic Evangelist.

I sometimes come across code that I consider posting to The Daily WTF, but I find the old career preservation instincts usually kick in. One of the most common problems occur when code is copy/pasted into lots of places. Tonight I accidentally stumbled upon this little tool for java - the copy/paste detector. Nice - the web site has links to the algorithims that are used. If I had the spare time I might get this working against C# code, but I have too many sourceforge projects open at the moment. Thinking about it, another great tool would be a plugin to word that could scan a document and hook into the google web api to find where sections had been lifted from - might have helped with 'that' dossier...

I'm loving the new Generics support in C# 2.0 - may I never use the vanilla System.Collections namespace again. Syntactically, generics in C# and Java are quite smiliar, but there are some important differences under the hood of the VM (or CLR). There is an intersting converstation between Bill Venners and Anders Hejlsberg posted here on the topic of the C# and Java implementations of Generics. Basically, the IL for .NET 2.0 understands generics, but the 1.5 JVM does not.

Fantastic, the EAP for Resharper 2.0 is open for public download. Cannot wait to install this as I've really been missing Resharper in Visual Studio 2005 (writing Using statements is soo 2003). I just hope ctrl-shift-n is in there already (remember, ctrl-shift-n is the new ctrl-n). Will post and let you know how I get on.

Bill Caputo has posted an interesting article discussing inheritance in C#, especially when using interfaces. I learned about OO while learning how to write Java, so I still struggle with the fact that methods in C# are not virtual by default. Bill explains why inheriting a class that implements an interface might not work as you expected.

The key statement here is:-

...the implementation of an interface method is implicitly 'final', just as if the 'sealed' and 'override' modifiers were present.

Anyway, worth reading. I think I might be getting myself a copy of .NET Essentials...

At the risk of prompting someone to ask for them, Dan Fernandez has posted a neat stored procedure utility that inspects your sprocs and can give you come basic code metrics on them as well.

If any managers are reading, just remember that code metrics are not a direct measure of productivity. If I spend a week on some existing code and the number of lines of code in it have gone down then I consider that to be a good week....

Thanks to a load of great work from Alexis Kartmann, the nDumbster project is now in a state that I would recommend it's use for real world projects.

nDumbster is a .Net port of the Dumbster project which is maintained by Jason Kitchen. nDumbster allows you to easily set up a dummy SMTP server in your xUnit tests and then query the messages that the server has received. Very handy for testing mail based code.

Anyway, if you are writing .Net based code that wants to talk to a mail server then take some time to look at nDumbster and let us know if you need it to do anything else. If you want to do a similar thing from Java then pop along to Jason's Dumbster project.

Update (Jan 2006): If you would like to use some of the features of VSTS from Eclipse then you might want to take a look at my new employer, Teamprise. They provide an eclipse plugin to access Team Foundation Source Control from Eclipse running on multiple platforms including Windows, Linux and Mac OS X.

Original Post: I have just managed to get a quick test java client talking to VSTS and retrieving a list of projects running on my Team Foundation Server. I know this is a small step towards the goals of the VSTSEclpse project, but I'm really happy I've got this working and wanted to tell everyone. Thanks to Davanum Srinivas, Dexter Wong, Evgeny Beskrovny and the rest of the folks over at the axis-user mailing list for all their help in getting NTLM authentication with a Windows Web Service working from a Java application. I've been trying to get NTLM authentication working for something I needed to do at work recently, so this came in doubly handy (another example of working on Open Source projects in your own time directly benefiting your employer).

The test client uses Axis 1.2.1, Jakarta Commons HTTPClient 3.0rc3, Jakarta Commons Codec 1.3 running in Java 1.4.2_08 on Windows XP SP2. The Commons stuff is needed to allow you to talk to the VSTS Web Services because they are protected with NTLM authentication. To get NTLM working from java, you need to do the following:-

Download a copy of the WDSL to a local directory. This is to allow you to run WSDL2Java over it to generate a starting point for your stubs. You can point WSDL2Java to a URL, but not if that is protected by NTLM ;-)

Run WDSL2Java pointing at your local WSDL file. For more information on WSDL2Java see the axis user documentation.

Edit the createCall method of your stub to include the credentials you would like to use in the call. Currently I have this hard-coded like:-

Obviously, this should be loaded from a configuration property somewhere.

Now, you need to configure Axis to use the Commons HTTPClient class for the HTTP transport. Luckily, the Axis folks have made it very configurable, so all you have to do is include a file called "client-config.wsdd" in the classpath. My client config looks as follows:-

Now all you need to do is write a client class to use your stub to talk to VSTS. Below is my first class which returns a list of the projects available in the team foundation server by talking to the Common Structure Service running on Team Foundation in the bisserver application.

Now the real work starts...

If you have ever tried interoperability between an MS environment and any of the others you are likely to have tripped over NTLM authentication before now. Obviously there is no RFC for this one, but Eric Glass has an excellent explanation of NTLM that was gathered by reading public available information and a bit of network sniffing. This information was used by the Samba developers when they were enabling Windows File Sharing to/from *nix which is such a great achievement I am still in awe that they ever got it done.

Just found this interesting article on documentation in Agile projects by Scott Ambler. As with everything in the Agile world, we must strive to do just enough to meet the business objectives - this includes documentation. One point which Scott makes very well is that "Your team�s primary goal is to develop software, its secondary goal is to enable your next effort."

Even when I am not working in an environment which requires a formal design phase, I often write stuff down during the design stages so I can get my head around a problem. These take the form of Design Notes, a logical UML diagram or a scribble on a whiteboard - whatever works best as a way of communicating the design to myself and the rest of the development team. In my view, the key to a good design is that each component should have a very discrete job to do (low coupling, high cohesion). I just wish I could figure out a painless way of sticking the artifacts produced during the initial design stages into a document that someone could pick up and get an understanding workings of the system.

I find presenting the design the best way of communicating, but that is probably because I enjoy talking. A document is quite often not the best form of documentation, but you sometimes cannot avoid them.

I'm currently thinking a lot about different testing strategies. Martin Fowler has an intesting post entitled Mock Aren't Stubs in which he also discusses the differences between State testing and Interaction testing. In summary state based testing is when you set up your test conditions and say that when you do x the result in the data or from your method is y. Iteraction testing is when you say that when I call the method it should call a, b and c passing the appropriate parameters.

In the days when I first started learning to write code professionally (on an MVS mainframe in PL/1) a test was described as a set of conditions that if your code passes then by definition it is working. This is classic state testing. We didn't do interaction based testing then, possibly becuase we couldn't - we didn't have nMock on the mainframe..

Stepping back into the modern world of TDD in dotnet, by doing state based tests IMHO you are more free to refactor and test all possibly branches of the code by adjusting the inputs to the public interface. With a state based approach you find yourself having to edit the tests much less and just wait for the magic green lights to say you have finished.

Interaction testing allows you a much finer granularity of control in your tests, leading to smaller, discrete sections of code. It also allows you to quickely identifiy which part of the code is broke (by following the single red light in xUnit). However, I've recently run into a few instances where problems have been spotted in the very final stages of programmer testing because assumptions were made about the interaction that turned out to be false.
I'm not saying interaction testing is bad. It is just emitting a slightly bad smell for me at the moment, hence why I am re-evaluating how I test.

Apologies for the lack of posts recently. We are due a new release of my latest project at work in a couple of weeks and in my spare moments I have been investigating my new play project, a VSTS plugin for Eclipse. Anyway, I have just uploaded the website for the VSTSEclipse project to sourceforge.

It is ages since I have done a static HTML website and not been messing around with some server side functionality or other. There are a couple of features I would like to add but didn't want to delve too deeply back into the world of php that I haven't touched for at least 4 years, possibly longer.

This site is not the most inspirational design I have ever done, but it is all div / css based, tested in IE 6 and Firefox 1.0.3 and I hope to post a handheld version of the stylesheet shortly. As ever there is hardly any content at the moment, but there you go - you would expect nothing less from me, it would all be spelt wrong anyway.

Take a look at let me know what you think. The project itself is really interesting and quite challenging. Looking forward to getting stuck in to it (just need to finish off installing Beta 2 of VSTS at home).

In an effort to keep my Java skills up to date while also using some of my new-found dotnet knowledge I've joined the team trying to put together a VSTS SCC plugin for Eclipse that talks to the source control functionality in Visual Studio Team System. It is early days yet but it is an interesting goal and I hope to learn some interesting stuff while putting something together that will be useful for the wider community. If the only thing I accomplish is to get the wonderful Eclipse Diff tool hooked into VSTS then I'll be very happy (the one in VS 2005 is still not as good as the Eclipse one, if a little better than the appalling diff tool that ships with VSS 6).

From Martin Fowler's site:

Detestable (adjective): software that isn't testable.

Heard about WebWork from a few different sources lately. I hate Struts with a passion (especially in large teams) and find the aspx framework in .NET gets in the way between what I know I want to do and the implementation. I have ended up writing a few of web frameworks in the past in different langauges (including Net.Data, remember that anyone?). Anyway this post is a note for me to go check out WebWork and see what it does. Any comments about it are gratefully recieved to the usual email address or in this post (if it is working - I am overdue a change in blog software aswell...)

Good post on Paul Vick's blog about the characteristics of Black Hole projects at M$. This got me thinking about some traits I recognise:-

• The goal of the system cannot be summed up in a sentance
• They involve more people than you can fit on a bus
• They must have completely unrealistic deadlines.


• Alternatively, they have deadlines more than 9 months away.

Posted by Martin at 08:33 PM | Comments (0)

Recently discovered these excellent UML 2.0 Stencils and Templates for Visio by Pavel Hruby.

While I am still waiting for my ideal UML drawing tool, I keep coming back to Visio. It is much better than Rose and better performing than Poseidon (you also don't get that daft fish dude in the background as you do with Poseidon exports).

With IBM's UML 2.0 article and the book UML 2 For Dummies, I can express what I want to without getting to bogged down in implmentation detail too early, yet I can give that level of detail when I need to. My only problem is when I turn up on customer sites having been sold as an Architect and I have a copy of UML 2 For Dummies under my arm. At least I am setting expectations early...

Just found out about the various extensions to RSS 2 that allow information about comments to be included in the RSS feeds.

I have revised my feed to include the full article along with links to the comments in it, numbers of comments etc. For more information on supporting comments in Moveable Type see this excellent HOWTO by Oleg Tkachenko.

The only caveat I would add is that you must remember to add the following namespace declarations to your RSS feed file (index.xml):-

    xmlns:wfw="http://wellformedweb.org/CommentAPI/"
    xmlns:slash="http://purl.org/rss/1.0/modules/slash/"


Been using Visual Studio 2003 lately, not a bad editor (no where near as good as intellij for Java for pretty good). Today, all of a sudden the autocomplete feature - called Intellisense (TM - M$) stopped working. Now normally, this is because of an error earlier in the code, not yet displayed to you (dumb). However this time my project was building fine, just no auto-complete. After much scratching of heads, the following seems to work:-

• Shut down Visual Studio .NET


• Open the project in explorere


• Delete the bin and obj directories from within the project that is causing problems (or delete all of them if in doubt)


• Start up Visual Studio