Recently in Programming Category

Earlier in the week I was on a support call with a customer who was seeing strange issues.  It later transpired that they had a TFS workspace configuration issue that we were not able to solve quickly over email.  When we were saying the words “TFS Workspace”, they had been translating this as “Working Folder Mapping” which is a perfectly sensible thing to do for someone new to TFS – but was sadly not helping in this case.  Once we’d ran the user through a couple of scenarios they instantly “got it”.

One of the many pleasures of working for Teamprise is that our end users are nearly all developers which makes our life much easier.  In this particular case, once the customer “got it”, they quickly forgot the time when they hadn’t and became a little apologetic blaming “User Error” for their troubles.

I don’t believe in “User Error”.  I’m not just being cute – I really don’t believe that it exists.  If you went up to a vending machine and instead of a tasty beverage you ended up with a nasty cut you wouldn’t say it was “User Error”.  Equally, when getting into an elevator to ride from the 20th to the 22nd floor, if you had to travel down 20 floors before going up to your destination it isn’t “User Error” that is causing the diversion but an issue with the user interface and the design of the application.  Especially if you go on this diversion more than twice.

Admittedly, there are times when users accidentally do dumb things – and there is a learning curve to all applications (the fun my son is currently having learning his numbers by pressing the buttons in elevators is an example of that learning curve).  However if more than one person is having the same issue with your application then it is a good sign that you have a usability fault and you need to make your application more transparent.  While some usability issues will probably always have to live in your software (especially if it is not a mass-market consumer application) at least try to file down some of those sharp edges in your code.  That way, if they do go off the beaten track then at least they shouldn’t get hurt.

Over the years, I have tried many different approaches to software estimation.  From the Ceiling and Weller method (look up to the ceiling, scratch your chin and say "Well-errr"), FITA analysis (Finger In The Air) through various variations (both formal and informal) of metric based estimation techniques to things like IFPUG etc.

Today, I had something happen that reminded me why software estimation is so hard. I've just wasted about 8 hours trying to figure out an issue with what I thought was some bizarre firewall problem with Windows Vista SP1 x64 when it turns out that some code that I wrote was actually working correctly and was picking up a HTTP proxy preference that was pointing to a server that no longer exists.  I'd just forgotten I'd fixed this bug, and that I had the proxy set in my preferences.

I took two lessons from this frustrating day.

1. We need a better error message when your proxy is no-longer available
2. Software estimation is hard

In discussions with my wife she frequently struggles when some nights I finish work complaining that I am 3 days behind and the very next afternoon I can be caught up or even a little ahead.

The best way I have come up with to explain this, is to get her to imagine that I had given her 60 Suduko puzzles all rated as 10 minute puzzles.  Should take you 10 hours right?  Now see how long it takes you to do each puzzle.

Suduko is the closest analogy to computer programming that I can find for "normals" - i.e. people that don't code.  This only really works if the person does Suduko puzzles, but my wife does so it works in our house.  Suduko is a numeric analytical problem solving activity.  While there are tricks and techniques to solving some puzzles, there is a significant challenge and difference to each one. Looking at the puzzle, it is hard to know if it is going to be hard or easy.  You can get stuck down blind alleys and have to start all over. Also when you "get into the zone" you can often make surprising intuitive leaps that often defy verbal explanation afterwards. Finally, solving a hard Suduko puzzle quickly involves a fair degree of luck and depends on your state of mind at the time of trying the puzzle.  There is a great amount of satisfaction to be gained from solving a Suduko puzzle along with a high degree of frustration when you cannot solve one - you know that it must be possible after all.

So.  Each task (solving a Suduko puzzle) should take about 10 minutes, with-in a certain error range.  In software estimation a really good developer doing a well known and well defined task can only hope to get their estimate to around 25% accuracy (a "hard estimate") - and then there are always the odd random occurrences that throw you way off.

I've been in interviews where the candidate swears blind that they always finish a task on time.  This tells me two things about that candidate:

1. They are a liar
2. They either never estimate how long something is going to take up-front or they over-subscribe to the Scotty principle of estimation. (Or as Microsoft are fond of saying, under promise - over deliver)

Does this mean that we should give up on estimation?  Of course not.  Planning a project is going to be pretty hard if you have no idea of roughly when you are going to be finished, what the end result is going to do or how much it is going to cost.  Not to forget that when estimating a large number of tasks you can rely on the fact that some of the work you finish early will offset the work that takes longer than expected.  However, understanding the nature of software development and how it differs from, say, laying bricks, helps you be more likely to succeed.

The problems inherent in software estimation also help me to understand why Agile software development methodologies work. 

• You are forced to break problems down into small bits that can be managed, tracked and measured.
• Small iterations means that you can only get so far behind.
• You re-estimate and re-prioritise work at each iteration when a hard estimate on that task is possible (and therefore your estimates are more likely to succeed)
• You frequently listen to the person who will be using the software
• You can load your iterations so that there is always work to do of the correct priority to the customer when you have finished a task early and things of the appropriate priority that you can drop off the list when something takes longer

There are also many other techniques that people can adopt from Agile methodologies or just from common sense to reduce the amount of time taken when a task is taking longer than expected, such as

• Limit distractions. You need time to concentrate on a single problem otherwise you will never finish it.
• Pair programming. Identify when a second pair of eyes is needed and quickly as for help in your team (can be hard when your team are distributed across multiple time-zones)
• Daily progress meetings. The guy that has been stuck for 24 hours on what he thinks is a firewall issue is quickly identified)
• Lack of ego.  I'm the "Windows guy" in our company, if I say something is a firewall issue with Windows Vista SP1 x64 and a probably collision with the new Eclipse 3.4 launcher executable - then the new intern who only knows about the Mac he used at college should feel like he can ask a "dumb" question and say "Have you checked your proxy settings after that bug you were fixing yesterday".

Anyway.  All common sense stuff and writing this post has been a nice way to get rid of the frustration of finding why I was stupidly stuck for the past day.  Now to stop distracting myself further and get on with some more work :-)

As we all know by now - the build system in TFS2008 was substantially improved.  Along with the many improvements came an official API for talking to the build system.  This is the same API that the Team Foundation Build UI in Visual Studio uses, however there are many additional methods that were added that were not for the UI at all but for potential consumers of the Build API.

In talking with folks at community events, and on the MSDN forums I have realized that there isn't a huge awareness of this API.  Having written a parallel implementation of the build API, but in Java for the Teamprise 3.0 release I have spent a great deal of time with the .NET API and have a few examples lying around of how to accomplish certain common tasks - so I thought I would run through a some of them.  If you have an example of something you would like to see with the build API then please leave a comment for this post or drop me a line.

So - let's start with a basic one.  How to queue a build.  This will introduce us to a few concepts with-in the build API that are common across all of the methods.

The Easy Way.

A quick look at the class diagram above will show you that the main interfaces you'll be dealing with in the Build API are the IBuildServer and IBuildDefinition interfaces.  To get started with these you'll need to add references to the Microsoft.TeamFoundation.dll, Microsoft.TeamFoundation.Client.dll and Microsoft.TeamFoundation.Build.Client.dll.

C#

TeamFoundationServer tfs = TeamFoundationServerFactory.GetServer("http://tfsserver:8080”);
IBuildServer buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));
IBuildDefinition buildDef = buildServer.GetBuildDefinition("TeamProject", "Build Name");
buildServer.QueueBuild(buildDef);

VB.NET

Dim tfs As TeamFoundationServer = TeamFoundationServerFactory.GetServer("http://tfsserver:8080")
Dim buildServer As IBuildServer = DirectCast(tfs.GetService(GetType(IBuildServer)), IBuildServer)
Dim buildDef As IBuildDefinition = buildServer.GetBuildDefinition("TeamProject", “Build Name")
buildServer.QueueBuild(buildDef)

I'm afraid that's the last example you'll be seeing in VB.NET.  Being a Java developer by day I tend to like my semi-colons.  However if you are a VB developer then hopefully you'll be able to follow along as the rest of the examples are more or less just method calls with the occasional bit of casting.

So - what's going on here.  Well we first get hold of a TeamFoundationServer object.  If this was server side code and we wanted to specify some credentials then we would use a slightly difference mechanism but in this case the constructor works well and this will automatically connect using the credentials of the current thread.

Next we get hold of the build service that implements the IBuildServer interface.  Finally we get a hold of the build definition by specifying a team project and build definition name (build definitions are unique per team project in TFS).  We then queue a build using the defaults for that build definition.  This API call is actually shorthand for the following...

The Hard Way

Supposed you want to queue a build, but you don't want to use the default build agent, priority, drop location etc.  Well in that case you need to look at the IBuildRequest interface.  Here you will find all the options to customize the build request, you can specify a build agent, drop location, priority etc like you can do from the Visual Studio UI.  You will also find other options such as being able to queue the build with a maximum acceptable queue position, pass a custom get version for the build or even queue the build in a postponed status.

In the following example, I am going to find a non-default build agent from the server and then queue a build using it.

        TeamFoundationServer tfs = new TeamFoundationServer("http://tfsserver:8080");
        IBuildServer buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));

        String teamProject = "TeamProject";

        String buildDefinitionName = "MyBuildDefinition";

        String buildAgentName = "MyBuildAgent";

        

        // Find our (non default) build agent.

        IBuildAgentQueryResult queryResult = buildServer.QueryBuildAgents(buildServer.CreateBuildAgentSpec(teamProject, buildAgentName));

        if (queryResult.Failures.Length > 0 || queryResult.Agents.Length != 1)

        {

          throw new Exception("Invalid Build Agent");

        }

        IBuildAgent buildAgent = queryResult.Agents[0];

        IBuildDefinition buildDefinition = buildServer.GetBuildDefinition(teamProject,buildDefinitionName);

        // Create a build request based on our chosen definition.

        IBuildRequest buildRequest = buildDefinition.CreateBuildRequest();

        // Optionally set command line args, drop location, priority, custom source version etc here

        // in this case just overriding default build agent.

        buildRequest.BuildAgent = buildAgent;

        buildServer.QueueBuild(buildRequest);

Pretty simple really and very powerful.  I encourage you to go take a look at the IBuildServer interface to see some of the stuff that you can do.  If you have any suggestions as to what you would like to see as a build API example then leave a comment to this post.

Jeffrey Fredrick just announced that CruiseControl 2.7.2 is now available for download: http://tinyurl.com/2zm9mz.

There are lots of bug fixes, lots of changes to the Dashboard and some new plug-ins, but the bit that is of most interest to me was (from the release notes)

TeamFoundationServer source control
----------------------
* Fix compatibility with Microsoft Visual Studio Team Foundation Server 2008 (CC-735). Submitted by Martin Woodward.

This was to work around an issue that came up when using CruiseControl (java version) to talk to a TFS2008 server (TFS2005 worked fine and still does).  If you are attempting to use CruiseControl with TFS 2008 then you should go with CruiseControl 2.7.2.  For that matter - if you are using CruiseControl.NET with TFS then you should also take a look at the latest release of the integration to TFS - as that contains the same fix allowing you to happily talk to a 2008 version of Team Foundation Server (also using the TFS 2008 client API's).

Anyway, congratulations to the CruiseControl team on the 2.7.2 release!

At EclipseCon 2008 this morning, we just announced that Teamprise 3.0 has been released!  If you've been wondering why I have been quiet on the blog lately - but also why anything I have been talking about is Team Foundation Build related, then you are about to find out why :-)  First of all, I'd encourage you to go visit the shiny new website at http://www.teamprise.com.  Our marketing team had too much fun putting that together, including getting a real, live, massive Teamprise power button made up and shipped in a huge crate from New York to be photographed and used as the new site/icon image.

The full release notes are available here, but as has been the tradition for the past few Teamprise releases, I thought I would give you a run down of my favourite new features in the 3.0 release.

At a high level, the features in 3.0 can be summarised as:-

• Full Team Foundation Build integration (including ability to execute Ant based builds)
• Check-in policy support
• Recursive folder compare
• Single sign-on (from Microsoft Windows machines)
• "Destroy" command for version control
• Show deleted items and undelete from Source Control Explorer UI
• much much more (see release notes)

While it is not my area, I should also mention that we've taken this opportunity to make our licensing more affordable for smaller teams.  We have been very pleasantly surprised by the number of people buying 1 to 20 licenses at a time.  Originally, Teamprise pricing was skewed to the Enterprise customers (i.e. simple, all inclusive and with steep volume discounts).  So we have done a couple of things to help out the smaller companies:-

• You can now purchase the various components (Teamprise Plug-in for Eclipse, Teamprise Explorer, Teamprise Command Line Client) individually as well as the Teamprise Client Suite which gives you the lot.
• We have lowered the initial prices for a single seat, meaning that people buying one or two licenses can now get the same discounts that used to only be available to folks purchasing 100.

If you have any licensing issues / queries then feel free to contact me, or you can talk to the sales team direct at sales@teamprise.com.  Anyway - back to the part of this release that I do know about - the technology. 

The first feature I want to talk about is one that I had no involvement with.  It's one of those features that many people will not notice because it just works but anyone who has done any Java to .NET web service interop work will instantly recognise as being a little bit clever.

Single Sign-On

The initial log-in screen has undergone a big overhaul.  On Windows machines you are given the option to use "default credentials", i.e. the username and password that you are logged onto windows with.  It obviously doesn't know your password, but does some JNI magic to get the native Windows API's to handle the authentication logic with Team Foundation Server.  While you are also on the login screen, you may notice the Profile feature.  This is an area that many people probably won't use, but we added for our power users and for ourselves.  Basically, the profiles feature allows you to store sets of servers/credentials that you commonly use to connect to Team Foundation Server and then you can bring up the details using a simple drop down.  Makes it much easier to switch between your production TFS instance and your CodePlex project for example - or switch credentials if you are a TFS administrator.

Check-in Policy Support

In Visual Studio, check-in policies are implemented as a .NET assembly runs every time a policy is evaluated or configured.  The policy also has full access to the .NET API's, the Visual Studio API's as well as anything it might want to pinvoke out to on the Win32 API side.  As you can imagine, this presented us some problems when we wanted to have check-in policies that ran the same in Eclipse on Windows Vista as Teamprise Explorer on the Mac or Aptana on Ubuntu - therefore we have had to develop a parallel Teamprise check-in policy framework.

As we were doing this, we took the opportunity to learn from some of the feedback folks have been having with the Visual Studio check-in policies.  While our framework and SDK will look very familiar to anyone that has developed a custom check-in policy for Visual Studio, you will notice some differences.

Firstly, we supply different policies out of the box.  The vast majority of custom check-in polices that people deploy are things like "Check for Comments" etc, so we just shipped the common ones our customers wanted to prevent them from having to write their own.

Secondly, we make use of the Eclipse plug-in framework to implement our policies as extension points.  This means that they are easy to deploy (using the Eclipse update site mechanisms built in to the IDE).  We have also separated the configuration (stored as a blob of XML data in our framework) from the implementation - represented by the plug-in deployed.  The again makes it easier to deploy, especially when it comes to version 2 of a policy...

Thirdly, all of our policies can be scoped by the path in version control to which they correspond - you are not limited to per Team Project scoping and you do not have to wrap your policies in a custom policy to get more detailed scoping like you do with the current Visual Studio framework.

Team Foundation Build Integration

Anyone that has been following this blog for a while, or who attended the Team Build talk I did at TechEd with Brian Randell, will notice that I have been increasingly involved in the inner workings of Team Foundation Build.  Now you can see the fruits of that labour.

In Teamprise we now have full integration with the shiny new build functionality in TFS 2008 as well as support for TFS 2005.  Backwards compatibility with the TFS 2005 server is very similar to if you were using a Visual Studio 2008 client, accept that ours is slightly more backwards compatible (you can create new builds on a TFS 2005 server as well as manage build qualities etc).  However it is with TFS 2008 that you get to see the majority of the features.  I could go on about this aspect all day as their are so small things that I am proud of, but at a high level you can:

• View existing build definitions
• Manage builds in Build Explorer
• Queue new builds
• View build report
• Edit Build Quality
• Delete build
• Manage Build Qualities
• Open Drop Folder
• New/Edit Build Definition

• Edit Retention Policies
• Keep Build
• Set Queue Priority
• Postpone Build
• Stop/Cancel Build
• Delete Build Definition

One of the smaller features I will call out is that from the build definition in the Team Explorer, you can right click and do a "View Build Configuration" that will open the Source Control Explorer at the place in which the TFSBuild.proj file is stored so that you can check it out and edit it.  A feature that I added solely for my own sanity during dogfooding :-). 

All this would be fairly academic, if you didn't have some way to do a cross-platform build using Team Foundation Build.  In the current release, we provide a the Teamprise Extensions for Team Foundation Build which basically Ant enables the Team Foundation build server.  The Teamprise extensions are a set of MSBuild targets that insert the Ant build process into the standard Team Build mechanism as well as a custom MSBuild We hope to extend this to support in the near future to some of the other common build/test tool-chains in the cross-platform world.  However, the Ant integration case will help a lot number of people out there.

Best yet, the Teamprise Extensions for Team Foundation Build are available free of charge for everyone - wether or not you are a Teamprise customer.  Also, if you want to see how they work and customize them to meet your own non-standard build system then the source is available under the permissive open source Microsoft Public License (MS-PL).

I would personally like to thank the Team Foundation Build Team (especially Buck Hodges and Aaron Hallberg) who have been incredibly helpful through the development of the build functionality in Teamprise 3.0 while they were also busy working on TFS 2008. 

Hopefully that gives you a quick flavour of Teamprise 3.0 and where we are going with this release.  If you head over to the new site now and take a look at the many improvements we've made, we'd love to hear what you think.

The fantastic team at CodePlex have just rolled out yet another great feature - this time it is statistics for your CodePlex project.  I dropped by my TFS Plug-in for CruiseControl.NET to take a look...

The project is currently averaging over 12 downloads a day and has had nearly 5000 downloads since I moved it to CodePlex in August 2006.  Not bad for a chunk of code I originally wrote on a plane..

This is incredibly motivating as a maintainer of an Open Source project.  I've been knowing that I need to give the CC.NET integration some love as there are a few issues and things that I need to address from the excellent feedback I have been receiving.  However - seeing how many people are still interested in the plug-in despite the excellent CI functionality in the 2008 release of TFS has made me realise that I need to get on this ASAP.  Thanks CodePlex!

Developer Developer Developer is coming to the beautiful west of Ireland in May with GAMTUG hosting the event on Saturday May 3rd.  If you are in or around Ireland at that time then I would encourage you sign up.  My good friend, and fellow Team System MVP, Mike Azocar will be doing a session on Scrum which should be well worth sitting in on.

There will be no Microsoft speakers presenting, just speakers from the .NET developer community, although I hear our local friendly Platform Evangelists (DPEs) will be on hand to help out and chat to everyone and to take in the atmosphere.

The day promises to be a great learning experience, but also a fantastic way to meet like-minded people - if anything else, you get a trip to the lovely Galway which is never a wasted journey.