Sabertooth - Strategies that hit the mark

There is a lot of hype about Extreme Programming lately. Just like it's name suggests, it's techniques and methods are extreme and because of that it has architects, developers and management on different sides of the fence (with abnormally few ON the fence).

5 cent tour of XP
For those who don’t know too much about Extreme Programming I will give you the 5 cent tour. XP was created by Kent Beck, Ward Cunningham, and Ron Jefferies while working on the now infamous C3 project at Daimler Chrysler a while back. They claim they came up with a better way to deliver software in a shorter time span, in an ever-changing requirements environment.

The XP methodology promotes the use of a number of lightweight principles to replace heavyweight design methodologies that require a lot of notational documentation (e.g. CASE tools). Some highlights include:

Iterative design instead or ‘single run’ design (e.g. waterfall)
CRC cards and system metaphors instead of CASE tools and bulky design docs
Frequent Small Releases instead of one large release at the end
Simplest design possible instead of building code you may (or may not) need upfront
Refactoring (improving upon simplest designs, reducing redundant code, etc.)
Paired-programming instead of individual coding
Test-driven design whereby unit tests are created before code instead of after
No overtime (40 hour week) except for final crunch week at end

The hoopla
Now many of those principles are extreme in terms of being near opposites of the standard way applications have been developed in the past. While a number of XP's principles such as iterative design, test-first coding, and refactoring have been assimilated into a growing number of software design methodologies, there is a single point of contention among many - the 'Simplest design possible' principle.

Alistair Cockburn, founder of the Crystal family of Methodologies and noted expert on the human quality of technology, jokingly commented on that principle with the statement that one of the top books NOT to write would have been ‘ignorance-driven software’ until now (cause it has been done in XP).

The subject of that joke has become the rant of many architects and analysts. They are fighting furiously to denounce XP’s value and get the software industry to stick with the heavy hand of design approach.

Argument against
Is XP really ignorance-driven design because it prescribes providing the simplest design that would work and then refactoring it later? Could you effectively end up refactoring 2 or 3 times for every simple design choice, reducing overall productivity? Would a thorough design exercise upfront prevent that from happening?

If the customer knows exactly what they require, and those requirements can be completely fleshed out, and the application being built follows a very predictable pattern – yes, a thorough design could prevent extra work later. It has been proven successfully many times (e.g. aerospace industry, military).

How many times do those conditions honestly apply in e-business though? Very few, in my experience, due to the fact that customers rarely know what they want until after they get it. This is because people, by nature, require a frame of reference before they can make a judgment.

As a result, the first time a customer develops an application they know very little about what they want or require. After they complete the first version, and the users have used it for a while, they have a good idea of what they want (or more likely don’t want), because they have a frame of reference to base their decisions on. With each version, or iteration, of the application the customer will better understand his or her requirements. In the end it may take several versions to end up with a product that really meets the customer’s needs.

Argument for
If our goal then is to only deliver what the customer asks for and thinks they will require at the time, then XP is right on the money. It prescribes meeting the current needs as fast as possible, and improving it as necessary. The ‘quick win’ theory does produce results. A wise project manager once said to me ‘it’s hard to argue with success’. That is XP’s fundamental strength and in that way it can be very successful. Customers that frequently see progress and success are much more apt to continue to fund a project to its completion, and be more at peace that the final outcome will be successful (not anxiously awaiting the end, hoping it will be successful).

So why then do architects and seasoned developers, having also developed many e-business applications themselves and assumingly knowing full well that customers frequently change requirements requiring redesign, persist that their way is still better?

Perhaps, as Mr. Cockburn often suggests, we need to look at the root of their desire. Many software architects and developers in general strive to create and develop the best possible solution every single time. Much like the craftsmen of old, software technologists make it their mission to do it right (a good number of them anyway). In that way, a ‘good enough’ approach to design is counter-minded to their goal of producing the best. Many IT professionals go out of their way to convince their clients to do something that although much better, robust, and forward thinking for their business is NOT what their client is asking for. It is also possible that although a much better solution is being offered, it is also NOT what the client requires right now.

Is good enough, good enough?
So really its not a question of whether XP can or cannot work (it has been proven as well), it’s a question of whether or not architects and developers can feel comfortable delivering only what a customer wants?

…And…

Is it the technologist’s job to deliver what they need (or may need), regardless of what they are asking for?

While we must always offer the best solution available based on our domain expertise, that solution must conform to the client's needs. How many of us are domain experts in our customer’s domain? More importantly how many of us have the inside scoop on the political decisions being made that ultimately govern any initiative’s success, technical or otherwise? Few, I would wager.

The key to this whole debate really lies in the difference between ‘value’ and ‘value-added’.

Value and value-added
If a solution meets the customer's current goals and requirements it is a success and offers value. Anything above and beyond meeting (exceeding) those requirements is value-added. There is a fine line between exceed and excess.

If those additional features are actually used then they will be perceived as added value, meaning they provide additional value beyond the base value expected. If they aren’t used then they will be perceived as excess baggage, or feature bloat as it were. The critical factor here, however, is that regardless of the success of those value-added features, it will be difficult to determine whether or not valuable time will be wasted on designing and implementing functionality that may or may not be used upfront.

Kent Beck of XP states ‘give me that time you would spend on features that may or may not be used, and I will use it to refactor what the customer IS using several times over’. Since the true test of whether an application is successful or not will always be in its use, Mr. Beck’s argument is very compelling. XP, and the adaptive approaches of the AGILE methodologies in general, all subscribe to the reality that no matter how well you design an application you may not meet your customer’s real requirements and it may need to be redone, enhanced or adjusted – so why not plan for that to happen and speed up development to a point where you can accommodate multiple changes in a timely fashion.

XP is saying don't rely on an idealistic view of design, but put mechanisms in place that will cope with the reality that requirements will change and that design will need to evolve. In that light XP is much more deserving of the title reality-based programming, rejecting the idealistic (and often unrealistic) view offered by the all-encompassing design upfront approach.