The term conflict is vastly misunderstood

"We never have any conflict in our team"!

For me that is a really, really sad sentence. I really love to work with teams without quarreling and bickering – but I very much hope that there is conflict.

The meaning of conflict

What is `conflict´ after all? According to the english Wiktionary conflict is:

Noun conflict (plural conflicts)

1. A clash or disagreement, often violent, between two opposing groups or individuals.
   [...]
2. An incompatibility, as of two things that cannot be simultaneously fulfilled.
   [...]

And while almost everybody thinks about the first interpretation, the second one is equally valid and extremely valuable to recognize.

It's all about managing conflict - and that's a good thing

This is yet another thing that popped up when Tom and I prepared our hand on Agile and lean practices course - a lot of the techniques in lean and Agile are there just to handle conflicts in a reasonably civilized way:

• We prioritize work-items because they are competing for development time - there is a conflict for the time
• We queue deployments - there is a conflict for the deployment machine
• We use A3 reporting to point out conflicting interests.
• etc.

Recognizing conflict is an enabler

Without recognizing conflicting interests as such there is no way to resolve the conflict - take the original idea behind (non-automated) continuous integration: Software developers in the nineteen-nineties realized that they ended up with incongruent systems, when they all just committed their work to a central repository whenever they had a working result.
Apart from testing mechanisms and some other technical details, there was also a hidden conflict - the conflict for exclusive access to the system for modification. Recognizing this conflict enables people to find solutions. In the original example from Kent Beck, integration happened only on one machine, thus mediating the conflicting request for access to the system by queueing them. Denying this conflict on the other hand prevents solutions.
(Or this denial leads into misguided attempts on Jidoka like continuous integration servers which should really be called build servers and more often then not are just used as a fig-leave ... but that is another story – which will be up here shortly).

Conflict, there's just no way around it…

…but there are many ways to handle it wisely - by applying tools and techniques to resolve or manage it. As with change in the subtitle of the first eXtreme Programming (XP) book, fighting it only makes it worse. In XP they say "embrace change" - the same is true for conflict if handled responsibly "embrace conflict".

Priorities 101: I have to fell trees ...

Once upon a time ... ... I was wandering around the woodlands, when I heard a faint noise that drew my attention. I closed in on the noise and coming closer I heard heavy breathing and moaning and swearing and cursing - and the sound of an old mechanical saw cutting through timber.
Then I finally came to a clearing in the woods and saw a huge guy covered in sweat, wielding and old and worn out buck-saw to cut slices from an enormous tree he had cut down earlier.
I greeted him friendly and inquired as to why he was swearing and cursing so much. He looked at me friendly and explained with great patience "You know, the work of a woodworker can be really fulfilling if you have the right tools and are able to really excel at your work. But me? I only have this worn out buck-saw and I have way more work to do than I ever can finish."
So I asked him: "But what about those sharpening tools over there? Why don't you use them to sharpen the blade?" He looked at me worriedly. "Haven't you been listening?" he sneered, "I have way too much work to do, I DON'T HAVE TIME TO SHARPEN THE BLADE". And with that he turned around and started sawing away at the giant tree again...

I have heard and told this story in a dozen different ways - but I constantly have to remind myself to take my own advice. It's just too easy to get carried away in the everyday struggle of "getting stuff done" to take a look back at the big picture.
When I worked on the Hands-on lean and Agile practices course together with Tom Breur I realized that I had a lot of tools in my back that I constantly introduced to other people but neglected to leverage in my own work. Re-introducing things like A3 thinking and root cause analysis into my everyday (business) life has proven to be quite effective – and I am really happy that I took the time to sharpen the saw a little bit.

Software development is such a new field?

Software 'engineering' is not such a new field anymore, is it? After all we have been writing software since the late forties – so our field is more than sixty years of age.

Strangely enough, I keep hearing the argument that this is such a young field of engineering and we "don't yet have the experience to make it an exact science."

Is that really true?
How about building planes? That's quite a new field as well, right? If we start the timeline with the Wright brothers, we would have to start in 1903 (if you'd want start with Otto Lilienthal I'll raise you a Babbage – and I would answer a DaVinci with a Leibnitz – or even a Pingala if I have to. So, just for the sake of the argument let's stick with powered flight and 1903). So the Boing 747 – which had it's inaugural flight in 1969 and is still in service with many airlines – was designed when the craft of building planes was roughly the same age as the craft of building software is today.

And I doubt that you would refrain from stepping aboard a Jumbo-Jet because it was designed at a time where "the field of aircraft development was still very young and the results where not always predictable."

So I think, we should put more emphasis on the things we already can do – let's not complain that our internet connections is slow on a transatlantic flight – let's be amazed that we do have an internet connection on a transatlantic flight!

And let's not complain that software development is not determinedly predictable but let's embrace all the techniques we do have to manage and navigate this uncertainty.

In essence: we do have the tools – we just have to use them!

BTW: When Tom Breur and I designed the hands-on lean and agile practices course we found that there are so many tried and proven practices even from our own hands-on experience that we had to actively limit the number of practices to make it a digestible sized course.

What's in a name? (a.k.a. "I don't think that word means what you think it means")

Why is it so hard to get the challenges of software-development across to people who are not from "our field"?
To be quite honest: I don't know. But I do have an inkling about some of the more obvious points. One of them, that has been pointed out numerous times, is that we tried to compare software-development with production for a really long time - instead of comparing it with, well, development.
But let's not linger on that aspect - let's look at another one that has not been covered quite as well. Language and mental models. In most other fields the words and their physical representations are put to the test quite early - even if the words themselves may be ambiguous. No one would really try to use hazelnuts or walnuts and cross-bow bolts when a manual speaks of "nuts and bolts".

So, what's different in Software development? A lot! One of the main differences though is, that we don't work with concrete material (nuts and bolts) but with concepts - words representing things from the physical world. Or even worse: words representing things from the physical world that represent concepts that represent intangible things - like money or time.

But let's just focus on the words for a second - they can mean a lot of different things to different people. For example my grandmother has quite a different notion of the concept "button" than my godchild does, as I am sure you can imagine.

Or, as Lewis Carrol put it in "Through the looking Glass":

'When I use a word,' Humpty Dumpty said, in rather a scornful tone, 'it means just what I choose it to mean — neither more nor less.' 'The question is,' said Alice, 'whether you can make words mean so many different things.' 'The question is,' said Humpty Dumpty, 'which is to be master — that's all.'

And this is not only true for the concepts we use inside our craft but also for the concepts we employ to create, envision and define our products.
Let's just look at some examples...

• For some people "A3" refers to a paper format – for others it refers to a way of thinking (actually also based on the paper format due to technical reasons, but that is another story).

• For some people "XP" is an operating System – for others it is a software development process.

• A Retrospective can be a very interesting art exhibition (for example on Ansel Adams) - or a very formal technique to optimize the way people work together.

And so forth...

So from my point of view one of the challenges we face is in the meaning we put in words and concepts and how much they can differ across organizations or even within a team.

What to do about it? Take the time to find your own meaning for the words, join industry groups to discuss those meanings and get a little bit better applying the concepts every day. And never rely on a word meaning what you think it means - always verify with the people with whom you're working, that you're talking about the same things - even if you use different words.

Shameless plug: And perhaps attend some trainings where you can experience some of the concepts behind some of the words first hand...

We don't have a problem ... IS a problem

"Do you have a problem?" – "No, we don't have any problems around here" - "Oh, that is a problem"

What? – I have to admit, that this conversation might need some explanation.

The dialogue above depicts a typical situation that arises when groups first start to introduce continuous improvement. In our culture, problems are so tightly connected to bad things that we tend to either block them out or try to solve them as quickly as possible - even if the latter means circumventing the problem in the first place... instead of really going to the root cause of the problem and in fact solving it.

But as Jerry Weinberg once stated "Once you eliminate your number one problem, number two gets a promotion." (The Secrets of Consulting, page 15) – so the real question is "Are you aware of your current number one problem?"

To achieve continuous improvement, it is important to be aware of your problems - and to keep in mind, that under changing circumstances, yesterdays solutions tend to become todays problems – which makes it even more important to take a close look for and at each and every problem with a fresh and open mind as unbiased as possible.

Cover all the Angles

What gave planning such a bad name?
"In 'Agile' we don't do estimates!"
"We do not plan ahead - we just do what is necessary at that Moment in Time"
"We don't know what we will find out, so let's not try to plan the un-plan-able"
Guys seriously... Of course the kind of planning that emerged in the 70s, 80s and 90s of the last century, where people try to plan what other people do a year in advance down to a very detailed level does not make sense - but does that really mean that we shouldn't plan anymore?
After all, the ability to plan ahead is what made it possible to evolve from hunter-gatherers to settlers. And this planning did not include assigning "Caveman A" to sharpen the sickle on February the 20th and "Caveman B" to have a delivery relationship on his task to "fetch some sharpening stones, suitable for stone age sickles" with an end-date "not later than February 19th". But the planning still included all those things that could be addressed - plus contingencies. In a reasonable manner. With all members of the relevant community being responsible for their respective contribution.

But somehow planning really got a bad name, when responsibilities got piled up on the shoulders of "the planner" and instead of covering all the sensible angles for the problem at hand, "the planners" started to cover all the angles of attack to their backs.

And instead of planning for things that can be planned, it became more important to produce plans that seem airtight so that the poor guy who wrote it can avoid being attacked.

But don't misunderstand planning in 'Agile' or lean - it's not "no planning" it is way more planning - it is just not done by someone from outside in advance, but occurs more often and is done by the people who are best suited to make the decisions.

On a sailing boat you don't necessarily plan for the exact wind that you find on the day of the regatta, but if the race is well planned you do have the right sails on board to enable the highly trained and specialized (!) crew-members to make the best of the situation. Each and everyone responsible all by himself!

Fortunately, the lean approach brings us some of the planning back. So now the topic of planning no longer is as much of a taboo as it used to be for a while.

One place to experience some of the methods and approaches for lean planning is the hands-on workshop on agile and lean practices Tom Breur and I plan to run in the autumn of 2013

I hear and I forget; I see and I remember; I do and I understand.

Benjamin Franklin... Benjamin Franklin? Or was it Confucius? He supposedly said:

Tell me and I forget, show me and I [might] remember, involve me and I understand.

Or was it actually Xun Zi (also written as 'Hsün Tzu', but not to be confused with Sun Tzu of "The Art of War" fame), the confucian philosopher who said:

"Not hearing is not as good as hearing, hearing is not as good as seeing, seeing is not as good as mentally knowing, mentally knowing is not as good as acting; true learning continues up to the point that action comes forth"

Wether you'd like to rely on the Wikiquote rendition of the saying or not, modern behavioral science shows us that there is actually quite a big measurable difference between learning by hearing and learning by doing.
That fits well with the recommendation that can be found in 'The Toyota Way':

"Start with action in the technical systems; follow quickly with cultural change [...] Learn by doing first and training second."

Quoted from the first two 'general tips' for organizations on a transition towards lean (emphasis added)

And perhaps this also is the reason why simulations (or games if you're not too heavy on political correctness) are such an important part of 'Agile' and lean trainings.
The only thing I am still missing in most of these simulations (except for the getkanban game) is the usage of the real techniques. That is one of the reasons Tom Breur and I created the Lean and Agile hands-on training with a strong focus on all the management methods, applied in some well known simulations.

Lean is not Agile, is it?

We live in a time where project management Methods exist in abundance and traditional methods compete with Agile (with a capital A) and lean approaches which in turn compete with each other.
But do they? First of all methods hardly ever compete - it's people who do the competing. The approaches just exist and either oppose each other, complement each other or have no relationship whatsoever. The Agile movement mostly was a counter-movement against the overly structured approaches to software development that came to live in the 1980s and basically this movement re-instantiated good practices from the 1950s and 1960s as for example Craig Larman points out in his article Iterative and Incremental Development: A Brief History.
The whole lean approach in turn surfaced as the Agile movement was taken over by the same people who enforced the rigorous methods of the 1980s and more and more problems in the value chain, but outside the core development efforts, became visible. David Anderson – one of the fathers of Software Kanban – calls Kanban (which is related to lean software development) a "post-agile" approach, which doesn't really help in differentiating Agile and lean but emphasizes the fact that they are not the same.
From my experience companies that focus on agile have highly productive teams but often lack an efficient flow across the value stream while companies focusing on lean tend to have a real efficient flow but tend to lack efficiency inside the teams. To employ agile techniques inside the stations along the value stream while managing the value stream itself in a lean manner to me seams to produce the best results.
To get a first hand experience of how to integrate lean and Agile approaches you might want to come to the workshop that Tom Breur and I will run on May 28th & 29th in the Netherlands

Real options revisited?

BETA * some proofreading necessary * BETA * some proofreading necessary * BETA * some proofreading necessary * BETA * some proofreading necessary

What does a ticket from Auckland to Tokyo have to do with "Real Options" and how can 10 Minutes investment in an option save several hundred Dollar? A little while ago (roughly 9 month) I promised to come back on my experiences from missing a bus in the "middle" of the desert in Dubai.
So, here it is.

The Basics

"Real Options" – or real options, as I will mostly call them for the remainder of this piece – is a popular concept inside the ideas behind different models for organizing the work of knowledge workers, namely in the agile and lean communities. So what't the difference between real options and normal options?

How I understood real options

From what I understood about real options in the beginning the idea revolves around three basic concepts.

Options have a value (and a price)

So while you may see a value in an option like "(or a specific trip) I can choose between train and plane at the same price" there is a price to be paid to even have that option. (In this case it may be possible to get both plane and train tickets at a very low rate if you book a ling time in advance, but there might be cancellation fees that are applicable as soon as you cancel a ticket - so basically the price of the option "both kinds of transportation" is that of one of the cancellation fees.)

Options expire (or can be released)

If you hold an option at some point in time it becomes an obligation. Consider a tentatively held appointment of the "let's meet on monday or tuesday night"-style. If you collect enough of these appointments you probably will run into a huge overhead of coordinating open options if there is no way to cancel them. As explained in the talk by Chris Matts and Olav Maassen (see below) each option also implies a commitment in a way. So it becomes important to be able to release options if you don't want to over-commit by keeping options open.

Never commit early unless you know why (but pay the options price to keep it "open")

The first part of that rule comes quite naturally to me, because of my natural tendency to decide as late as possible (a dangerous tendency that I share with a huge part of humankind as explained by Dan Ariely in his book "Predictably Irrational") - but the second part makes it really something that I am willing to put active effort in. Especially, and that is what Dan Ariely talks about in his book, if the cost of keeping the options open is higher than the (perceived) value of the options.

How real options are explained in "the community"

When I just scanned the available slides and papers for the first time I was quite confused because they mostly state only the three concepts without the part in parenthesis

• Options have a value
• Options expire
• Never commit early unless you know why

but actually after listening to the complete talks I realized that the parts I put in parenthesis in my understanding where present – the where just not printed out. I guess in their book Commitment - A novel on managing project risk Chris Matts and Olav Maassen will actually go into all these details.

Practical example

Where did real options hit me?
Last year I retrained some of my development skills while traveling the world - completely on my own whim, so I should have had all the options I could have wished for. As I learned in Dubai that was not quite true, but the full applicability of the real options model – and the price I had to pay in hindsight for keeping my options open – hit me when I tried to jump on a plane from Charlotte, USA via San Francisco, USA to Auckland, New Zealand.
Since I had no one to answer to I thought it would be possible to just fly when and where ever I wanted - as long as I was equipped with the right visa. Little did I know about New Zealand's policy on that. Although citizens of the EU are eligible for a visa-on-embarkation there is a subtle twist. The organization who brings a person into the country is responsible for their departure after the visa expires. Which means that Air New Zealand required an onward ticket upon embarkation to the Air New Zealand plane.

I only realized that when the boarding staff in my layover airport (San Francisco) announced that all non Air New Zealand ticket where subject to exchange now. "Now" in this case being 40 Minutes prior to the scheduled "doors closing" time. Did I mention that I thought I had "all the options I could wish for"? Suddenly my Options decreased and became very exactly priced. I could either buy a (refundable) onwards ticket to Australia, or plan my onwards trip on the spot. Given the fact that I had a really, really slow internet connection I wasted about 30 minutes searching for an onward flight to Tokyo. Which I found. At a high price, but without having to buy the way more expensive refundable ticket to Australia. Why would I do that? Because buying the ticket to Australia - even though it was refundable - would have taken another huge bite off the monthly limit of my credit card - which I would have to use to get the ticket to Tokyo anyway in that month.

Back to real options: the investment of ten minutes on the same afternoon while I still had a decent internet connection to book a refundable ticket to Tokyo would have kept my options open - at a price but since I didn't invest that time the price for the option "go anywhere I like after New Zealand" changed from $0 to roughly $1000 inside 30 minutes - but still had an intangible (but subjectively high) value for me. So inside 40 Minutes I experienced the value of options, the price, the fact that they expire and that you have to act early to keep them open and enable yourself to push the decision to a late point in time.

Take away

From my point of view it is inevitable to take the parts in parenthesis into account - only if you take those aspects into consideration the options become real to me.

Ressources

Thanks a lot to Yves Hanoulle for the pointers!

• These videos do a great job of introducing "Real Options" as the term is currently used in the lean and agile space.
  • Chris Matts - Olav Maassen - IT Risk Management with Real Options and Real Liquidity
  • Jürgen De Smet - Yves Hanoulle - Improve your decision making by using Real Option
• When it's out, this book by Olav Maassen and Chris Matts will probably be the reference on the topic Commitment - A novel on managing project risk
• The original paper that introduced me to the ideas can be found at the infoQ article '"Real Options" Underlie Agile Practices'

The "Last Responsible Moment" may be sooner than you think

Now that was a nice one - as a practitioner of agile 'stuff' I have been a big fan of postoponing decisions to the last responsible moment for quite a while.
Suits my way of looking at things too.
The question of course is: When exactly is the last responsible moment? Usually when building software it is tempting to work with the assumption that the last responsible moment is right before I write the line of code related to the decision. Works well with the way 'agilistas' despise Big Design Up Front too.

Funny when you start to compare software development with the real life. (After all: That's what object orientation is all about, right? Modelling the real word with software representations? But let's take that at another time and come back to the subject at hand.
While I literary roamed the world in 2011 the concept of planning with regard to the "Last Responsible Moment" made the rounds on some of the mailing lists concerned with agile software development practices.
I on the other hand tried to live a 'normal life'. writing software and preparing material during the daytime, while spending the evenings with everyday pasttimes - in my case Aikido. Never mind if you have never heard of Aikido - although you might want to look it up at Wikipedia.

The question in case is the one of the latest responsible moment - in this case the moment for the decision on whether to hitch a ride with one of my fellow aikidoka, take a cab or take the bus.

To cut a long story short - in the real world the latest responsible moment is not when you reach a bus stop a couple of miles of the central public transport axis of dubai, shortly after your last training buddy has left.
Especially if the area temporarily has no cell phone reception and that particular bus stop is only serviced until half an hour earlier at this time of the year (mentioned in the small print)

16 elevator pitches for scrum as a contribution to @boeffi's compilation

As an answer to a recent query for scrum elevator pitches on twitter from @boeffi (Who also runs German blog on software development) I asked myself (and him subsequently) a couple of questions regarding the setting and desired result to target the 140 character accordingly...
Unfortunately (for me) his answer was "All of them".
Please join the discussion at his site

Challenge accepted!

Given that I asked about four dimensions I think 16 "pitches" of less-than-140 characters will do for starters...

The collection of Scrum Elevator Pitches

Please read the disclaimer below - don't use them literally

• The physically realized earned value from delivering fully functional slices in scrum takes ROI from estimatable to measurable
  MSLM: Management, Sell, Long term, Money
• Proccesses like Scrum can help in the transformation of the whole organization towards lean by providing transparency throughout.
  MCLM: Management, Convince, Long term, Money
• By concentrating on the most valuable parts in shippable slices Scrum reduces time to market considerably - often to a few weeks.
  PSLM: Users, Sell, Long term, Money
• Instead of investing the subject matter expert's time in the test at the end Scrum involves him early to build a fitting product
  PCLM: User, Convince, Long term, Money
• One scrum project, budgeted for 10 Mio $ was ended after spending only 2 Mio $ – because it had delivered all the important features
  MSSM: Management, Sell, Short term, Money
• Scrum's focus on a "potetially shippable" increments ensures that the produced product-parts provide real value to the users
  MCSM: Management, Convince, Short term, Money
• The proven prioritization models of Scrum enable you to get high value from the development team early for your time investment
  PSSM: User, Sell, Short term, Money
• The continuous effort you put into specifications can be turned into continuously valuable and tangible results by Scrum
  PCSM: User, Convince, Short term, Money
• Scrum can give organizations the information they need for a fundamental reorientation - or help to make them ready for it
  MSLB: Management, Sell, Long-Term, Buy-in
• Having crunch times and lulls can be countered by adopting the basic Scrum rules - they create a self balancing, sustainable system
  MCLB: Management, Convince, Long-Term, Buy-in
• Getting Scrum to both sides - developers and users - gives users a real strong lever over the scope of the final delivery
  PSLB: User, Sell, Long-Term, Buy-in
• If all the effort that goes into upfront specs went into a Scrum-like collaboration the results would be closer aligned to the needs
  PCLB: User, Convince, Long-Term, Buy-in
• Scrum is a set of techniques that generates transparency and makes projects predictable and controllable again.
  MSSB: Management, Sell, Short term, Buy-in
• With scrum both parties of product development lay out rules that ensure - but also requires - mutual commitment and engagement
  MCSB: Management, Convince, Short term, Buy-in
• With Scrum you get the opportunity to set the direction of the development at least once a month.
  PSSB: User, Sell, Short term, Buy-in
• If you want to gain a real and direct influence over the priorities in the development team you might want to have a look at scrum.
  PCSB: User, Convince, Short term, Buy-in
  

Explanation of the Dimensions: Audience, Goal, Timeframe, Required Action
Audience: [Management=M, user representative or possible Product owner=P]
Goal: [Sell=S, Convince=C]
Timeframe: [Long term=L, Short term=S]
Required Action: [allocate Money or resources=M, Buy in=B]

Disclaimer

These are examples of pitches for generic situations - don't use them! Forge your own given your specific situation and requirements.
In my opinion there is no such thing as one right elevator pitch for anything - it's all about the context.

One Inbox or Inbox Zero?

Since when does "Inbox Zero" refer to email?

When I first heard of the idea of an empty inbox it did not refer to you email inbox - instead it referred to a very physical inbox and to the management of workflows. Merlin Mann did an excellent Job of tying the inbox metaphor in the context of inbox zero to the email mailbox with his article series.

In GTD (getting things done) the inbox is something quite different

From what I understood from the GTD book by David Allen the point of an GTD-Inbox is twofold. First to get things out of your mind quickly and secondly to have one place of reference for your tasks/todos. Once you have that you can apply the two-minute-rule and other things from the GTD toolkit.
So actually in this model the inbox is a logical container for all the stuff you have to do.

Using the email inbox as a todo list harms your work organization

If my inbox is tied to the email system my work is tied to my computer. Assuming I wanted to set up a workflow system that is more pull oriented and does not necessarily contain only virtual to dos I would have to somehow bend your email workflow and it would be hard to use a different tool for your workflow management. Although I have to admit that there was a time when I had email folders called "Pending", "In Progress", "Waiting" and "Done" - but they didn't have WIP-Limits and it was at a time before David Anderson had even begun to write the Software Kanban Book.

Goals of the Inbox

Having one place to collect all todos is far more important for me than having an empty "unsorted pile" - which basically is what the "inbox" really is.

"Email Inbox Zero" is quite random IMO

Why the heck are so many people possessed with getting their (email) inbox to zero? What about their Twitter Inboxes, their Skype Inboxes their paper inboxes, their... you get the picture. So instead of trying to achieve inbox Zero I just scan my emails whenever they arrive or I have the time to do it. If I think I have to act upon one of them I create a ToDo in a secondary system. If I need to reply, i create an empty response, save that (so that don't have to search for the original) and create a ToDo in a secondary system.
The todos in this secondary system are completely independent from my mail workflow, my inbox contains literally tens of thousands of emails and I don't have any problem with that.
And I don't spend hours of thinking about ways to find an effective way to abuse my email system.

Plain text is king? Pen and Paper?

So what magical secondary system do I use?
None at all.
Right now I work with a three tiered approach consisting of Pen and Paper for the first tier, and two documents in plain text that get synchronized between all my devices.
I guess everybody has to find their own solutions. But that is not the point. The point is: It is not you E-Mail inbox and copying, deleting or moving hundreds of emails per day might even eat up time that you could spend getting things done.

Cheers
Michael

(This post is v0.9 but the ship date has arrived...)

Pre-flight checks are possible on planes but not on Enterprise Applications - why?

Unit testing is all pretty well but what about delivering these tests into production?
Is it a good idea? Or is it "nice for toy applications, but you can't do that with Enterprise Applications, because there is to much money/security/personal data at stake"?
And what kind of tests are we talking about anyway?

Basically this post is meant to debunk some believes that I encountered during a couple of Enterprise Application projects. Lets look at them:

1. We write throw-away code for unit tests that that we wouldn't need if we didn't write unit tests


2. [The] tests are only usefull during development - they /must/ be removed from production code


3. We don't need to simulate the connected systems once we get the real systems connected


4. Throw-away code is effort that's useless and could/should be avoided

While these claims seem very convincing at first sight and might be very appealing to managers who try to get a development team to be "more productive" their physical world counterparts tell that story quite differently. And my own - albeit to some extent seemingly counterintuitive - observation is that these believes seem to be quite wrong.

Now - if you're still with me (or against me - as long as you care about the matter) - let's debunk them one at a time

(Unit-) Tests require otherwise useless code - I don't think so!

Of course you need to write some things to automate the tests that would not be necessary if you did something else instead. Like testing the application manually. But that doesn't mean that you win anything by not writing "those things" in the first place. You'd just replace a checkable, repeatable way to test your program by one that’s dependent on the alertness of the tester (in fact: your alertness!), her or his mood, interruptions etc.

And let’s look at "those things" that you have to write. For me, three things come to mind - the tests themselves, some kind of test harness and some special diagnostic extension point where it is possible for the tests to get access (read and write) to otherwise unexposed information. I'll come back to the exposition of otherwise unaccessible information in the next section but the other two can be dealt with right away.

The tests themselves? Well, of course I could just test the System manually. But even if I was willing to re-test all possibly affected portions of my system each time I change my code - which I am not - it soon would become economically unsound. Apart from the sheer amount of time spent doing that, for any system that has reached a substantial size I would have to keep records of the test so that I can repeat them easily and know which parts have been retested since the last change and which still need some love. Just writing the tests is way less effort than the combined effort of writing specs, keeping track of the test I just did manually and the manual testing itself.

The test harness? I think this is addressed by the same consideration that hold true for the tests themselves. Plus the fact, that there often is no way to test without a test harness since the system under development relies heavily on surrounding components.

Test should be run only outside live systems - I don't think so!

So let's come back to those extra accessors we wrote for our test that are "only there for the tests". Well sometime they /are/ only there for the tests. Unfortunately. They could be so much more. Let's look at the real - sorry, physical - world for some comparison:

Airplanes and automobiles.
Have you ever wondered what all those dials on a typical airplane dashboard of the last century indicate? Well a lot of them show things that /should not be necessary to measure/ for example the exhaust gas temperature (egt). This also is a function of the manifold pressure and the fuel intake - both of which are displayed by other gauges. But measuring each of the components separately - which really is redundant if everything works as designed - gives the pilot the option to manually lean or enrich the fuel for optimal performance. And it serves as an indicator of possible related problems.

I still have to see an Enterprise Application where there is e.g. a counter for request, a counter for inserts and a counter for updates, that get monitored in production. Such counters usually are there - for the tests. And they are used - in the tests.
So why not take them to the production system?

Or take another element of pre flight checks (not applicable to jets), the mag-check. Usually airborne engines use a magneto based system for spark generation and two separate circuits with separate magnetos and two spark plugs per cylinder. The "normal operation" is to have both spark plugs running. But - and this is where Enterprise Applications differ hugely - the pilot can control this. Even in mid-flight. And it's part of the pre-flight check to test those unwanted conditions.

Like "Both" - Revs should be 1400, "Left" - Revs should drop (only one plug), "Both" - Revs should rise again, "Right" - … you get the picture.

But I've never seen something like that done to an Enterprise Application:
"Cache Available" => All pages should show in less than n milliseconds
"Cache not Available" => Pages x,y and z should show much slower
"Database available" => Data should be written to the database
"Database not available" => Data should be written to files

But wait - that's not possible.
Because we removed all the rods and knobs that enable direct access to those internals from the production code.

And more often than not it's not even possible to take the delivered application an debug it because the debug information - and even the symbol tables where applicable - have been removed. So it's necessary rebuild the production version including debug information from the source control and try to recreate the failing scenario.

If automobile engines where build like that you would have neither the yellow "general engine failure" light on the dashboard nor the option to go to the garage and have them plug in the diagnostics computer and tell you exactly that there are air bubbles in the fuel pipe every time the engine revs over 5521. (that's an example of course…)

Instead the garage would have to rebuild an engine exactly like the one you've got in your car and analyze the error from that one. Or at least completely disassemble your engine, insert probes, reassemble it and run it in an environment simulating your car.

Although there are numerous implementations where you can probe a running system - especially on Apples OS X - they are mostly aimed at system level functions or the core frameworks. Actually I think it's very sad that it is so uncommon for programs and especiall Enterprise Applications to reveal relevant information about their internals.

It's preferable to use The Real (connected) System for development - I don't think so!

Do you really think that, in the car supply industry, they fit each wiper motor in bodies of all targeted cars before they ship them? Last time I checked they used test rigs that have nothing in common with cars except for the interface to the wiper motor - but they might have some additional features like an instrument that measures the torque of the motor and another one that measures the current drain. And by mounting the wiper motor in that rig and measuring those values they can verify that it actually confirms to the contract to which both the car manufacturer and the supplier designed their part. (Which opens the road to Design by Contract - but that would be another story)

If you write code that get's deleted later than it's waste - I don't think so!

There are lots of examples from the ream world that tell a different story. The simple one of course is the paper cup you get with your coffee to go. It does get 'deleted' later on, when you throw it away, but I don't think it's feasible to do completely without it. At least I would either burn my fingers or at least ruin my shirt if I tried to take "only the coffee and nothing but the coffee" from my local takeaway. (Replacing the throw-away cup with a mug for multiple usage might be a good idea from an environmental point of view, but replacing it is not the same as "just don't do it”)

Another examples - that is perhaps more suitable from production point of view - is almost anything that has to do with casting metal. A wide range of casting techniques requires a cast that gets destroyed in the process. Those are made up of a multitude of materials and usually require serious work before the casting process can begin. And after the casting and cooling and hardening it often gets destroyed by brutal force - with hammers and pickaxes if needed. I really don't think you could build those metal things without the cast. So if I write code that for example simulates a calculation that will be build much later and gets thrown away once the more complex "real" function is implemented that - for me - is the mold that allows me to form the rest of my code around it.

References:
Theres a ton of related material out there, but "PPP" by Uncle Bob and the original edition of "Extreme Programming explained" by Kent Beck do a pretty good job of conveying the fundamental concepts. And throwing in some TDD by Kent Beck, BDD by Dan North plus Refactoring by Martin Fowler et. al. for good measure wouldn't do much harm either - but that's on my 2¢ and of course YMMV.

PPP = Agile Software Development, Principles, Patterns, and Practices by
Robert C. Martin
TDD = Test Driven Development: By Example by kent Beck
BDD = http://dannorth.net/introducing-bdd/
Refactoring = Refactoring: Improving the Design of Existing Code by Martin
Fowler, Kent Beck, John Brant and William Opdyke

Re: [Airbnb Community Support] Re: Airbnb.com iPhone Support Request

Dear Airbnb, (CC'd to the same Blog as before)

please do me the courtesy of at least reading what I write.

Which part of "I don't want you to help me track the information down" did I write in such a way that it was so easily misunderstood?

Now I am not only a customer who feels that he is not in Control of his data but also a customer who infers from your actions that his concerns are of no real importance to you.

Yours, Michael

On 28 Apr 2011, at 17:07, Cathy H <notifications-support@airbnb.zendesk.com> wrote:

-- Please respond above this line. --

Cathy H, Apr-28 08:07 (PDT): Michael, Our records indicate that your reservation with Hannes in Vienna has been accepted. As payment you supplied us with a Mastercard ending in nnnn and were charged: $nnn / €nnn EU This charge was pushed through manually a moment ago, because when you entered your payment information you forgot to select the country you live in. We've done that for you and now you should see the charge on your credit card. Your request with Eva in Vienna is no longer possible because she did not respond to your request within 24 hours. You were not charged for that. Please let us know if we can provide further assistance. Kind regards, Cathy H www.airbnb.com 855.4.AIRBNB (Toll Free) or 415.800.5959 Help Airbnb win a Webby Award, vote now! http://bit.ly/gpDDsW Michael Mahlberg, Apr-28 07:14 (PDT): Dear Airbnb, (CC'd to one of my blogs) In the meantime the registration has expired and /after that/ the payment information is visible on the website - for a brief moment until I confirm the information. Then it's inaccessible again! So, in the name of open Business-2-Customer relationships, here is what I would expect: I don't want you to help me track the Information down - I want control over my data! I want to be able to see /exactly/ which data you have on file for me. I want to be able to see, which transactions are pending right now! As much as I like the idea of your service, I don't feel comfortable if you play it so unreasonably close to the chest. Cheers Michael ( @MMahlberg on twitter ) Cathy H, Apr-28 06:57 (PDT): Hi Michael, We'd be happy to help you track this down. Can you please provide your reservation code? Best, Cathy H www.airbnb.com 855.4.AIRBNB (Toll Free) or 415.800.5959 Help Airbnb win a Webby Award, vote now! http://bit.ly/gpDDsW Michael Mahlberg, Apr-26 09:18 (PDT): Please enter the details of your support inquiry below: User Information Name: Michael M. ID: 533213 How can I find out, which of my credit cards I used to make a reservation? Situation in case: 6 credit cards, one booking with airbnb - no movements on any account. Host has not yet (21 hours) responded. Which c-card is on your files? which one can I max out? Cheers Michael User Information Name: Michael M. ID: 533213 (von unterwegs gesendet) Airbnb Community Support Phone: 1-855-4-AIRBNB | 415-800-5959 FAQ: www.airbnb.com/help

Why Google-Checkout sucks - at least for me

I just tried to buy the electronic version of David J. Anderson's Kanban book, because I'm traveling the world quite a bit right now, left the paper book at home and wanted to look up a few specifics for the design my own personal kanban system.

Two things delayed my decision to buy it - and actually will delay it for a couple of days to come. Which is unbelievable for a process that takes a mere minute on amazon and the pragmatic bookshelf.

1) I have to buy the eBook twice to read it in iBooks and on the Kindle [see below]
2) I have to become a Google-Checkout Customer (which is the dealbreaker)

The Problems with Google-Checkout

• They (Google Checkout) added my credit card info to the currently logged in google account
  Really? The currently logged in account? That's just silly.
• They (Google Checkout) stored the credit card information

Actually Amazon also stores the credit card data. But amazon is a store - Google is a search engine and an information broker. So I'm supposed to give my credit card information to an information broker, who adds new services to it's portfolio every day?
So, what I'll have to do now - to get the electronic version of a book I already own - is to get a prepaid credit card, put some money on it, connect that to google checkout, buy the book, disconnect the card from google-checkout.
[Aka: No, I don't want others to be able to control my credit card and identity management. That's where I really want a push system]
What a waste of time!

The Problem with buying the book twice

I use the Kindle for reading and the iPad for Browsing, and since the Kindle UI is only mediocre compared to iBooks I would love two have both formats available as it is possible with the pragmatic bookshelf. I admit that the Kindle UI is not as bad on the iPad as it is on the Kindle itself, but still it's a drawback compared to the iBook experience. So the current option for buying the book online isn't "quite" as pleasant as it would be to buy it from the pragmatic bookshelf - or perhaps I'm just spoiled by the latter's focus on customer value.