Recently, I read a post that described purposly doing things in a way that wouldn't scale, thus hoping:

  • to provide better insight into the problem domain
  • to reduce investment until confidence of the return is known.
    In particular, Joel suggested manually executing processes that could be automated, at least at the beginning of a project. This could then be automated if needed later on.

During my current early activity into developing my first (hopefully successful) iPhone application, I came across a couple of processes that my earlier insight into Agile / Lean development would suggest to automate. However, following Joel's advice, I've decided to execute them manually, and here's why:

Don't Test Drive Everything

Wow, I thought this would be quite controversial, but there's been quite a lot of discussion about not using TDD recently. When I first learnt about TDD and 'saw the light', I thought all code must be Test Driven, if only for the regression capabilities.

However, now exploring more and becoming a little more wise, I use TDD slightly less. Specifically, I always use TDD when

  • The code to be implemented is complex, and thus risky. The benefits of getting it right, through testing is essential, and the ability to refactor in confidence pays back the cost of the Unit Tests.
  • I don't know what the code to be implemented will look like. Test Driving will drive out design based on baby step learning of the problem domain. TDD enables you to understand subsets of the problem initially and learn step by step rather than trying to load the entire domain into your head in one (impossible) go.
    There's one other reason why I will always Test Drive - when I am developing a component where it's consumer doesn't exist. I was going to exclude this as it's a sign that I'm not developing Outside > In. But I do sometimes develop Inside > Out. Why? Because in some teams, one colleague maybe developing the UI at his/her own pace whilst I can get on with the Service Tier. I will generally only adopt this collaboration after substantial white-boarding on how the UI will work, and what dependencies on the service it will have. It also helps that most apps I work on are fairly CRUD based.

However, there are several times when I don't Test Drive even on production code.

  • When the cost of learning Unit Testing frameworks for a particular user interface technology will cost significantly more than manually testing the app, even taking account appropriate regression over the probable lifetime of the application.
  • When I'm not sure exactly what the application should do. I know, I know, this should be addressed through spiking rather than developing 'production code' but again, sometimes you have to be pragmatic. Learn through developing, iterate and iterate until you understand how the UI should work. Then, based on risk, I will decide whether to retrospectively apply Unit / Functional Tests. Bear in mind, that in this situation, whilst I might not Unit Test the full app Outside > In, some of the more 'known' components might be Test Driven.

Don't automate complicated processes before understanding them through manual execution

I don't want to go too deeply into this one as I don't want to discuss my iPhone app in detail but there's a very important validation process that must be implemented to make the app successful. Without this process, the app will have little value. However, I'm not exactly sure how this validation should work. There are just too many variables to taken into account, that to develop the process in code would probably take maybe 2 man months to code.

I could of course spend 2 man months coding it, but I would no doubt will get it wrong - several times. Therefore, I'm going to follow Joel's advise about not automating payment gateways, or email notifications. I've decided to manually execute this validation. What does this mean?

Well, firstly it means that I will still have to develop a little code to help me manually validate the data. This work will only take about a man week to develop however, and will help  me learn about other areas of the system so isn't pure cost.

Mostly importantly however, it will mean that I will learn the data and problem domain quite well before trying to commit to writing code to model it. Infact, I may never do so even if the app does take off. I might crowd source the validation, or delegate to moderators etc. Or maybe discover the validation isn't actually required!

Again, the lesson is this - If I spent time developing automated  validation, the code would almost certainly be wrong, and could prevent the application even being completed.


comments powered by Disqus