Specification by example – the missing link?

I’ve been thinking. That statement alone will be sure to put fear in the heart of a lot of you… But if you have continued on this far, here we go.

Learning programming stuff

During the last year or so I have been reading a lot. I have read stuff on XP, on good design DDD and TDD. This reading has affected me and my coding style way much more than I first thought. I simply cannot write code anymore without the test first, interface first, thinking of SOLID etc..

Learning lean stuff

At the same time I have change role at Avega. I am now an AvegaCoach. This means that my time is divided between my regular (often coaching) assignments with customers and Avega and Elevate. Since my fellow AvegaCoaches (Joakim and Christophe) and me are interested in agile and lean stuff we have focused our joint efforts on that.

This has led that I have learned a lot from them about lean and Kanban. And this has expanded my understanding of Scrum and how to work agile.

Frustrations

But around here the frustration started to kick in. I felt that these two areas; technology and process stuff could marry in a great way. But how? And how much does the it affect each other?

Also, a specific thing that I have been really frustrated in is how to include test in an agile world. Testing are left out in planning, are left to try to keep up with development and the regression stuff will eventually kill them. So “testing sprints” are introduced, “we’re running a sprint behind the rest of the team” and other solutions like that is used to try to get around it.

Surely – this cannot be the best case? So I started read about agile testing. I loved the book by Gjoko Adzic. This book introduced me to the term agile acceptance testing and specification by example.

The solution?

I have heard and learned quite a lot about BDD. I knew that BDD and “outside-in” thinking can help me design my application in a maintainable way. But BDD can be applied at any level. It could be used to spec out your unit test. Another way to do TDD (AAA becomes GWT)

But I had missed the fact that the specifications and features should could be written in collaboration with customer or business analysists. This will turn the specs into executable specifications. (Those words still gives me the shivers. It’s so cool!)

That’s the thing! With executable specifications you have a living specification to code against, and the moment you’re done (as in done done) with the story the specification is “magically” turned into an acceptance test.

And the specifications finally reflects what the code is doing. I can never forget a quote by a customer how very proudly showed me the folder with all the specifications (use cases in this case) and then told me:

“And the best thing… they are almost up to date with what the system does”.

The really scary part about that is that it was actually the best I ever seen. Most documentation and specifications are a violation to the DRY principle. But with executable specifications you’re as close as it gets.

But it doesn’t stop there: since the specification with any current tool is written in plain English you can workshop around it to get everybody's (business analysis, testing, coding, deploy etc) views on the matter. And you minimize the misconceptions by using real world examples.

And that in turn closes the gap for me; agile testing is hard since I am still viewing testing as something separate from coding. And the same goes for specifying… But it’s not! You can do this in agile way, a little slice of functionality at the time. A way to do that is specification by example.

Recommended watching and reading

Read Gojkos book – Bridging the communication gap. Stop reading whatever you reading now and start reading this. It was a eye-opener for me.

Watch (anything you can get your hands on) this webcast by Dan North. There are lots of great stuff in here on BDD and design.

I’m reading Agile Testing right now. That has really showed me that testing is something that any agile person (developer, tester, business analysist etc.) should care about. Or as Deming put it:

“Quality is everyone's responsibility.”

The value of an ubiquitous language

A few days back we had a mail-wise discussion on the subject; why should we care about an ubiquitous language?

For me the question falls into two parts; for the whole company/business or for the application.

• An ubiquitous language or common domain model for a complete business have never felt right with me. I know, I know; I have been defending it, striving for it and even forced it on some customers – but I don’t like it!
  It will never work! The same entity, (customer) for example, can have very different meanings in different context.
  A better and much more detailed discussion can be found here:
  • http://devlicio.us/blogs/casey/archive/2009/05/14/commercial-suicide-integration-at-the-database-level.aspx
  • And in Dan North’s excellent SOA for the rest of us
• An ubiquitous language for an application is a completely different thing and is something that I think should be strived for. But that basically means that all the members of the team (customer, analysts, developers, code, tester, documentation) referrers to the same things with the same name.
  For me it has to do with speed and accuracy in communication within the team.

But don’t take it for me. I did a quick search through the net and found some articles that gives better points on why an ubiquitous language is a Good Thing:

• It starts and ends with Eric Evans book: Domain Driven Design. With Google Books you can read the relevant chapter here.
• James Shores take on the subject, with a Haiku even…
• Kent Becks words
• A more formal definition and description, from www.domaindrivendesign.org
• A long but worthwhile read on the subject. Search on Eric Evans to go to good parts. I told you that it starts and ends with him… ;)

Oh yeah – thanks to Oscar and Jocke for the discussion.

ÖreDev day #4 – morning

We’re of to a brand new day. Feel well rested although last night was quite late…

Keynote: What drives design?

This can be very interesting, if focus on the last two D in any [x]DD-technique (TDD, BDD etc). I’ll make a summary in the end.

This was a interesting historical overview to start with. Pretty cool that our industry is so young that the people who “started it all” are still not that old. Cool to hear about their troubles and stumbling on their way to greatness.

Rebecca Wirfs-Brock talked a lot about the RDD (responsibility driven design) and the patterns behind it. Then said compared it to other DD-techniques, such TDD, BDD, FDD and DDD and so forth.

Making the sausage

Now this should be interesting if by no other reason because of the speakers; Dan North, Neal Ford, Stuart Halloway and Tyler Jennings. They are going to talk about BDD in something called Clojure. It promise to be great!

It was a bit disappointing in a strange way; it’s was great but I had a hard time keeping up with their ideas. But after a while I get it but not well enough to re-loop it here. Sorry.

Functional languages are strange in itself and four functional programmers running through code they wrote a late night…  You can probably understand my confusion.

Test-Driven Web UI Development

Scott Bellware is doing this. It’s the first time I’ll hear him, except in the local pub…

The talk was about lessons learned on trying to bridge the gap between testers and (TDD-)developers in an agile team.

Here are some random stuff that I picked up:

• Selenium looks like a cool UI-automation testing framework. It free!
• Selenium can actually generate test-code in a programming language of your choice. Beware that the tests are just a starting point – not the full blown test suite to use for ever and ever…
• As programmers we shouldn’t do things that decreases productivity for the testers. It’s the whole teams productivity that counts.
• The tests should describe the product not the implementation.
• Good BDD-tests should be like a Table of Contents in a book. You don’t want to go into details if not needed.
• Pair programming with a tester and a developer may be a way to find issues really quick. But I think a good, solid understanding of the test framework is needed.

I learned some about of Ruby and some on automating tests. I liked it but we didn’t reach all the way, sadly. James Bach had some comments that I would have loved to heard more around.

DDD and Naked Objects

I’ve just learned about this and don’t want to loose the thought and links surrounding it.

The first encounter I had with DDD was when Jimmy Nilsson did a presentation at Elevate. One of the things he said did a profound impression on me, especially the part when he talked about the database being a consequence of my domain model, not the other way around.

OK – this got me thinking. Wouldn’t it be nice if it also was that way with the GUI? The GUI being a consequence or reflecting what my domain model captures?

Here is, to my very narrow knowledge, the solution – DDD with Naked Objects. Naked objects is an architectural pattern that adds a principle of the GUI being automatically generated (or generated on the fly I presume) from the model.

OK – but that cannot be used in production, of course – I hear some people way in the back saying. No but – to quote a late post the Swedish DDD-list on Google (my translation):

A story is considered done when there is an implementation of the Domain Model and a basic GUI, in which you can do anything, but only in one way and without any major afterthought. The users can then test the story with this GUI and think about how they really want to work.

Pretty cool, eh? Thank you Tomas.

There are some frameworks to help you (be sure to check the videos out, especially this one and the follow on), some books to read and whole lot more to learn about this.

But I like the idea so far.

AutoMapping with FluentNHibernate

I ran across this post by Ayende and it pretty much sums up where I want to reach with my persistence ignorance efforts:

“After that, you are done. Just create an entity in the proper place, hit the /database/create and have a lot of fun.”

I of course like the fun-part of the quote the most. :) But seriously – that what I want to reach – to configure my conventions. And then simply code the model as I want it and let the framework (NHibernate in this case) figure out how to store it.

Well, as it seems, the Fluent NHibernate framework has been updated since the Ayende post. So I’ve read, and read, and read and discovered some shortcuts. By the way – here is an article on how to make the transition from the “old” convention-style into the new.

I have put together an updated sample for what Ayende did in that post above. I also added a feature to be able to automatically set cascades for OneToMany- and ManyToMany-relationships.

Here is the most important parts of the code.

[UPDATED]
Here is the complete sample.

Did I mention that I LOVE Fluent NHibernate? Well I do!

Implementing Unit Of Work with NHibernate

The more I work with it the more stupid I feel around what I have been doing before; why have I written so much SQL? It feel just … unnecessary. Who said stupid? Not me…

Jimmy Nilsson said something like; “I just want the database and storage problems to be a consequence of my domain model. I don’t want to think about it”.  And that is one of the goals with DDD and the solution is, for example, NHibernate.

I have been doing some labs with NHibernate the last couple. And as often with me – I think I am in love. NHibernate really rocks. The mapping files are quite hard to chew off at first and I suspect that there are many tricks and traps for me to find. But still – so beautiful. 

Of course the Net has helped me a lot on my quest. Here are some articles that I’ve used:

• The NHibernate API Reference
• A great first-time tutorial that helped me in the early stages
• Here is a great couple of articles describing the unit of work pattern implemented with NHibernate (part 1, 2, 3)
• More advanced mapping scenarios
• Description on how to map many-to-many entities, without having to specify a connection entity that matches the connection table.

A bit on the sad side (or NOT) is that we were trying to create a Unit Of Work implementation for NHibernate. Not only has this been done before, but also apparently NHibernate hooks into the System.TransactionScope implementation of the .NET framework.  Thanks a lot to the gurus of level 3 – Magnus and Calle who show me that. All my code – down the drain… :)

OK – this has been a great learning experience. If you want the code – drop me a line and you’ll have it in a jiffy.

Parental leave reading

I am slowly but certainly coming to an end of my parental leave. Six months without working! They have disappeared so quickly… It’s a bit sad.

But I also feel “hungry” for work. Especially to put my newly acquired skills and knowledge into use (TDD, ASP.NET MVC, BDD, DDD etc.)

I have done some reading during my leave, (I am actually proud of reaching a lot of the goals I set up for myself on the last day of working) and I thought that I could give a short review of the books here.

Clean Code by Robert “Uncle Bob” Martin was really good in one sense and quite annoying in another. Don’t get me wrong – this is a classic and a very good read. There are loads and loads of good tips and trick on how to find bad code ("Smells” as Uncle Bob calls them) and what to do about it. There are also some really good examples.

But in the end all the tips stack up to an unreachable amount. I don’t know if that says more about my code or the book but I felt that it would be impossible to grasp all those rules at once. Maybe use the book to check back to from time to time.

Especially when you decide in your team what clean code means to that team. That’s what really important, I think, that the team has consensus concerning the style and rules that applies.

Patterns of Enterprise Application Architecture by Martin Fowler. Now this is a great book! I’ve learned a lot from this book and also learned the name and true form of some of the patterns that I have been using without knowing.

Ok – the book is in two parts; the narratives and the patterns. The narratives speaks in more general terms about architecture and when the patterns applies, while the patterns part is a more detailed look on each of the patterns mentioned.

The book is quite old in IT-time (from 2002) and some of the patterns mentioned has now been solved with frameworks and products (System.Transaction, NHibernate etc.)

I like the style of Mr. Fowlers writing and the way you are guided through the text.

I said to someone that this book came along perfect for me. I actually don’t think that I would have understood it just a few years back. You’ll have to reach a certain understanding to be able to understand the next step I think.

I could easily recommend both these books to every IT-professional that want to progress in knowing their business, in the large (Fowler) and in the small (Martin).

DDD – what’s the deal?

After doing DDD (on a very basic level) for sometime now, I now realize that I could benefit with some sort of sum up. Often when I reach this point I quickly realize that many people has tread this paths before and that I’m reinventing the wheel…
Well, in that case I do it for me. I like my wheels or at least my way of understanding them…

So here is my take on what the deal with DDD is (i hope that it will change over time but here is how I see it today):

I think first that the whole idea of DDD can be divided into two aspects (actually Jocke pointed me in this way but i like it):

• first code - a bunch of design patterns that help you produce great object-oriented, testable software
• then the implications on the way you can do software projects.

For the code part there is not much to add, or rather not much that could fit here. But what I mean is that to do DDD is “simply” to apply the patterns Domain Model, repository, aggregate, services, factories and more.

It can be quite a challenge to grasp all these patterns and put them into use at once, so I recommend a some reading, that will set your mind into DDD-mode and most of the patterns will fall into place by themselves. Or you can grab hold of an example architecture and build from that.

The other part of the story is the way that setting your brain into DDD-mode will change the way you can be agile in your project. The main idea (as I founded out, and here) is of course to focus on the model and let implementation details such as database integration, technology or even GUI be of secondary importance. Of course you cannot take in the complete universe of your problem at once so doing a small part of it and letting that evolve with time is a recommended approach. See this from Jimmy Nilsson.

This approach lend itself very well to do in the form of for example user stories and TDD. In this way you only need to focus on the part of the model that the user story is about. The next user story might tell us more about the same part of the domain model, but we don’t care now. The tests will assure us that we’re not breaking anything that already works as we progress.

So by doing DDD you automatically get a step-by-step evolving architecture and also a very good way to use an agile approach in the project.

With BDD (a subject I know very little about) you can even go further and get specifications that you verify with tests. But that is out of knowledge.

Sprint Planner Helper – final

I have had some real problems to get going with the Sprint Planner Helper Project after the trip to Ethiopia. I am not sure why – but it made me think about the project, the scope and purpose.

First and foremost – I am not working, i am on parental leave. That means this is my spare time project and I promised myself that I would end it if I didn’t feel up to it for some reason. Since I have found it hard to restart I conclude that I have reached that point.

Secondly – the main reason and purpose of the project was to learn, not to produce a wonder-application (although that would have been nice :)). I think that I have learned quite a lot during this project and that I am at a junction right now. You cannot pursue at depth in all directions.

OK – what was my learning goals and where have they taken me?

DDD

The main purpose was to look into Domain Driven Design. I feel that I have got a nice grasp around that subject – and I most certainly want to learn more. I’ll write a separate post about my thoughts on DDD and it’s implications on your project.

[UPDATED]

Here is my post on DDD

TDD

When I started the Sprint Planner Helper project I liked test. Now they are a part of the way I code. Over and over again you are struck by the fact that doing test first actually improves your code quality and helps you find flaws in your code early. I like that. Might be a post on these findings also.

ASP.NET MVC

Finally a web framework from Microsoft that I understand! I’ve been doing WebForms on and off during several years and still don’t get it. Actually I don’t think many do (how many developers does fully understand the implication of this…)

After doing ASP.NET MVC during this project I can honestly say that I will surely continue and never look back. Also the community is exploding with examples, code and projects that will assist you in creating great code. And of course the introduction Nerd Dinner chapter.

I am ridiculous proud of being the number one hit on this search. Before the actual book it self!!!

Extreme OOP

Eeeh – you can’t win them all, now can you? Actually this was the one thing that I pretty soon abandoned. Not that I don’t like the rules and the way they improve your code quality but it was one thing to much to think about.

I think that if you want to learn this you’ll need to do a separate project that solely focus on extreme OOP. And that would be quite fun.

Actually, in the PDF i am linking to above, you find an exercise that’ll get you started.

Where to go now

Some other reasons for me to stop is that I have found some great resources that is much more ambitious than my feeble tries. Here are a short list of them. Be sure to check them out, I know I will:

• Sharp-architecture – an base architecture that let’s you build web applications with ASP.NET MVC and NHibernate. Here is an introduction video.
• ASP.NET MVC Contrib – this is a project on CodePlex that collects “nice to have” features for ASP.NET MVC, such as grid helper methods, validators, routing engines and view engines etc.
• InfoQs book on Domain Driven Design

So I hope that anyone who has read this has enjoyed it for as long as it lasted. I know that I have learned a lot and that doing this project has made me a better developer.

The code will still be available at http://sprintplannerhelper.codeplex.com/, if not contact me to get it. I might even re-start the project later.

Sprint Planner Helper – Session 17

When you learning something new you’ll expect some progress and the occasional setback. If you are on your own, as I am in this project, that is even more likely to be the other way around.

My approach to learning this is very much like the way we played with the chemistry box when we were small – try something you believe will work and expect the whole thing to blow up, possibly with a stinging sensation to your eye…

So… I guess that you see where this is going. I have to redo something. Actually it’s two. And actually I am not that worried; the whole idea of agile TDD, DDD and all this is that it should be easy to change.

The first thing is not biggie; my Entities has a Guid as ID right now. It’s mega-ugly when I am passing it around on the querystring etc. I’ll change that into an integer. The nice thing about Guid’s is that they are guaranteed to be unique. But not very human friendly…

The second thing is a more delicate matter in DDD. It has to do with aggregates and something called aggregate boundaries (which I understand as: “where do I stop, the whole database is knitted together…”).  This post, by Casey Charlton, explains the background in a nice, compact way. Actually the whole site rocks, with a the relevant posts as a book even.

For me Jimmy Nilsson summed it up in a nice way:

The parts of an aggregate have no reason to exists without their parent.

In my case I have the Product-class that contains a ProductBacklog which is a list of ProductBacklogItems. The ProductBacklogItems have no reason to exists without being attached to the Product. So the Product is my aggregate root.

And as far as I understand (here comes the chemistry box part :)) it should be no ProductBacklogItem-repository. Add how to add a ProductBacklogItem to the ProductBacklog should be an operation on the ProductBacklog.

When the whole Product (aggregate root) is saved the repository will have to save all parts of the aggregate. Which can be quite tricky in the update-case I presume, if the aggregate is big…

Seems like I will be learning how tricky in a while. On to the code!

[Some time passes by while Marcus is coding…]

Eeeh – that took much longer than anticipated and I will stretch this session over two sessions. The main reason for that is that was forced to invent some kind of auto-increment functionality for my fake repository. It was quite simple when I had it figured out. I simply rewrote the add method into this:

public void Add(T entity)

        {

            if (entity.ID == BaseEntity.NEWID)

            {

                // get the new highest id

                entity.ID = FindAll().Max(x => x.ID) + 1;

            }

 

            dictionary.Add(entity.ID, entity);

        }

But I still have the Update-functionality for my Product aggregate to figure out. More simple tests, please. 

[Again some typing, head scratching and thinking takes place…]

Ok – that was much easier than I thought – which of course have to do with that I have a in-memory implementation of the repository right now… So the very simple Update-method:

public void Update(T entity)

        {

            dictionary[entity.ID] = entity;

        }

… took care of it for me. It will probably be a bit more advanced when I have to map this to a relational database.

OK – but now I can create new product backlog items.

Great progress today – I’ve learned a lot about DDD today.

Sprint Planner Helper – Session 16

I am now almost ready with the first user story of my own backlog.

1. As user I can create a product that describes the product and the product owner so that the purpose of the product is known.

Each product has a product backlog that describes what is left to do.

Each product backlog item requires at least an Number(1.2.1), description, initial priority.

Other properties are story points (how big is this compared to other) and a document with additional business rules.

The one thing that is left is to be able to create new backlog items.

My GUI for editing a product right now looks like this:

and as I said before I will not put to much effort into the design of the GUI, but I have noticed that the MVC-pattern opens up for GUI-design decisions to be change later on. I mean – there is a product backlog item create action that can be outputted as a pop-dialog or on a own page or via AJAX/Flash or …

That is a really nice feature.

Come to think of it – this is one of the main thing of DDD. You are given the freedom to choose (and change) how to solve when the time is right. This goes for all the ends of the application design; the database and the GUI for example.
The important stuff (ie. the model) is what you need to worry about.

OK – on to the code! Lets write some simple test…

I brushed on some extension methods today. They are cool but a bit messy since they don’t belong in the class they are extending… I am not sure if I like them or not.

I came as far as I could create the form for creating new product backlog items. Tomorrow I will save them.

PS. Passed 100 unit tests today. Yeah!

Why DDD rocks – the marcusoft.net version

Today I had very interesting conversation with a colleague and friend. He is very accomplished in matters concerning Application Lifecycle Management (ALM). And the more we talked about that and the things I am learning with the Sprint Planner Helper (TDD, DDD and ASP.NET MVC) I got sure of that I am on the right track. It’s the way to go – especially if you want to be agile.

Just take my experience with this project. I have now coded for some time (ca 15 hours) without having to lock down to a database-design and data access strategy. Not even near – I am working out the model. I am even planning to do the database-stuff at the very end.

You could choose to view the data access strategy and database as a “necessary evil”. I know I will need one to persist my data – but I will do as little with it. Hopefully generate it or leave it to people better suited to fine tune a database.

The same goes with the UI – I am by no means a GUI-designer. So I know I will need a good UI, but I’ll leave it as it is right now (with as little work as possible for me) and maybe let someone else look at it later.

So what am I doing, you might ask… I am doing the real stuff. The Domain and it’s problems. Trying to keep it in a state that will hold for future changes.

Get on the DDD-train – it’s good for ya! And your code…

DDD – the coin drops for Marcus

I have now tried to grasp the DDD-concept without fully “getting it”. I must confess that I have thought: “What is the deal with this… I have done it before”.

But today we had a most prominent guest at Elevate (the education initiative at Avega) – Jimmy Nilsson.  And what he said got the coin to drop for me.

The heading of his presentation was something like “A new era for data access”. And the things he said that got me to understand was something like this – not a quote but rather his reasoning:

Traditionally we focus on the database design in application development. So very early we want to get the database design in place. “We need it to start to do code”. Since we did that design early in the process we often got it wrong and inherited those error through the process. And we are reluctant to change the database later on because its hard to test and overview what the change may break.

In DDD we do the opposite – we start with the domain model, as far as we know. We prepare for change – we want it to change when we get more knowledge about it. We will not even use a database for quite some time in the project, when we have got to “know” the model through test and code.

Ok – that was my interpretation of Jimmy’s reasoning. But I got it – when starting with the pure domain model you don’t need to worry how it’s stored.
And also I understand why I liked the DDD-concept from the start. It goes perfectly with SCRUM – you don’t strive to do it correct from the start but rather expect and prepare for change. I like it!

I also learned some other stuff that I will need to incorporate into my design:

• Aggregate – use aggregate to logically divide and group your model. An aggregate root is the only way into the other objects in the aggregate. They cannot be accessed from the outside.
• Start modeling with what you know – I have started out with the first user story and find out what I can about the model in that story, preparing for changing it down the road.
• Behaviors – I have not put the behaviors on my entities yet. Need to do that.
• TDD is the way to go – with DDD and TDD together you get to “feel” the model in a more direct way than to write about it.

So that was quite a lot of information to me and to the Sprint Planner Project. No code today – but quite a lot of nice input.

Here’s a last one: since I only code one hour a day I often find it hard to remember what to start with. Leave a RED test – was the very simple and resolute tip from Jimmy.

Thank you Jimmy – for everything I take with me from your presentation.