On Measuring Agility, Craftsmanship, and Everything Else

There is some heated debate in the wind at the moment on the idea of an Agile Maturity Model (AMM). See here for more on the subject, or go straight to the heart of the matter by reading through this slide deck from Ross Pettit at ThoughtWorks.

I have taken the time to study what Mr. Pettit proposes and to read some other opinions around the sphere. What I have found doesn?t jive for me all the way around.

To put it mildly, there are differing opinions on the ability to measure Agility with a maturity model. The most common rebuttal (rooted in Lean Thinking)  is we should ?measure up? and simply evaluate Return on Investment (ROI).

Measuring Organizational Success

Human nature and then the stock market figured this out eons ago. How do we reliably measure an organization?s success in a free market? Profitability. Essentially, ROI.

Simple measurements like ROI look great on paper and are notoriously elusive to use in improvement because of the vast number of mitigating factors involved in deriving the final number. But it sure is tempting, so sayeth the Lean Thinkers.

In order to demonstrate ROI of a team, we simply look at cost and revenue directly attributed to that team.

Easy, right?

Not so fast, Gomer. What about the fact that what really happened during the year in question is that an aggressive sales team enables a boat load of sales of an essentially crappy product?

We all know that happens all the time. Billions have been made on spaghetti code while at least as much is lost on elegant solutions. So how do we define success?

Lean offers many other ways to view an organization?s effectiveness. Value stream, flow, ROI, Kaizen, and pull systems can genuinely eek every miniscule efficiency in your company. Indeed, in my opinion taking a holistic Lean approach in your organization holds the highest promise of making things better. But is this measuring Agile?

I?ll go with ?No.?

Measuring Agility

Every since the first teams called themselves ?Agile? and delivered in increments, business analysts  have been relentlessly following these teams around with slide rules and calipers measuring their excrement. Some of the first sessions I attended years ago at Agile conferences looked at varying models for measuring Agility and effectiveness of teams.

Q: Why so fascinated with measurement?
A: Because it is necessary to being taken seriously.

Many attempts have been made (many successfully) to co-opt Agility to mean something beneficial to the person using the term. In my mind the only true definition of Agility comes down to what we find in the original manifesto.

If we are to truly measure Agility, one must therefore be able to quantitatively measure the degree to which an organization holds true the Agile principles. To some extent, that is what Mr. Pettit?s AMM endeavors to do. After thinking about it, I must conclude that this is a fruitless and nearly hopeless task.

On a scale of 1-5 how much does your organization value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

Well, that?s gonna be a tough nut to crack, isn?t it?

Measuring Craftsmanship

There is no question that the Agile movement has been pivotal in the evolution of software development. Both in methodologies (process) and in engineering (practices) we have evolved the software development field more in the last 10 years than the previous 30.

The most obvious effect for coders on the ground is in the practices side and includes subjects like Test First Development, emergent design, automation, Continuous Integration, and other practices that are inarguably good for teams to practice.

Thanks to some vocal proponents and notable cantankerous grouches (thanks, Uncle Bob) we are now starting to talk about these things as Craftsmanship.

So now the question becomes, can we measure craftsmanship? To this I resoundingly reply, ?Yes!?

In fact, this has been done for thousands of years. We can point to any trade and see the evolution of the craft and the teaching of modern practices in it. An 18th century blacksmith?s apprentice had specific skills to master to qualify as a practitioner. Mastery of those skills is inherently measurable because they are skills, not art.

The guild model for craftsmanship just works here. Here is a fictitious scoring sheet for a blacksmith?s apprentice.

 

Skill	Score (1-5)
Ability to gauge furnace temperature	3
Proper use of all common tools	2
Shaping thin metal	5
Creating and using metal casts to form simple objects	4
And more?	?

Is it really such a stretch to construct similar evaluation areas for software development? I think not. In fact, many are in existence today in the form of thoughtful job descriptions.

Although the skills desired and measured will undoubtedly be guild specific, there are fundamentals that shouldn?t be too hard to agree on.

 

Skill	Score (1-5)
Effectively apply Test First Development tools	3
Appropriately identify, construct, and apply fundamental design patterns	2
Demonstrates ability to compose a cohesive object model for a simple application domain	5
Appropriately creates, disposes, and manages memory	4
And more?	?

Pick a Guild

The guild spats are (and should be) over the specific skills deemed important. Fine, let?s let the discussion spiral to that, but can we agree on some basics?

This is why I actually like the Application Lifecycle Management model proposed by Microsoft. I wrote about it here (and here), and have since used it to actually work with teams to better their skills. Sure, it is guild specific, but so will be many of the instruments to come.

The important thing is to use one, and to use it deliberately with a genuine Kaizen approach.

1. Pick a skill
2. Assess your proficiency
3. Make a decision to modify your behavior in a deliberate way
4. Perform this new way
5. Evaluate your skill level
6. Repeat

Simply put, we need these things as a way to make deliberate and lasting change in our skills, and to hold each other accountable for that advancement. I?m just saying.

Like It or Not, Tools Play a Role

Have you ever watched ?The New Yankee Workshop?? If so, you?ll no doubt recognize that Norm Abram is a master craftsman. Notice anything else? The man has a different tool for every minor job in the workshop. Further, he has more than one! And he has mastered the use of each one.

Norm can obviously execute a tenon joint with perfection. Once those pieces of wood are joined, let no man break them apart! And watching the show indicates Norm has a preferred way to execute the joint.

He uses a $3000 table saw.

That doesn?t mean he doesn?t know the principles behind a solid tenon. In fact, he has probably carved many of them with chisels in the past. That doesn?t make his tenons executed on the table saw any weaker, though.

Given a choice, Norm chooses the tool he enjoys and the one that makes quick work of the job. He doesn?t need to carve the thing with a pocket knife every time.

Nor does a software guild member need to limit their mastery of a principle to always rolling their own. Indeed, tool mastery is significant part of the program.

Will the budding apprentice need to learn Make, FinalBuilder, MSBuild, NAnt, Ant, and Maven in order to demonstrate mastery of build automation principles? No.

But they will need to use something, just like Norm. And the tool they use will likely be a function of the guild. It will probably be the tool the shop steward already has laying in the corner. It is important to note that one need not master all the tools available for a given job in order to execute the job well. One must invest in a specific tool set on the road to mastery.

We?ll be comparing test frameworks and IDEs for years to come just as Norm surely argues with his cronies about the best chisel set. Just recognize the debate for what it is and move on.

Conclusion

We have the tools we need to move forward in front of us already. An Agile Maturity Model doesn?t resound with me as a necessary thing.

Using instruments of Lean as the rubrics for organizational improvement is sound. Further, there is more to come. We are really only getting started with Lean.

Using guild-specific instruments like Microsoft?s ALM assessment as a rubric for personal and team performance holds intrinsic value. Want different skills? Try a different guild.