There is some heated debate in the wind at the moment on the idea of an Agile Maturity Model (AMM). See here for more on the subject, or go straight to the heart of the matter by reading through this slide deck from Ross Pettit at ThoughtWorks.
I have taken the time to study what Mr. Pettit proposes and to read some other opinions around the sphere. What I have found doesn?t jive for me all the way around.
To put it mildly, there are differing opinions on the ability to measure Agility with a maturity model. The most common rebuttal (rooted in Lean Thinking) is we should ?measure up? and simply evaluate Return on Investment (ROI).
Measuring Organizational Success
Human nature and then the stock market figured this out eons ago. How do we reliably measure an organization?s success in a free market? Profitability. Essentially, ROI.
Simple measurements like ROI look great on paper and are notoriously elusive to use in improvement because of the vast number of mitigating factors involved in deriving the final number. But it sure is tempting, so sayeth the Lean Thinkers.
In order to demonstrate ROI of a team, we simply look at cost and revenue directly attributed to that team.
Easy, right?
Not so fast, Gomer. What about the fact that what really happened during the year in question is that an aggressive sales team enables a boat load of sales of an essentially crappy product?
We all know that happens all the time. Billions have been made on spaghetti code while at least as much is lost on elegant solutions. So how do we define success?
Lean offers many other ways to view an organization?s effectiveness. Value stream, flow, ROI, Kaizen, and pull systems can genuinely eek every miniscule efficiency in your company. Indeed, in my opinion taking a holistic Lean approach in your organization holds the highest promise of making things better. But is this measuring Agile?
I?ll go with ?No.?
Measuring Agility
Every since the first teams called themselves ?Agile? and delivered in increments, business analysts have been relentlessly following these teams around with slide rules and calipers measuring their excrement. Some of the first sessions I attended years ago at Agile conferences looked at varying models for measuring Agility and effectiveness of teams.
Q: Why so fascinated with measurement?
A: Because it is necessary to being taken seriously.
Many attempts have been made (many successfully) to co-opt Agility to mean something beneficial to the person using the term. In my mind the only true definition of Agility comes down to what we find in the original manifesto.
If we are to truly measure Agility, one must therefore be able to quantitatively measure the degree to which an organization holds true the Agile principles. To some extent, that is what Mr. Pettit?s AMM endeavors to do. After thinking about it, I must conclude that this is a fruitless and nearly hopeless task.
On a scale of 1-5 how much does your organization value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
Well, that?s gonna be a tough nut to crack, isn?t it?
Measuring Craftsmanship
There is no question that the Agile movement has been pivotal in the evolution of software development. Both in methodologies (process) and in engineering (practices) we have evolved the software development field more in the last 10 years than the previous 30.
The most obvious effect for coders on the ground is in the practices side and includes subjects like Test First Development, emergent design, automation, Continuous Integration, and other practices that are inarguably good for teams to practice.
Thanks to some vocal proponents and notable cantankerous grouches (thanks, Uncle Bob) we are now starting to talk about these things as Craftsmanship.
So now the question becomes, can we measure craftsmanship? To this I resoundingly reply, ?Yes!?
In fact, this has been done for thousands of years. We can point to any trade and see the evolution of the craft and the teaching of modern practices in it. An 18th century blacksmith?s apprentice had specific skills to master to qualify as a practitioner. Mastery of those skills is inherently measurable because they are skills, not art.
The guild model for craftsmanship just works here. Here is a fictitious scoring sheet for a blacksmith?s apprentice.
| Skill | Score (1-5) |
| Ability to gauge furnace temperature | 3 |
| Proper use of all common tools | 2 |
| Shaping thin metal | 5 |
| Creating and using metal casts to form simple objects | 4 |
| And more? | ? |
Is it really such a stretch to construct similar evaluation areas for software development? I think not. In fact, many are in existence today in the form of thoughtful job descriptions.
Although the skills desired and measured will undoubtedly be guild specific, there are fundamentals that shouldn?t be too hard to agree on.
| Skill | Score (1-5) |
| Effectively apply Test First Development tools | 3 |
| Appropriately identify, construct, and apply fundamental design patterns | 2 |
| Demonstrates ability to compose a cohesive object model for a simple application domain | 5 |
| Appropriately creates, disposes, and manages memory | 4 |
| And more? | ? |
Pick a Guild
The guild spats are (and should be) over the specific skills deemed important. Fine, let?s let the discussion spiral to that, but can we agree on some basics?
This is why I actually like the Application Lifecycle Management model proposed by Microsoft. I wrote about it here (and here), and have since used it to actually work with teams to better their skills. Sure, it is guild specific, but so will be many of the instruments to come.
The important thing is to use one, and to use it deliberately with a genuine Kaizen approach.
- Pick a skill
- Assess your proficiency
- Make a decision to modify your behavior in a deliberate way
- Perform this new way
- Evaluate your skill level
- Repeat
Simply put, we need these things as a way to make deliberate and lasting change in our skills, and to hold each other accountable for that advancement. I?m just saying.
Like It or Not, Tools Play a Role
Have you ever watched ?The New Yankee Workshop?? If so, you?ll no doubt recognize that Norm Abram is a master craftsman. Notice anything else? The man has a different tool for every minor job in the workshop. Further, he has more than one! And he has mastered the use of each one.
Norm can obviously execute a tenon joint with perfection. Once those pieces of wood are joined, let no man break them apart! And watching the show indicates Norm has a preferred way to execute the joint.
He uses a $3000 table saw.
That doesn?t mean he doesn?t know the principles behind a solid tenon. In fact, he has probably carved many of them with chisels in the past. That doesn?t make his tenons executed on the table saw any weaker, though.
Given a choice, Norm chooses the tool he enjoys and the one that makes quick work of the job. He doesn?t need to carve the thing with a pocket knife every time.
Nor does a software guild member need to limit their mastery of a principle to always rolling their own. Indeed, tool mastery is significant part of the program.
Will the budding apprentice need to learn Make, FinalBuilder, MSBuild, NAnt, Ant, and Maven in order to demonstrate mastery of build automation principles? No.
But they will need to use something, just like Norm. And the tool they use will likely be a function of the guild. It will probably be the tool the shop steward already has laying in the corner. It is important to note that one need not master all the tools available for a given job in order to execute the job well. One must invest in a specific tool set on the road to mastery.
We?ll be comparing test frameworks and IDEs for years to come just as Norm surely argues with his cronies about the best chisel set. Just recognize the debate for what it is and move on.
Conclusion
We have the tools we need to move forward in front of us already. An Agile Maturity Model doesn?t resound with me as a necessary thing.
Using instruments of Lean as the rubrics for organizational improvement is sound. Further, there is more to come. We are really only getting started with Lean.
Using guild-specific instruments like Microsoft?s ALM assessment as a rubric for personal and team performance holds intrinsic value. Want different skills? Try a different guild.
Craftsmanship is more about personal preference, I don’t see it playing a significant factor in the success of a company. ROI is an accurate guide for maturity because successful bosses worship ROI, and for good reason. Businesses are about making money, success is about making money. Craftsmanship is about passion. If that happens to make a bit of money that’s grand, but don’t expect to convince money-focussed corporations that craftsmanship is a good idea unless you have figures to back it up.
As for the summing up the parallels between software craftsmanship and traditional craftsmanship. Take a look at where traditional craftsmen and guilds are headed… 95-99% of things people use every day, that make companies/people money are mass-produced. ROI and processes like Six-Sigma are gods in this realm. Craftsmen still exist making Ash rocking chairs vs. Ikea’s Poängs. You can debate all you want about which one is “better”, but it’s pretty clear which is more successful.
@Steve Py
“Craftsmanship is more about personal preference, I don’t see it playing a significant factor in the success of a company.”
You might want to mention that to Chrysler.
I struggle with the more crap vs. less quality culture we have built for ourselves, too. That is also an age-old problem facing craftsmen. I really enjoyed this book on the subject that expounds on both of our points.
http://www.nytimes.com/2008/04/06/books/review/Hyde-t.html
It has little to nothing to do with software development, but makes the points we are discussing here better than I can.
@David Starr
And for every Chrysler there is a hundred companies that pull it off. Every company is going to have it’s ups and downs. The fact that you have to consider with Chrysler is that the impact of their failing is as big as it is, *because* Chrysler was as successful as it was. They didn’t make their Billions by hand-assembling and inspecting each car off the line then colapse because they started using robots operated by grad-11 drop-outs.
IMO “craftsmanship” is not the same as “quality”. Craftsman are generally very focused on quality, but companies can certainly be successful, very successful without going to that extent. Quality is about delivering and responding to customer needs, and a process of continuous improvement to that end. Craftsmen have that same goal, but approach it at a very personal, almost spiritual level. “I make what I make, how I make, because I love it.” That’s not a bad attitude to have towards software, but it is not a necessity to producing successful (read: Highly-Profitable) software. I’d argue it could be an obstacle to building highly-profitable software.
Besides, just because something isn’t built by a “craftsman” doesn’t automatically delegate it to the dragon-infested land of “crap”. A lot of it is actually quite good. (I love my Poäng) A crafted piece of furniture would probably last longer, but it will also be more expensive up-front. As a customer I’ll decide whether that extra up-front cost is justified. With software, putting on the Devil’s Advocate hat, I don’t care if it’s easier for you to maintain, modify, or extend if all I want is what I know I want right now. If yours costs me $200k but I can get that same functionality for $150k from someone else, (with reasonable quality) even if further enhancement is more expensive with the second option I’d likely go with it. Just as YAGNI applies to software development, it also applies to software consumption. If you can’t convince the customer, you can have the best-crafted, highest quality piece of code, but a niche customer base, or living out of other “successful” organization’s pockets like OSS “pets”.
Damn you, making me think hard…
OK, so I get it. I have been the guy making your argument. I swear. My wife has a friend who runs a catering company and she refuses to put cheaper potato salad on the menu, even when customers request it.
My statement to my wife was, “Well, good luck with her business. She isn’t giving customers what they want.”
Her reply was, “It is a quality issue. She refuses to make that compromise and is willing to take the hit that it costs her.”
I get both arguments. How about this for a leveling statement? “Craftsmanship is quality as measured by the creator, not the consumer”
So, this balance is what I am trying to strike. I refuse to believe that we can’t serve the ideals of craftsmanship and ROI. These things are not mutually exclusive.
Further, tools of craftsmanship applied well further the goals of the ROI focused organization. I do not believe that well built software cannot be the most efficient to build. Quite the opposite. Why do you think Norm uses a table saw? The same reason we save money when we build with TDD and CI. It is faster to produce something of higher quality. No?
@David Starr
Fair enough. Though a table saw is just a tool. A craftsman will spend the effort to make the most out of whichever tools their heart desires. A fool with efficient tools will just end up making a mess more efficiently. 🙂
Reducing waste and improving efficiency can be boiled down into one phrase from carpentry: “Measure twice, cut once.” In software that means spending the time to get the requirements right. This is something learned from experience. Irregardless of your code craftsmanship, or how well your can cut a dovetail joint and bring the grain out of a nice piece of Blackwood, it won’t mean peanuts if the customer wanted mitres in Cherry. But that digresses into a completely new discussion.
I believe the passion of one’s chosen craft and ROI will always be a bit polar. Learning how to use tools efficiently may be on the path to craftsmanship, but at some point that path must diverge towards ROI to be successful. The Craftsman side of you will say “it can be better”, the ROI side will say “it’s good enough to sell.” That’s a tug-of-war I see, or read about every other day.
I don’t know too many people that would admit to staring in awe at what a master did in 40k lines of Cobol in 1989. I doubt anyone will give any more care for looking at 50k lines of C# with 98% coverage in unit tests and an automated build environment running on a cluster of P4s, in 2020. But we marvel at a Stradivarius or a Monet. To me, that’s the difference.
I can see both sides of the argument. ROI is ultimately what will allow you, your company/department/project/etc. to stay in business. You have to stay in business in order to express your passion for your craft. How many open-source projects were started by someone with a lot of passion and that went nowhere because it was not the right project for anyone other than the author?
On the other hand, you don’t build a culture that is dynamic, fun, and passionate about the work that they do by bashing ROI over everyone’s heads all the time. ROI is the proper focus for Marketing, Sales, and ultimately for the one overseeing everything. But craftsmanship and passion are fundamental to the ones building the development team. They need to understand the money and the reason ROI is important. But if that’s all they know, and have no passion for the craft that they are in, there’s no way to build a culture around that.
Successful software and business require both, IMO, and striking the right balance is the key to succeeding. The trick is to understand that the balance point is not always the same depending on the lifecycle of the company/department/project.