One Thing DevOps Does to Scrum Teams

Yes, DevOps has fundamentally changed the way we operate in a good and healthy way. Development teams being part of the care and feeding of the systems they produce increase quality and will eventually automate the CICD pipeline as close to production as possible, if not all the way.

Even so, many of the Scrum teams I work with feel overwhelmed and even distraught about the work they don’t get done during the Sprint on their product. This is most common among teams who assume ad-hoc or interrupt work will not disturb their capacity to get work done on their core product.

There are 2 common sense ways to deal with this problem.

  1. Pull less work during Sprint planning.
  2. Pull less work during Sprint planning.

Most teams come to this realization on their own, but then struggle with the minutia of how to track the 2 types of work they are now responsible for: operations work and product development.

There are 2 main ways I have seen teams mitigate this tracking of work. One looks like the image below, and teams associate all operational work with a reserved User Story for just this purpose.

Scrum Board

These teams will often pre-size the Ops User Story based on previous experience of how much operational work comes into the Sprint vs. product development.

I’ve seen examples of operational items with up to 50% of the team’s capacity reserved. This is a team in trouble. They are on the verge of becoming an operational team that no longer adds features to a product and thus no new value to the customer.

At any rate, in my experience most teams settle into about 10% of their time being reserved for operational duties in DevOps environments. This informs the size of the operational Sprint item during Sprint Planning and tends to work well enough.

The thing I don’t care for in this model is that it doesn’t shine a bright light on the operational work, which is, in fact, bad. If we had no operational work, after all, we would be adding product features.

Since we are focusing on the tracking mechanism, another other way I have seen teams visualize the operational work they must address is through a fire lane. A fire lane may take the form of the board we see below.

Fire Lane

I far prefer the fire lane model as does not obscure the operational items and makes them very visible. This can lead to pattern recognition. “We need to solidify our integration testing,” or “We need to close the gap on automated container deployments.”

This technique makes each and every item visible to the team and can be examined during retrospectives, potentially causing a team to expand its definition of done.

While there are undoubtedly many other ways to track operational vs. product development work, the technique to do so should draw a very clear distinction between the two and highlight operational work in such a way as it can be mitigated over time. One step at a time, just like paying down technical debt.

Team Safari – Beating the Central Services Team Problem

Safari

The Central Services Problem

In large enterprises, a customer-facing product development team often has dependencies on other teams for frameworks, underlying systems, and (very often) centralized platform APIs. The most common scenario in large enterprises is a central Platform Services team that provides APIs to product teams for access to underlying data and service workflows.

The point of this post is not to critique that model, but instead to accept this model exists and work with it as best we can. These central Platform Service APIs are a product in themselves, with customer-facing product teams as customers. 

Delivering a full slice of functionality of a given feature is the only true example of delivering value, but this can be challenging when team dependencies like this exist, which they often when dealing with scale. This central services model can produce frustrated customer-facing application development teams because they may be waiting on a Platform Services API team to provide those services or endpoints they depend on. So, what does the product team do? They can:

  1. Mock out the Platform Services API based on proposed contract (which may change)
  2. Put in an abstraction layer over the Platform APIs that would be unnecessary should the services be directly available, because it’s easier to mock and doesn’t predict service contracts. Then mock that, adding a layer of premature abstraction to your design.
  3. Send the Product Owner over to fight for positioning in the Platform Service team’s backlog. (This should probably happen anyway, but let’s assume that fails because the product team needs aren’t at the top of the list).
  4. Submit a Pull Request to the Platform Service team’s repository to provide the service capabilities the product team needs.

Ding, ding, ding. That last one is the right answer. The product-facing team submits a PR to the Platform Services API repository. What a great solution! The ultimate prescription for success here is to use an internal open-source workflow model and submit code to the Platform Services team to accept into their codebase.

Then the PR is rejected for a myriad of reasons that boil down to the PR-issuing team not understanding the conventions and policies of developing code for the Platform Services team APIs.

A Team Safari

Consider a Team Safari (we call these Agile Safaris at GoDaddy).

A Team Safari is when one or more members of a team leave their home team to join with another team for a Sprint or two to solve a problem and learn more about that team’s software systems.

Why would a we do this? After all, it decreases the capacity of the team left behind for the time the team-member is away. Further, the receiving team capacity is eaten up teaching the new developer how everything works.

The short answer is everyone benefits from high-quality PR being accepted. If the product-facing team can issue PRs that get accepted to the Platform Services team, everyone benefits. The Platform Services team grows its API services footprint. The customer-facing team gets the service it needs.

A Successful Safari Recipe

When a Team Safari is used, there are some ground rules to help things go much better than for a developer to just move a desk over and announce, “Here I am! Safari me!” In addition to core platform capabilities, the visiting engineer will greatly benefit from gaining a working knowledge of all aspect of the visited team practices, patterns, underlying architectures, etc.

First and foremost, ensure the Engineer on Safari is there to solve a specific problem, or learn a specific skill. This should be summarized in a clear and concise “purpose statement” everyone involved understands.

If the following are approached methodically, once the engineer returns to the home team, he or she will later be able to issue successful PRs to the team visited. The following knowledge will help ensure those PRs are accepted the first time.

  1. Standards, conventions, and architectural guidelines of the Platform Services APIs public interfaces.
  2. Standards, conventions, and architectural guidelines of underlying systems capabilities including data access, logging, security, etc.
  3. The underlying language (probably a prerequisite to the Team Safari anyway), documentation, and coding standards.
  4. Full understanding of the CICD team’s CICD pipeline, to ensure appropriate unit and integration tests are included with the PR and pass appropriately.
  5. The specific test-first or other developer testing practices used by the visited team.
  6. The greater architecture (if applicable) of the visited system, framework, or technology if appropriate. Not just solving for the stated purpose, but doing so in the visited team’s context.
  7. Any teams and system the Platform Services team depends on, and how those dependencies relate to each other. 

Paying Back on the Investment

Not only does the Team Safari inform the developer who went on Safari, but that knowledge eventually gets transferred back into the home, or product-facing team then the developer returns.

And now, we don’t have to wait for the Platform Service team to work our Product Backlogs to the top of the queue. The newly informed team has someone who can create real PRs that are accepted and deliver on the promise of working software each Sprint.

As stated, the visited team now has the added capability the accepted PRs provide. Everybody wins.

Do you and your teams use Team Safaris or something like them? Let e know in the comments.

Agile’s Coming of Age

Now that the term “Agile” is sufficiently compromised as to be near meaningless, Agile Software Development is old enough to stand on its own, make its own business case, and demonstrate its value. But it still isn’t a mature adult. Agile Software Development is a hormonally unbalanced pre-teen with ugly spots, occasional outbursts of irrational anger, and the promising potential of smart-assed intelligence.

Giving birth to Agility and parenting it to pre-pubescence was a miraculous feat. The thought leaders who brought us the manifesto and subsequent culture shift deserve our thanks for seeing the need and creating the right message at the right time. Thanks to these revolutionaries Agile is not a footnote, but the most promising path forward to improving our profession.

One focus of the last 10 years of agile discussion has been, “There are better ways to develop software.” While the state of spaghetti code in the universe is still a big problem, progress has been made in this area. It is no longer radical to think in terms of test-first, pattern-based, or *-driven. Many teams are just acting more professionally, and that’s a wonderful thing.

The other primary focus of the last 10 years has been teaching technologists to actually speak human. You know, with emotion and stuff. And there’s great news; as a profession, we developers aren’t as jerky now as we were 10 years ago. See? It’s working.

The Agile Consensus

I get to see many implementations of Agile Software Development. Not surprisingly, most teams out there aren’t living the dream, but are actively trying to improve. Indeed, recent studies and surveys have noted that projects using Agile methods now outnumber plan-driven, or waterfall, projects.

Sam Guckenheimer calls this change the Agile Consensus in his recent book with Neno Loje (disclosure: I was a technical reviewer on this book). The idea behind the Agile Consensus is simply this: Agile won. Creating software with plan-driven techniques obviously falls short of the advantages of developing with a focus on humanity and exploiting shorter feedback cycles.

Bottom line: Agility has crossed the chasm and is no longer only the domain of developers. The enterprise wants big-A Agility.

Not Crazy Anymore

Whether or not a given practice is crazy or “edgy” depends on who you talk to. Organizational acceptance of a given practice is typically rooted in the values that practice supports, and the practices here seem to have reached a level of general acceptance in our industry. Mostly.

Daily Team Meeting

Whether you call it the Daily Scrum, the Daily Standup, or simply a daily team meeting, the practice of a quick, informal team meeting held at the same time and place each day has really caught on. The reason for this is simple; when done well, the daily team meeting helps teams be more productive and improves situational awareness in complex environments. And regardless of what kind of software development you do, odds are it is fairly complex.

The daily team meeting is one of the most misunderstood and poorly executed practices mentioned in this article, but I digress.

Bottom Line: Daily Team Meetings are commonplace and useful. They aren’t considered crazy by most teams anymore.

Sprints

Sprints, or iterations, are simply short periods of dedicated time within which teams will deliver working software. These time boxes in which teams agree to deliver something have changed the way we think about long, death march projects. Many teams know that progressing toward the broader goal of releasing software is often best managed by delivering working software all the way through a delivery pipeline, with increasing amounts of functionality each time.

Businesses leaders often love Sprints because Sprints are an obvious way to manage risk. More on this later.

Bottom line: While there are other ways to organize work, doing it in small batches within a Sprint of 30 days or less is a model that has proven itself time and again.

Test as We Develop

Lean thinking encourages “testing at the point of work”, which is a broad concept we’ve apply to software development to derive TDD, BDD, ATDD, and other forms of simply proving that software works as we create it. We’ve learned along the way that we don’t even need to think of this as a verification process. Test-First practices have proven to not only help developers build the “right” software, but to build the right software well.

Not every developer has drunk this Koolaid, but most understand the basic value behind Test-First as either a design tool or a verification tool. Those of us who really drank deeply see Test-First development as the de facto way to write code. Sure, I’ll bang out a quick shell script without an accompanying automated test harness, but that’s a simple fit-for-purpose decision.

Bottom line: When making software that needs to be work right and be crafted well, many developers see Test-First practices as indispensable.

Deliver Frequently

Delivering working software frequently allows development teams to actually deliver something, and that’s half the battle. Frequent delivery of working software enables the most valuable feedback loop in software development. The conversation around this feedback loop is simple and sounds like this:

“Here’s what we made. Here is how it works. What do you think?”

And then comes the tricky part – Actually listening to the response. This helps teams build the right thing next. I loved watching eBay a few years back as it changed its color scheme and skin layout gradually over a few months. They design a destination look and feel and migrated to it with frequent delivery of small changes in their UI. This allowed the product to evolve, rather than re-release. This is how software delivery is evolving and the most extreme form is called “Continuous Delivery.”

Delivering frequently may sound a lot like “Sprints”, but there is more going on here. Sprints are simply a forcing function for frequent delivery. There are other ways to pull it off. Frequent delivery all the way to customer feedback can change the way a company engages its customers and plans its strategic moves.

Bottom line: Frequent delivery is craved by executives for the business advantages offered and by technical teams because it makes actually shipping ubiquitous.

Still Crazy

While some Agile practices have crossed into “just plain old good ideas,” many are still seen as edgy, or extreme. Despite evidence that these practices offer real value and better alternatives to traditional thinking, the old ways of looking at the world are just so ingrained that these practices provide fodder to skeptics.

Pairing

No technical practice has drawn more fire than Pair Programming. Hard data has begun to emerge about the practice of pairing, and all that data shows (to varying degrees) how pairing creates higher quality and simply better software. A paper

There are also a ton of human advantages, like increased learning, knowledge sharing, and removing single points of failure within a team.

Why then has formal pairing been relegated to the domain of roman sandal wearing hippie agilistas? Most development team leaders or managers simply see pairing as an investment of two people doing what one could accomplish. I won’t try and convince you otherwise in this article, but I will mention this:

Barry Bohm has made a very distinguished career of studying software development. In Balancing Agility and Discipline, he asserts that 60% of all software defects in production could have been caught with a peer review. Pair programming is continuous peer review. You do the math.

Finally, most developers treasure their alone-time with the code. Sharing the way I approach problems or write code can feel like a job interview every day. That kind of scrutiny can feel very uncomfortable unless I am in an environment of absolute trust. That ties the success of this technical practice to the culture of the team and company.

Bottom line: Pair programming is still seen as eXtreme, and the transparency it forces can terrify many developers.

Funding Alternatives

Companies spend a lot of time and energy developing golden plans for the next year. Strategic planning is a dependable activity of middle-management in those months counting down to the end of the current fiscal year.

We know good and well that we can’t predict the evolution of a software project beyond a few months in most thriving businesses. Change just happens. Why then do we persist in thinking Big Funding Up Front is any different than Big Design Up Front? Some are making inroads with models of T&M funding, fixed cost, adjustable scope, and other techniques like incremental funding. However, for the most part we remain stuck in annual funding models because business Agility, the real promise of Agile, remains elusive.

Bottom line: Software development projects are still funded when we know the least about how we’ll be spending that money.

Strategic Iteration

While Sprints, or iterations, are very popular on the operations side of the house, few companies see them as the strategic advantage they really are. Sprints are loved by the business because they reduce risk, but actually refining the scope, plans, and functionality based on an iterative feedback model is a foreign idea. Iterative delivery provides a regular cadence that can be interpreted as “milestones” by traditionally trained most project managers.

The innovation companies could have with regular Sprints is lost because of the aforementioned Big Up Front Funding that causes Sprints to be seen as a tool of operations.

Bottom line: The potential value of iterative incremental teams is being wasted by a determination to fund fixed-scope projects up front.

The Next Challenge

Professionalism

The profession of software development is reflecting upon itself right now and the question of what it means to be a software professional is coming to a head. The craftsmanship movement has a genuine toehold with many introspective developers; Universities are actively looking beyond computer science programs to fill the supply void of industry; and my mom thinks she’s “writing code” when her excel macro runs without error.

Writing solid code is now table stakes for being a software professional. The expectations we have of true professionals are becoming appropriately greater. As technology matures and abstractions go higher, the productivity of development teams should be through the roof. Yet, it isn’t necessarily the case and hiring organizations are desperate for some way to assess prospective developers en masse.

One desperate attempt at identifying professionals is the ridiculous history of Scrum certification. Certification teases with the allure of simply trusting a credential. Unfortunately, this isn’t working for any known certification yet, university, private, or otherwise.

Bottom line: Professionalism in software is finally being demanded by those creating it, and by those asking for it.

Maturity

Just like the allure of hiring a professional, the temptation of the perfect development process is just too tempting for the ignorant to ignore. The success of simple frameworks like Scrum and Kanban provide just enough structure to get things done, without providing prescribing specific practices. That scares plan-driven organizations that value control over creativity.

To get real traction with Agile methods means getting not just permission, but support; and support means money. Before bureaucracies spend money, they want assurances and guarantees. False ones will do; look at how well RUP and MSF sold.

Providing any compelling story for change requires supporting data. The willingness of good leaders to instigate and support true change will start with the end in mind. While the end state of an Agile transition can’t be predicted, case studies and measurements of established Agile teams are the catalysts for getting Agile transitions started.

The demand for reassurance will drive development of tools like assessments, maturity models, and formal adoption programs. As older and more established industries explore Agile, these tools will be in heavy demand by those wanting to make data-driven decisions.

Bottom line: Agility is moving into more mature organizations and Agile itself will need more accessories of maturity.

Book Review – Extreme Programming Explained

Last week I finished reading the first edition of Extreme Programming Explained, written by Kent Beck. I explicitly wanted to read the first edition, that I purchased via AbeBooks, which is also the first book that appeared in the series of books on XP. I’ve read a couple of book reviews of the second edition on the interweb where some have stated that the second version is somehow less ‘extreme’ compared to the first edition. That’s why I went for the first edition as I wanted to learn from the ‘original’

While it seems that Scrum and Lean are getting all the buzz in agile software development these days, why on earth would I want to pick up a book on Extreme Programming that has been written back in 1999? Well, it’s my current feeling that XP is the ‘core’ agile methodology. While there was some buzz around this software methodology in the late 1990s and early 2000s, it has never been commercialized nor pursued by hypesters, at least not on the same scale as Scrum and Lean. I’m not saying that Scrum and Lean are therefore bad and should be avoided. On the contrary! I’ve been part of a Scrum team for many years now where we’ve been complementing this project methodology with XP engineering practices like unit testing, continuous refactoring, continuous integration, etc. … . This has been working because of these XP practices, not the other way around. But what I’m seeing more and more lately are teams that are adopting Scrum without any of these solid engineering practices, especially in enterprise environments. Therefore I think it’s important to learn more about XP, which is one of the agile methodologies that somehow managed to preserve its values throughout these years without losing its balance. I’m with Uncle Bob when he states the following in his blog post on Bringing Balance to the Force

“I, for one, consider the agile movement to have been inundated by a vocal and enthusiastic cohort of project managers, waving their scrum-master certificates, or their Lean and Kanban books.  They have overwhelmed the original movement and changed it into something new.   Agile is no longer about a balance between the technical and non-technical parts of development.  Rather it has become a discussion almost entirely dedicated to non-technical issues.  Agile is no longer about healing the divide, or closing the crevasse.  The agile movement now represents one side of the crevasse.”

Anyway, back to the book review. I personally found this book to be a very fascinating read. It’s divided into three major parts:

  • The Problem – This first section sets the stage for XP. The chapters in this section discuss the value proposition for XP while also showing the disciplines behind it. The four values of XP are Communication, Simplicity, Feedback and Courage while the fundamental principles are Rapid feedback, Assume simplicity, Incremental change, Embracing change and Quality work and the four basic activities are Coding, Testing, Listening and Designing. These values, principles and activities are broadly discussed throughout this section. Basically, here you’ll get a high-level view of Extreme Programming.
  • The Solution – Given the values, disciplines and activities from the first part, the second section elaborates on the strategies and practices that make up XP. The practices discussed are The Planning Game, Small releases, Metaphor, Simple design, Testing, Refactoring, Pair programming, Collective ownership, Continuous integration, 40-hour week, On-site customer and Coding standards. The book doesn’t provide an explanation on how to do these practices. It’s more about why these practices are important and why they’re under the XP umbrella.
  • Implementing XP – The third and final section of the book deals with a number of various topics like how to adopt XP, when to use it, when not to use it and what it takes from people that are part of an XP team. I personally got the most value from the chapters in this section.  

Something that the author kept repeating throughout the book was the importance of how the office furniture is arranged for an XP project team. I never really thought about this, but after reading this book I’ve come to realize how important this can be. Being able to easily pair program and get the customer to sit with you can in fact be a game changer.

Here are some of my favorite quotes that I picked up while reading the book:

“No single action takes the life out of a team or a person more than being told what to do, especially if the job is clearly impossible. Primate dominance displays work only so long in getting people to act like they are going along. Along the way, a person told what to do will find a thousand ways of expressing their frustration, most of them to the detriment of the team and many of them to the detriment of the person.”

 

“So you code because if you don’t code , you haven’t done anything. You test because if you don’t test, you don’t know when you are done coding. You listen because if you don’t listen you don’t know what to code or what to test. And you design so you can keep coding and testing and listening indefinitely.”

 

“Code gives you a chance to communicate clearly and concisely. If you have an idea and explain it to me, I can easily misunderstand. If we code it together, though, I can see in the logic you write the precise shape of your ideas. Again, I see the shape of your ideas not as you see them in your head, but as they find expression to the outside world.”

 

“The difference is between playing to win and playing not to lose. Most software development I see is played not to lose. Lots of paper gets written. Lots of meetings are held. Everyone is trying to develop "by the book", not because it makes any particular sense, but because they want to be able to say at the end that it wasn’t their fault, they were following the process. Software development played to win does everything that helps the team to win and doesn’t do anything that doesn’t help to win.”

I highly recommend that you pick up and read this book. It’s only 166 pages and a bargain on AbeBooks or Amazon. By reading this book I learned a lot about agile software development and more importantly, how to strike a balance in customer collaboration and technical stuff. The book contains a tremendous amount of knowledge and wisdom. I sometimes had a very hard time putting it away as its also very nicely written. Two thumbs up!

The Social Network

Over the weekend I watched this movie called The Social Network. As a geek, I felt that I was somehow mentally obligated to see this movie :-). While I was watching this movie, I noticed a subtle detail that was shown a couple of times throughout the film. This subtle detail showed how this small team of geeks maintained and preserved the productivity and efficiency of their team. Basically, whenever someone barked into a room where geeks were working, team members that were not working or having a break protected their peers that were "in the zone" from any kind of interruptions from this person. So the individual that marched in there wasn’t allowed to talk or interrupt someone that was working. If this individual did try to interrupt a working member of the team, he got told that this person was not available and to leave him or her alone. If the person working somehow still got distracted, the non-working team member reminded him of his responsibility by firmly telling him "you’re working", trying to get his peer back to whatever he was doing.

I don’t know whether this is actually part of the team culture at Facebook, and frankly, it doesn’t matter. I just find this to be a cool and fascinating idea. I’m intrigued by this notion of a team that takes responsibility for preserving their flow and velocity, protecting other members of the herd from interruptions and any other kind of events that prevent them from effectively doing their job. Sounds pretty self-organizing to me.

I have to admit that this is also the first time I actually learned something useful from a Hollywood movie. I must say that this feels weird too ;-).

Breaking Even Small Things Up

I’ve talked about why smaller is better and especially why this pertains to product backlogs, or rather fatlogs, but I wanted to zoom in for a second to one of the most valuable applications of the rule: writing the code.

Typical workflow

What do you do when you get a task or a backlog item to work on?  Do you just pick a starting point and start writing code?  Do you break it down into a few pieces and start working on one?

It probably depends on the size of the work.

When I run into an item that is sized a little too large, problem admiration can set in if I am not careful.

What is problem admiration?  It is when the problem you are trying to solve seems too big and overwhelming, that you just sit and stare at it instead of attacking it.

This can be a really big problem if you don’t have a good workflow that tackles it.

breakwindow

Winning the battle

So how do we combat this problem?

One of the strategies I use, is very similar to a mini-agile process in the agile process. 

When I am working on a piece of functionality, I will have my own sticky notes nearby.

I’ll slice up my work, whether it be a task or a backlog, into distinct pieces of work that need to be done.  I’ll even write a few different units of work on a single sticky note.

I might have a sticky note that says “localize the screen.”  Or one that says “recheck unit tests to make sure they are accurate.”  Or “Populate drop down x.”

Sometimes I refine the notes and break them down further.  My goal is always to get the current task at hand to such a small difficulty that I can easily accomplish it.

You are always much more effective doing many easy tasks rather than trying to do few difficult tasks.

As I accomplish a task, I cross it off the list.

If I think of a new task, I add it to the list.

Why is this so important?

In general, developers, or rather humans, share a weakness of having a hard time accomplishing something that is large and not fully understood.

I have seen some very knowledgeable and skilled developers that were never successful, because they could never push through and get things done.

Time and time again I have seen them become paralyzed by the work that needed to be done instead of actually doing it.

You might not have this problem in general, but you would probably be lying if you said you don’t suffer from it from time to time.

You might not even recognize it.  It might show up as a lack of motivation, or “burn out.”

Many times when I don’t break down the tasks into smaller pieces I have a tendency to overlook some part of the work that needed to be done, because I either forgot about or didn’t take the time to consider it.

If you take the time to break apart your work into very small pieces, you will probably find, you are more motivated to get the work done, feel like you have a better direction in which to go, and will miss fewer requirements.

As always, you can subscribe to this RSS feed to follow my posts on elegant code.  Feel free to check out my main personal blog at http://simpleprogrammer.com, which has a wider range of posts, updated 2-3 times a week.  Also, you can follow me on twitter here.

Scrum and the C-Word

There has been much deriding of both Scrum and the word “certification” in the past few months. It has become quite vogue to look down on formal training of any kind it seems, particularly among the ALT.NETerati. I get it, and I think several of the criticisms are deserved. Some aren’t.

Disclosure

  • I am a Professional Scrum Developer Trainer. That is: I have been certified by Microsoft and Scrum.org to teach others about Scrum and modern engineering practices on the Microsoft platform. More information about all of that is available here and here.
  • I teach for a living. Mostly Visual Studio ALM along with topics I believe are focused on making better software craftspeople.
  • I have watched teams get it wrong (and sometimes right) for about 20 years now. That admission makes me feel very old.

A Bit More Background

Choosing to enter the Microsoft and Scrum.org program and be part of the certification process for developers was not an easy one for me. Indeed, as several people I irritated with my waffling for over 6 months will attest, it was a decision I spent a long time considering. In the end, I obviously elected to become an instructor for certification courseware and this in this post I will tell you why.

On Scrum

I first practiced Scrum in or around 2003/4, if memory serves. I didn’t see it done well for several years after that.

On paper, everything sounded just spiffy when I studied Scrum and considered bringing it to my team and organization. Who wouldn’t want the benefits it promises of frequent feedback, clear accountability, and no-nonsense shipping of product? To get started we hired Jeff Sutherland to come in and teach the original Certified Scrum Master course offered through the Scrum Alliance. I was baffled that at the end of the 2 day discussion to receive this thing called a “certification”. My colleagues and I actually joked about it at the time as being proof that we didn’t fall asleep in class. As Ken Schwaber pointed out to me in an interview recently, we probably could have nodded off and been fine, actually.

My team struggled a great deal trying to use Scrum well. I won’t bother including the long list of mistakes we made, but we eventually got traction and settled into a cohesive rhythm. After we figured things out, Scrum became an amazing vehicle for delivering product, which was the original intent.

To my knowledge, Scrum is still used today in that organization. At the time of my departure was in fairly healthy shape.

Looking back on Scrum’s contribution to my professional life and its contribution to our industry as a whole, I am humbled. Scrum as an idea has done more to drive agile software development than perhaps any idea other than the original manifesto itself. TDD ranks right up there, but fewer CEOs have heard of TDD. There are a lot of reasons Scrum has won the marketing war, but the simple fact is that our profession is a better place for the presence of the Scrum framework and the people who provided stewardship to it, Ken Schwaber and Jeff Sutherland.

I know good and well that other techniques are very effective. Kanban is a wonderful way for many teams to operate. Feature Driven Development is a powerful model for the teams who need to formality of it. And of course, the gaping hole left by the absence Extreme Programming reminds us that zealotry has its costs. Scrum, as it advertises, is not a silver bullet. It won’t fix anything, in fact. Only people of good intent who come together and try to do something well can make that happen. Scrum can merely provide the rules of engagement.

Scrum is popular for several reasons. When it comes to the mechanics of the framework:

  1. It is easy to understand.
  2. It is easy implement and follow the basics.
  3. It can show dramatic return in a short period of time.
  4. The first Scrum in an organization is a highly visible Kaizen Blitz.

With all of those benefits, Scrum is plagued by the same problems as every other process or methodology framework in the world.

  1. Human beings are the ones who try an implement it.
  2. Human beings are the ones who try an implement it.

Scrum is great when wielded for good. It is the lowest barrier-to-entry agile technique I know (excluding engineering practices, like TDD and CI). Teams who currently live in plan-driven organizations derive great benefit from Scrum and so do the organizations themselves. Scrum can be very, very effective.

I chose to work with Scrum.org specifically because I know from experience there are more shops out there that need to change than those who don’t. I believe most of those shops cannot handle something like Kanban. [See this for more] My professional life as a trainer is focused on helping developers love our craft again. Those of us who got a thrill from lighting up a pixel many years ago have just been beaten to hell and back since deciding to do this for money. Although Scrum is only one arrow in the quiver, Scrum as a great vehicle for improving people’s professional lives.

On Certification

Ken has readily admitted that using C-word in the beginning of the Scrum Alliance has resulted in a complete and total mess. Primarily because certification involves a demonstration of something, typically knowledge. Unfortunately that requirement was never required to obtain a Certified Scrum Master “credential”. We can all recognize this is not a credential in it’s current form as offered by the Scrum Alliance, but merely a roster of class attendees. ‘Nuff said.

I also see there is value in demonstrating knowledge through a genuine certification process. For example, we all take driver’s tests. I look for ASE (Automobile Service Excellence) certified mechanics to work on my car. I trust a BMW certified technician to work on my motorcycle. What is a university diploma if not a certification award?

In almost any field, it is impossible to certify competence. Certification can demonstrate knowledge, but not ability, talent, experience, or wisdom. It is exactly for this reason that I never got hung up on a college degree as a hiring manager but went to great lengths to try and know a person. A college degree did show me what that person experienced, though, because I basically know what it means to get a university degree. There is value in that credential because it had to be earned.

Given the current state of the industry, there are 3 logical progression paths for Scrum certification.

  1. Change nothing and remain a mockery.
  2. Drop the certification word and idea altogether.
  3. Add meat to C-word by actually testing for it.

Scrum.org is taking the 3rd option and I am going for that ride. My reasons include:

  1. Working for change within a system is typically far more effective than pushing from the outside.
  2. Scrum as an idea continues to help teams and individuals improve.
  3. I respect the people involved in the stewardship of Scrum.org.
  4. “Certification” means a demonstration of knowledge by a taking a test in this crowd. Not perfect, but a good start.
  5. I believe the teams and other trainers I work with have the best interest of the craft at heart. Really.

On The Future of the Craft

In the last 60 years of programming computers, we have have learned that developing in small batches and delivering incrementally works. Scrum gives us a model for doing exactly that, and it will still be helping teams until the next great thing is found. Now that I know Ken a bit better I can say that Scrum itself will likely evolve to incorporate whatever works better.

Ridiculing certification or Scrum may well be the in-crowd thing de jour, but I see that Scrum addresses a core need in our industry, and genuine certification has its own benefits. Lean offers some wonderful directions for us, and shows all the signs of providing a path forward. But, let’s not forget that Scrum is an inherently Lean idea.

But I digress.

Another time, perhaps.

If Self-Documenting Code is the What, Unit Tests are the Why

It is not enough to just write code that is clean and self-documenting.  Eliminating comments and replacing them with clear variable and method names, will tell the reader of your code clearly what it is doing, but it will not properly express why it is doing it.

If you recall from my original post on Elegant code, I stated that elegant code is:

Something that is simple yet effective, delivered with grace.

By writing self-documenting code we are hopefully able to achieve some of the aspects of simplicity and grace, but it tells us nothing about the effectiveness of the code.

Bringing the dead to life

zombies

Throwing out comments is a debatable topic.  It raises quite a bit of controversy, as you can see from some of the comments on my post about the subject.  The reason, perhaps, that so many people are so staunchly opposed to the idea, is because they are worried, and rightly so, that throwing out comments will reduce the documentation of what the code is doing.

Think about that for a second.  Is documentation of what the code is doing important?  It better be.  What we are trying to accomplish by writing self-documenting code instead of comments, is to take the vessel, which is the code itself, and make that same vessel be the documentation of what the code does.  It is by doing this that we creating a living representation of the functionality of the code.  Comments are akin to a dead representation of the functionality of the code because they do not change automatically with the code.

I diverge to talk about comments so that I can draw a parallel to another form of documentation which is valuable and should be considered necessary.  That other set of documentation is requirements.  Have you ever written requirements documents?  Have you ever captured requirements from the customer and put them into UML diagrams, or perhaps plopped them into some templated Word document, which is never updated again, and no one reads, because no one trusts it is correct?  Wouldn’t it be great if we would replace that dead document or set of documents, with a living one?

The good news is we can, and some of us have.  And I fully expect this recommendation will be just as controversial, because I am suggesting that we don’t have to write low level specifications of the system at all, instead we can write good unit tests. When we do this, we are taking a dead form of document and bringing it to life.

Unit tests tell us why and how our code is effective

If

i = 5;  // Widget count defaults to five.

is replaced with

widgetCount = DEFAULT_WIDGET_COUNT;

then

1.1.5 When a new widget is created, the widget count should be increased by one.

is replaced with

public void When_NewWidgetIsCreated_Then_WidgetCountIsIncreasedByOne()
{
     WidgetFactory.Count = 3;
     WidgetFactory.CreateNew();
     assertEquals(4, WidgetFactory.Count);
}

The unit test is replacing the low level requirement, which is somewhat open to interpretation and untied to the code, with an absolute and completely tied-to-the-code, specific requirement.

The unit test is telling us why our code is built the way it is.  It is telling us what requirement that particular structure of code is trying to address.  It is telling us what end result that code is trying to accomplish and what the expectations are on the results of a given condition.

The unit test is telling us how the code is to be used in order to achieve its result.  It is serving as a living reference to the syntax and use of the code we built.  Instead of providing instructions on how to use our API, we are providing a living example, and proof that it will indeed work. Examples are often much more effective than instructions anyway.

What exactly are we replacing?

At this point we are not trying to replace all the documentation of the system (at this point).

With self-documenting code, we sought to eliminate comments, and replace them with good variable and method names.

With unit tests, we seek to eliminate technical documentation and requirements of the system, that is targeted at a developer or technical audience.

We are NOT trying to eliminate high level specifications of the system, or use cases. Unit tests DO NOT replace that kind of documentation! We will talk about replacing that kind of documentation when we talk about automated functional or system tests.

Simply put, any kind of documentation that you would give to a developer so they understand how the system works and how to use the code or APIs, can be easily replaced by writing good unit tests.

Our simple goal is to replace the “dead” documentation (meaning that it does not update automatically with the thing it is documenting), with “living” documentation in the form of unit tests.

One last point.  I want to be clear that I am not saying that having unit tests automatically results in the ability to replace technical documentation.  Just like having long-named methods and variables doesn’t automatically replace the need for comments.  It is a matter of the quality of the unit tests that are written, just as it is a matter of the quality of the self-documenting code.

As always, you can subscribe to this RSS feed to follow my posts on elegant code.  Feel free to check out my main personal blog at http://simpleprogrammer.com, which has a wider range of posts, updated 2-3 times a week.  Also, you can follow me on twitter here.