One Thing DevOps Does to Scrum Teams

Yes, DevOps has fundamentally changed the way we operate in a good and healthy way. Development teams being part of the care and feeding of the systems they produce increase quality and will eventually automate the CICD pipeline as close to production as possible, if not all the way.

Even so, many of the Scrum teams I work with feel overwhelmed and even distraught about the work they don’t get done during the Sprint on their product. This is most common among teams who assume ad-hoc or interrupt work will not disturb their capacity to get work done on their core product.

There are 2 common sense ways to deal with this problem.

  1. Pull less work during Sprint planning.
  2. Pull less work during Sprint planning.

Most teams come to this realization on their own, but then struggle with the minutia of how to track the 2 types of work they are now responsible for: operations work and product development.

There are 2 main ways I have seen teams mitigate this tracking of work. One looks like the image below, and teams associate all operational work with a reserved User Story for just this purpose.

Scrum Board

These teams will often pre-size the Ops User Story based on previous experience of how much operational work comes into the Sprint vs. product development.

I’ve seen examples of operational items with up to 50% of the team’s capacity reserved. This is a team in trouble. They are on the verge of becoming an operational team that no longer adds features to a product and thus no new value to the customer.

At any rate, in my experience most teams settle into about 10% of their time being reserved for operational duties in DevOps environments. This informs the size of the operational Sprint item during Sprint Planning and tends to work well enough.

The thing I don’t care for in this model is that it doesn’t shine a bright light on the operational work, which is, in fact, bad. If we had no operational work, after all, we would be adding product features.

Since we are focusing on the tracking mechanism, another other way I have seen teams visualize the operational work they must address is through a fire lane. A fire lane may take the form of the board we see below.

Fire Lane

I far prefer the fire lane model as does not obscure the operational items and makes them very visible. This can lead to pattern recognition. “We need to solidify our integration testing,” or “We need to close the gap on automated container deployments.”

This technique makes each and every item visible to the team and can be examined during retrospectives, potentially causing a team to expand its definition of done.

While there are undoubtedly many other ways to track operational vs. product development work, the technique to do so should draw a very clear distinction between the two and highlight operational work in such a way as it can be mitigated over time. One step at a time, just like paying down technical debt.

Leftover Work in Scrum

I recently received an email from someone who had taken one of my Scrum courses on Pluralsight. He asked:

  1. For the next Sprint, how is this unfinished User Story/Documentation task handled and is the User Story re-estimated for its remainder to arrive at Story Points?
  2. If the User Story is not considered for Sprint 1 Velocity given that it is not complete, and if its considered entirely in the second sprint when complete, this will skew the Velocity. As organizations love to measure Velocity and would like this to be fairly consistent, it does not help. Wouldn’t it help to have the story points prorated for Velocity discussions? If Documentation would have taken 2 % (we can use task estimations in hours to find out the relative contribution), we can consider 98% of the story points for velocity. What do you think?

WHEW

This was my reply to this poor soul who is clearly being beaten down by a manager or someone else who doesn’t understand what this whole velocity thing is all about.

I do not advocate for re-estimating the work. Just leave it on the board and let it finish, taking the Story Points in the next Sprint. It will all come out in the wash. Just try to limit this by learning to plan better to your team’s capacity.
 
Trying to arrive at a consistent velocity is a dysfunction. Take the averages and don’t work about Sprints being consistent. What happens when people go on vacation, become sick, etc.? Velocity is a historical record and lagging indicator. It is a poor predictor as a leading indicator because of the reality of life.
 
I hope this makes sense. In short: Don’t let the process beat you to death. Let it flow and don’t worry so much about perfection 🙂

Agile’s Coming of Age

Now that the term “Agile” is sufficiently compromised as to be near meaningless, Agile Software Development is old enough to stand on its own, make its own business case, and demonstrate its value. But it still isn’t a mature adult. Agile Software Development is a hormonally unbalanced pre-teen with ugly spots, occasional outbursts of irrational anger, and the promising potential of smart-assed intelligence.

Giving birth to Agility and parenting it to pre-pubescence was a miraculous feat. The thought leaders who brought us the manifesto and subsequent culture shift deserve our thanks for seeing the need and creating the right message at the right time. Thanks to these revolutionaries Agile is not a footnote, but the most promising path forward to improving our profession.

One focus of the last 10 years of agile discussion has been, “There are better ways to develop software.” While the state of spaghetti code in the universe is still a big problem, progress has been made in this area. It is no longer radical to think in terms of test-first, pattern-based, or *-driven. Many teams are just acting more professionally, and that’s a wonderful thing.

The other primary focus of the last 10 years has been teaching technologists to actually speak human. You know, with emotion and stuff. And there’s great news; as a profession, we developers aren’t as jerky now as we were 10 years ago. See? It’s working.

The Agile Consensus

I get to see many implementations of Agile Software Development. Not surprisingly, most teams out there aren’t living the dream, but are actively trying to improve. Indeed, recent studies and surveys have noted that projects using Agile methods now outnumber plan-driven, or waterfall, projects.

Sam Guckenheimer calls this change the Agile Consensus in his recent book with Neno Loje (disclosure: I was a technical reviewer on this book). The idea behind the Agile Consensus is simply this: Agile won. Creating software with plan-driven techniques obviously falls short of the advantages of developing with a focus on humanity and exploiting shorter feedback cycles.

Bottom line: Agility has crossed the chasm and is no longer only the domain of developers. The enterprise wants big-A Agility.

Not Crazy Anymore

Whether or not a given practice is crazy or “edgy” depends on who you talk to. Organizational acceptance of a given practice is typically rooted in the values that practice supports, and the practices here seem to have reached a level of general acceptance in our industry. Mostly.

Daily Team Meeting

Whether you call it the Daily Scrum, the Daily Standup, or simply a daily team meeting, the practice of a quick, informal team meeting held at the same time and place each day has really caught on. The reason for this is simple; when done well, the daily team meeting helps teams be more productive and improves situational awareness in complex environments. And regardless of what kind of software development you do, odds are it is fairly complex.

The daily team meeting is one of the most misunderstood and poorly executed practices mentioned in this article, but I digress.

Bottom Line: Daily Team Meetings are commonplace and useful. They aren’t considered crazy by most teams anymore.

Sprints

Sprints, or iterations, are simply short periods of dedicated time within which teams will deliver working software. These time boxes in which teams agree to deliver something have changed the way we think about long, death march projects. Many teams know that progressing toward the broader goal of releasing software is often best managed by delivering working software all the way through a delivery pipeline, with increasing amounts of functionality each time.

Businesses leaders often love Sprints because Sprints are an obvious way to manage risk. More on this later.

Bottom line: While there are other ways to organize work, doing it in small batches within a Sprint of 30 days or less is a model that has proven itself time and again.

Test as We Develop

Lean thinking encourages “testing at the point of work”, which is a broad concept we’ve apply to software development to derive TDD, BDD, ATDD, and other forms of simply proving that software works as we create it. We’ve learned along the way that we don’t even need to think of this as a verification process. Test-First practices have proven to not only help developers build the “right” software, but to build the right software well.

Not every developer has drunk this Koolaid, but most understand the basic value behind Test-First as either a design tool or a verification tool. Those of us who really drank deeply see Test-First development as the de facto way to write code. Sure, I’ll bang out a quick shell script without an accompanying automated test harness, but that’s a simple fit-for-purpose decision.

Bottom line: When making software that needs to be work right and be crafted well, many developers see Test-First practices as indispensable.

Deliver Frequently

Delivering working software frequently allows development teams to actually deliver something, and that’s half the battle. Frequent delivery of working software enables the most valuable feedback loop in software development. The conversation around this feedback loop is simple and sounds like this:

“Here’s what we made. Here is how it works. What do you think?”

And then comes the tricky part – Actually listening to the response. This helps teams build the right thing next. I loved watching eBay a few years back as it changed its color scheme and skin layout gradually over a few months. They design a destination look and feel and migrated to it with frequent delivery of small changes in their UI. This allowed the product to evolve, rather than re-release. This is how software delivery is evolving and the most extreme form is called “Continuous Delivery.”

Delivering frequently may sound a lot like “Sprints”, but there is more going on here. Sprints are simply a forcing function for frequent delivery. There are other ways to pull it off. Frequent delivery all the way to customer feedback can change the way a company engages its customers and plans its strategic moves.

Bottom line: Frequent delivery is craved by executives for the business advantages offered and by technical teams because it makes actually shipping ubiquitous.

Still Crazy

While some Agile practices have crossed into “just plain old good ideas,” many are still seen as edgy, or extreme. Despite evidence that these practices offer real value and better alternatives to traditional thinking, the old ways of looking at the world are just so ingrained that these practices provide fodder to skeptics.

Pairing

No technical practice has drawn more fire than Pair Programming. Hard data has begun to emerge about the practice of pairing, and all that data shows (to varying degrees) how pairing creates higher quality and simply better software. A paper

There are also a ton of human advantages, like increased learning, knowledge sharing, and removing single points of failure within a team.

Why then has formal pairing been relegated to the domain of roman sandal wearing hippie agilistas? Most development team leaders or managers simply see pairing as an investment of two people doing what one could accomplish. I won’t try and convince you otherwise in this article, but I will mention this:

Barry Bohm has made a very distinguished career of studying software development. In Balancing Agility and Discipline, he asserts that 60% of all software defects in production could have been caught with a peer review. Pair programming is continuous peer review. You do the math.

Finally, most developers treasure their alone-time with the code. Sharing the way I approach problems or write code can feel like a job interview every day. That kind of scrutiny can feel very uncomfortable unless I am in an environment of absolute trust. That ties the success of this technical practice to the culture of the team and company.

Bottom line: Pair programming is still seen as eXtreme, and the transparency it forces can terrify many developers.

Funding Alternatives

Companies spend a lot of time and energy developing golden plans for the next year. Strategic planning is a dependable activity of middle-management in those months counting down to the end of the current fiscal year.

We know good and well that we can’t predict the evolution of a software project beyond a few months in most thriving businesses. Change just happens. Why then do we persist in thinking Big Funding Up Front is any different than Big Design Up Front? Some are making inroads with models of T&M funding, fixed cost, adjustable scope, and other techniques like incremental funding. However, for the most part we remain stuck in annual funding models because business Agility, the real promise of Agile, remains elusive.

Bottom line: Software development projects are still funded when we know the least about how we’ll be spending that money.

Strategic Iteration

While Sprints, or iterations, are very popular on the operations side of the house, few companies see them as the strategic advantage they really are. Sprints are loved by the business because they reduce risk, but actually refining the scope, plans, and functionality based on an iterative feedback model is a foreign idea. Iterative delivery provides a regular cadence that can be interpreted as “milestones” by traditionally trained most project managers.

The innovation companies could have with regular Sprints is lost because of the aforementioned Big Up Front Funding that causes Sprints to be seen as a tool of operations.

Bottom line: The potential value of iterative incremental teams is being wasted by a determination to fund fixed-scope projects up front.

The Next Challenge

Professionalism

The profession of software development is reflecting upon itself right now and the question of what it means to be a software professional is coming to a head. The craftsmanship movement has a genuine toehold with many introspective developers; Universities are actively looking beyond computer science programs to fill the supply void of industry; and my mom thinks she’s “writing code” when her excel macro runs without error.

Writing solid code is now table stakes for being a software professional. The expectations we have of true professionals are becoming appropriately greater. As technology matures and abstractions go higher, the productivity of development teams should be through the roof. Yet, it isn’t necessarily the case and hiring organizations are desperate for some way to assess prospective developers en masse.

One desperate attempt at identifying professionals is the ridiculous history of Scrum certification. Certification teases with the allure of simply trusting a credential. Unfortunately, this isn’t working for any known certification yet, university, private, or otherwise.

Bottom line: Professionalism in software is finally being demanded by those creating it, and by those asking for it.

Maturity

Just like the allure of hiring a professional, the temptation of the perfect development process is just too tempting for the ignorant to ignore. The success of simple frameworks like Scrum and Kanban provide just enough structure to get things done, without providing prescribing specific practices. That scares plan-driven organizations that value control over creativity.

To get real traction with Agile methods means getting not just permission, but support; and support means money. Before bureaucracies spend money, they want assurances and guarantees. False ones will do; look at how well RUP and MSF sold.

Providing any compelling story for change requires supporting data. The willingness of good leaders to instigate and support true change will start with the end in mind. While the end state of an Agile transition can’t be predicted, case studies and measurements of established Agile teams are the catalysts for getting Agile transitions started.

The demand for reassurance will drive development of tools like assessments, maturity models, and formal adoption programs. As older and more established industries explore Agile, these tools will be in heavy demand by those wanting to make data-driven decisions.

Bottom line: Agility is moving into more mature organizations and Agile itself will need more accessories of maturity.

Introducing Professional Scrum Foundations

Today marks the launch of the Professional Scrum Foundations course, from Scrum.org. I am excited to see this course offered because:

  • I wrote it, in collaboration with Ken Schwaber
  • I have seen it be a great way to bootstrap a better work life for teams who are NOT living the dream
  • It is the first step in my path of answering the following question:

Me: Most Scrum teams are struggling. They know what Scum is, but not how to use it well. That doesn’t do anything but make Scrum seem like it doesn’t work.

Ken: Agreed. What are you going to do about it?

What This Is

Professional Scrum Foundations is exactly that. A way to set the foundation for Scrum with your team by practicing for 2 days. This isn’t just a talking head at the front of a classroom, but a very hands-on experience for participants that has them create software *gasp* in teams.

Knowledge is not enough to be successful. In addition to knowledge, I need experience. At the intersection of those 2 things lies skill. That’s the focus of this style of teaching, which is heavily influenced by the ideas set forth in a great book that all teachers should read, Training from the Back of the Room.

Skill = Knowledge + Experience

And so, I wanted to make something that provided skills, not just knowledge, as has been the case for most Scrum training I am aware of to this point.

Further, I liken this to faith vs. religion. Traditional Scrum training tries to sell faith. “Believe what we believe and your life will be better.”

I say “bollox”. Teach me how to behave and I’ll try it. If I find that that makes a better life, I may start to believe.

This class teaches behavior, not faith. That’s for you to decide on your own after you try out the lifestyle.

Who Cares?

This is specifically for people unfamiliar with Scrum, and causes attendees to understand the pressures Scrum often applies. It also proves the tendency of humans to cut quality in the face of that pressure, but more on that another day. I went through great lengths to ensure executives, technical writers, developers, testers, and anyone else will be comfortable in the room and find value. You don’t have to be someone who writes code to participate, although delivering software is the focus.

If you are someone who already understands Scrum, self-organization, and empiricism, this isn’t for you.

Why I Did This

As I stated in an earlier post, my early experiences with Scrum (2003-2004) were challenged by understanding theory and principles, but having little guidance on how to enact them. In other words, I had no skill.

A simple example is our early attempts at setting a Sprint schedule. Our teams would target a set of features to deliver from a Product Backlog, and then determine a Sprint length that would accommodate the work of delivering those features. Of course, this is a terrible practice for many reasons and one that undermines many of the advantages one might get from Scrum. Our training (2-day CSM at the time) did not provide guidance around this all-to-common mistake. We eventually learned not to do this, but the source for the learning was blogs, books, and many mistakes. No one simply told me, “that’s a bad idea and here’s why.” I still feel a bit miffed that the training we received (and spent a lot of money on) didn’t better prepare us for the reality of life with Scrum.

Obviously there are more resources today for teams looking to implement Scrum, but I still find teams making the same mistakes I did. In fact, I spoke with a team yesterday making the variable Sprint length mistake and not to belabor the point, but it is 2011. C’mon.

It is practices like these that make this statement true:

“Of the teams claiming to use Scrum, it is likely true that only 10% or so actually are.”

The simple fact is that until you’ve done something “by the book” for awhile, you aren’t competent to claim that the body of knowledge in the book isn’t sufficient to your needs. This is true of learning to drive a car, using software design patterns, cooking a sophisticated dinner, and using Scrum to manage work. Accordingly, Professional Scrum Foundations derives it’s content directly from the Scrum Guide, available at Scrum.org.

What Next

In addition to a commitment to blog more, I am teaching Professional Scrum Foundations in Boston in July. There are other trainers who are also teaching the class, and you can see that here. If your team is interested, hit me up.

Finally, I am working on other projects with Scrum.org that will include more classes, but also other things that don’t involve sitting in a classroom.

Dear Silver Bullet Seeker

Dear Silver Bullet Seeker,

I understand that you want a guarantee that using a new development process will make everything better. Who wouldn’t want to claim great improvements with little to no effort on your part? Unfortunately, your team is likely comprised of human beings, which makes behavior non-deterministic, like your own. This means there is no way to accurately predict how much this new agile thing you are trying will work, if at all.

Your team can certainly understand Scrum easily. How it works, what steps to perform, and the tools the Scrum framework offers. As a manager, you can even learn how to work with the Team. This can be a life-changing event in your organization. It can also be another tangent you take for a year or so until you ultimately discard Scrum in favor of Yet Another Potential Silver Bullet. We know, after all, that it takes about a year for the average organization to admit that Scrum will not do their jobs for them. Then they try something else.

Teams and organizations that actually modify their behavior causing Scrum to thrive report increased value being delivered to their customers, deliberate progression in accomplishing goals, and improved quality in the products they ship. Teams dedicated to genuine change report dramatic changes in these qualities and improved culture within the broader organization. Best of all, Scrum-driven organizations report a renewed ability to proceed deliberately rather than reactively in their business.

Teams and organizations that merely declare Scrum is in place and continue exhibiting old behaviors are far more common than those who actually commit and change. This means very few teams ever realize the benefits of Scrum. It isn’t that Scrum itself is hard, but changing ingrained behaviors is. Indeed, more organizations misuse or fail with Scrum than succeed. This isn’t surprising because Scrum asks everyone involved to:

  • Work in a collocated or at least highly connected space
  • Self-organize to find the correct answer
  • Avoid interruptions with extreme prejudice
  • Commit to new behaviors
  • Jettison ego in favor of rational decision making
  • Trust that everyone will do their jobs to best of their ability
  • Follow the rules we all agreed on in the first place

So, Silver Bullet Seeker, just pass this all by. In fact, go ahead and criticize the process while pretending the fault of failure is not your own. That works for most people.

Sprint Review. Cheater, cheater, cheater!

I have just been accused of cheating at last week’s sprint review. Let’s see what you think.

The way our sprint reviews work, everyone in the company gets to rotate between teams who show their work in 15 minute segments. Picture a science fair with parents moving between booths and spending 15 minutes at each one. This has been a hugely successful format because teams get to have more intimate conversations with people seeing their work. Discussions are more interactive because there is a smaller group watching the demo and folks are more likely to speak up.

From the presenter’s point of view, this is showing the same thing 4 times in a row. This is okay, but gets a bit tedious. Worse, because each group brings up individual issues there may be things about the feature that get skipped in a given 15 minute segment.

What I Did

I recorded a feature walk through on Camtasia and played it during sprint review. Four times. This left 3 minutes for discussion, but with the transition times, it pretty much took up the entire 15 minutes.

The most common feedback was, “Cool idea. It was too long.”

Noted. And, I think the idea of doing a complete walk through as a recording has merit because it sort of forces that all areas I want to show will get shown.

It is important to realize that I made the recording a mere hour before Sprint Review. Thus, the software was real and did do what I showed.

What I Will Do Next Time

  • Limit the Camtasia video to 5 minutes and leave plenty of time for discussion.
  • Be more animated. No monotone voice.

Your Thoughts? Am I a big, fat cheater?

Electronic Scrum Boards with Jeffrey Palermo

I found myself eating lunch with a group that included Jeffrey Pallermo one day at Tech Ed. We were discussing our favorite sessions at the conference and I noted my interest in the new TFS Scrum process template from Conchango and the electronic Scrum board they will soon be bringing to market. I actually posted on it here.

Jeffery weighed in that he prefers low tech note cards on the cork board because of the high tactile nature of using them.

I pointed out that our teams have huge electronic dashboards for use during their daily Scrum meetings, and the dynamic nature of the Scrum board I saw from Conchango would work similarly to a a cork board experience.

“Yeah,” someone interjected, “and you could get a touch screen to touch the work item cards. Touch it, move it, hold it and be mad at it, whatever.”

“That’d be sweet,” I thought.

Jeffrey responded, “You just told me how to spend a whole lot of money so the Scrum master doesn’t have to do 15 minutes of data entry per day and write out the cards.”

Pause.

Pause.

Damn it, he’s right.

The geek in me really wanted to like the bigger, more expensive, technical solution to this problem. You know what, though? The Scrum Master is there to facilitate the team’s success and to be the gate to management.

I will freely admit right here and now that I have been slacking. I was recently working as Scrum Master for a team back home and I taught the team how to use TFS to manage their own work items. I also asked them to update their time remaining on SBLIs before our daily Scrum. You know what? That was wrong of me. That was flat out laziness. They even had a cork board in their work space.

I resolve right here and now that when I get home I will make sure the team has their Scrum board. I will make sure the team doesn’t have to deal with time tracking. I will cover the administration of the team; that’s my job as Scrum Master.

The team needs frictionless tools. They’ll get them. It’s important.

If you are thinking I am declaring project management or cost accounting tools unnecessary, you are dead wrong. The transparency into the team and the overall planning of features and work items is hugely important. I just know it is the job of the Scrum Master to deal with it, not the team’s.

Frankly, I am willing to bet this kind of data management model will result in more accurate and detailed reporting anyway.

Lesson learned. Thanks, Jeff.

Conchango’s Scrum Process Template 2.1 for Team System

I went to my favorite session of Tech Ed today, but it was for personal gratification reasons that I enjoyed this so much. I am so happy about the work coming out of Conchango that I could pop. Why? Because when I get home I have the task of implementing TFS 2008 for my organization using the Scrum process template. I was nervous about this because of the tremendous amount of customizations I have been making to the Scrum work items in the prior version of the template I had been working with.

Colin Bird, CTO for Conchango, showed the 2.1 version of the process template today. It solved almost 100% of the problems I had been struggling with in my customized versions of the template.

  1. Support for different teams on every work item type. This is HUGE because it allows for all teams to pull work from a single backlog.
  2. Burn down reports for sprint, product, and release, that all split against teams. This is the biggy. If you haven’t tried working with reporting in TFS, you don’t know how HUGE this is. Awesome.
  3. Code churn reporting
  4. Value flow diagrams
  5. Elimination of the Sprint data element in a work item in favor of using a release model based on iterations. Thank you.
  6. Bugs as work items that appear on the product backlog

Honestly, there is very little left to do! I can only think if a few things I will be adding to this already existing functionality.

  • Epics and Themes as work items and reports to support them. Even this will get better in Rosario with hierarchical work items.
  • Tagging

That’s about it.

img006 Colin also showed us an awesome new product that Conchango will be charging for and I would happily pay for. It is a WPF application that is a Scrum board working right off of TFS directly. There are other attempts at solving this same problem in the open source world, but this one is fully baked and soon to be available. It looks very usable. I can’t wait to see this thing in my shop. It will help the daily standup so much, and the overhead of making out physical cards will be a thing of the past.

I took a picture of it with my cell phone, but I don’t think I was supposed to do that. Oh, what the hell. Here it is.

You can actually drag the cards around and they change state and work remaining. If Colin weren’t English I would have hugged him. And then tickled him.