Agile’s Coming of Age

Now that the term “Agile” is sufficiently compromised as to be near meaningless, Agile Software Development is old enough to stand on its own, make its own business case, and demonstrate its value. But it still isn’t a mature adult. Agile Software Development is a hormonally unbalanced pre-teen with ugly spots, occasional outbursts of irrational anger, and the promising potential of smart-assed intelligence.

Giving birth to Agility and parenting it to pre-pubescence was a miraculous feat. The thought leaders who brought us the manifesto and subsequent culture shift deserve our thanks for seeing the need and creating the right message at the right time. Thanks to these revolutionaries Agile is not a footnote, but the most promising path forward to improving our profession.

One focus of the last 10 years of agile discussion has been, “There are better ways to develop software.” While the state of spaghetti code in the universe is still a big problem, progress has been made in this area. It is no longer radical to think in terms of test-first, pattern-based, or *-driven. Many teams are just acting more professionally, and that’s a wonderful thing.

The other primary focus of the last 10 years has been teaching technologists to actually speak human. You know, with emotion and stuff. And there’s great news; as a profession, we developers aren’t as jerky now as we were 10 years ago. See? It’s working.

The Agile Consensus

I get to see many implementations of Agile Software Development. Not surprisingly, most teams out there aren’t living the dream, but are actively trying to improve. Indeed, recent studies and surveys have noted that projects using Agile methods now outnumber plan-driven, or waterfall, projects.

Sam Guckenheimer calls this change the Agile Consensus in his recent book with Neno Loje (disclosure: I was a technical reviewer on this book). The idea behind the Agile Consensus is simply this: Agile won. Creating software with plan-driven techniques obviously falls short of the advantages of developing with a focus on humanity and exploiting shorter feedback cycles.

Bottom line: Agility has crossed the chasm and is no longer only the domain of developers. The enterprise wants big-A Agility.

Not Crazy Anymore

Whether or not a given practice is crazy or “edgy” depends on who you talk to. Organizational acceptance of a given practice is typically rooted in the values that practice supports, and the practices here seem to have reached a level of general acceptance in our industry. Mostly.

Daily Team Meeting

Whether you call it the Daily Scrum, the Daily Standup, or simply a daily team meeting, the practice of a quick, informal team meeting held at the same time and place each day has really caught on. The reason for this is simple; when done well, the daily team meeting helps teams be more productive and improves situational awareness in complex environments. And regardless of what kind of software development you do, odds are it is fairly complex.

The daily team meeting is one of the most misunderstood and poorly executed practices mentioned in this article, but I digress.

Bottom Line: Daily Team Meetings are commonplace and useful. They aren’t considered crazy by most teams anymore.

Sprints

Sprints, or iterations, are simply short periods of dedicated time within which teams will deliver working software. These time boxes in which teams agree to deliver something have changed the way we think about long, death march projects. Many teams know that progressing toward the broader goal of releasing software is often best managed by delivering working software all the way through a delivery pipeline, with increasing amounts of functionality each time.

Businesses leaders often love Sprints because Sprints are an obvious way to manage risk. More on this later.

Bottom line: While there are other ways to organize work, doing it in small batches within a Sprint of 30 days or less is a model that has proven itself time and again.

Test as We Develop

Lean thinking encourages “testing at the point of work”, which is a broad concept we’ve apply to software development to derive TDD, BDD, ATDD, and other forms of simply proving that software works as we create it. We’ve learned along the way that we don’t even need to think of this as a verification process. Test-First practices have proven to not only help developers build the “right” software, but to build the right software well.

Not every developer has drunk this Koolaid, but most understand the basic value behind Test-First as either a design tool or a verification tool. Those of us who really drank deeply see Test-First development as the de facto way to write code. Sure, I’ll bang out a quick shell script without an accompanying automated test harness, but that’s a simple fit-for-purpose decision.

Bottom line: When making software that needs to be work right and be crafted well, many developers see Test-First practices as indispensable.

Deliver Frequently

Delivering working software frequently allows development teams to actually deliver something, and that’s half the battle. Frequent delivery of working software enables the most valuable feedback loop in software development. The conversation around this feedback loop is simple and sounds like this:

“Here’s what we made. Here is how it works. What do you think?”

And then comes the tricky part – Actually listening to the response. This helps teams build the right thing next. I loved watching eBay a few years back as it changed its color scheme and skin layout gradually over a few months. They design a destination look and feel and migrated to it with frequent delivery of small changes in their UI. This allowed the product to evolve, rather than re-release. This is how software delivery is evolving and the most extreme form is called “Continuous Delivery.”

Delivering frequently may sound a lot like “Sprints”, but there is more going on here. Sprints are simply a forcing function for frequent delivery. There are other ways to pull it off. Frequent delivery all the way to customer feedback can change the way a company engages its customers and plans its strategic moves.

Bottom line: Frequent delivery is craved by executives for the business advantages offered and by technical teams because it makes actually shipping ubiquitous.

Still Crazy

While some Agile practices have crossed into “just plain old good ideas,” many are still seen as edgy, or extreme. Despite evidence that these practices offer real value and better alternatives to traditional thinking, the old ways of looking at the world are just so ingrained that these practices provide fodder to skeptics.

Pairing

No technical practice has drawn more fire than Pair Programming. Hard data has begun to emerge about the practice of pairing, and all that data shows (to varying degrees) how pairing creates higher quality and simply better software. A paper

There are also a ton of human advantages, like increased learning, knowledge sharing, and removing single points of failure within a team.

Why then has formal pairing been relegated to the domain of roman sandal wearing hippie agilistas? Most development team leaders or managers simply see pairing as an investment of two people doing what one could accomplish. I won’t try and convince you otherwise in this article, but I will mention this:

Barry Bohm has made a very distinguished career of studying software development. In Balancing Agility and Discipline, he asserts that 60% of all software defects in production could have been caught with a peer review. Pair programming is continuous peer review. You do the math.

Finally, most developers treasure their alone-time with the code. Sharing the way I approach problems or write code can feel like a job interview every day. That kind of scrutiny can feel very uncomfortable unless I am in an environment of absolute trust. That ties the success of this technical practice to the culture of the team and company.

Bottom line: Pair programming is still seen as eXtreme, and the transparency it forces can terrify many developers.

Funding Alternatives

Companies spend a lot of time and energy developing golden plans for the next year. Strategic planning is a dependable activity of middle-management in those months counting down to the end of the current fiscal year.

We know good and well that we can’t predict the evolution of a software project beyond a few months in most thriving businesses. Change just happens. Why then do we persist in thinking Big Funding Up Front is any different than Big Design Up Front? Some are making inroads with models of T&M funding, fixed cost, adjustable scope, and other techniques like incremental funding. However, for the most part we remain stuck in annual funding models because business Agility, the real promise of Agile, remains elusive.

Bottom line: Software development projects are still funded when we know the least about how we’ll be spending that money.

Strategic Iteration

While Sprints, or iterations, are very popular on the operations side of the house, few companies see them as the strategic advantage they really are. Sprints are loved by the business because they reduce risk, but actually refining the scope, plans, and functionality based on an iterative feedback model is a foreign idea. Iterative delivery provides a regular cadence that can be interpreted as “milestones” by traditionally trained most project managers.

The innovation companies could have with regular Sprints is lost because of the aforementioned Big Up Front Funding that causes Sprints to be seen as a tool of operations.

Bottom line: The potential value of iterative incremental teams is being wasted by a determination to fund fixed-scope projects up front.

The Next Challenge

Professionalism

The profession of software development is reflecting upon itself right now and the question of what it means to be a software professional is coming to a head. The craftsmanship movement has a genuine toehold with many introspective developers; Universities are actively looking beyond computer science programs to fill the supply void of industry; and my mom thinks she’s “writing code” when her excel macro runs without error.

Writing solid code is now table stakes for being a software professional. The expectations we have of true professionals are becoming appropriately greater. As technology matures and abstractions go higher, the productivity of development teams should be through the roof. Yet, it isn’t necessarily the case and hiring organizations are desperate for some way to assess prospective developers en masse.

One desperate attempt at identifying professionals is the ridiculous history of Scrum certification. Certification teases with the allure of simply trusting a credential. Unfortunately, this isn’t working for any known certification yet, university, private, or otherwise.

Bottom line: Professionalism in software is finally being demanded by those creating it, and by those asking for it.

Maturity

Just like the allure of hiring a professional, the temptation of the perfect development process is just too tempting for the ignorant to ignore. The success of simple frameworks like Scrum and Kanban provide just enough structure to get things done, without providing prescribing specific practices. That scares plan-driven organizations that value control over creativity.

To get real traction with Agile methods means getting not just permission, but support; and support means money. Before bureaucracies spend money, they want assurances and guarantees. False ones will do; look at how well RUP and MSF sold.

Providing any compelling story for change requires supporting data. The willingness of good leaders to instigate and support true change will start with the end in mind. While the end state of an Agile transition can’t be predicted, case studies and measurements of established Agile teams are the catalysts for getting Agile transitions started.

The demand for reassurance will drive development of tools like assessments, maturity models, and formal adoption programs. As older and more established industries explore Agile, these tools will be in heavy demand by those wanting to make data-driven decisions.

Bottom line: Agility is moving into more mature organizations and Agile itself will need more accessories of maturity.