Technical Currency Metric

I was recently presented a rather interesting question, ?Are our products technically current??

The easy answer is to look at something like whether or not our web applications will run on the latest version of IIS and whether or not our applications perform properly under the latest browser versions. That?s the easy way out though.

It turns out to be a far more layered question as you begin asking these sorts of follow-on question.

1. How far back must we provide functionality to our clients? Do we still support that version of SQL Server, Oracle, IIS, or Apache?
2. How much time between a new product releases (say, Internet Explorer 8) and our public support for it? 3months, 6 months, a year?
3. How much customer adoption must a given technology have before we try and exploit it with our products? Think SilverLight here. Obviously this will become a winner, but when do we join the band wagon?

Goals

So in pursuit of this question, I sought to create a way to measure ?technical currency?, along with my boss (our CTO). Here is what we are after.

• Define a ?technical currency? measurement (TCM) used to assess products as part of their product management dashboards
• Keep the TCM methodology consistent over time for the benefit of managing products and product lines
• Keep it simple, meaningful, and straightforward (ie, no 100 question surveys)

What We Found

Several industry methods for assessing technology were examined and found to be too complex or not relevant.  Specifically the following were examined:

Technical Readiness Level (TRL)

A method of classification used by many US Government agencies and global companies to assess the maturity of new technologies.  For our purposes this methodology is too focused on the future and does not assess current or past classifications and hence can not be used to assess a product as ?lagging?.  See http://en.wikipedia.org/wiki/Technology_Readiness_Level

Gartner Hype Cycle

A classification of the maturity of technologies and how far they are from mainstream acceptance.  The Hype Cycle is a five step range from ?Technology Trigger? to ?Plateau of Productivity?.  Too proprietary and does not assess current or past classifications.  See http://en.wikipedia.org/wiki/Hype_cycle

Chasm Group Classifications

A popular mid-90s book called Crossing the Chasm detailed a technology adoption scale of Innovators, Early Adopters, Early Majority, Late Majority, and Lagards.  It classified Innovators and Early Adopters as Early Market and  Early Majority and Late Majority as Mainstream Market.  There is no specific product assessment methodology defined.  See http://en.wikipedia.org/wiki/Technology_adoption_lifecycle

A Simple Proposal

What I really think is that our TCM:

• Is a reasonable thing to try and quantify. This should be measurable and meaningful.
• Needs to be specific to our industry and our customer base, not latest interests of developers.
• Needs to be deliberately managed. That means that when we are out of an agreed upon compliance level, fix it.

It is typically reasonable to be technically compatible one version back on any given vendor?s product, and it is also a good idea to be looking forward to the next version to come, whatever that is.

This gives us a 3 tiered model to look at current, previous, and next versions of things to be compatible with. The v.Next versions are versions made public by the vendors. If a technology has no official ?announced? upcoming release version, we can assume current to be the latest version.

Choosing the technologies to monitor is crucial. I propose adding a technology to the list if you are currently using it, or if you are planning to release a feature that exploits it in an upcoming release.

Here is a simple table laying out a few technologies critical to the product.

Let?s look at some of the columns in this table:

• Internet Explorer is on v7 today, but we know for sure that v8 is on the horizon, therefore, it is included.
• IIS 7 is pretty much the latest release of IIS. We don?t know yet what the next version will really change, so there is no way to start an initial to ?get compatible? with it.
• .Net 3.5 is pretty much the same story
• SQL Server and Fire Fox are recently released products with no buzz about upcoming versions.
• WCF could be considered part of .Net 3.5, but maybe its use is important enough in our product to explicitly call it out.Also, since it it really at v1.0, it takes up the whole column There is no genuinely public v.Next yet.
• XForms is only at official v1.0, but a new version is in recommendation phase fro the W3C. This means we have enough information about the next version to start looking at supporting it, so it makes the list.

As for scoring, a product gets 3 points from each column. We might want to score our clients to see where they are and then score ourselves to see if we score similar to them. Looking at this fictitious product:

This product scores 14.5 out of a possible 21 points, given a half point for the split for the XForms. Now, does this mean we are technically current? Not really, but it does give us a breakdown of particular attributes to watch for. For instance, what if there were no green cells in a particular column? Obviously we would need to start focusing a bit on improving that situation.

Also, if we are scoring at 4 out of a possible 21, that starts to look a bit more troublesome. The real value is in trending because a declining TCM over time indicates we are lapsing on technical currency.

Is this crazy?