Why Software Development Will Never be Engineering
I always find it rather interesting when academics try to quantify generalized metrics about software development.
Things like: per lines of code, there will be X number of bugs.
Statements like: it has been empirically proven that “blah” affects the development of software in some way “blah.”
These are all interesting thoughts, but software development will never conform to rigid engineering principles like many other engineering practices. The longer I work in the field, the more and more I realize that software development has nothing to do with engineering. We just happen to attend some of the same math classes in school.
Building bridges
One of the main arguments I hear people make about the current state of software development as an engineering practice is based around its relative maturity to other engineering fields.
There is a huge problem with this line of thinking.
The argument goes something like this: “Engineering has been around for hundreds of years and has matured to the point where it is a measurable science, but software development has only been around in earnest for the last few decades or so, therefore it is relatively immature as an engineering discipline.”
On the surface this argument seems sounds. And certainly in 50 years software development will be different than it is now. Heck, it is much more different than it was just 10 years ago.
But, here is the problem. We build much more software than we do bridges.
Let me give you an example that came to light in another area of my interest… poker.
The poker quickening
An interesting thing happened in the poker community in the last 10 years; poker pros with decades of experience started getting beat time and time again by 19 year old poker prodigies.
From an outside perspective it looks like the poker world opening up just widened the search for talent and there it was. But, a closer examination of the evidence leads to the truth.
Before the online poker industry was born, poker was played in card rooms and casinos in states where it was legal and occasionally at small home games throughout the US.
A poker pro might play poker tournaments perhaps 100 times to 150 times a year. The sum knowledge of the poker community and the meta game around it was based on this base of professional poker players playing tournaments and gaining experience at this mediocre rate.
Once the internet poker rooms started opening up, things changed, and they changed rapidly. Now anyone could play poker from their home PC. Not only could they play poker, they could play poker tournaments at all hours of the day.
Here is a list of some of the major changes that took place:
- Hands are dealt at 60-80 hands per hour instead of 10-30
- Players can play in many tournaments at one time, 10 tournaments at once is not unheard of
- Players can analyze hand histories and historical data about their play and opponents play through the use of software
- Age requirements are out the window, technically any one of any age could be playing online
- Players can play 24 hours a day
- The professional online tournament player could easily play over 3000 tournaments in a year, while a brick and mortar pro would be lucky to reach 100 in a year.
So where am I going with all this?
The point is that as a whole more hands were dealt and more poker knowledge and experience was obtained in 1 year of online poker than probably the entire history of poker before that. We reached such an accelerated pace of play that all previous knowledge of poker became obsolete.
The strategy of tournament poker completely changed. The game evolved perhaps 500 years into the future in a matter of 5 years time.
The same has happened with software development.
Back to the bridge building
So let’s take that poker example and look at it through the lens of software development.
Let us compare the engineering maturity to software development to the engineering maturity of the engineering discipline of building bridges.
I think most people would agree that bridge building is a very mature engineering discipline but many people would argue that software development is not.
How many bridges do you think have been built in the world?
Well there are about 600,000 bridges in the United States alone, so the world figure must be at least somewhere around 10 times that number perhaps 20.
How many software programs have been written?
This is a very hard number to estimate, but lets take a rough guess based on how many software developers there are in the world.
If we say there are about 12 million software developers and each software developer has written approximately 3 programs, we can estimate that a large amount more programs have been written than bridges built.
My point is not to knock bridge building, we are pretty good at it as a whole, but rather to show that collectively, even though we have been building bridges for hundreds of years, we have probably devoted an equivalent amount of time to building software.
This line of thinking may lead you to argue back that software development and bridge building are very different. Bridge building has a fixed set of requirements that are pretty close to the the same for each bridge you build, but software development is a big open void of whims and ambiguously contradictory statements.
I agree with you 100%! And in essence that is my point.
Software development is different
And we have had enough time to realize that. Waiting for software development to gel into some kind of engineering discipline like other engineering disciplines is like waiting for water without gelatin mix to turn into Jello.
It’s just not going to happen!
In my mind it is clear that the argument that we haven’t given it enough time is just wishful thinking. The nature of software development, just like online poker, leads itself to rapid evolution.
Consider what direction software development is evolving. Is it even evolving in the direction of rigid engineering practices or is it evolving in the exact OPPOSITE direction?
Ten years ago, we tried to use UML diagrams and CASE tools to develop software. Ten years ago waterfall was all the rage. Ten years ago, we thought that ten years in the future we would have programs that would allow us to build software in the same way that CAD tools allow for building machine parts.
Not only did it not happen. It went completely the other way. Now we are all talking about Agile. Now people don’t even remember what CASE tools are. Now we are building software without trying to define the whole system in UML diagrams.
The fact of the matter is software systems are unruly beasts!
In my mind it comes down to one simple distinction. Software is living, bridges aren’t. When you are done building a bridge, you are done building the bridge.
Sure someone, probably not you, will have to come along and do some routine maintenance on it. Sure, some small things might change about it, but for all intents and purposes the work is done.
In most software development scenarios, this is not the case. In most software development scenarios, releasing V1 is not even close to the end. Sometimes V1 and V2 don’t even look that same at all. Software development is about operating on a living breathing thing and all the while keeping it alive.
The truth is, we software developers have more in common with surgeons than with other engineers.
First of all, your numbers regarding software development are just out of the hat. If you see the world as ten times larger that US, then I’ll say you’ve never seen the world.
As for bridges… they are living beasts to. They have their share of failures too. Let’s take the George Washington Bridge (GWB) that when it was completed in 1931, it consisted of only one level.
This bridge handled the traffic at that time, however, the builders could not have anticipated how much more traffic this bridge would need to carry. Today when you approach the GWB, you must decide between taking the upper or the lower level. With the bridge slowly reaching its capacity, the builders added a new level just below the existing one, to meet the growing traffic needs.
The situation at the GWB project bridge is something that occurs in software engineering quite often. Even though engineering disciplines are similar in many ways, some major differences distinguish them.
If you’d like to have a better comparison just think about house building. Comparing every piece of software a programmer can roll-out in a night of heavy coding to bridge construction is just too far out.
The majority of the article is spent onto proving how separate the world of IT and real engineering are. It’s interesting to learn about the changes of the poker game, and I must admit, it seems to be at least partially applicable to the IT world. However, the final conclusion, the comparison to a surgeon, is supported by only one sentence with no in depth looking at it and it spoils the whole article.
Online poker players can play more games then the pros, because they have more opportunity to play and gain experience, they can spend more time at it. This is not the case for software developers.
Poker players are incentivised to get better and analyse the past games, otherwise they loose all there money, this is not the case with software developers. The analogy does not fit.
The analogy does not fit.
Good sir, I think you’ll find that Engineering has been around a lot longer than you have assumed. http://en.wikipedia.org/wiki/Egyptian_pyramids
In the post you’re telling what software development is and isn’t. But you’re just proposing another ‘model’ for software development focusing on some different aspects. This model again will not be ideal. It might be good for describing some particular behavior, other behavior will not fit into this model. Software development is considered to be engineering task because like in engineering you have a problem and you should provide some technical solution for it. It may be compared to the work of a surgeon as you did, just the same way as it may be compared to thousand other things, focusing on its different aspects. The question is WHY we may want this or that comparison, what problem are we addressing with it, and the best solution for the problem can be driven by the right model for THIS PARTICULAR CASE. So what problem are you trying to solve now?
I do not agree with your statement that software development will never be engineering. I have the opposite feeling. The more I work on the field, the more I understand that it is “engineerable”. It is just a matter on how you define Engineering. On the other side, I do agree that one size does not fit all: there is no definitive and unrefutable software engineering approach. See my post at: http://bit.ly/ktS97Q
The civilized world is about 10 times the size of the US. It is a pretty reasonable estimate to think that most of the places where significantly engineered bridges would exist would be in these places. (What I mean by this is that most of the world is not densely populated and urbanized.)
Good point about GWB. I still think the amount of change in that case is much less than your average software though.
That is my exact point, software development has become so common place and easy to do from any location due to the high availability of computers and technology that the field has evolved at a very fast pace.
Consider how often the need arises to build software vs the need to build a bridge. My point is that we have had a such a surge in software development that advances that should have taken place would have taken place, and it did not push software development any closer to engineering. It actually pushed it father away.
The main issue with software is that software must correctly observe a huge number of rules which can be both arbitrary and a bit fuzzy, and it must observe them all simultaneously all the time. This “combinatorial explosion” massively increases the number of ways in which software can fail.
By “arbitrary”, I mean that many if not most business rules are not backed by laws of nature that provide a firm grounding, something that unambiguously informs the engineer that something is wrong and must be fixed.
A bridge must satisfy the same law – gravity – over and over again, whereas a piece of software must satisfy many many “laws”.
I think you might be pigeon-holing engineers a bit in terms of what they do – even civil or mechanical engineers. I’m an Industrial Engineer and most of what we do is more akin to writing/maintaining software than any physical, tangible object. We tend to work on designing/improving physical processes or systems, where we have to understand a change in one component’s effect on the entire system.
I also write a lot of software (related to process/system modeling & simulation) and find the parallels between Industrial Engineering and Software Development to be striking. By the way, where do you think the Lean tools for software came from?
I think you can call software development an engineering discipline, presuming you think like an engineer. It doesn’t matter if what the engineer is working on is a physical, tangible object, or if it’s a business process or software. It’s basically designing a system such that components in that system don’t screw each other up.
Hi Jon, some valid points you have there. I guess it depends on what you define engineering to be.
My argument is against calling software engineering the same kind of engineering that could be measured and predicted in the way that engineering projects like bridge building are. (Hence why I used that example.)What I am really saying in this post is two main points:
1. You can apply statistical analysis to software development like you can other engineering disciplines. (In that regard software development is not engineering.)
2. Software development has already advanced further than most people think in terms of evolution. If we are waiting for software development to “mature” and look more like other “mature” disciplines, it is not going to happen.
I see an engineer more like a problem solver. An engineer would go for the most convenient solution to solve a problem. Software could possibily be such convenient solution.
Secondly, it seems you are greatly underestimating the proces of an engineer. This is at least what I understand. An engineer does not just draw something on paper and the day after they are constructing the following product. An product needs to be tested multiple times (for some products on lower scale) before it can be realised.
Also there are multiple versions of products created by engineers. Do you think that hundreds years ago we had exactly the same bridges as we now have? No, we developed our skills by making new bridges and evaluating on the previous ones. These improvements are not only improvements in maths but also improvements in design and the ergonomics. And what about cars? They are also made by engineers, and the develope like crazy.
Last, I may be wrong, but I really hope that a surgeon just operates by the book and not experiment like a sofware developer does.
Thus I still define software developing as enginering.
As someone who develops software AND has designed bridges, I believe you are mistaken regarding the bridge design process and the engineering process in general. In fact, bridge design is very much like the software development process you describe. You see, every single bridge is different. Every one. Each bridge design has a very broad set of criteria that varies wildly. In fact, you might say that the bridge design process is filled with whims and and ambiguous contradictory criteria! The end product is built upon those criteria, but there is no set method or correct answer for how the final product is attained, or what the final product will actually be. Why one bridge is designed as a suspension bridge while another is designed as a truss has more to do with the ideology of the engineering team than any sort of mathematical rigor.
What you have actually done through your description of software development is very clearly describe every field of engineering. The mistake is in assuming that other engineering fields are somehow rigid “cookbook” professions. They are anything but, and the fact that so many people view engineers as human calculators is a major contribution to the overall shortage of engineers in this country.
Interesting. If engineering, (including that of a bridge), is what you describe, then of course I agree with you.
But, it begs the question then, who is coming up with empirical statistics for other types of engineering projects such as bridge building?
I don’t know much about other engineering fields, so I definitely could be convinced that software development and engineering as you describe it are the same, but that would mean that the general definition of engineering is wrong and that even engineering isn’t engineering. If you get my point.
I have written an extended critique of this article at http://esr.ibiblio.org/?p=3345
I’m not an engineer, but I’ve played one on tv (jk, married to one)
There’s another interesting aspect to your bridge analogy. While there may have been fewer bridges built than software applications, they are all available for study and improve the discipline. Like in any profession, more is learned from failures. We really haven’t had that capability in our profession. I wonder if open source will help with that.
I completely agree with Dave. The way I see it is that engineer and software are like an art because the engineers and developers create stuff. When you create stuff that never existed before, there are mistakes and no rules can be followed. It is always iterative. I have designed both hardware and software and the methodologies are very similar.
Sorry John, I did not read this before I wrote my comment. I do not mean to spam here but I have to respond.
I can give you examples from EE circuit design: we have design rules on how transistors are laid out. If the UV light shines at a slight angle or if an extra molecule of the doping material falls to the side, the pre-calculations become useless. Thus, the statistical models emerge. Notice that those statistical models are more comparable to Microsoft collecting bug stats using their feedback tool. They are indeed useful for the next revision but it does not make hardware design a calculated field.
By the way, engineering as I was taught is the art of solving problems within the given constraints. Often times the problem is not defined and never is the solution so obvious that it can be found from a book. Thus, I do agree with Dave that engineering is still still engineering:-)
By the way, great post. There topics are great to discuss as they help us understand our field better.
> I wonder if open source will help with that.
Oh, it already does. Rather a lot, actually.
This comment is a little off-topic but it is a concept I want to hammer on a bit and this looks like the perfect audience.
A meta-level comment about what you say in the beginning:
“I always find it rather interesting when academics try to quantify generalized metrics about software development.
Things like: per lines of code, there will be X number of bugs.”
First, it is not just the academics who want to quantify this stuff. Companies would/should do it too. Second, trying to define a metric does not require that the system be well-defined. In fact, one can argue that until we bring quantification into software engineering, we will not be able to make the right decisions.
The best example is medical science. They deal with a very unpredictable
system but they still quantify everything just to be able to logically
argue about stuff. All medical decisions –every single one of them– are based on probability and statistics via experiments that are often flawed by the scientific metrics. Yet, we trust modern doctors more than oriental medicine because they use the “scientific process” and at least have some evidence to support what they are doing.
I architect computer chips and we often need some metric to argue whether a particular feature in the machine will make programmer’s life easier or harder. The lack of a well-defined metric (good or bad) makes such arguments religious ego wars.
IMO adding some direction to our industry, even if it is stupid, can be better for the advancement overall.
Bridge building wasn’t always an engineering task. It was
partially art, partially (black | white) magic, partially empirical knowledge. As
recently as 150 years ago Engineer, The Bridge Builder, was staying under the
bridge when the first train was entering his Creation – it tells you something
about bug fixing…
Software engineering as a human activity is gradually
entering the stage where Software Engineers are not required to stay under the
bridge. Well… may be I’m overoptimistic, and we are not entering that stage yet.
Still, very few of us would really risk staying under our own bridges.
Bridge building wasn’t always an engineering task. It was
partially art, partially (black | white) magic, partially empirical knowledge. As
recently as 150 years ago Engineer, The Bridge Builder, was staying under the
bridge when the first train was entering his Creation – it tells you something
about bug fixing…
Software engineering as a human activity is gradually
entering the stage where Software Engineers are not required to stay under the
bridge. Well… may be I’m overoptimistic, and we are not entering that stage yet.
Still, very few of us would really risk staying under our own bridges.
Indeed, in software there are much more parameters what makes it more complex. It starts with the requirements. A bridge has to stay there and allow traffic to pass. A little piece of software needs database connections, screens, print outs…
The difference between software and the brigde builder is that you can calculate if a bridge is stable and can support the weight and withstand the wind… I never could prove a piece of software to be correct with mathmatics. Just by using it and see if it didn’t cause errors.
I agree. Now we can calculate if the bridge can support the weight of a train. There is no equivalent for software to check if it won’t crash under any circumstance. First somebody had to invent a new type of mathmatics.
I agree. Now we can calculate if the bridge can support the weight of a train. There is no equivalent for software to check if it won’t crash under any circumstance. First somebody had to invent a new type of mathmatics.
Well I am going to assume that you are neither a software developer or engineer.
Well, and it is supposed surgeons to based on science and engineering or just magic and intuition?
Hi John,
I can’t agree. You can apply statistical analysis to software development, particularly as we do for scene processing object detection in vision systems. Secondly, maturity of the tools is a red herring; software development is innately part of systems’ development, for which there are well established methods that have to be used, particularly in a regulated environment such as automotive engineering. The SW designer-as-artiste doesn’t cut much ice in that context. I’d love to spend my time exploring weird and wonderful creative possibilities, but customers demand “a system” and not “a software”, so I use system development principles.
Hi John,
I can’t agree. You can apply statistical analysis to software development, particularly as we do for scene processing object detection in vision systems. Secondly, maturity of the tools is a red herring; software development is innately part of systems’ development, for which there are well established methods that have to be used, particularly in a regulated environment such as automotive engineering. The SW designer-as-artiste doesn’t cut much ice in that context. I’d love to spend my time exploring weird and wonderful creative possibilities, but customers demand “a system” and not “a software”, so I use system development principles.
Hi John,
I can’t agree. You can apply statistical analysis to software development, particularly as we do for scene processing object detection in vision systems. Secondly, maturity of the tools is a red herring; software development is innately part of systems’ development, for which there are well established methods that have to be used, particularly in a regulated environment such as automotive engineering. The SW designer-as-artiste doesn’t cut much ice in that context. I’d love to spend my time exploring weird and wonderful creative possibilities, but customers demand “a system” and not “a software”, so I use system development principles.
There are formal semantics for software in some languages, but your point is valid in the sense that the number of variables that have to be fit in many software, are greater and more importantly they are changing more rapidly. So it is not that we don’t have equational foundations increasingly used in practical software engineering at least by those on leading edge of software engineering (e.g. category theory, monads, programming paradigms, agile practices, agile enforcing compilers, formal logic, pure lambda calculus and the referentially transparency, behavioral static typing, etc), it is that the dynamic complexity of variables in software is insatiable, because the better our software, then the more variables of demand are created by the new possibilities revealed. Software is unique from other engineering disciplines in that it is applicable to all of them.
Software is the encoding of human knowledge, which is always progressing.
There are some important equational foundations that we need to popularize so that competitive cooperation on software can accelerate.
Here’s my take on it; http://www.cs.usfca.edu/~parrt/doc/software-not-engineering.html
agree, that software biz is much more exposed to users, than most engineering is.
Although engineering is catching up: e.g. in automotive usage features have been becoming more and more important over classic car engineering features, like horse power etc. It took car engineers quite a while to realize this.
think, they can still learn a lot from software dev. here.
Agreed, Software Development is not at all like Bridge Building. It’s also not like materials engineering, aerospace engineering, electrical engineering or any other engineering disciplines… neither are any of these engineering disciplines really like each other. They all have different constraints, different models, different sets of maths that they need to learn, and different costs to pay in order to test their outputs. If you look for differences, then you’ll find them very easily.
However, if you abstract in a slightly different direction and take engineering as a disciplined approach to problem solving, applying the appropriate models and techniques to solve problems within a set of constraints… it starts to sound more like software development (or at least what we should be striving for in software development, IMO 😉 ).
So yes, Software Development is not like Bridge Building… not really a surprise as we aren’t building bridges. We don’t need to abandon the engineering aspect though… we just need to stop committing the grave fallacy of assuming that all engineering is like bridge building and therefore something that doesn’t produce bridges isn’t engineering.
I think I can understand that perhaps John has never developed software that requires engineering principles. Rather he accustomed to using libraries for which other engineerers applied software engineering principles, such as a library geared to accessing a database or designing and implementing a tool that creates databases (MySQL, SQLServer, etc) or a library designed to implement computer vision algorithms.
It’s one thing to use tools to build programs (software development/IT) and quite another to build the tools that create programs (software engineering).
It depends on what your trying to prove to be correct. You certainly cannot prove that a program works for “all cases” of ‘n’ without mathematics.
Again, perhaps this article is more a distinction between engineering and IT, rather than whether or not you can apply engineering principles to software development.
Nice writeup. Loved your surgeon analogy. I agree 100% with you that SW creation is not engineering, today …
Though do you think that your conclusion [about SW not being engineering] is a good thing, or a bad thing. Should we strive for engineerizing SW development, or should we just accept that it’ll always be shrouded in mystery and ‘black art magic’ …?I think not!Depending upon how you perceive ‘engineerizing’ !I think that just like masonry evolved from the ‘magicians’ that built the great cathedrals of Europe and into sky scrapers, or even better; literacy, and how it spread after the invention of the Printing Press, we need to evolve SW creation, to the point where it’s a commodity, for the masses. Where everyone can ‘express’ themselves, like the photo-camera gave common men [and women] the possibility to express themselves as ‘artists’ …SW and IT is mankind’s first success at synthesizing, and strengthening, through robotics, mechanical and electrical processes, our ‘brain muscles’! While all previous innovations have been about strengthening our _body muscles_ …… meaning, it is important that it becomes ‘common knowledge’ ASAP, and not stay on a few ‘magicians hands’ like it is today …!That’s my definition of ‘engineerizing SW creation’, or rather to be accurate; A ‘side-effect’ of the engineerizing process’.I think from that approach we all have a divine obligation towards making it happen, even though you unfortunately currently are right, and shows great insight into a lot of interesting things, which you pinpoint out in your article … 🙂
What I don’t see in most of these articles about whether or not software development is an engineering discipline is the definition of “engineering.” I think once we’ve established what “engineering” really is, it’ll become easier to establish whether or not software development is an engineering discipline. I think software development can be approached as an engineering discipline. However, it’s a lot more than just engineering.
I’ve expected an answer like this, even if I am not an engineer. As a software developer I know the intricacies of programming and the perils of management, but I would wager that engineers have their own obscure problems and need for creative thinking. After all, if engineering would be like engineering, then we could build a program to do it 😉
@92eb8e053fbae0a6193cfce34144b2f1:disqus I like the wikipedia definition of engineering personally:
‘Engineering is the discipline, art and profession of acquiring and applying technical, scientific, and mathematical knowledge to design and implement materials, structures, machines, devices, systems, and processes that safely realize a desired objective or invention.’
With that definition, it becomes very obvious that Software Development is exactly embodied as an engineering discipline… just one that produces software, not bridges (and so analogies to how bridges are built will always be flawed). For example: building a bridge from designs is very expensive, but building software from designs (i.e. compiling it) is very cheap. Testing a fully built bridge is very, very, very expensive and very impractical, while testing a fully built piece of software is reasonably cheap.
When you get down to considering these very important differences, you can easily class software development as engineering… but one that relies much more on empirical testing of the produced artifacts and rebuilding them if they’re wrong than one that relies very heavily on models and design before the build because it’s massively expensive to rebuild the bridge if something is wrong with it.
And always remember – software development is more analogous to being bridge designers, where the binaries produced by a compiler are the actual bridge 😉
US: 308 millon people (http://en.wikipedia.org/wiki/United_States) – World; 6.93 billion (http://en.wikipedia.org/wiki/World_population) (more than 20 times)
“Ten years ago, we tried to use UML diagrams and CASE tools to develop software. Ten years ago waterfall was all the rage.”
That’s historically incorrect. Waterfall, UML, and CASE are as prevalent now as they were 10 years ago. Netscape and the dot-com era/first-generation web applications developers did not use waterfall, etc. Linux was and still isn’t designed with that approach in mind. Those techniques were and are primarily used at certain large organizations, and those organizations haven’t drastically changed their policies in the last 10 years.
I read your article with patience waiting for you to express your points, but you went off on tangents about nothing at all.
Lots of comments, let’s see if I have anything new to add…
I agree with John completely that building software is not
like building physical things like bridges and buildings; but the other end of
the argument is that writing code is an art, which I disagree with (but would
like to hear your thoughts on it sometime).
The space between these two poles is vast. My experience of
too many years is that software can be developed using engineering principles,
and be better for it; but it can also be developed without those principles and
still be working software. We all use such software without considering this, but probably would not want to drive over”version 1.0″ of a bridge created without engineering.
My experience with Information Engineering, and indeed the
whole industry’s experience, was that the value of engineering and CASE tools
was that they made extension and enhancement easier and faster, addressing the
fact that the most money spent on s/w is not development but maintenance over
its useful life. They did not make delivering
the original software faster, but that is what the market (always) wanted. And
after IBM’s useless AD-Cycle tainted the
CASE tool market, companies mainly turned not to new ways of writing software
like OO (Agile was a few years off) but to buying the new integrated s/w
packages like SAP.
So, the ways of developing software will continue to evolve.
I would only suggest that some application of engineering principles can help,
especially modeling the end product before building it.
I think you’re confusing the amount of bridges that exist with the amount of bridges that have ever existed since we started building bridges… The USA is a relatively young country.
most of those 6.93 billion people live in areas without complex engineered bridges.
most of those 6.93 billion people live in areas without complex engineered bridges.
most of those 6.93 billion people live in areas without complex engineered bridges.
most of those 6.93 billion people live in areas without complex engineered bridges.
most of those 6.93 billion people live in areas without complex engineered bridges.
Man, I can’t disagree more. As Alexander Stepanov said in his ‘Elements of Programming’:
“As with other areas of science and engineering, the appropriate foundation of programming is the deductive method. It facilitates the decomposition of complex systems into components with mathematically specified behavior. That, in turn, is a necessary precondition for designing efficient, reliable, secure and economical software.”
Wrong; a bridge has just as many parameters as your average piece of software. There’s a lot of “my work is harder than your work” commenting going on in this thread, at least from the programmers’ side.
Suppose all a bridge has to do is “stay there” (which means earthquake resistance, wind resistance, perhaps hurricane resistance) and “allow traffic to pass” (which means enough lanes to avoid congestion, perhaps space on either side and infrastructure in case expansion to more lanes is needed, structural support so that it can handle large trucks in all lanes at once, signs posted with the maximum weight limit if necessary, those signs posted early enough so that heavier trucks can take an alternative route,…). That’s quite a lot of implicit requirements already. If you then add that the bridge should “look pretty”, that’s another dozen requirements. Perhaps the construction of the bridge should even “create local jobs”: this might not affect the materials used in the bridge, but it might affect where they come from. Certainly if the bridge is an overpass, you’ll need to worry about the height of the road below, and also schedule the construction work so it minimally interferes with traffic on the road below.
And so on and so on. And that’s just for one bridge!
Compare this to your average “little piece of software”, where the only explicit requirement is “talk to the database and give me the information I requested.” All that stuff about connections, screens, and printouts is implementation detail, and standard to boot. You might as well say that a bridge “needs” concrete, rebar, those funky squiggly metal guardrails…
Bridges aren’t the only feats of engineering.
I think he was referencing the idea that the human condition is ever changing, akin to his explanation of software development. Try not to look too much into it.
Almost have your commenters spend the time trying to disprove your article. I guess they must not be software developers. Haha. Great article John. Will definitely be reading your blog more often.
Oh, I forgot… never say never…
I don’t really understand the point of this article. Certainly, the author makes a bunch of assertions. But why? What is it that he is being stopped from doing? What is it that he wants to be different? Why is he motivated to write this?
I have been developing software for over 20 years. During that time, I have heard diatribes such as this over and over again portraying this creative/artistic/romantic vision for software development and vilifying any attempts to incorporate structure/discipline as by definition subversive and evil.
For me, invariably, these articles come off as a self-righteous, emotion based sermons from the bully pulpit. If the author wishes to disregard any of the learning or practices from the field of engineering when he is building his software, he is more than free to do so. No one is stopping him, and unlike bridges or buildings, there is no legislation in place for software development that dictates practices or certifies quality. As such, his work will be measured on its own merits… development productivity, defect rates, and fit to the intended urpose. He can attempt to achieve that however he wants. But the thing is, I don’t know why he seems so fixated in judging and dictating the way others might choose to achieve these same objectives.
YOu need to chillax buddy, and stick to writing code.
One does not study Bridge Building (at least not in the program I went through). It is a subset of Civil Engineering. So your comparison should include hydraulics, transportation, geo-technical, and structural.
To the meat of the argument though: web development will likely become cookbook production soon. As there are more and more security breaches, we will see the government start to regulate precisely how a website should be secured. Other ‘best practices’ will follow.
I read your article and then looked at the comments and thought someone should say that they enjoyed and agreed with the article. I don’t know why so many comments are nit-picky or downright wrong.
Thank you 🙂
Yes it does, I’m going back and forth to write software as I read this because the barrier of entry is just one keypress to switch to my text editor. Building a bridge definitely is not so easy to do.
Man, people gotta be haters! I think it’s a well written article that makes some very good points. Although I still think it is engineering, you are completely right in that it is unlike any other type of engineering before. In no other engineering discipline can one smart guy design a revolutionary new tool, and the next day someone halfway around the world can be using it to improve their own tools.
This leads to an incredibly increased rate of development than for example electrical engineering where a new idea requires a team and complex manufacturing processes, and many iterations and testing before it will be available for others to improve their own products.
Those people talking about ‘oh yea this number is wrong’ are completely missing the correct bigger picture implications of the article.
Man, people gotta be haters! I think it’s a well written article that makes some very good points. Although I still think it is engineering, you are completely right in that it is unlike any other type of engineering before. In no other engineering discipline can one smart guy design a revolutionary new tool, and the next day someone halfway around the world can be using it to improve their own tools.
This leads to an incredibly increased rate of development than for example electrical engineering where a new idea requires a team and complex manufacturing processes, and many iterations and testing before it will be available for others to improve their own products.
Those people talking about ‘oh yea this number is wrong’ are completely missing the correct bigger picture implications of the article.
…interesting article; also, the comparison between software & civil engineering is not without irony – software vs civil engineering; the principles of engineering are almost identical. in fact in the early 70’s (coincidentally round the time info tech was really taking off) a consortium of civil engineers compiled a manual outlining requirements gathering(needs analysis) against which all technical design is based. the manual was such a success that it revolutionized the practice of civil engineering and was in fact so comprehensive that other industries adopted the very same manual. one notable and pertinent one in particular is ……the IT industry. yes! its true! the same design principles – its just the media is different. the process is the same. modify the design, implement the changes. software requires lines of code, server locations for deployment, contractors to write & manage the code. civil engineering would only swop the writing code with actual physical building tasks. a surgeon is in fact a highly specialised form of engineering in that it requires a high degree of technical skill and also requires a broad & continuous dialog with industry peers in order to stay on the cutting edge. no pun intended. a little busy right now, will try post a link for further reading.
Because the article is thoroughly illogical. I hope he codes better than he argues! If you set out to disprove prove that “Software Development” is as mature as “Engineering” you can have an interesting discussion. Unfortunately, this writer instead decided Engineering = Bridges. It’s crazy talk!
It’s as stupid as plucking an arbitrary subset of software engineering and pretending it’s the entirety. Software engineering is completely immature because the number of Cobol-based MP3 playing apps in existence is far fewer than the number of air conditioning units ever built. Therefore, it’s clear that the air conditioning industry is more advanced than the programming industry.
Interesting,
but you do know that that Colleges and Universities teach and issue degrees in
Software Engineering. In some cases these learning institutions have been
offering degrees in Software Engineering since the early 90’s.
In Canada,
(and Texas 😉 Software Engineering is a legislated professional practice and
you can be licensed as a Professional Software Engineer. http://www.apeg.bc.ca/reg/pengappdocsrequired.html#other
I am a
licensed and practicing Software Engineer. http://www.apeg.bc.ca/members/LLScopes2010/M_Barnett%20Certificate%2010.pdf So how should I react to an article
that says it no such thing exists and waiting for it is never going to happen?
With due
respect, I feel that you should look into the practice of professional Software
Engineering to see that it is alive and well and growing in numbers. I ask for
your help in educating others that such an engineering discipline actually does
exist and is a respected professional practice.
So why not
give our industry a boost with your audience reach by promoting Software
Engineering. According to our industry,
Software Engineering has already happened.
You can get a degree in it and a professional license to practice
it. What more do you want?
I would say that software engineering is just a name, and doesn’t necessarily make it equivalent or even similar with civil or mechanical or biomedical engineering. This article isnt trying to say that what software engineers do is any less important or difficult than civil engineers so don’t be offended, but rather operates on very different principles and requires a very different process than most if not all other types of engineering.
Obviously bridge building is just an analogy for the rest of engineering. After reading the article you should them be thinking about how it applies to the rest of engineering rather than immediately rejecting it because you think the scope is too small. Are you really saying you expect a single article to touch on every field of engineering and compare them to software development?
Of course not. By quantifying the number of bridges, the author is making a direct comparison between a single element of X and the entirety of Y while pretending he’s comparing X & Y.
The poker aside was an interesting analogy but it was accurate (# of online tournaments entered and games of poker played vs # of non-online tournaments entered and games of poker played). Made sense. He didn’t compare # online contest entered vs # of packets of blue-backed poker playing cards sold. That’s effectively what he tried to do for the engineering argument.
Software
Engineering is a legal and professional practice on equal footing with other
engineering disciplines or at least that is how Canada law and legislature sees
it.
Operates
on different principles? No, as a former electronics engineer, the same
principles apply for software engineering. The only thing that is different is
that one is electronics and the other is software. Having requirements, design,
implementation, test, deploy, operation and maintenance life cycles are very
similar in all engineering practices.
The
thought that developing software is somehow different than every other
engineering discipline is what’s wrong… and actually preventing our software industry to
mature 🙂
Software
Engineering is a legal and professional practice on equal footing with other
engineering disciplines or at least that is how Canada law and legislature sees
it.
Operates
on different principles? No, as a former electronics engineer, the same
principles apply for software engineering. The only thing that is different is
that one is electronics and the other is software. Having requirements, design,
implementation, test, deploy, operation and maintenance life cycles are very
similar in all engineering practices.
The
thought that developing software is somehow different than every other
engineering discipline is what’s wrong… and actually preventing our software industry to
mature 🙂
Software
Engineering is a legal and professional practice on equal footing with other
engineering disciplines or at least that is how Canada law and legislature sees
it.
Operates
on different principles? No, as a former electronics engineer, the same
principles apply for software engineering. The only thing that is different is
that one is electronics and the other is software. Having requirements, design,
implementation, test, deploy, operation and maintenance life cycles are very
similar in all engineering practices.
The
thought that developing software is somehow different than every other
engineering discipline is what’s wrong… and actually preventing our software industry to
mature 🙂
Software
Engineering is a legal and professional practice on equal footing with other
engineering disciplines or at least that is how Canada law and legislature sees
it.
Operates
on different principles? No, as a former electronics engineer, the same
principles apply for software engineering. The only thing that is different is
that one is electronics and the other is software. Having requirements, design,
implementation, test, deploy, operation and maintenance life cycles are very
similar in all engineering practices.
The
thought that developing software is somehow different than every other
engineering discipline is what’s wrong… and actually preventing our software industry to
mature 🙂
Software
Engineering is a legal and professional practice on equal footing with other
engineering disciplines or at least that is how Canada law and legislature sees
it.
Operates
on different principles? No, as a former electronics engineer, the same
principles apply for software engineering. The only thing that is different is
that one is electronics and the other is software. Having requirements, design,
implementation, test, deploy, operation and maintenance life cycles are very
similar in all engineering practices.
The
thought that developing software is somehow different than every other
engineering discipline is what’s wrong… and actually preventing our software industry to
mature 🙂
Software
Engineering is a legal and professional practice on equal footing with other
engineering disciplines or at least that is how Canada law and legislature sees
it.
Operates
on different principles? No, as a former electronics engineer, the same
principles apply for software engineering. The only thing that is different is
that one is electronics and the other is software. Having requirements, design,
implementation, test, deploy, operation and maintenance life cycles are very
similar in all engineering practices.
The
thought that developing software is somehow different than every other
engineering discipline is what’s wrong… and actually preventing our software industry to
mature 🙂
Bridges may be an analogy, but I think the point is he sets out to somehow prove that “Software Developemnt will never be Engieering”. He does not define Software Development or Engineering, just talks in vague mataphors, and then introduces surgeons as the answer in the last sence.
I disagree.
The article doesn’t suggest that Software Engineering is different from other fields of Engineering. The author explicitly sets out to prove that “Software Development will never be engineering”. The thesis, as near as I can make out, is that Software Engineering is an oxymoron.
I also disagree with you that Software Engineering is just a name. People spend alot of time, energy and money to become software engineers. For them it is very real. And as Mitch says, in some jurisdictions it has legal meaning.
So, you can say that Software Engineering is a bad idea and we could debate that, but Software Engineering does exist, and is certainly more than just an arbitrary name.
Maybe you could list out the “Principles of Engineering” and your point would be come a bit clearer?
I understand your point. But I think in general the reliance as a society we have on a certain class of people, I’ll label as academics is far too strong.
Just because a board or a committee or a university dubs something as truth or law doesn’t make it correct.
I think a large amount of readers have assumed that by suggesting software development is not engineering, that I am suggesting that strict rules and discipline as well as best practices do not apply to it.
That could not be further from the truth. I am one of the most stringent sticklers about best practices in software development of anyone I know. I recently wrote a post on this site called “Why Rules Rule.”
What I am really trying to point out here is that we are trying to push software development into the same direction other true physical based engineering disciplines have gone.
Software development will never go that direction. Instead software development is going a completely different direction, because it is a completely different kind of thing that really is like no other comparable thing.
The design work we do as software developers is completely in the abstract. Sure we produce a semi-tangible product, but really the design of the user interface or functionality of a software product is different than it’s architectural composition.
Most software development work. At least the hard stuff is spent in this pursuit. Building ideas. Object oriented development doesn’t model the real world. It models an imaginary world that we create. We create rules upon rules to operate in that imaginary world to eventually produce a real result.
Software development is about weaving an intricate story made out of webs of ideas into something that can produce a result in the real world. If you were to look at software development from the perspective of what the output of a software program, then I could relent and call that engineering, but that is truly the simple part of the job.
The true feat is the creation of the intangible. That feat is fueled by creativity and construction of a virtual world with self created rules.
Engineering disciplines are based on real world rules and limitations, we cannot simply bend gravity to our wills, but with software we can.
I appreciate your point of view. However, having spent 10 years as an electronics design engineer and 20 years as a software engineer, I am not convinved of your point and nor am I am an academic.
I have designed enough electronics in the real world to know that sometimes things just work even though the theory says it should not. Ask any RF Engineer about theory and not being able to bend gravity – you might be suprised at the response.
Software design is really hard and often very abstract. But like everything in the world, it too is bound by the laws of physics – all of those electrical 1 and 0’s come to mind… And software is based on real world rules and limitations – programming languages come to mind. But even on the pure software side, I can’t just take a chunk of code and expect it to run on every device. In fact in some ways, compared to my electronics world, software is even more limiting.
I don’t know why you think software will never go that direction. You yourself are practicing software engineering (but may not know it) by saying Why Rules Rule.
I think your point comes across alot clearer here.
You are saying that Software Development has alot more artifically imposed constraints rather than the absolute and universal physical ones assocated with more traditional forms of Engineering.
I agree that is a key difference.
But I am still at a loss as to why you are making this point.
A theme throughout your paper is that something bad is happening as a result of this difference.
I geuss that is where you lose me, I have trouble connecting the dots between the distinction regarding types of constraints you have identified above, and bad things happening in Software Development.
You are also right, that I originally thought it was structure and discipline that stifled you, but as you say, it isn’t. So, John, I am left scratching my head.
You suggest your software development is being pushed in an undesirable direction by someone/thing and the root cause has something to do with engineering.
The 2 examples you site are Case Tools, and waterfall development processes, yet you then say they existed 10 years ago and you have moved in the opposite direction now.
So, specifically, what is it that you are being pushed into doing against your better judgement? What force is at play in your work life that is exerting that kind of pressure? And why cann’t you just ignore it if it detracts from your ability to produce quality software in a productive way? And how does all of that relate to traditional forms of engineering being constrained by absolute and unviersal physical forces.
If you could answer that, I think the dots would start connecting on for me.
Most of the civilized world has more than 200 years of history. Many bridges had colapsed since we started to build them.
On the other hand I do not consider extreme engineering rope-like bridges like the ones in Indiana Jones films. I am sure they were pretty dificult to build but not very safe.
I suppose my answer is “nothing.” I’m a pretty happy camper. I was just thinking one day and the idea that we have built more software than bridges occurred to me.
I thought it was a perspective that I hadn’t heard anyone mention before.
I had also heard in a podcast someone talking about how software development hadn’t matured enough as an engineering discipline.
If I am fighting against anything, it would be trying to do silly things like taking empirical data for software development projects and trying to make sense out of it in order to make statements like. “TDD is good, or TDD is bad.” Because each software development project is so entirely different that most empirical data about software development is just meaningless…
Yes, I geuss that was the subject of your opening paragraphs… Metrics that is.
And I do see that “objective decision making through imperical measurement” is a pretty fundamental principal of Engineering.
I don’t agree that each project is entirely different. And that their can be no global learning carried from project to project.
If that were true, then your assertion that CASE tools don’t work for software development would be nonesensical. You would have to say that “As it turn out Case Tools didn’t work on the last 10 projects, but they may well work on the next project, because one software project has no relation to the other.” I don’t think you would say that.
So, I am not an Academic, and I am not a Professional Engineer. I do however use metrics extensively within projects and infer alot of useful meaning from them across projects. I run a large development team, we work on lots of projects. Our goal is to continually increase our customer satsifaction (somewhat subjective), productivity and quality.
Throughout a project we are continually measuring these things, both imperically and anecdotally and adjusting our activities to improve them.
We take learnings from our past projects, trends in the industry and yes look at the historic imperical data and look at the realities of the current project and make initial decisions as to how we will engage.
This is not some academic spouting off about theroretical time and motion studies. We develop software under fixed price contracts every day. Trust me, it doesn’t get any more real than that.
There is a certain irony that I have percieved in our industry. I have never met a software developer who considers themself below average in terms of productivity or quality. Hmmm. Yet, the vast majority of them can not begin to quantify how productive they are or their rate of defects.
Just like you don’t rely completely on a developer to ultimately certify if his code has no defects, you get an idependent test of the system. You don’t rely completely on a developer to perceive how productive he is being, you measure it if you really want to know. Human nature just works that way. And that doesn’t mean that the developers opinion on the realities of quality and productivity aren’t relevant, just not very good on their own. That is how you drive improvenement.
So, let’s take your example, Test Driven Development. If I were to interpret you literally (which I will for the sake of argument), you are suggesting that there is no way I can make a decision as to whether TDD would be relevant to a new software project. Your words suggest that experience with previous projects would provide no relevent guidance, because all projects are entirely different.
That would be a pretty hopeless world. I don’t think you really beleive that. I think you go into a project and at the beginning determine whether TDD would be benefitial or not based primarily on past experience and the realities of the project.
I certainly know that is what we do. And that quantitaive data we have from previous projects helps us in the decision. You will note I say, helps, because we definately supplement it with our anecdotal experience, as well as readng about other peoples experinces in the industry. And yes we factor in the realities of the particular project. Very important, but in our experience, although all projects have differences, few projects projects are entirely different. In fact, I always beleived that was the art of a good software architect was percieving similarities and differences between things and treating them accordingly.
You tend to paint a world with metrics as very totalitarian, with academics as the fascists. Also not our experience. We would not take an academics word carte blanche. We certainly would not take metrics literally from an academics. We create our own metrics, but we would never infer from metrics that TDD was good, therefore stop thinking, stop questioning, stop measuring, the numbers are in. That would be as silly as not measuring.
We use metrics, not so we don’t have to think, but rather as one factor in our decision making process and to guide us on what we should be thinking about.
So, you may not beleive in metrics. They may not work in the type of software development you do. They may not work with the mindset of folks on your team. I could understand that.
But I don’t think that you can make the assertion that using imperical measurments has no positive impact on software development. The reason I say that, is that it has for us. I know our teams our benefitting from it because they are making decisions that have allowed them to become above average over time and are improving. How do I know that? Because we measure it of course.
I think the smart a55 question at this point would be how do you know metrics don’t work. Have you measured it?
Good point! Excellent point!
I agree with you 100%.
I am not against measuring, but we have to decide what to measure based on a project or team.
I am against applying metrics across projects without a really good reason for doing this.
I believe more in the aspect of measurement when applied to an iterative cycle. For example, in Agile methodologies we usually have a retrospective. At which time we look at measurement and make adjustments.
I would be against trying to take measurements from my teams retrospective and applying them to some other random team.
So, here is the thing.
That practice of “Making objective decisions based on imperical measurment” is a fundamental principle of engineering.
Having “Strict rules and discipline around the way things are done” is another fundamental prinicple engineering.
When you think of engineering principles, you thing of the law of gravity or ohm’s law. I think of things like
the last 2 paragraphs.
Someone above pulled the definition of Engineering from American Engineers Council and cited it above. It speaks of these too, not the immutable laws of physics.
So, yes, there are academics in the dustry who don’t know what they are talking about. And yes some of them are engineers. And yes they can spin metrics to a false conclusion. But as the saying goes “Liars figure to make figures lie”, that happens everywhere.
When this happens it is the academic who is wrong. Engineering isn’t wrong, it is just bad engineering, and Software Development is fraught with that.
The other thing is that literally 99% of engineers are practictioners, not academics, and they make decisions in their field using the same “Fundamental Principles” as I describe above.
Which in my mind has always added up to “The fundamental principles of Engineering have alot to teach me as a Software Developer”. Not everyone things that I know, but as we have wrestled with your words and fundamentally what engineering is, I sense you beleive that too.
So, maybe the thesis of your paper shold actually be.
“Don’t listen to anyone who is spouting immutable facts about software development, particularly if they are not a practitioner. Software Developent is at a stage where it is evolving rapidly on many fronts. Maybe it willo always be there, but we know taht very few aspects of software development today will remain unchaged moving forward. This is why we need to incorperate the fundamental principles of engineering to continually evaluate our suituation imperically and make objective decisions for the current time/place, and then execute in a very disciplined and consistent way untill new information becomes available and we will refine it, and of course keeping our eyes open for opportunity along the way as any good engineer does.
Don’t tell avionics software companies that V1 and V2 don’t even look that same at all.
Oh… and don’t tell their customers. They would be so scared…
This is one of the reasons why every README or licence file includes a disclaimer…
Way back when I first started studying computer science and engineering in school I came to the realization that what we do is more akin to what doctor’s do. I mean we work on a machine (the computer) which in many ways is modeled after living breathing organisms. It has a brain, a pulse, a heart.
Are we not engineers, well I’m not sure I agree with that. We certainly aren’t civil or mechanical engineers, but electrical engineering has certainly advanced at a pace similar to that as software engineering (which probably explains why I took one civil engineering class in school and many more EE classes).
In this post you make some very informative points. Keep up
the great work!
Everybody visit this website collect some information for software solution. Sure someone, probably not you, will have to come along and do some routine maintenance on it. Sure, some small. This site is attractive for software user.VOIP, SIP, SKD, PBX, softphone are more popular in people.
PBX,softphone,SDK,SIP softphone, VoIP SIP client, IVR SDK