No Silver Bullet: Essence and Accidents of Software Engineering

Fred Brooks comments:

Much of present-day software-acquisition procedure rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software-acquisition problems spring from that fallacy. Hence, they cannot be fixed without fundamental revision--revision that provides for iterative development and specification of prototypes and products.

Incremental development--grow, don't build, software. I still remember the jolt I felt in 1958 when I first heard a friend talk about building a program, as opposed to writing one. In a flash he broadened my whole view of the software process. The metaphor shift was powerful, and accurate. Today we understand how like other building processes the construction of software is, and we freely use other elements of the metaphor, such as specifications, assembly of components, and scaffolding.

The building metaphor has outlived its usefulness. It is time to change again. If, as I believe, the conceptual structures we construct today are too complicated to be specified accurately in advance, and too complex to be built faultlessly, then we must take a radically different approach.

Let us turn nature and study complexity in living things, instead of just the dead works of man. Here we find constructs whose complexities thrill us with awe. The brain alone is intricate beyond mapping, powerful beyond imitation, rich in diversity, self-protecting, and selfrenewing. The secret is that it is grown, not built.

So it must be with our software-systems. Some years ago Harlan Mills proposed that any software system should be grown by incremental development. [10] That is, the system should first be made to run, even if it does nothing useful except call the proper set of dummy subprograms. Then, bit by bit, it should be fleshed out, with the subprograms in turn being developed--into actions or calls to empty stubs in the level below.

I have seen most dramatic results since I began urging this technique on the project builders in my Software Engineering Laboratory class. Nothing in the past decade has so radically changed my own practice, or its effectiveness. The approach necessitates top-down design, for it is a top-down growing of the software. It allows easy backtracking. It lends itself to early prototypes. Each added function and new provision for more complex data or circumstances grows organically out of what is already there.

The morale effects are startling. Enthusiasm jumps when there is a running system, even a simple one. Efforts redouble when the first picture from a new graphics software system appears on the screen, even if it is only a rectangle. One always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build.

Brooks, Frederick P., "No Silver Bullet: Essence and Accidents of Software Engineering," Computer, Vol. 20, No. 4 (April 1987) pp. 10-19.

Microsoft Vista: Scrum or Not-Scrum

Peter Krantz rants on Scrum, Lies, and Red Tape at the Microsoft Vista project (see below). He's in Stockholm and I'm on a SAS flight over Greenland returning from two weeks in Sweden and Denmark doing several ScrumMaster Certification Classes. While Microsoft adopted Scrum on many programs large and small, I'm not aware of the Windows Vista team using it. Scrum is built on truth, transparency, and trust and was designed to emulate the productivity of the Borland Quattro project where the each developer delivered 1000 lines of quality production C++ code per week. Vista only delivered 1000 lines per developer per year.

Philip Su thinks Vista is the largest software project in the world and maybe the longest. However, Jim Coplien was in my Copenhagen ScrumMaster training last week and pointed out that he delivered weekly releases into a 100 million line code base deployed worldwide by ATT so that was at least twice as large as Vista. Jim also wrote a case study of the Borland Quattro project that was the inspiration for the Scrum Daily Meeting while he was at ATT. It was interesting to dive into the details of that project, but I digress. Here's Peter:

Scrum, Lies, and Red Tape by Peter Krantz

Philip Su from Microsoft gives us a glimpse of the inner workings of one of the most complex software projects in the world. It is interesting to see that the same problems that sometimes plague small waterfall projects (lies, red tape) exist in an organization that have put a lot of effort into their development methodology... Didn’t Microsoft adopt Scrum a year go? Maybe they skipped the part about transparency.

Broken Windows Theory by Peter Su

Vista. The term stirs the imagination to conceive of beautiful possibilities just around the corner. And “just around the corner” is what Windows Vista has been, and has remained, for the past two years. In this time, Vista has suffered a series of high-profile delays, including most recently the announcement that it would be delayed until 2007. The largest software project in mankind’s history now threatens to also be the longest...

Let's see if, quantitatively, there's any truth to the perception that the code velocity (net lines shipped per developer-year) of Windows has slowed, or is slow relative to the industry. Vista is said to have over 50 million lines of code, whereas XP was said to have around 40 million. There are about two thousand software developers in Windows today. Assuming there are 5 years between when XP shipped and when Vista ships, those quick on the draw with calculators will discover that, on average, the typical Windows developer has produced one thousand new lines of shipped code per year during Vista. Only a thousand lines a year. (Yes, developers don't just write new code, they also fix old code. Yes, some of those Windows developers were partly busy shipping 64-bit XP. Yes, many of them also worked on hotfixes. Work with me here.)

Lest those of you who wrote 5,000 lines of code last weekend pass a kidney stone at the thought of Windows developers writing only a thousand lines of code a year, realize that the average software developer in the US only produces around (brace yourself) 6200 lines a year. So Windows is in bad shape -- but only by a constant, not by an order of magnitude. And if it makes you feel any better, realize that the average US developer has fallen in KLOC productivity since 1999, when they produced about 9000 lines a year. So Windows isn't alone in this.