Stepwise Refinement

"You eat elephants one spoonful at a time."
- Bryce's Law

In a nutshell, the concept of "stepwise refinement" is to take an object and move it from a general perspective to a precise level of detail. Architects have used such an approach for years, as have engineers building products. But to do so, they realized they cannot simply go from the general to the specific in one felled swoop, but instead, in increments (steps). The number of steps needed to decompose an object into sufficient detail is ultimately based on the inherent nature of the object. To illustrate, for architects designing a building, the typical steps include:

1. Develop artist rendering (to consider viability).


2. Design foundation and superstructure.


3. Design Floor plans.


4. Design electrical and plumbing diagrams.

In other words, before the first shovel of dirt is dug on the project, the architect knows precisely what the building will look like and how it will work. All of the guess work has been eliminated.

Engineers building products such as automobiles follow similar yet different steps:

1. Develop artist rendering (to consider viability).


2. Design major assemblies (e.g., chassis, body)


3. Design subassemblies (e.g., engine, transmission, electrical, etc.)


4. Design operations (e.g., the various components of the engine).

Like the architects, the engineers know precisely how the automobile will look, the parts needed to produce it, and the sequencing of assembly. All of the guess work has been eliminated.

"Stepwise refinement" ultimately represents a "divide and conquer" approach to design. In other words, break a complex object into smaller, more manageable pieces that can be reviewed and inspected before moving to the next level of detail.

There are those in the I.T. field that feel such an approach is impractical to implement, and instead of engineering your way to success, you should take an initial stab at developing a program, then continue to modify it until you have developed something to pacify the user's needs. This of course is a hacker's approach to development and may be fine for developing an innocuous little program but definitely not suitable for developing anything of substance.

Can the concept of "stepwise refinement" be applied to a single program? Absolutely. As a matter of fact, it lies at the core of the structured programming movement of the 1970's-80's. But can it be applied on a grander scale, such as an enterprise-wide Information System. Again, the answer is Yes. In fact, it is the logical way of attacking such a major endeavor.

Before we can tackle an information system, it would make sense that we first understood the inherent properties or structure of the object of our attention. It has always been our contention that an Information System is a product that can be engineered and manufactured like any other product. To this end, we see an Information System as a four level hierarchical structure consisting of the following components:

LEVEL 1 - SYSTEM - representing the overall product to be built.

LEVEL 2 - SUB-SYSTEMS - representing the business processes associated with the system (one or more).

LEVEL 3 - PROCEDURES - representing the work flow of each sub-system. There are essential two types of procedures; Administrative - representing procedures executed by humans; and Computer.

LEVEL 4 - PROGRAMS - representing the programs needed to execute each computer procedure.

As an aside, there is also a Level 4 for Administrative Procedures (to write the instructions/steps pertaining to the procedure), but for the purposes of this paper, let's put this aside for the moment.

Under "stepwise refinement" the levels are decomposed top-down during the design process, and implemented bottom-up; a common engineering/manufacturing technique.

To implement this approach, a "blueprinting" technique is used which is actually not too dissimilar with that used by architects and engineers in other fields. Here, the blueprints are used to express the succeeding levels in the system hierarchy. To illustrate, consider the following diagram, where the work from the preceding phase feeds the next level in the system hierarchy:

What this means is that all of the guess work has been eliminated. So much so, that the creation of executable programs should be a relatively simple task. Again, the specifications for the programs were developed naturally in smaller and more manageable increments, not all in once.

CONCLUSION

The concept of "stepwise refinement" is not exactly new and has been used successfully in the engineering/manufacturing of products for many years as a means to manage complexity. It has only been in the last thirty years that people have been trying to implement the technique in the development of systems and software.

In the absence of "stepwise refinement" on a complex system, the "brute force" approach is typically used, whereby superficial work is done in the earlier phases and substantially more time is spent in programming second-guessing what is to be developed. This is one reason why companies today rarely tackle major systems development assignments, and are content with attacking it in piecemeal.

But if you can assimilate a system as a product, and believe it can be engineered and manufactured like any other product, than "stepwise refinement" is a pragmatic solution you can definitely use.

If you would like to discuss this with me in more depth, please do not hesitate to send me an e-mail.