The New Conference Room Pilot

This article is also available on our website: PROACTION - Generating Best Practices. It is an excerpt of a paper originally written by George Miller, Founder of PROACTION. It has been modified and updated by Paul Deis, PROACTION CEO.

Overview

Making major business systems work requires planning, vision, leadership, teamwork and great effort. This presentation explains the role of the Conference Room Pilot (CRP) in helping to ensure success, how to help re-engineer the company business system, how to set one up and operate it.

What is a Conference Room Pilot?

A methodology to develop and simulate operation of a system, to learn how it works/should work and how best to manage the business with it—prior to "live" implementation.

On-Line, interactive, integrated systems and software confound traditional evaluation and implementation planning methods. Various "friendliness" issues and software "personality" are much easier to evaluate and understand in a live simulation than through documentation review, flow charts and vendor sales pitches. It can be on line or on paper.

The CRP has traditionally been associated with the testing and business modeling with new computer software, but can also be used to handle evaluation of policies, procedures, organization, forms, training and performance measurements. It can be used to test changes to existing systems. The closer one gets to reality, the more effective the training experience. It is becoming more common to leave a CRP database in place to use as a permanent test bed and education/training tool.

Applications

Some people use the term CRP to mean testing software prior to implementation. While this is an excellent application, the list below indicates that there are far more uses:

The CRP may be used to:

• Explore policy/procedure issues, alternative solutions and initiate improvements.
• Partially or completely re-engineer the company system for improved performance.
• Train employees in the operation of the system/software- policies, procedures, software, forms, and reports.
• Gain a practical understanding/evaluate how the software really works- approach, strengths, and weaknesses.
• Test accuracy of vendor proposals, claims, and documentation.
• Help develop a workable plan for conversion, implementation and utilization of the software as part of an overall system.
• Provide education in how a manufacturing system and software could work- especially interdepartmental relationships.
• Facilitate constructing system descriptions and demonstrations of regulatory compliance.

The CRP may be used during one or more system lifecycle phases:

• Identification and development of policies, procedures and operating approach.
• Shakedown cruise to refine/debug software, policies, procedures, training and even organizational approach.
• Ongoing training tool.
• Ongoing test bed for new/modified software, policies, procedures, application areas, including system integration.

There are several major advantages over other approaches:

• Evaluate, test and learn at minimal risk.
• Test more alternatives, more comprehensively.
• Easier to control conditions.
• Far faster and cheaper than full simulation, "live" pilot, parallel operation or "cold turkey"/"big bang" (total implementation all at once).
• Much more realistic than program/module technically- oriented testing.
• The CRP is slow, but it will usually result in better, faster implementation results in the long run.

Disadvantages exist as well:
• CRP's are time-consuming.
• They initially slow down the implementation cycle, but can yield improved results.
• Require high degree of discipline, but so does implementing a major system - think of it as good practice.

How It Works

Recommended key elements of a successful CRP are outlined and briefly explained:

A. Planning/organization

• Objectives - It's a good idea to reiterate the system objectives or write them if you haven't already. Good objectives are clear and quantifiable where possible. They directly support company strategic plan goals and objectives. CRP objectives should be selected from the applications section above and tailored to company objectives.

• Scope - It's important to state what's included - and not included - in your CRP. State the applications, organizations affected and the depth of involvement desired. For instance, do you want to completely re-engineer the process to eliminate all non-value added operations, or merely automate or convert an existing approach? Does the project encompass all company business systems, or just purchasing, MRP or accounts payable?

• Organization - Who will be responsible for directing, performing and acting upon the results? What is the role of each player? What degree of autonomy/empowerment is assumed? How will MIS, vendors, consultants be involved?

• Approach - How will the CRP project work? How will players go about their assigned tasks? What tools will be employed to accomplish the process?

• Project plan - Create major milestones, dates, supporting activities, responsibility assignments, precedence relationships, durations, resource requirements (people, equipment, outside support). The plan becomes the principal project management tool and may actually be a subset of a larger plan, such as MRPII implementation. Each application area needs to have its own milestones, responsible people and activity plan. We have found it works better to use at least some people whose regular job responsibilities lie in the affected areas.

B. Requirements definition/documentation

Some people say this shouldn't be part of a CRP. It appears here for three reasons:

• Make sure it's not forgotten.
• Put a business rather than a technical emphasis on the CRP.
• Put structure into the CRP effort.

Without a methodical foundation for the planned operation of a new system, the CRP will tend to wander aimlessly and will be subject to manipulation by vendors or others with a different agenda. If requirements are defined already—great. However, my experience is that most companies claiming to have this complete have done a superficial or incorrect job. Don't dive into the software before this important homework is done first!

It's imperative to know where you're going and where you're coming from when embarking on the perilous journey of major system change. The as-is and to-be systems definition approach can only be shortcutted so much before running into trouble. While we don't think that an existing system, if it's to be replaced, should be exhaustively and elaborately documented, it is necessary to understand how it operates and what issues and problems exist. Following that, a more detailed and comprehensive first cut "to-be" definition should be created. It may be successively refined as the program moves forward.

Experience has shown that most initial system documentation efforts do not reflect the actual richness, complexity, ambiguity and diversity of the real system operation. People usually assume that the current system is logical and rational, but it isn't always so. First pass analyses often appear "single threaded", in that they show what would happen if everything went right. In real life, there are many, many exceptions, requiring complex branching and alternative activities. To make matters worse, not everyone handles things the same in all cases. In fact, there is usually a good deal of ambiguity and even blind alleys in most de facto systems. MRB, customer returns and outside processing are good examples of this in most companies.

Conventional development and documentation tools are often not used in a way that deals with the above effectively. To better deal with these problems we have used the "living flow chart" method with a fairly high degree of success for some time. Instead of documenting systems in boring, complex diagrams that get hidden away in books, try this:

Employ systems users/project team members, not systems professionals, to construct the charts. Use the systems people as facilitators. Why?

• Instills user ownership.
• Users know better "where the bodies are buried."

Project management and systems people are often real nervous about doing this because users often come up with a poor analysis, unrealistic requirements, or worse yet, attempt to automate the status quo.

How to avoid these pitfalls

• First, provide education in modern systems approaches and provide a thorough grounding in the applications to be studied. Set up well qualified and well balanced teams. Expose people to methods used by more successful companies, but allow for differences, making sure that they don't claim that "it won't work here because we're aerospace" or some similar nonsense. Next, carefully explain how you want the documentation done.

• Then, set goals to be achieved, such as: reduce paperwork 50% in order processing, or cut procurement time by 33%. This will force people to look past the current approaches to achieve the desired approaches. It will also help ensure that the project will generate a healthy return on investment. Emphasize that processes need to be re-engineered, not just replicated on a new computer system.

Try putting flow charts up on the walls, life size, using actual forms, screens and reports. Connect the flows together with highly visible arrows. Record cycle time, responsibilities and applicable policies/procedures for each process. Put notes on the wall explaining what is being done, how and why. Review these flows with various departments, auditors, even customer and government people- anybody who will listen and provide feedback! The day before this was written, the President of one of the companies using this approach was out with the team as they were working on the wall charts- making suggestions! When's the last time that happened at your company? An amazing array of people have contributed to the process at this company.

Now here's a key point...

Have people write down all their suggestions, their name and the date on little cards or yellow "Post-It"™ notes and slap them up on the wall where they apply. The project team can record, classify, edit and prioritize them on an issues list, used to drive change activities. Talk about empowerment! Once the as-is configuration is documented, discuss issues and direction, then start right in on the to-be.

The to-be charts may be reworks of the as-is, but it seems to work even better with new charts, side by side with the old ones. By the way, these charts can become massive. We saw one that consumed eleven large walls. The re-engineered "to-be" version was less than half of that. More words of advice: don't let specialists work in isolation- employ cross-functional teams to get a balanced picture and full knowledge of system interactions.

Also, It stimulates the team's creative juices if they are provided some challenging objectives, such as: "reduce cycle times and administrative paperwork by 50%."

C. Administration

Competent administration contributes significantly to a successful CRP. It is recommended that an administrator be assigned to provide a focal point for all CRP activities. In a smaller project, this person might very likely be the overall project manager or an MIS person. In larger projects, this would typically be an all-consuming activity for months.

Working to the project plan for task assignment, scheduling, control, follow-up and reporting helps keep the program on schedule, on budget and on track.

The administrator should plan the CRP, set process and documentation standards ensure that functional leaders publish meeting announcements, release comprehensive meeting minutes, keep adequate records of meetings/decisions and refer unresolvable issues to the steering committee. This person should also monitor the process to ensure that it's being followed as prescribed.

Maintain system parameters - defaults, settings for things such as netting methods, costing methods, exception reporting rules, posting rules, default charge accounts, etc.

Ensure that databases are maintained:

• CRP databases, used for structured business testing of various likely scenarios.
• "Play" databases for free form/ad hoc experimentation.
• Training database, used for formal and informal instruction.
• MIS should maintain live (production) databases for the various organizations employing the software after implementation.

Required maintenance of CRP databases includes such things as maintaining parameters per instructions of project management. For example, the administrator may be instructed by a functional leader to change a parameter, such as default inventory posting account number. Before that is done, it's advisable to review the impact with the team and take appropriate steps before/after implementing it. It might be necessary to inform the other team members of the impact of this change. Changes tested in the CRP may be subsequently moved to other databases after the team has indicated that is the direction to go. For best results, MIS should probably perform this after evaluating the impact and consulting with the team, which should have already consulted with key systems users.

The administrator may also be asked to maintain archives of CRP databases and to roll back/restore various versions at various times. MIS should give the administrator a free hand and the proper tools/utilities to accomplish it efficiently and quickly.

D. Equipment/facilities

The establishment of proper facilities can enhance results, reduce costs and time required to successfully complete the CRP. A project headquarters area should provide adequate space for teams to assemble, perform wall flow charting activities, conduct classroom training and simulation testing. Figure 2 depicts a facility employed for a medium sized company implementing an integrated MRPII/financial system.

An adequate number of terminals/workstations and both line and slave printers should be made available in close proximity to each other and to the room facilities (the slave printers are needed to document screen contents at critical test points). This permits more effective interaction of the group performing its tasks. Equipment can later be recycled for production purposes, unless a small ongoing CRP test bed is maintained.

E. Technical

• MIS needs to provide databases as described above, back them up regularly, provide roll back/restore capabilities and/or services, adequate equipment and support services.
• Provide help with batch job streams.
• Continuing MIS CRP participation in providing technical and applications support will help ensure favorable results. Technical consultants, software and hardware vendors can play an important role here as well.

F. Operational Phase

There are many ways to organize the CRP activities. Here's a good one:

• Functional leaders direct construction of as-is and to-be flows, identify issues.
• Team works on prioritization, assignment and resolution of identified issues.
• CRP administrator does an informal run through of system flow charts, trial runs the software on-line and develops a "story line" to guide the scenarios to be written.
• CRP administrator lays out overall CRP plan and schedule.
• CRP administrator lays out CRP scenario guidelines. Keep test cases small. You want a good cross section of products, processes and organizations involved, not a massive data entry exercise.

Functional leaders construct detailed functional scenarios using guidelines, own knowledge of functions, issues list and advice from system users, managers and other legitimate interested parties. Others review and augment this product.

Functional leaders walk cross-functional teams and potential system users through flow charts. Issue handling is discussed. Leaders take group through on-line scenario exercises. Various questions and additions to the foreseen problems are raised and alternative approaches are tested. Most issues should be resolved by the team during the exercises or delegated for further action. Occasionally, issues will be referred to the steering committee when the group is unable to resolve them. This should be unusual if the group composition is correct and they are empowered to address most issues. It is advisable to have had at least some of the vendors software training and certainly "generic" or industry-specific education before attempting these steps.

It's a good idea to do the walkthroughs and resolve most issues on paper first, before starting the computer simulation. Why: Too hard to keep reloading the database, re-doing exercises and resetting parameters all the time.

Don't be afraid to try ad-hoc approaches to issues encountered. If worried about contaminating the database by deviating from the tried and true, try tests under different contracts, parts, accounts, or even on a separate database. This is one of the reasons we recommended having a "play" database earlier.

Don't fall into the trap of testing individual functions or software vendors' "modules." Run a comprehensive, integrated test of the entire system, with data that flow through all portions, so that interaction can be tested in an environment as close as possible to the one to be implemented.

Don't just run through the vendors' screens and reports, or slavishly replicate your existing approaches. Use all policies, procedures and forms that will be used to actually run the business, but look for improvements/streamlining in the process. The software is only one business tool and will only work as part of an integrated whole.

Document results of tests and discussions, using screen and report copies as well as written notes and published summaries.

In most cases, the group will begin to realize how little is really known about the process and how many issues have gotten away from it in the past. The CRP process permits knowledge to expand rapidly, and will enable them to better resolve issues raised.

G. Disposition

Bringing closure to the effort is key to benefiting from all the work to be done. A structured, methodical approach, documentation of results and control of open issues will help make this happen.

It's important to capture the results and issues raised and to relentlessly pursue their resolution to continually improve the system approach. Issue resolution is what really drives the change process.

Use the empowered team to resolve issues whenever possible. Maintain a dialog with affected organizations. Use the steering committee as the "secret weapon" to overcome obstacles when they can't.

Don't just "automate the mess you already have." Focus on re-engineering the process for improved performance, but don't get carried away. In some cases, it's possible to make major improvements after the initial implementation.

Put mechanisms in place to rapidly translate findings and recommendations into change. The best way to do this: competent team members should be drawn from affected areas, be empowered by their management to make changes, given guidelines for rapid change implementation and encouraged to do it! To break bureaucratic logjams, we suggest that suggested changes be approved by default, if entered to a regularly published issues resolution log, discussed at a project meeting and remain unchallenged for a specified time.

Conclusion

It may be evident from the preceding material that the CRP approach provides an excellent opportunity for evaluation, testing, development, education and training. This is a great way to build a trained team ahead of time, to ensure a more successful implementation.

It has been noted by veterans of successful system implementations that testing and other conference room pilot activities took much more time than originally planned, but that it was worth it. Some of these same people noted that it was necessary to run through the CRP more than once, in order to incorporate lessons learned into another try(s). Some unsuccessful project teams have remarked that they should have spent more time on these.

We have found that almost nobody initiates these on their own unless they have been exposed to them before or unless someone experienced helps them introduce the process to the project. In fact, there is sometimes active resistance or indifference from some of the uninitiated, who feel that the approach would waste too much time. While not intuitive, this and other CRP approaches are found to be logical, easily understood and accepted once tried with an open mind.

The Conference Room Pilot is the single most effective way we know of to truly learn and understand a system for evaluation, education, testing and implementation planning purposes.