Selecting Mission-Critical Suppliers for your CE Team

by William J. Pardee
Bill.Pardee@pardee-quality-methods.com
www.pardee-quality-methods.com

 

Abstract

An inappropriate or ineffective supplier of a critical system can cause enormous delays, increased cost, and lost market opportunities. This paper describes a Quality Function Deployment (QFD) based method to identify the desired benefits accurately and completely, screen a large field of candidates efficiently if necessary, and accurately determine the value of the remaining candidates. Example applications are mentioned.

Introduction

An unsuitable supplier can have terrible consequences for quality, cost or schedule, yet it is often difficult to anticipate all the ways in which a supplier can fail and to identify one who won't. More positively, a great supplier will solve problems you hadn't anticipated and enable you to exceed your own goals. This article describes a procedure based on Quality Function Deployment (QFD) that has been used to discover, screen, rank and select suppliers for a variety of critical roles. Its applications include selection of:

Each of these projects was initiated by an engineering manager to evaluate tradeoffs in an external procurement of a mission-critical system. Each time the manager sought a relationship in which the selected supplier would be a partner contributing ideas, not just following directions.

The method focuses the selection team on what they need from the system, and evaluates the supplier only as the supplier's characteristics indicate the likely long term satisfaction with the system. It does so in two stages, a minimum (screening) level that quickly eliminates unacceptable suppliers and their systems, and a more detailed ranking level to evaluate tradeoffs among the acceptable suppliers. It need take no more calendar time than one engineer writing a specification, gathering data and writing a sole source justification, and it requires perhaps twice the man-hours. A team of six might work 1/3 time for six weeks instead of one engineer full time. The method's usefulness depends on the supplier's importance to your success more than on the system's cost.

The underlying methodology, Quality Function Deployment1,2 (QFD), is a family of methods that link actions to goals in a series of numerical tables. The supplier selection procedure differs largely in task emphasis from the product development decisions to which QFD is more often applied3. It differs first in identifying goals, because almost everyone has blind spots in describing his own needs. We are often blind both to qualities that we expect unconsciously to be present and to those that we have never seen but which could help us. It's impossible to interview or observe oneself, so supplier selection QFD must use other methods to identify the requirements comprehensively. Second, the supplier selection team compares complete alternative solutions, while product development QFD can often succeed by creating just one good solution. Third, the supplier selection team often needs to reduce a long list of prospective suppliers to a small number that can be evaluated in detail. Product development has the analogous task of filtering alternative concepts, but it doesn't need information from each of 50 companies to do so. The last difference is that supplier selection evaluates both the supplier and the system supplied.

The procedure has four kinds of tasks which are described in the following sections:

Manage the Procedure

Management of the procedure is similar to management of any other project. It entails defining the problem explicitly, delegating responsibility, providing resources including training as needed, planning and reviewing. A few of these are slightly different or more important in supplier selection.

Delegation is one of the most important decisions. A well-chosen team reduces blind spots through its diversity and promotes cross-functional acceptance of the eventual choice. The team can manage far more details carefully than any one person, and, with facilitation, it can reach a decision confidently in less time than any one of its members making an unfamiliar choice.

It is seldom possible to include on the team everyone who might have insight. Diversity is valuable in reducing blind spots, however, and an engineering team may have blind spots involving business or manufacturing issues. Purchasing departments are often very helpful with the business issues, and in the examples above, the purchasing members seemed to see in this method a welcome opportunity to influence the specification earlier than usual.

Two other management actions produce disproportionately better results and lower selection effort. These are the written definition in one or two sentences of the internal customer and his problem.

For example, in selecting the critical assembly line station mentioned above, the process development engineers on the team initially thought of themselves as the customer. After discussion, they realized that their only goal was to ensure the stations effectiveness in manufacturing. The team, therefore, defined the manufacturing engineers, managers, operators, and safety, quality and regulatory personnel as the customers. That choice influenced subsequent technical decisions.

The most useful problem definition statements describe the difference between the situation as it is seen now by the [internal] customer and as that customer would like it to be. That customer gets better solutions by stating the problem not as: "Buy a robot welder," which is not a problem but a solution to a problem that might be, "We have insufficient skilled welders to achieve our production goals." Notice that buying a robot welder is not the only solution that could be imagined to this problem, which is a good indication that this is a problem statement.

Identify Desired Benefits

A supplier benefit 4 is more than something the system will do or have or be; it's something so obviously good for this customer that no "Why?" could be asked. Instead of "3/4 inch punch depth," a benefit might be, "Able to compress any size tablets that we will manufacture." There may be nothing wrong with the first statement as a criterion; used, after careful analysis, to provide that benefit. The benefits list is used to ensure that such criteria are correct and complete, and to estimate their relative value.

The benefits identification process can change even an expert team's perception of the key issues in ways that would not be likely if they went directly to criteria as is common in non-QFD specification procedures. For example, the information systems selection team initially sought to increase internal efficiency, but discovered even greater corporate value lay in providing information that would increase its customer's performance.

The following three subsections describe the three distinct elements needed to document benefits:

Discovery

The first task is to identify comprehensively the benefits sought from the supplied system and from the supplier, itself. The most reliable discovery methods are the observation and interview of people affected by the prospective system and of experts on possible consequences. In some of the projects above, the company had never used the technology before and did not have internal experts on all aspects of it. They knew the benefits they wanted, however, such as consistent high quality from the integrated circuit supplier, and they hired a retired Vice President of QA from the semiconductor industry as a consultant to define criteria that were good predictors of consistent high quality.

The specific discovery methods, obviously, can be tailored to the projects size and needs. For small projects, the project leader and facilitator jointly interviewed internal experts to learn the problems they faced. In the largest project, the facilitator trained the team members who observed and interviewed.

Particularly for the acquisition of major systems, or for a long term relationship, discovery must capture intangible as well as tangible benefits.. Examples of intangible supplier benefits include, "Creative," "Innovative," "Responsive," "Flexible." The section on criteria will mention methods for measuring such benefits.

The interviewers culled the typed notes for individual benefits which they printed on index cards with a title, a one sentence description and one or two sentences of rationale in a different font. Figure 1 shows an example of such an index card from the information management system selection process. The number at the top is a reference to the notes that were the source of the benefit. The interview, observation and documentation of unanalyzed benefits usually takes one to four weeks.

Data Analysis

The internal experts met to scrub the data on these index cards. Each took a stack of cards. One person read a card, and they discussed it for clarity and significance. The person reading that card reworded it if necessary, then threw it in the center of the table and the next person read a card. Team members consistently enjoyed these sessions. It allowed them to learn and understand what mattered to everyone in a situation where they didn't have to decide, yet, how important it was; they needed only to agree that it was clearly a benefit to someone. These discussions improved the data and built shared understanding.

When all the cards had been read, the participants moved around the table silently associating related index cards into groups of no more than five. If a group exceeded five, they found a distinction that allowed them to separate it into two or more groups. Silence encouraged them to search for other possibilities when they disagreed with someone else's association. When the index card groups stabilized, the participants drafted a name for each group that described the collective benefit and wrote that name on a red index card. They put the red index card on top of the stack and bound the stack with a rubber band.

A typical project might generate 80 to 800 initial benefit index cards. This first step reduced it to 25 to 250 red cards, and they repeated the grouping and naming process, this time with a green card. Then they used blue cards.

That process, called affinity analysis, creates a tree of benefits bottom up. The aggregate benefits at each level are defined so they manifestly represent a real benefit, not say, "Physical properties." All objects have "Physical properties." Classifications like "rugged," "attractive," "compact," "portable," in contrast, are manifestly beneficial for appropriate problems. As a rule of thumb, it is a benefit if it makes sense in the sentence, We'll really like this product because it has <benefit name>. Creating these manifestly beneficial categories enables the team to build a comprehensive data-driven intuition for the best system, and because the benefit is manifest, the team can more easily set minimum performance standards and the value of higher performance.

Two actions particularly contribute to the usefulness of the benefits hierarchy. First, of course, is thorough data gathering. Second, is careful choice of the names of the groups of benefits. Careful definition of a group's meaning, as in the "Physical properties" example, produces groups with only truly related elements, and those meaningful groups help team members to recognize gaps and to estimate value more accurately.

A team can group and regroup multi-colored index cards easily, and that flexibility lets the data drive the solution, but to review the results, the team needs a document. The designated archivist (who is sometimes the facilitator) organizes the stacks of multi-colored index cars into a structured text file. The benefits document is more like a dictionary than a textbook. It shows the benefits hierarchy with, at each level, a name, one simple sentence declaring the meaning, one or two simple sentences explaining the reason for that benefit, and up to five component benefits with the same structure. The team gets a fresh view of their benefits hierarchy when they see the document, particularly the table of contents of the document. They often recognize better names, missing elements, and sometimes better ways to combine groups. Affinity analysis and review of the document can be completed in two or three days.

Define Performance

With the benefits document complete, the facilitator guides the team to establish consensus minimum performance standards for every benefit. Many people confuse performance and value at first. Performance describes how well the benefit is provided. Value, ultimately, is how much the company would pay. The customer may consider performance superb, and it still may not be valuable to him. Microsoftš Word and Excel, for example, have excellent macro capability. Most users, even if they know those capabilities are very good, never use them and probably would not pay extra for them. That benefit has high performance, but low value to those users.

The minimum is a level that the company might accept if other qualities were extraordinary, not the target level. Don't define a minimum standard unless you're willing to reject a supplier who doesn't meet it. Some benefits may not have an explicit minimum standard because the absence of the benefit could be accepted. In most of the above examples, the team also defined a target performance that represented the level they wanted and an ideal performance that would be worth still more.

Define Value

The team then estimates the value of higher performance for each benefit in order to rank suppliers that exceed the minimum performance. That value makes possible explicit trade-offs, and it is rooted in the real value to the company. Several variations were employed to make the task as easy as possible consistent with adequate accuracy for the project. Working top down, they estimate how much it's worth relatively to improve from minimum to target performance, which can be thought of as the penalty for being at the minimum standard instead of the higher, target standard. For one selection project, the engineering oriented team rated the top level categories as follows:

Desired Benefit Relative Penalty
Strong technically...............................61%
Good business relationship.................20%
Effective quality systems.....................16%
Responsive delivery.............................4%

The same team also evaluated the reward for being well above target for those same benefits, expanding the table to show: 

Desired Benefit Relative Penalty Relative Reward
Strong technically 61% 50%
Good business relationship 20% 30%
Effective quality systems 16% 14%
Responsive delivery 4% 6%

The team then considered the relative value of the benefit components of each of these. For example, responsive delivery included on-time delivery, short lead time for new orders, and adequate capacity to meet any likely level of demand. The team rated the penalty for falling below target (though still above minimum) as, respectively, 50%, 20% and 30%. The net penalty for capacity below the target level was then 30% times 4% or 1.2%.

Primary Benefit   Component Benefits Relative Penalty Penalty
Responsive Delivery 4% On-time delivery 50% 2%
Short lead time 20% 0.8%
Adequate capacity 30% 1.2%

The use of penalty and reward is discussed in more detail elsewhere.6 Sometimes it's so difficult to reach the target that only penalty is sufficient to rate the desired outcomes.

Assess Accurately

A criterion is an objectively true or false declarative English sentence. The objectivity contrasts to the benefits, which may be very subjective. For example, a criterion might read, "The prospective supplier has supplied at least three references to clients in our industry for whom the supplier has installed systems within the last three years." The team will write both screening criteria and ranking criteria.

Screening to Eliminate Unsuitable Suppliers

For every benefit with a minimum performance standard, someone on the team must write one or more criteria that are equivalent to the minimum standard of the benefit, in the sense that a supplier who satisfies the criterion will meet the minimum standard on that benefit. Many of the resulting criteria will be simply requirements that almost any supplier could meet if it is specified. Those criteria don't help reduce the list of candidates or to rank them, but they should be saved to include in the eventual contract, so move them into a separate file called something like "Acceptance Requirements." For your first questionnaire, pick the criteria that are most likely to elicit a negative answer. Screening criteria need no weights, since all acceptable companies answer true to all screening criteria.

A single answer of "False" to a screening criterion eliminates a candidate supplier. That's very helpful when you need to reduce 50 suppliers to five to be evaluated carefully. Such reduction can normally be done with a list less than a page long which you can fax to suppliers with a simple cover letter. You use your resources and those of the prospective suppliers best by eliminating unsuitable ones as easily as possible. You can't make a mistake as long as the property on which you screen is objective, represents something you really must have, and will be answered in the negative by some respondents. Satisfaction of the remaining criteria can be verified in either a follow-up questionnaire if they are in doubt, or by the selected vendor prior to signing a contract.

Ranking Criteria

Ranking criteria are more difficult. Some important intangible benefits, like "Innovative," are not suited for screening because of their complexity, but may make a big difference in ranking. One of the above teams, looking for a supplier to develop an automated production facility for a type of product nobody had ever made before, signed contracts with three finalists to do a preliminary design and then evaluated the designs. They emerged with a supplier who was very successful and responsive in the many changes needed before production. Another team, knowing that the supplier that they had used during development was not able to meet production quality or volume requirements, asked candidates how they would redesign the custom integrated circuit their first supplier had developed.

To determine which of the acceptable suppliers offers the best combination of benefits, the ranking criteria must completely represent the benefits. The team uses a QFD matrix to connect the list of criteria to the list of benefits to verify completeness and to obtain the weight for each criterion. For those of you who haven't seen such a matrix, it can be explained simply. It is a table with the rows (horizontal) labeled by the benefits and the columns by the criteria. The intersection of a row and column is left blank if satisfaction of that criteria has no important relationship to that benefit, given a 1 if the criterion has a significant effect, 3 for an important effect, and 9 for a criterion that is very effective in assessing whether the benefit will be provided. The team estimates the effectiveness of the criterion with these numbers. To be valuable, a criterion must be effective in measuring one or more valuable benefits.

To infer that criterion value, these teams divided the value of each benefit among all the criteria that contributed to it in proportion to the criteria's 1, 3, 9 effectiveness ratings. That's a modified7 QFD matrix which results in a perfect supplier receiving a score of 100%, instead of the indefinite and much larger number that conventional QFD produces.

Evaluation

To apply the ranking criteria, the team needs data from the suppliers and, often, their current and previous customers. These often cannot be obtained just from sales literature. This is the time for meetings, visits and even trial contracts like those mentioned above.

These teams assembled the answers to the criteria in a spreadsheet, putting each group of related criteria, such as quality, business stability, etc., on its own separate sheet in a single workbook. They used one sheet in that workbook to add the scores from each category. When a candidate appeared weak, that weakness could be traced first to a group and then to specific criteria. The team could consider, first, whether their evaluation was accurate, and, second, whether the deficiency could be corrected.

Two primary correction methods were considered. First, would the supplier accept assistance in, for example, achieving the desired level of process documentation? Second, could the supplier be combined with a second supplier with a different kind of expertise to provide both, say, high quality chip manufacturing and sophisticated testing? If only small differences remained within the finalists - not a bad result - the team knew that any of the finalists would do a good job, and the decision could be made on price.

Conclusion

The team has to work very hard to determine the benefits, the minimum standards on each and the value of higher performance. That is a fact of the problem, not a limitation of the method. Writing criteria for screening is relatively easy. Writing criteria for ranking and linking them to the benefits is a significant effort, but it has high value in ensuring completeness and providing accurate weights. The screening and selection processes, themselves, then go relatively smoothly and the decision can be made confidently. Because of the depth of the benefits analysis, the procedure can produce enormous savings, by, for example, in one of these projects preventing the loss of tens of millions of dollars in sales and delays of months or years if a supplier didn't meet quality standards or long term production rate.

The method's value lies in making possible successful choices through accurate knowledge of your own problem, even when you're not deeply knowledgeable about your supplier's business. It also promotes broader based acceptance of the choice, an acceptance that can smooth implementation later.

About the Author

Bill Pardee is an author, speaker and project facilitator on the subject of customer requirements and the use of customer value in systems decisions. Before starting Pardee Quality Methods in 1992, he was Principal Scientist of Information Sciences at the Rockwell International Science Center. He was a 1987 Rockwell International Engineer of the Year. His book, To Satisfy and Delight Your Customer: How to Manage for Customer Value, was published by Dorset House in 1996.

1 Bob King, Better Designs in Half the Time, 3rd edition, GOAL/QPC, Metheun (1989).

2 Lou Cohen, Quality Function Deployment: How to Make QFD Work for You, Addison Wesley, Reading (1995)

3 William J. Pardee, To Satisfy and Delight Your Customer: How to Manage for Customer Value, Dorset House, NY (1996).

4 Many versions of QFD call these benefits the root needs, the needs that underlie the customers requests.

5 The author of this article was the facilitator on each of the projects described. That facilitator role, a process guide who helps the team make good decisions without representing any specific technical perspective, can be played by someone inside the corporation.

6 William J. Pardee, To Satisfy and Delight Your Customer: How to Manage for Customer Value, Dorset House, NY (1996).

7 W. J. Pardee, op. cit.

Return