There are many process performance metrics that we can use to measure the current and future value of our processes.

Before we look at the metrics, though, let's review some of the terms that we'll be using. It's important that you understand these terms as they're used in Six Sigma.

Terminology

  • Defect: An end result (often a product) that doesn't fall within a pre-defined acceptable range of values. For a physical product, these values might be strength and/or size measurements. For a service, these values might be KPIs like turnaround time. In a general sense, think of a defect as a failure to meet customer expectations of quality.
  • Opportunity: A chance to add value for a customer. This applies to any situation in which your company can improve the perceived value of a product or service. For example, if you're manufacturing a product to customer specifications, your opportunities might be:
    • Matching customer's physical requirements
    • Meeting deadlines
    • Providing a free courier service straight to the customer.
  • Unit: A tangible result to a customer: a single service or product. For example, a customer's phone call to your service department is a unit.
  • Yield: The yield is the percentage of opportunities that were successfully met. Another way to look at yield is the percentage of processes that were defect-free.
    Yield = (Opportunities - Defects) / Opportunities

Opportunity vs Unit

It's easy to get opportunities and units mixed up. The simplest way to differentiate them is to think about customer needs and requirements.

The item they need is the unit.

The specifications for that item (color, size, materials, time to delivery) are the opportunities.

You might have several opportunities per unit. For example, think about a customer calling a service line to get help with a piece of software. The unit, in this case, is the phone call. The opportunities could be:

  • Answering the phone call within an acceptable time range.
  • Having the phone call directed to an appropriate person.
  • Being able to help the customer with their problem.
  • Leaving the customer with a positive experience.

Defects per Opportunity (DPO)

DPO stands for Defects per opportunity. This is a simple ratio. Take the number of defects you have in your process (usually found by sampling) and divide it by however many opportunities there are. Then express the ratio as a percentage.

DPO = Defects / Opportunities

Complex processes and items often have many opportunities to add value. Hence, you might have several defects per process or item.

For example: a fast food drive-through order. The customer has a lot of criteria on which they'll judge the experience as a whole:

  • Staff attitude
  • Speed of service
  • Accuracy (do the received items match the order?)
  • Presentation
  • Taste of the food
  • Freshness/heat of the food.

Each of these criteria is an opportunity to add value for the customer. A failure in any of these criteria detracts value and counts as a defect. If the customer has to wait ten minutes (defect in speed of service), receives the wrong burger (defect in accuracy), and their burger is luke-warm (defect in freshness/heat), three defects have each detracted value from the end result.

Potential gotcha: Make sure that you're using defects and opportunities for the same population. For example, if you're counting defects in a sample of 500 products, the number of opportunities must also apply only to that sample of 500 products.

Worked example of Defects Per Opportunity

Let's say Joe's Burgers serves 1,000 customers a day. The company has identified its opportunity types as:

  • Accuracy
  • Speed
  • Freshness
  • Taste.

For example, in a single day, 50 customers had the wrong order, 75 felt they waited too long, 25 said their order was cold, and 50 more said their burgers just tasted bad.

Some of that feedback might have overlapped; customers might have received the wrong order and waited too long for it. That doesn't matter for this metric, because each order has multiple opportunities attached to it.

To calculate the number of opportunities, multiply the number of orders (1000) by the number of opportunity types (4) to get 4000.

DPO = number of defects / number of opportunities = 200/4000 = 0.05 = 5%

Defects per Unit (DPU)

Defects per unit is a similar calculation to DPO, but instead of looking at opportunities, we're looking at units. See Opportunity vs Unit above if you're getting confused.

Let's look at the Joe's Burgers example again, but this time from a DPU viewpoint.

Worked example of Defects Per Unit

Let's say Joe's Burgers serves 1,000 customers a day. In a single day, 50 customers had the wrong order, 75 felt they waited too long, 25 said their order was cold, and 50 more said their burgers just tasted bad.

Some of that feedback might have overlapped; customers might have received the wrong order and waited too long for it. That doesn't matter for this metric. We're just looking for a basic ratio.

(Note: it would matter if we were talking about defective units per 1000, for example.)

The number of units is the number of orders (1000).

DPU = number of defects / number of units DPU = 200/1000 = 0.2 DPU = 20%

Defects per Million Opportunities (DPMO)

DPMO is based on DPO, but in real-world manufacturing type numbers. This is an important metric, because it's used in Six Sigma to measure the performance of a process. For example, Joe's Burgers would use DPMO to figure out how successful their current serving process is, and how it compares to other serving processes they might have tried.

Defects per Million Opportunities is typically extrapolated from a sample. In the worked examples above, we looked at Joe's Burgers using a single day's orders as a sample – 1000 units. If you've already worked out DPO for a sample, you can calculate DPMO by simply multiplying the decimal result (not the percentage) by one million.

DPMO = DPO * 1000000

Alternatively, you can use this equation:

DPMO = (Defects / Opportunities) * 1000000

Or:

DPMO = (Defects / (sample size * opportunities per unit)) * 1000000

Worked example of Defects Per Million Opportunities

Joe's Burgers serves 1,000 customers in a day. The company has identified its opportunity types as:

  • Accuracy
  • Speed
  • Freshness
  • Taste.

In a single day, 50 customers had the wrong order, 75 felt they waited too long, 25 said their order was cold, and 50 more said their burgers just tasted bad.

Number of defects: 50 + 75 + 25 + 50 = 200 Sample size: 1000 Opportunities per unit: 4 DPMO = (Defects / (sample size * opportunities per unit)) * 1000000 DPMO = (200/(1000 * 4)) * 1000000 DPMO = 50000

How to Calculate 6 Sigma level based on DPMO

There are two basic ways that you can determine the Six Sigma level from your DPMO figure:

  • Look at an appendix table.
  • Use the equation.

Six Sigma Level Equation

Use this equation to calculate your process's Six Sigma Level based on its DPMO:

Level = 0.8406 + √(29.37 – (2.221 * ln(DPMO)))

Note: If you're wondering where the numbers 0.8406, 29.37 and 2.221 come from, they are constants that help us calculate the DMPO or level simply.

Process Sigma = 0.8406 + SQRT(29.37 – 2.221 * (ln(DPMO))).
Reference: Breyfogle, F., 1999.Implementing Six Sigma: Smarter Solutions Using Statistical Methods. 2nd ed. John Wiley & Sons.

Worked example of calculating Six Sigma Level

Let's start with the DPMO figure from Joe's Burgers: 50000.

Firstly, if you haven't come across ln before, it means that you need to find the natural logarithm of the number – in this case, the DPMO. Use a scientific calculator. In this case, the natural logarithm of 50000 to 4 decimal places is 10.8198.

Secondly, we need to plug that into the equation:

Level = 0.8406 + √(29.37 – (2.221 * ln(50000)))  Level = 0.8406 + √(29.37 – (2.221 * 10.8198)) Level = 0.8406 + √(29.37 - 24.0307) Level = 0.8406 + √(29.37 - 24.0307)  Level = 0.8406 + √5.3392 Level = 0.8406 + 2.3106 Level = 3.1513

First-Time Yield (FTY)

First-time yield is the most common way to calculate the process yield. It is the number of defect-free units coming out of a process, compared to the number of units manufactured. In other words, it is the probability of a defect-free output from a process.

FTY does not capture how many defects are reworked in each stage. In other words, It doesn't include units that need to be reworked in the defect-free units. It will not detect the effect of hidden factors.

Simply, FTY is the number of good parts produced divided by the total number of parts going into the process.

How to calculate FTY for a process

  1. Calculate the first-time yield for each step in the process, based on the number of defect-free units going into each step (typically each step will have fewer units than the preceding step).
  2. Multiply the FTYs together to get the total first-pass-yield.
FTYt                = FTY1                * FTY2                * FTY3                * ... * FTYn

Where:

  • FTY t is the total First Time Yield for the process.
  • n is the number of steps in the process.

Worked example of calculating FTY

Example: In a manufacturing plant, 100 parts are entered into the first process, 2 are scrapped, and 98 defect-free parts go to the next stage. 98 parts enter the second process, 5 are scrapped, and 93 defect-free parts go to the next stage. 93 parts enter into the third process, 10 scrapped and 83 defect-free parts are going to the next stage. Find the First time yield.

  • First process: 100 parts enter into process; 2 scrapped, So output 98 parts; FTY1 = (100 – 2)/100 =98/100= 0.98
  • Second process: 98 parts enter into process; 5 scrapped, So output 93 parts; FTY2 = (98 – 5)/98 =93/98= 0.9489
  • Third process: 93 parts enter into process; 10 scrapped, So output 83 parts; FTY3 = (93 – 10)/93 =83/93= 0.8924

Total First Time Yield = 0.98*0.9489*0.8924= 0.829 ~ 83% yield

FTY also can calculate the final number of defect-free parts divided by the number of parts in the start of the process = 83/100 = 83%

Rolled Throughput Yield (RTY)

Rolled Throughput Yield is a great way of seeing how healthy a process is. RTY provides a probability that a unit will be generated by a process with no defects. In other words, it's the probability that a multi-step process will produce a defect-free unit.

Rolled Throughput Yield is more powerful as it is sensitive to defects, which means instead of being based on the yield on units, it uses the number of defects at each step (even the defective part is corrected, but still it will count in the calculation of RTY). It is valuable to the organizations as most of them consider only the successful/passed units although inherent muda are present. RTY considers total defects in the entire process.

Before calculating RTY, you need to complete two important steps:

  • Map the process so that you know how many steps it has.
  • Take samples at each stage of the process and test for defects, so that you have data for the calculation.

Example

A sequence of 3 operations has first pass yield (right first time) rates as follows:

  • 1st step: 93%
  • 2nd step: 87%
  • 3rd step: 92%.

In other words, the first step in a process has a 93% chance of completing correctly. The 2nd has only an 87% chance. And the third process step has a 92% chance.

The first pass yield rate for the whole process is the chance of each step multiplied.

RTY = 93% * 87% * 92% = 74%

Each step by itself had a good chance of being acceptable. But when you take a look at the entire system, you see that the cumulative errors take a toll. In the above example, any item that the process produced only had a 74% chance of passing through without error or rework.

Worked example of calculating Rolled Throughput Yield

Example: In a manufacturing plant, 100 parts are entered into the first process, 2 are scrapped, and 5 are reworked to get the 98 parts to the next stage. 98 parts enter into the second process where 5 are scrapped and 10 reworked to obtain 93 parts. 93 parts enter into third process, where 10 scrapped and 5 reworked to get 83 parts. Find the Rolled throughput yield.

  • First process: 100 parts enter into process; 2 scrapped and 5 reworked, So output 98 parts; RTY1 = (100 – (2+5))/100 =93/100= 0.93
  • Second process: 98 parts enter into process; 5 scrapped and 10 reworked, So output 93 parts; RTY2 = (98 – (5+10))/98 =83/98= 0.85
  • Third process: 93 parts enter into process; 10 scrapped and 5 reworked, So output 83 parts; RTY3 = (93 – (10+5))/93 =78/93= 0.84

So, the Total Rolled Throughput Yield = 0.93*0.85*0.84= 0.664 ~ 66% yield

Calculating DPU from Rolled Throughput Yield

In addition to the Defects Per Unit calculations above, you can use a process's RTY to calculate its Defects Per Unit. Use this equation:

DPU = -ln(RTY)

Reminder: ln is the natural log.

Worked example of calculating DPU from RTY

From the above manufacturing example, RTY is approximately 69.5%. Then to get the DPU, we plug that into the equation:

DPU = -ln(0.695) DPU = 0.3638

Using Process Performance Metrics

Once you understand the process performance metrics available, and how to use them, then the next step is to decide which metrics to actually use.

Primary vs Secondary Process Performance Metrics

Primary and secondary metrics aren't static. Each organization will have its own priorities when it comes to process performance metrics, leading to different lists of primary and secondary metrics. These lists will even change from time to time, as an organization's focus and business objectives change.

Generally speaking, then, primary process performance metrics are the ones that your organization decides to focus on most. The rest are secondary process performance metrics.

Questions to Ask When Designing Your Process Performance Metrics Program

The following information was emailed to me and attributed to Will Poats of the CEB.

Accounting and Reporting teams are always looking to include the most relevant metrics in their management reports, so they should avoid the three following missteps when incorporating metrics: wrong metric selection, lack of process ownership, and ineffective reporting practices.

Below are 13 questions you can ask yourself and your team members when designing or strengthening your metrics program.

  1. Have we chosen no more than 12 to 15 metrics for decision making among those suggested by the corporate finance leadership team and business partners?
  2. Do we agree on the definition for each metric, the reason for tracking it, and the target we are aiming for?
  3. Can we link each metric back to a business or corporate goal?
  4. Have we tested the metrics with our stakeholders to ensure they understand the definition and have a clear sense of how they will use the metric to make decisions?
  5. Check whether we incorporated both qualitative and quantitative metrics into our decision-making framework?
  6. Do we know which drill-down options are most useful for each metric?
  7. Also, do we know which styles of metrics presentation each stakeholder prefers?
  8. Have we run each metric during a trial period to establish a baseline?
  9. Check whether we agreed on refresh triggers or cycles for each metric to prevent them from becoming obsolete?
  10. Do we have a metric that can predict future problems?
  11. Have business partners feel comfortable alerting Finance when metrics need to change and adapt to the new environment?
  12. Do we eliminate old metrics when new ones become more representative?
  13. Finally, Do we update our business partners on metrics we will no longer be tracking when the dashboard changes?

Six Sigma Black Belt Certification Process Performance Questions:

Question: Which of the following performance measures is most appropriate for evaluating the tangible effects of a six sigma project? (Taken from ASQ sample Black Belt exam.)

a) Cycle time

b) Team member absentee rate

C) Employee morale

D) Unsolicited compliments from customers

Answer:

Question: Which of the following is the correct formula for DPMO?

(A) D/TOP
(B) DPO X 1,000,000
(C) D X U XOP
(D) DPU/DPO

Answer:

ASQ Six Sigma Green Belt Process Performance Questions

Question: Which of the following measures is used to show the ratio of defects to units?
(A) DPU
(B) DPO
(C) DPMO
(D) PPM

Answer: