Visual Six Sigma: making data analysis Lean

Contact (s)

By Malcolm Moore, Andy Liddle & Andrew Ruddick | Published: 20 Feb 08

Introduction

This paper introduces the idea of “Visual Six Sigma”, a practical and pragmatic approach to data analysis and process improvement. This approach has been developed in response to a growing business need to broaden the use of six sigma type thinking beyond the realms of highly trained and “statistically savvy” Black Belts and Green Belts.

In the typical business environment of process improvement, what people are looking for today are simple to use tools that can be widely used by everyone at all levels to rapidly explore and interpret data, and then use that understanding to drive improvement. By making these tools highly visual and engaging we can accelerate the process of analysis and eliminate the need for advanced statistical analysis in all but the most complex of situations.

We can also broaden and deepen the application of Six Sigma thinking in the organisation by making the tools intuitive, easy to use, and the results easy to interpret.

This article describes and illustrates the Visual Six Sigma approach based around a case study, but first lets set the scene typical of many business environments and ask a critical question:

So Be Honest... How Much Heavy Duty Statistics Do We Really Need To Drive Process Improvement?

Many of you will be familiar with the Dabawalla story. A story of Bombay’s extraordinarily efficient lunch system which has operated for more than a century. Last spring The Times (UK) reported :

"Just after 11am every workday, Bombay’s famous dabbawallas stream off the city’s railway network into the downtown business district to deliver hot, home-made meals to an army of hungry office workers. Carrying tiffin boxes lovingly packed by wives and mothers in nearly 200,000 surburban kitchens, these 5,000 lunch delivery workers are part of one of the world’s most admired distribution systems. Employing a complex colour-coded logistics process, the dabbawallas (can-carriers) complete a door-to-door service across 15 miles (25km) of public transport and 6 miles (10km) of road with multiple transfer points in a three-hour period."

In a system finely tuned over 120 years they maintain an error rate of only one in eight million ( >7 sigma performance)... and they do this without statistical analysis at all!

In a recent analysis of lean six sigma deployment in a large multinational we also made some very interesting observations (see figure 1). Not only are about 80% of the typical business population either terrified or very uncomfortable with statistics, also > 80% of the project value comes from projects where only very basic tools and/or modest statistical analysis was required to identify and deliver the improvement.

Figure 1: Comfort levels with statistical methods

So, if most people are terrified of statistical methods and try to avoid the methods taught at black belt level and above, how can we make data analysis simple, quick, intuitive, practical and engaging for the typical business so that they achieve data driven solutions rapidly and with minimum overhead.

In this article we describe the approach we call “Visual Six Sigma” based around exploiting the capabilities of JMP software. This approach focuses heavily on using a range of very powerful and easy to understand visual tools to rapidly identify “Hot Xs”. The statistical rigour can be used (but only as much as required) to underpin this and then to easily build models to simulate and do “what ifs” to assess improvement opportunities.

Not only is this software analysis package easy to use – Its visual capabilities and accessible output makes the findings very easy to communicate and to engage with leaders in getting support for the improvement activities….. another challenge in many continuous improvement deployments!

The Visual Six Sigma approach is described in section 3 and then demonstrated in some detail in section 4 based around a fictional (but fairly typical case study).

Lean Data Analysis Process

Figure 2 presents our lean data analysis process. This starts by framing the problem with regard to the process inputs and outputs that need to be measured, this data is then collected and managed using measurement system analysis and data management methods. Once the data is clean and free of large measurement errors, visualisation methods are used to uncover the hot X’s (the process inputs responsible for driving variation in product quality or associated with variation in product quality). Statistical models are optionally developed for more complicated problems, our process knowledge is then revised using the visual and statistical models developed in steps 3 and 4, this increased understanding is also utilized to improve product or service quality.

Figure 2: Lean Data Analysis Process

Case Study

A fictional case study based on simulated data is presented, a copy of which is available on request from the authors. The scenario around which the data has been simulated is fairly typical of call centres, the situation is not based on any particular case, but does try to reflect the realities of analysing and improving call centre processes.

Situation

The particular scenario relates to the handling of customer queries via an IT call centre. Prior to initiating a call, a customer may or may not have attempted to resolve the issue via alternative contact mechanisms such as FAQ via a web site. Due to customer demand, the particular call centre does not operate the traditional answering service whereby the details of the call and issue are logged and then passed onto a service engineer capable of solving the problem. Instead the first call is taken by a service engineer who becomes responsible for finding a solution.

Benchmark analysis (not presented here) indicated that “The Company” had lower customer satisfaction ratings than best in class competitors and that higher levels of customer satisfaction are driven by call centre performance with respect to the speed with which calls are answered and problems correctly resolved. Further customer satisfaction is the top driver of product revenues, and it is estimated that 8% revenue growth is possible if “The Company” matches call centre performance of best in class. The performance goals to match best in class performance are:

Time to answer should be no more than two minutes.
65% of calls must be solved in one iteration with a maximum service time of 1.5 hours; 25% of all calls must be solved with no more than two iterations with a maximum total service time of 5 hours; and 99% of all calls must be solved with no more than three iterations with a maximum total service time of 15 hours.

Figure 3 summarises the distributions of time to answer, and time to solve problems for service calls received in the prior month and indicate performance alongside these specifications is disappointing. It takes more than 2 minutes to answer 93.7% of calls and very few problems are solved within the required time, irrespective of the number of times a customer calls into the call centre before the problem is solved (iterations or cycles). Substantial reductions in time to answer calls and time to solve problems are necessary to meet the new performance specifications.

Figure 3: Process Capability Analysis

A team was commissioned to investigate the call centre process and dramatically improve process capability. The team applied the lean data analysis process in figure 2 and used a Cause and Effect Analysis to frame the problem with regard to identification of the potential causes of excessive time to answer and time to solve as illustrated in figure 4. After voting the team circled the process inputs in red that were prioritized as the most likely causes and were readily available in the call centre database.

Figure 4: Key inputs associated with excessive variation in time to answer and time to solve.

Data on these inputs along with time to answer, time to solve, and number of cycles were collated for service calls received in the prior calendar month, which resulted in a data set consisting of 2836 rows.

Figure 5 shows simple histograms of each variable with calls having a time to answer of under two minutes identified with darker shading. This tells us that the only time we answer calls in under two minutes is on Tuesday and Wednesday afternoons when two staff are on duty. Further we never answer calls in under two minutes when three staff are on duty.

Figure 5: Visual Exploration of Relationships with Time to Answer

This prompts us to determine what is different about the operation of the call centre when three staff are on duty by clicking on the bar for three staff on duty to hi-light the circumstances associated with calls received when three staff are on duty as illustrated in figure 6. This shows that we only have three staff on duty on Friday mornings which is one of the busiest periods.

Figure 6: Calls received when three staff are on duty highlighted

The distribution of time to answer when there are three staff on duty has a mean of approximately 2.6 minutes with a range from 2 to 3.2 minutes. Thus it appears that on some occasions the staff loading of the call centre has been adjusted to compensate for higher call volumes, however the loading has not been optimised to achieve the new performance goals. To determine the impact of increasing staff loading by one person it is necessary to perform the comparison between periods of similar call volumes. The tabulation in table 1 indicates an equivalent call volume occurs on Monday morning.

Table 1: Volume of Calls by Day, Time of Day and Number of Staff on Duty

By alternately clicking on the Monday morning and Friday morning cells within the summary table, we see the impact of increasing the number of staff on duty by 1 between periods of equivalent activity as illustrated in figure 7.

Figure 7: Visual Comparison of Time to Answer on Friday Morning vs Monday Morning

The impact of increasing number of staff on duty by one is to reduce mean time to answer by roughly one minute. This visual model is confirmed by clicking on other cells with similar call volume but differing number of staff on duty, e.g. Wednesday afternoon vs. Thursday afternoon, but is not shown in the interest of brevity.

The distribution of time to solve has three clusters of data points: the first with a mean of around 2 hours is associated with calls solved in the first cycle; the second with a mean of around 7 hours is associated with calls solved in the second cycle; and the third with a mean of around 24 hours is associated with calls solved in the third cycle. Thus a separate visual analysis was performed for each of the three sub-groups defined by number of cycles.

Figure 8 shows simple histograms of each variable when number of cycles is one with calls with a time to solve of less than 1.5 hours identified with darker shading. This indicates that staff experience followed by type of problem, are the top drivers of variation in time to solve.

Figure 8: Visual exploration of relationships with time to solve when number of cycles = 1

Another useful visual exploratory tool is recursive partitioning. This method repeatedly partitions data according to a relationship between the input variables and an output variable, creating a tree of partitions. It finds the critical input variables and a set of cuts or groupings of each that best predict the variation in batch failures. Variations of this technique are many and include: decision trees, CARTTM, CHAIDTM, C4.5, C5, and others.

Figure 9 shows the resulting decision tree using recursive partitioning to explore the main drivers of variation in acceptable time to solve when the number of cycles is 2, i.e. what is associated with calls with a solution time of less than five hours when number of cycles is 2. The hot X’s are confirmed as staff experience and type of problem. Each node of the decision tree represents a sub-group, the criteria by which the sub-group was determined and the probability of solving problems in less than 5 hours. The four nodes of the decision tree tell us:

All calls routed to staff at level 3 experience are solved in less than five hours.
56% of calls routed to staff with level 2 experience are solved in less than five hours when the problem is category B or C.
41% of calls routed to staff with level 2 experience are solved in less than five hours when the problem is category A.
None of the calls routed to staff with level 1 experience are solved in less than five hours.

Figure 9: Recursive Partitioning Decision Tree

These two visual data exploration methods have collectively identified the hot X’s:

Day, time of day and number of staff on duty are the drivers of variation in time to answer.
Staff experience and type of problem, are the drivers of variation in time to solve and analysis of time to solve is conditional on the number of cycles.

Visual analysis has also indicated some potential solutions:

Determine staff loading required in each of the 10 time periods defined by the combination of day and time of day to ensure calls are consistently answered in less than two minutes using the approximate rule that an increase in one staff on duty will reduce mean time to answer by one minute.
Ensure time to solve is acceptable by training call centre staff to the equivalent of level 3 experience.

The effects of this subset of input variables upon time to answer and time to solve were investigated in more detail using multiple linear regression in figure 10. Day, time of day and number of staff on duty are confirmed as the drivers of variation in time to answer, explaining 80% of the variation. Staff experience, type of problem and number of cycles are confirmed as the drivers of variation in time to solve, accounting for 98% of the variation.

Figure 10: Multiple Regression Analysis Summaries

Only two of these variables are under the control of “The Company” – number of staff on duty and staff experience. To understand if a viable solution is possible by controlling these variables alone Monte Carlo simulations were performed using the regression model as the transfer function between the key process inputs, time to answer and time to solve. The results of the simulation when three staff are on duty in all periods and the experience of these staff is level 3 are indicated in figure 11. Zero defects with regard to performance levels for time to solve are achieved, however 42% of calls have an answer time in excess of two minutes.

Figure 11: Monte Carlo Simulation - click to enlarge (opens in pop-up window)

Figure 12 shows a distribution analysis of the simulated data with calls with an answer time in excess of two minutes hi-lighted, which indicates the poor performance with regard to time to answer is primarily occurring on Monday and Friday mornings.

Figure 12: Visual analysis of drivers of excessive time to answer from simulated data

The simulation model was re-run with four staff at level 3 experience for Monday and Friday mornings and predicted 4.8% of calls with an answer time in excess of 2 minutes. A defect free process is predicted with the following settings:

Staff experience level equivalent to grade 3 at all times
5 staff on duty on Monday and Friday mornings
4 staff on duty on Monday and Friday afternoons; Tuesday, Wednesday, and Thursday mornings
3 staff on duty at all other times

Summary

In the case study a variety of highly visual data exploration techniques were used to identify critical process parameters . This was combined with data mining tools to identify tighter control ranges of key parameters that result in more consistent service quality. Multiple regression modelling and Monte Carlo simulations then identified tighter control regions of key process variables that predict a near defect free process

Hopefully, from this example you can see how this visual approach really facilitates rapid exploration of the data to quickly drive focus onto the Hot X’s. These methods are simple to understand and train and are capable of being deployed intelligently by a wide community of users. The modeling and simulation capability then builds on that to allow very rapid “what iffing” and investigation of different improvement options.

So, if you have a concern that your data analysis is taking too long, or that your six sigma programme relies too heavily on a small number of highly trained and highly skilled BBs to carry out the data interpretation & statistical testing, then we would recommend that you consider the “Visual Six Sigma Approach” as a practical and pragmatic alternative.

If you would like to contact the authors, you can do so by using the form at the top of the page, or the menu on the right hand side.

Attachment	Size
CATA-visualsixsigma.pdf	2.3 MB

Popular forum topics

Language list