«Using Poka-Yoke Techniques for Early Defect Detection Harry Robinson Paper presented at the Sixth International Conference on Software Testing ...»
Go to Poka-Yoke Page
Using Poka-Yoke Techniques
for Early Defect Detection
Paper presented at the Sixth International Conference on Software Testing Analysis and Review (STAR'97)
Poka-yoke is a quality assurance technique developed by Japanese manufacturing engineer Shigeo Shingo. The
aim of poka-yoke is to eliminate defects in a product by preventing or correcting mistakes as early as possible.
Poka-yoke has been used most frequently in manufacturing environments.
Hewlett Packard currently develops its Common Desktop Environment software to run in twelve locales or languages. Traditional testing of this localized software is technically difficult and time-consuming. By introducing poka-yoke (mistake-proofing) into our software process, we have been able to prevent literally hundreds of software localization defects from reaching our customers.
This paper describes the poka-yoke quality approach in general, as well as our particular use of the technique in our localization efforts. Poka-yoke is providing a simple, robust and painless way for us to detect defects early in our localization efforts.
Poka-yoke History Poka-yoke (pronounced "POH-kah YOH-kay")  was invented by Shigeo Shingo in the 1960s. The term "poka-yoke" comes from the Japanese words "poka" (inadvertent mistake) and "yoke" (prevent) . The essential idea of poka-yoke is to design your process so that mistakes are impossible or at least easily detected and corrected.
Shigeo Shingo was a leading proponent of statistical process control in Japanese manufacturing in the 1950s, but became frustrated with the statistical approach as he realized that it would never reduce product defects to zero.
Statistical sampling implies that some products to go untested, with the result that some rate of defects would always reach the customer.
While visiting the Yamada Electric plant in 1961, Shingo was told of a problem that the factory had with one of its products. Part of the product was a small switch with two push-buttons supported by two springs.
Occasionally, the worker assembling the switch would forget to insert a spring under each push-button.
Sometimes the error would not be discovered until the unit reached a customer, and the factory would have to dispatch an engineer to the customer site to disassemble the switch, insert the missing spring, and re-assemble the switch. This problem of the missing spring was both costly and embarrassing. Management at the factory would warn the employees to pay more attention to their work, but despite everyone's best intentions, the missing spring problem would eventually re-appear.
Shingo suggested a solution that became the first poka-yoke device :
In the old method, a worker began by taking two springs out of a large parts box and then assembled a switch.
In the new approach, a small dish is placed in front of the parts box and the worker's first task is to take two springs out of the box and place them on the dish. Then the worker assembles the switch. If any spring remains on the dish, then the worker knows that he or she has forgotten to insert it.
The new procedure completely eliminated the problem of the missing springs.
Shingo went on to develop this mistake-proofing concept for the next three decades. One crucial distinction he made was between a mistake and a defect. Mistakes are inevitable; people are human and cannot be expected to concentrate all the time on the work in front of them or to understand completely the instructions they are given. Defects result from allowing a mistake to reach the customer, and defects are entirely avoidable. The goal of poka-yoke is to engineer the process so that mistakes can be prevented or immediately detected and corrected. Poka-yoke devices proliferated in Japanese plants over the next three decades, causing one observer
to note :
It is not one device, but the application of hundreds and thousands of these very simple "fail-safing" mechanisms that day after day has brought the quality miracle to Japan. Each one is relatively simple -- something you easily could do on your own. It is the totality, the hundreds of devices, that is almost frightening to behold.
Categories of poka-yoke devices Poka-yoke devices fall into two major categories: prevention and detection.
A prevention device engineers the process so that it is impossible to make a mistake at all. A classic example of a prevention device is the design of a 3.5 inch computer diskette.The diskette is carefully engineered to be slightly asymmetrical so that it will not fit into the disk drive in any orientation other than the correct one.
Prevention devices remove the need to correct a mistake, since the user cannot make the mistake in the first place.
A detection device signals the user when a mistake has been made, so that the user can quickly correct the problem. The small dish used at the Yamada Electric plant was a detection device; it alerted the worker when a spring had been forgotten. Detection devices typically warn the user of a problem, but they do not enforce the correction.
We are surrounded every day by both detection and prevention poka-yoke devices, though we may not usually think of them as such. My microwave will not work if the door is open (a prevention device). My car beeps if I leave the key in the ignition (a detection device). At few years ago, some cars were designed not to start until the passengers had buckled their seat belts (a prevention device); but this mechanism was too intrusive and was replaced by a warning beep (a detection device).
Characteristics of good poka-yoke devices
Good poka-yoke devices, regardless of their implementation, share many common characteristics :
they are simple and cheap. If they are too complicated or expensive, their use will not be cost-effective.
they are part of the process, implementing what Shingo calls "100%" inspection.
they are placed close to where the mistakes occur, providing quick feedback to the workers so that the mistakes can be corrected.
Judged by these criteria, the "small dish" solution to the missing-spring problem is an excellent poka-yoke device:
It was simple.
It was cheap, involving only the cost of a small dish.
It provided immediate feedback about the quality of the work; corrections could be made on the spot.
Further reflections on the small dish solution
The small dish solution used at Yamada Electric is typical of many poka-yoke devices:
It did not merely examine switches at the end of the operation; it changed the procedure for assembling switches. The additional step of putting the springs into the dish slowed down the individual operation, but the increased reliability of the assembly eliminated the need for rework and therefore sped up the overall process.
It was designed to stop a particular mistake -- a worker forgetting to insert a spring. It did not stop all possible mistakes. It would not detect, for instance, a situation where the worker, after removing the springs from the dish, accidentally dropped one on the ground without noticing.
Being a detection device, the small-dish solution was not completely error-proof. It could only warn of a problem, relying on the worker to correct the situation. Unlike the 3.5 inch diskette, this solution did not make it impossible to assemble a switch incorrectly. A worker wishing to ignore the warning could do so.
The solution dealt with aspects of the assembly that were necessary, though not sufficient, for correct operation of the product. This poka-yoke ensured only that each push-button had a spring under it; it did not attempt to detect whether the springs were the right height or made of the proper materials.
Finally, the quality check done was independent of the actual, eventual use of the switch. The poka-yoke device was oblivious to the overall goal of a properly assembled switch. Instead, one could argue that the small-dish solution was actually implementing a crude form of syntax checking enforcing the one-to-one correspondence between push-buttons and springs. I will return to this notion of syntax-checking later in this paper.
Poka-yoke and Software Quality Being mainly a manufacturing technique, poka-yoke has only rarely been mentioned in connection with software development, but the philosophy behind poka-yoke has never been far from the heart of software quality.
Gordon Schulmeyer  and James Tierney  refer to poka-yoke explicitly, but many software quality authors have championed detection and prevention methods in software.
In 1990, Boris Beizer wrote in Software Testing Techniques :
We are human and there will be bugs. To the extent that quality assurance fails at its primary purpose -- bug prevention -- it must achieve a secondary goal of bug detection.
In 1993, Steve Maguire echoed a similar sentiment in Writing Solid Code :
All of the techniques and guidelines presented in this book are the result of programmers asking themselves two questions...
How could I have automatically detected this bug?
How could I have prevented this bug?
Prevention devices in software From a poka-yoke perspective, the development of computer languages could be viewed as a prevention device, since one objective of these languages is to prevent us from creating code that can be error-prone. High level languages prevent self-modifying code. Structured programming rescues us from spaghetti code. Objectoriented programming keeps us from stepping on each other's data.
Detection devices in software
Software testing is a form of detection device, but traditional system testing occurs too late in the process to allow quick, corrective feedback on mistakes. Unit testing and "smoke testing"  come closer to the notion of poka-yoke, in that they are located close to the source of the potential mistakes and the quick feedback they provide can keep mistakes from moving further along in the process.
The tools in software that most closely resemble poka-yoke devices are the programs such as lint, printfck, cchk, clash  that examine the syntax of programs and alert the programmer to a possible mistake in need of correction. Static analysis utilities are simple and cheap to run; they aim to eliminate certain classes of common mistakes; and they concentrate on the syntax of the program rather than the program's function.
Some of Hewlett Packard's recent work in localizing software applications has illuminated areas that yield more readily to a poka-yoke approach than to traditional testing. The sections that follow describe our application of poka-yoke principles to solve a problem that defied a traditional software testing approach.
Poka-yoke and LocalizationSome background on message catalogs and localization
To create POSIX-compliant software that runs in multiple locales, developers store locale-specific strings in files called message catalogs.  Rather than hard-code a text string into the application, a developer stores the text string in the message catalog and references it by its message set and message number. A message set and message number uniquely identify any message string in the catalog.
Localization is the process of creating a message catalog for a particular language. Hewlett-Packard currently localizes its Common Desktop Environment software for 11 locales: French, German, Italian, Korean, Spanish, Swedish, plus 2 Japanese locales and 3 Chinese locales.
Localization is typically done after the development of the software has stabilized, and it is typically done by people external to the core development organization. These people, called localizers, receive the application's message catalog from the development organization. They then translate each message string into its equivalent expression in the target language.
Localizing a software application is a difficult job. The localizers may be unfamiliar with the application. They may be located halfway around the world from the development organization. They may not even be familiar with programming. Usually, therefore, the localizer performs translations based almost exclusively on the contents of the message catalogs and the information provided in the localization documentation.
Testing localized software
Testing localized software poses a unique set of challenges. The localizers know what the translated messages say, but since they may never have seen the application run, they cannot know if their translation is correct. The development team knows what the translated messages are supposed to say, but since they are not familiar with the target languages, they cannot know if that is what the translated message actually says. Given the constraints of time and distance, it is difficult for localizers and developers to work together, especially when localization is being done in 11 languages.
The usual testing approaches do not offer much relief. Running the tests manually can become tedious when there are 11 foreign locales to test. Testers become fatigued running the same test in multiple locales.
The traditional way to test the message catalogs is as follows:
the test team receives the translated message catalog from the localizer the test team installs the new message catalog and executes the test plan in the target locales obvious mistakes are referred back to the localizer and incorporated into a later release of the catalog.
This approach has several drawbacks. It is difficult to execute a test automatically in multiple locales. For one thing, image comparisons are not portable across locales, since by the definition of localization something should change on the screen when moving to a new locale. The alternative -- recording golden images in a dozen locales
-- would be a maintenance nightmare.