jump to navigation

Why Keep Testing If You Never Find Defects? September 26, 2012

Posted by Tim Rodgers in Process engineering, Quality.
Tags: , , ,

When I started working at the factory in China I inherited a quality system that included long checklists of visual inspections, almost all of which had been specified by our customer. I’ve never been a big fan of inspections, especially those that rely on subjective human judgment, but I’ll admit that in some cases they’re a quick (but costly) way of detecting defects.

Anyway, one set of inspections seemed especially puzzling. At the end of the production line the customer required an audit of a sample of the finished goods, including the accessories, localized printed materials, and the final packaging before everything was loaded on pallets for the shipping container. Boxed units were taken off the line and opened, and everything inside that had a barcode was removed from the box and hand-scanned to verify that the box contained everything it was supposed to contain.

What made this a head-scratcher was that the end of the production line was ten feet away, where the finished goods, the accessories, and the localized printed materials were each individually barcode-scanned and then put in the box. If the operator tried to put the wrong thing in the box, something that wasn’t on the approved bill-of-materials, it would be detected by the scanner. I wasn’t really surprised to discover that the end-of-line audit never found any missing or wrong parts, and we had a discussion with our customer about the value of the audit.

This brings me to my question: if you never find any defects, is the test or inspection routine still effective or useful? Can’t we get rid of it? Is this test any good?

Let me pause here for a moment and emphasize (again) that you don’t achieve quality by testing or inspections, especially at the end of the production line. Nevertheless, I think we can agree that an audit program, strategically placed in the process flow, can be a useful tool for verifying that customer requirements are being met.

When you’re considering eliminating or changing a test, I think you have to start by asking: What customer requirement does this test correspond to? What defect is this test supposed to find? If the test never fails and doesn’t find those defects, that either means there are no defects (or, at least no defects of that category), or the test isn’t capable of finding them, possibly because of bad test design or bad test execution; which is why the capability of the test should be investigated and verified. By the way, that’s how we discovered the problem with the barcode-scan audit described above: bad design and bad execution.

Assuming it’s a good test, correctly implemented and capable of finding defects that are linked to customer requirements, there are several options if you’re still not finding defects:

1. You can raise the quality standards and tighten the spec limits that define failures. That may give you more failures and an opportunity to eliminate a root cause or reduce variability somewhere in the process. Reducing variability is always a good thing, but you have to consider the cost to do so, and whether this is really a high-value opportunity.

2. You can reduce the audit frequency. Maybe you really have a design and a process that doesn’t generate defects, but maybe it would still be a good idea to check on it from time to time, wouldn’t it?

3. You can eliminate the test altogether. This is a risky move because you’re voluntarily giving up an opportunity to verify that some customer requirement are being met. Before eliminating the test I’d make sure there’s some other way to verify the customer requirement.

Reducing the cost-of-poor-quality by shifting from appraisal activities to prevention activities is certainly a worthy goal, but we shouldn’t be too quick to stop testing just because we aren’t finding any defects.



No comments yet — be the first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: