Tag Archives: problem solving

Fix the Problem XIII: What’s Different? What changed?

This is the latest of a series of case studies and examples from my career, where I attempt to summarize a problem solved, how I did it, all with an eye of passing along useful information… while still, of course, making a good faith effort to protect any confidential information. I hope this, and the others in the series – found HERE – will prove useful and education, illustrative of my abilities, and inspirational as to how I might fit into a new employer.  I am, after all, on a job search!

And a brief disclaimer: These cases are “a while ago” so it’s possible I am slightly off on some of the exact details – but the broad sweep of each case is correct.


As a part of teaching at U Mass / Lowell’s Plastic Engineering Department, one of the tacks I’ve tried to stress is that the students’ program and thoughts need to be aimed – ultimately – at solving real-world problems.  I’ve told my students that when solving a problem in production, or returns from the field, two critical questions often assist in delving quickly to the root cause(s) of the issue.  Specifically, What’s different? and What changed?.

To that end, I’d like to put forth three examples where these two questions were instrumental in helping to find the issues involved.

Retaining Rings Wouldn’t Retain

In one position in my career I worked as a floor-level Manufacturing Process Engineer. I was constantly challenged (read: hammered) with issues ranging from fixtures to field returns to, well, you get the picture.  Lots of “opportunities”, as my then-boss used to say.

One of these was a long-standing issue predating my arrival in the department. On the smaller, hand-held units there were several pieces in a family of similar products.  Some would be sent out of our department and rarely come back from retaining ring issues (essentially, a female piece installed over a mating male piece and held on by that retaining ring).  Others would come back having “spontaneously disassembled”, sometimes even before having left our factory.  This was a constant sore spot in my area’s weekly quality issues, not to mention warrantee report.  Thus it became a priority to solve and get one high-profile issue off the list.

I sat down with a sample of each product: the male, the female, and the retaining ring, laid out to compare and contrast. Visually, on a first-pass look, there seemed to be no differences aside from subtle variations in overall lengths.  More to the point, arranging the pieces from small to large, and putting either an OK or a NOT OK post-it below the pieces, based on their problem frequency, there wasn’t an obvious pattern (e.g., if it had been the two largest ones, or the two smallest ones, that could have been a clue – but there wasn’t).  Since some worked, and some didn’t… “What’s different?”

Being a fan of the Value Engineering discipline, which drives me to think functionally, I asked “What holds these retaining rings on?”  Answer: The groove geometry.  The groove depth, the central diameter, the width… in point of fact, each assembly used the same retaining ring.  Clearly, the ring was not the issue.

So I pulled prints for everything. One possibility was that there was not enough clearance, or the tolerances were wrong, and somehow the retaining rings were being pushed off.  But looking at the nominal and “worst case” dimensions showed that was not the case.  Also note that the company had good machinists, and a strong SPC program; things were in control on that score… and each groove had the exact same dimensions and tolerances.

But doing the “sit and stare” at the drawings laid out side by side I realized that some of the drawings had a callout for the external corners of the groove: SHARP. Some didn’t.  And the ones that didn’t have that callout were the ones with the issue.  Aha, a clue.

The company had a default callout requiring that all corners be broken by – going from memory – a chamfer of .010-.015 inch unless otherwise specified. In looking at the retaining ring groove design recommendations from the supplier, they stated that the edge at the outer diameter of the groove needed to be called out SHARP and could not have a break, whether radius or chamfer.  And the parts showed it; parts without that callout did, indeed, have that edge broken.

I showed this to the Design Engineer who acknowledged the issue, concurred with doing an ECO, and I wrote it up to put through.

Result: The problem went away… because I laid the good and bad parts out, l considered multiple potential factors, but the thing that was different was a design detail missed on the problem assemblies.

It Was Good, Now It’s Not

At that same company, in the same position, another product had a significant percentage of leak test failures. A far more complex assembly than the one above, it had multiple potential leak path failure locations.  Again, an inherited sore spot.

My path was to systematically take failed torches and block off one possible leak path after another, attempting to isolate which of the multiple possible potential leak locations it could have been was the culprit. My goal was to systematically examine the leak location(s) once I’d identified them.

But in one meeting in discussing the larger area, of which mine was a part, one of the managers said “We used to have no leak failures. Let’s find out what changed and change it back.”  A detail I had not known at the time.

It turns out that a design change had been made, with the best of intentions, that resulted in moving several o-rings axially by – IIRC – about 20 thousandths of an inch, which created the potential for them to move under pressure and thus lose their sealing ability.

However, ECOs are not done for no reason.  As I recall, a more careful re-examination of the initial ECO and its reason for being done found that the change could be made with a reduction  in the positioning and seating of the seals – leaks not being considered the first time – and thus maintain the ECO but eliminate the ripple effect that created the leak test failure issue.

Result: The problem went away. And the key lesson learned is to identify the time frame when things change for the worse, and ask “What changed?”  Not just materials, personnel, processes, etc., but consider Design changes too.

Cracked Handles… Sometimes

In one company, the plastic we sold was injection molded into large trash cans – the kind that are used for homes and often picked up by an arm to be dumped into a truck. The company that made them was receiving complaints from customers that the handle used by the truck lifter was cracking.  This was creating quite a problem for them and we were asked to investigate.

My initial role was to do a stress analysis of the handle. I obtained load forces, etc., and built several finite element models of the handle simulating both centered and eccentric loadings.  None showed stresses high enough to create cracks although the stress hot spots were where the cracks were forming.  I also considered impacts, not just static loads – again, while the stresses were in the right places for the cracks they weren’t high enough to exceed the yield strength.  I increased the mesh density – refining the model as sometimes that can affect the stress levels, but the numbers held.  Based on the force and impact load cases I was given, there was no reason for the handles to be cracking.

Our initial thought was that some kind of chemical might be attacking the material, and we started to inquire about possible chemical contacts, but then a clue arrived through our sales group. Only one color of the several the company offered to the end users was having the issue.

Aha! “What’s different?”  We supplied the base resin; at the molding location the customer would blend in colorant masterbatches to create the color variations.  Pursuing this further, we learned that the base resin used by the masterback colorant provider was, for this one color, significantly lower in average molecular weight.  I.e., by blending in this color masterbatch the molder was introducing a weaker and less impact-resistant material to be blended into the base resin.  (Material note: plastic strength is directly related, as a general correlation, to molecular weight of the polymer chains.)

Looking back with more experience under my belt, I’d ask two questions: 1: “Can the failure be reproduced?” And 2: If the answer to the first question is YES, “What happens to products molded from uncolored resin?”  Assuming YES was the answer to the first question, and the failure didn’t happen with uncolored resin, that would at least have eliminated the base plastic as the sole cause of the problem.

When a different masterbatch blend with a material molecular weight like the other masterbatch colors was tried, the problem went away.

Again, the key clue was learning that one color didn’t work while the others did… “What’s different?”  (And in the back of your mind you can add the potential for different colors to be questioned.)

Adding to the Toolbox

Multiple tools exist to aid in systematically looking for the root causes of a problem, e.g., Ishikawa/fishbone diagrams, the Five Whys (and Two Hows) questioning, etc.  Adding “What’s different?” and “What changed?” as appropriate can add an important new tool to your problem-solving toolbox.


© 2016, David Hunt, PE


Fix the Problem X: Learning from Interview Problems

Interviews are where companies and candidates learn about each other to see if there is a fit between a candidate’s background and personality, and the company’s needs and culture.

One of the things candidates, in particular technical candidates, get asked on interviews is to solve problems.  Often these are company problems that have been already solved, in an attempt to see how the candidate approaches problems.  Sometimes these are questions about extant issues.

Plusses, Minuses, and Danger Zones

On the positive side, this is a chance for candidates to show their abilities – ideally referencing other accomplishments as well to build a case for their skills.  On the negative side, such questions are often about problems that took multiple people hundreds, if not thousands, of man hours to identify and resolve.  Thus they can be very difficult to actually solve, in particular as a candidate is not as familiar with the details of the product and the processes by which it is made.

And if the issue is one currently extant, there is a danger that the line of inquiry is being used to obtain free consulting.  I’m not accusing all companies of doing this, but the possibility does exist.  So in the process of answering a candidate does need to be on guard.

A Two-Way Street

These problems, however, are an opportunity for candidates to learn as well.  I will discuss three situations where I was presented with problems in interviews in the hopes that these experiences will be of use to my readers as well.

Fiber Optic Bubbles

Back in 2001 after being in a massive RIF from Visteon, I interviewed with a company that made fiber optic cables.  The process was fascinating!  They would take raw, uncoated optical fibers off reels and pass them through a coating bath, lining seven fibers next to each other; this bath material would coat the fibers and adhere them together side by side into a ribbon.  Then, two of these seven-fiber ribbons would be lined up edge to edge, passed through another coating bath, and stuck together.  These ribbons would then be stacked up, one upon another, to form a rectangular bundle, and run through a machine that extruded a multi-layer protective sheath around the whole thing.

The issue they were having was bubbles in the coating around the fiber; bubbles distorted the coating and exerted a force on the fiber which, apparently, would allow light leakage out of the fiber – degrading the signal.  They handed me one of the 14-fiber ribbons to look at.

I have discussed looking for patterns in prior essays, for example here and here; I immediately noticed two things.

First, the bubbles were only present at the centerline bond between the two smaller optic-fiber ribbons.  This was an important clue as to where in the process from raw fiber to end cable this defect was being introduced.

Second, in looking at the bubbles, the spacing between them was remarkably consistent.  These bubbles, then, were not randomly formed, but something in the process was doing something that created a periodic “hiccup” creating these bubbles.  Using a ruler I got the distance between the bubbles, IIRC an inch-and-change, and in asking for the speed of the ribbon I was able to calculate the time between bubble formation.

My guess was that something was oscillating in the system, but probably not smoothly; there was likely “something” sticking slightly, with the sticking creating jerking motions as that “something” stuck and unstuck.

Since the serpentine path the ribbons took was visible, my approach would have been to use a high-speed camera and/or strobe light to identify whatever parts of the system were oscillating at that identified frequency as the place to focus attention.  (E.g., if the frequency was 10 Hz, light the strobe off with a frequency of 20 Hz.  The “something” – my a priori guess was one of the slack take-up reels – would appear to be shifting back and forth between two positions after some experimentation to synchronize the strobe with the “something’s” cycle.)

They were excited by my idea.  Unfortunately, I never did hear anything further about whether the idea contributed to the solution of the problem (I asked a couple of times over the next year).  I suspect my idea contributed to improving someone’s performance review.

Lesson: Strobe lights and/or high speed photography can be enormously useful if a frequency-dependence of a problem can be identified.

Cavity Numbering

Another interview was with a company whose supplier – in China – was molding their components and shipping them to the US for assembly.  There were eight cavities in each of two molds; these two components would then be assembled to form the product.  Some of the pieces, A and B, would not fit together.  Measurements indicated that there were dimensional issues causing a problem with some cavities of part A not fitting into part B.

Here was the problem: they didn’t know which cavity was which – translating to an inability to determine which cavities worked interchangeably and which ones didn’t.  They already were pressuring their supplier to take the molds offline for a day or so to put in some kind of cavity marking.

Lesson: If you have multiple cavities in an injection mold, number them to aid in troubleshooting when confronted with situations like this.

Engineering Fundamentals Always Apply

In one very thorny problem presented to me, a glass-encased thermocouple was placed into a blind hole machined into an aluminum piece and potted in place with an elastomeric material.  These units were subject to thermal cycling as part of the unit’s operation.  Failure analysis showed the thermocouple’s glass casing was cracking.

This is where engineering fundamentals comes in.  What happens when things get hot?  They expand; each material has its own expansion coefficient.  Resins – from which the potting material was made – typically have a higher expansion rate than metals, in this case aluminum.  With the resin trying to expand faster than the material around it, its attempt to grow was constrained by the hole walls – introducing a compressive strain from the difference in expansion rates.

Stress – force per unit area – is strain times Young’s Modulus.  The expanding resin, being constrained from growing outward, squeezed inward on the glass and fractured it.  The solution was to change to a softer resin with a lower Young’s Modulus.  Even though the thermal expansion strains were the same, because the resin was softer, the forces were lowered and the glass didn’t crack.

Lesson: Sometimes you really need to get back to basics.  In this case, mechanics of materials.

Interview Problems: An Opportunity

Not all interviews work out (alas).  But the presentation of a problem to solve can be an opportunity – not just to prove your mettle, but to take advantage of the experiences of others to improve your own problem-solving toolbox by being walked through a real-life case study.

© 2014, David Hunt, PE

Fix the Problem XIII – The Problem Toggle

(Author’s note: The “L” key on my computer – old one, new one finally ordered – was not working, and I missed it while posting this.  Apologies for the multiple revisions getting all the Ls into place in the title and link…)

While at Ford Motor Company, our group’s manager found a book that he thought was so good he recommended it to everyone, and even paid for our individual copies.  This book, Manufacturing Solutions for Consistent Quality and Reliability, was excellent.  But the book made one fundamental point about problem solving – whether in performance of a product or a process – that has stuck with me ever since (paraphrased):

You can only claim to have truly solved the problem when you understand it well enough to turn it on and then back off again.

With this in mind I’m going to review two instances from my career where this happened.  (I will revisit this theme in future essays to discuss more case studies.)

Plasma Cutting Torch: Mysterious Leaks in the Field

This particular torch was part of a plasma cutting system; I was the Manufacturing Engineer in charge of the torch area, and working on identifying the root cause of field failures was one of my responsibilities.  This particular torch had a significant rate of return; these returns, responded to by sending a new torch, was a large part of my department’s total warrantee cost.

The issue was that the problem could not be duplicated. We would receive the torch which, according to the customers, would have coolant leaking out the front “business end” of the torch.  No matter what we did, we couldn’t get returned items to do it in our lab.

Lesson One: Unless you can duplicate the failure, in order to experiment with what does and does not trigger it, you are flailing in the dark.

So one day we get a torch back where we’ve been told the coolant is gushing out in a waterfall.  We go to our own test machine, connect everything up, turn the coolant flow on… and nothing (like always).  We exchange parts of the system for new ones.  Nothing.  Go back to the original, returned pieces.  Nothing.  Try as we might with permutations of new parts and returned parts, we cannot duplicate the reported failure.

Let me take a moment to say that I trust intuition.  Something was nagging at the back of my mind.  I couldn’t even quantify it, but something was bothering me.  Just as the technician was about to turn off the coolant flow, I said “Hold it, I want to try something.”  I reached over to the torch, grabbed the retaining cap that threaded on and which held everything inside, and started to turn and loosen it.  I had barely touched it when coolant started to jet out of the opening.

I tightened it.  Nothing.  I started to turn it, and again, coolant flowed copiously.  AHA!  I tightened it to its hard-stop and made a mark.  Then I slowly started to loosen it until, like a floodgate opening, the flow started again.  I marked that too.  It took, maybe, a 0.25” of distance, as measured on the outside diameter, to make this difference.

Armed with this information I went to the prints and calculated that – going from memory – the axial translation of the cap being unscrewed was on the order of 0.020”.  This information was passed to the Design group, which found that the issue was design-related (details omitted for confidentiality reasons).  Designs of a few, key components were tweaked and prototypes made.

The result: The redesigned torch wouldn’t leak despite backing the cap off double what I had done.  After the change, warrantee returns started to drop dramatically as the redesigned torch was propagated into the customer base.

Lesson two: Intuition and hunches are often based on a subconscious stew of disparate facts coming together.  While you shouldn’t just go with them – a systematic approach like an 8D Problem Solving Process is needed – don’t ignore those tickles at the back of your mind.

One other thing to note.  In retrospect these symptoms and the “no problem found” status made sense.  Plasma cutting is often a dirty environment with grit, metal chips, etc., around.  Likely what happened is that people took off the cap while changing the consumables – the “razor blades” – putting it down in such a way that grit got onto the surface that was supposed to be flush with the surface that provided the hard stop when installing the cap.  This created an inadvertent shim that coupled with the design issue to create the leak.  In the process of being shipped to us the grit would fall off, removing that inadvertent shim and resulting in a torch that would function as intended with no problem.

Lesson three: When troubleshooting, think about the environment where the failure is occurring.  Ideally, go and watch.  There’s nothing like seeing the precise situation for generating data, even if that data only goes into the aforementioned subconscious.

O-ring Rollout: Leak Failures in Manual Assembly Area

At the same plant where I first was given this book, one of the products was a carbon canister assembly that fitted into the fuel cell.  Functioning to absorb gasoline vapors coming off the fuel tank for emissions control, some of them had several hoses with male attachments that would be inserted into a female port.  The work time standard was strict and people pushed hard to meet it.  (Note that I have a portfolio page about this problem.)

At issue was the fact that the O-rings forming the seal at these ports would roll out of the groove they were in, creating a leak path.  This assembly defect was internal to the female port and so was not visible.  The first indication there was an O-ring rollout was that the unit would fail the leak test.  The unit would then be methodically disassembled until the rollout was found.  Then it was reassembled after reseating the O-ring, and retested.  As you might imagine, this was quite time-consuming (not to mention not value-added).

Having been asked to look into the problem, one of my first actions was to look at the Design Guidelines.  Since my Master’s Research was in Design for Manufacturing and Assembly, I had – stated immodestly! – a pretty good grasp of the dynamics of how things go together.  One thing I noticed was the design of the lead-in.  Although “perfect” from a molding standpoint, having a radius as a lead-in was not so great from an assembly standpoint.  The reason being is that the O-ring needed to slide along the surface without being “grabbed” by friction.  The governing equation is:

Arctangent(angle) < coefficient of static friction

With a visual:

angle image

Note that in no case will an angle greater than 45 work.

What I found, in a detailed examination, was that it was possible to misalign the male insert sufficiently so that the O-ring would hit on the part of the radius where static friction would dominate.  This would then “grab” the O-ring and, as the insertion progressed, it would roll out of the groove.

In a Design for Assembly analysis I’d written before on my blog I referenced Fitt’s Law.  I applied it in this case and found that it was a difficult task for a person to do reliably – which explained the high rework rate.  If I redesigned the lead-in to be a 30 degree chamfer, as shown in the portfolio page (referenced again for convenience), I essentially made it impossible to NOT get the O-ring on a sliding surface.  (NB: a 15 degree angle is my “perfect” recommendation for this situation.)

Lesson Four: Very often there is a Design issue at the root cause of a production problem.  Not always, of course – but in my experience the probability has been very high that Design is a contributor to the issue.  Note that Design is the foundation: Reliability, Functionality, and Quality all start with Design… one can have production issues even with a good design, but one cannot have good production with a bad design.

Based on my write-up we made an insert for the mold of the female port (fortunately the mold boss forming the core of the port was an insert that could be easily changed!) and tried it.  Leak failures from O-ring rollouts fell to nothing.  But there’s one more lesson… I got a call from the Design group asking me why I was proposing changing the design, including altering the Design Guidelines.  When I asked if he’d seen my write-up, he said yes.  When I asked if there was a problem with it, or with the results showing it worked, he snarled – literally snarled – through the phone: “You’re just a Manufacturing Engineer, what can you know?”.  Needless to say I was tempted to retort, but again returned to the successful results and appealed to his “We’re all one company, right?” spirit.

Lesson Five: People can get very protective of “their turf”; keep that in mind as you propose changes, especially if the changes are in someone else’s department.  (In retrospect I should have involved the Design group from the beginning to have them on the team and involved once I figured out this was a Design issue.  Thus I would have avoided turf battles, toe-stepping, and bruised egos.)

The Problem Toggle

In both instances changes were made that turned the problem off.  In both instances we understood the root cause well enough that if we had gone back to the old design, the problem would have returned… and why it would have returned.  In these two cases there was just one true “root cause”, Design, but in other cases I’ve experienced there were multiple factors that worked together to create the problem.

Only by systematically working through a formal process, often including tools like an Ishikawa Diagram which can be very useful, testing each identified possibility by duplicating the failure conditions to see if the change affected failure rates, can problems be declared solved.

Otherwise, solutions become a variant of “I’ve got everything just right; don’t touch anything!”  And that’s not a way to design and produce in today’s hypercompetitive world.


© 2014, David Hunt, PE

Fix the Problem III

This is the third of an ongoing-series of case studies from my experience, looking at problems in Design and Manufacturing and how a solution was achieved.  Parts I and II are here and here, respectively.

As a part of a material cost reduction team in the climate control division of Visteon Corporation, I had joined the group just as several large projects were coming to fruition.  One, under the direction of another team member, was the replacement of a nylon material with a polypropylene material for the housing of a control panel for a high-volume automotive air conditioning product.  All the testing had been done to verify that the direct substitution of one material for another, lower-cost, material did not compromise product performance, durability, or the ability to produce it from raw materials to final product… a dock-to-dock approach to the validation of a new material was often required for this kind of project.

It was in this last phase – when hopes were high for this cost-savings project, with a projected annual cost savings of over $200K per year – that the project hit a snag.  Each piece had a code sprayed on it with the production date and other important tracing information on it.

Essentially, although the ink would dry in place as before, it was not adhering to the surface as before and would easily smudge; thus, the necessary information the markings contained would be obscured.  This should have been expected; polypropylene, like all polyolefins, has a very low surface energy which makes it hard for inks and glues to adhere to it.

The most common surface treatment for this is a primer.  Essentially, the process goes like this: a choice is made for a polyolefin in a product; a component that needs to be glued to another.  In talking with a glue maker, who should do due diligence in understanding the application, the substrates being glued together will be mentioned.  Immediately a concern about adhesion will be voiced and – lo and behold! – the glue maker has a solution: a primer.

Certainly this makes sense from the glue maker’s point of view.  They come across as proactive, voicing a concern for the application and having a ready-made solution available.  The ink maker similarly proposed a primer, also of their own manufacture (clearly they’d seen this situation before!).  But this was an open-air existing assembly line; there was no room to put in an enclosure to spray a volatile-based primer upstream of the marking process.  Never mind that the design and engineering of such an enclosure would take significant time, add a material requiring toxicology approval to the plant’s mix, and the capital cost and ongoing variable cost of the new material being applied would significantly eat into the projected savings.

But based on my own experience and having taken a continuing education class, Adhesives and Adhesion, at U Mass/Lowell, plus other research I had done faced with similar gluing problems I knew there were other solutions: flame, corona, or plasma treatment.

I quickly researched and found a corona spray equipment company – it was not a “Joe’s Fish, Chips, and Corona-Treatment Equipment” company, but a well-established one with some longevity.  Their sales representative visited, saw the line and the application, and arranged for a loaner unit to be overnighted to the plant.  It was quickly installed and test pieces were run down the line, each unit spending several seconds being sprayed by the ionized coronal discharge directed at the precise location of the markings.

And it was a smashing success.  Not only did the ink adhere initially and not smudge, but in the aging and “shake and bake” tests done, the markings lasted as durably as the ones on the original nylon substrate, which were tested side-by-side with the new pieces to ensure an apples-to-apples comparison.  This project went from “show-stopper crisis” to success… because I knew something nobody else on the team did – thus highlighting, yet again, the benefits of hiring from outside.


  1. Adhesion to plastics is a common problem, especially low-surface energy materials like polyolefins
  2. Glue makers will understandably race to recommend a chemical primer
  3. In many instances other techniques exist to enhance the surface energy to get better wetting and adhesion

Before running to add yet another chemical to the production mix, along with its own MSDS, traceability, and hazards, consider corona, plasma, or flame surface treatments when faced with a problem of poor adhesion to plastics or rubbers.


For reference and information, here are two articles about adhesives and surface treatments – from the medical device world, but certainly applicable everywhere.

Why and How to Use Gas Plasma Technology For Surface Treatment in Medical Devices

Using Adhesives Effectively in Medical Devices


© 2013, David Hunt, PE

Fix the Problem II

In Part I on this topic I looked at identifying the source of variation in an assembly cell making automotive headlights, and then outlined the proposal I made to neutralize the effects of the variation on the quality of the assembly since the variation could not be eliminated.

In this installment (of what will probably several case studies of both production issues as well as product failure troubleshooting) I will look at my troubleshooting work on a sterile blister pack consisting of a syringe, alcohol and iodine wipes, and some ancillary items.  The problem was critical: these blister packs are meant to be used in a hospital, and the package contents are supposed to be sterile for immediate use – often in an emergency room situation where there is no time to worry about checking for sterility.  The reality of sterility needed to match the assumption.

These blister packs consisted of a thermoformed polyethylene film pocket, the contents, and a Tyvek film covering which would be peeled back and the contents removed and used (Tyvek was used as it is permeable to the sterilization gas that is used).  The company was receiving field returns where the pockets had punctures in the pocket corners, allowing outside – and obviously non-sterile – air to penetrate into the pocket thus ruining the sterilization and the product’s usability.  (Side note: I use thermoforming as a generic term; thermoforming and vacuum forming are, functionally, the same process IMHO: a film that, through a pressure differential side-to-side, is formed to a shape – whether over a male protrusion or into a female cavity.)

In order to identify the root cause of the problem it was first necessary to figure out where the punctures were being introduced.  Begin at the beginning was my approach.  I obtained, straight off the production line, a sample of blister packs of, probably, a thousand formed pockets.  An exhaustive examination found no perforations.  Since we knew from returns that this was a significant problem, comprising several percent of production, a sample size of 1000-odd pockets should have turned up punctures if this was the source.

The next step was the assembly process – the various pieces were put in manually by a team of operators, each responsible for a couple of items.  Again, a sizable sample (several hundred products) was exhaustively checked to search for punctures.  Note that I did not forewarn people I was grabbing samples off the end of the line.  I merely showed up (with my boss’ written approval of course) and took them, which prevented any special care people might have taken had they known that a certain part of their production would be taken for testing.

This right-off-the-line sample had no punctures.  While not as large a sample size as the unfilled pockets, we should have seen something had the root cause been in the assembly stage.  Clearly the punctures were happening post-assembly.

One theory that had been proposed just before I started there was that the alcohol and iodine swabs, which were in flat foil packets, were creating the punctures through vibration of shipping causing them to shift and slide, and the sharp corners of the packets were – as the theory proposed – cutting the corners.  This was plausible as the two swab packets were placed in the bottom of the thermoformed pocket… although, in looking at the field returns we received it didn’t look like the punctures were clean cuts, so I was personally skeptical about this proposed failure mode.  It still needed to be tested.  So we did a test where the order of placement of pieces was changed to put the two wipe packets on the top of the cavity, thus keeping any foil corners away from the pocket bottom.  These samples were then packed as normal and shipped off to a vibration test simulating shipping and handling; IIRC we had five boxes of these parts with several hundred individual product packets.  To be complete, though, we also put five boxes of “normal” production pieces through the same testing process.  The idea was that if these foil packets were the root cause of the punctures, we should see (theoretically) no punctures in these, and the punctures in the standard product.

The results were not as expected, though.  Both the test sequence and regular production product had, statistically, the same number of punctures.  It wasn’t the product placement – it was the rough-and-tumble vibration of pieces moving in the box.  Aha!  At least we knew where in the product’s life cycle from start to end-user the problem was happening.  But we still didn’t know why.  We only saw punctures in the corners, which begged the question “Why only in the corners?”

Anyone who knows anything about thermoforming (or vacuum forming) knows that the thickness of the part is not uniform, but the wall thickness gets thinner the farther the film has to stretch, and in particular it gets thin in corners.  Measurements showed the thickness in the corners, which had far tighter radii than I would have designed knowing this process limitation, was 2-3 thousandths of an inch (mils).  This is not a robust thickness!

The next step was to outline recommendations for fixing the problem; these could have been used in combination but I’ll address each separately.  Note that all three options would require revalidation of the sealing process.

Option I: New tooling – with, IIRC, triple the radius size at the bottom of the tool cavities that made the pockets.

Advantage: The film itself would not change, so there would be no variable cost-of-material change.

Disadvantage: New capital cost for tooling; only a modest increase in film thickness.

Comment: This did, however, create information useful for the next generation of pockets that needed new tooling; specifically, use generous radii at the bottoms of the tools.

Option II: Thicker polyethylene film.

Advantage: The adhesion between the Tyvek and polyethylene material was known and
quantified.  No new capital cost required for tooling.

Disadvantage: The thicker film was more expensive, and the thicker film created the possibility of requiring a marginal increase in sealing time to get the interface up to the needed temperature.

Option III: New material for the film.

Advantage: There were any number of materials that were more puncture resistant for a given thickness.

Disadvantage: New film materials are more expensive than the old one; stronger materials might also affect cycle time.

The Final Action

Given the long lead time for new tooling and the need for a fast solution, as I recall we adopted a double-hit approach.  A thicker film made from nylon rather than polyethylene was chosen as a “swing for the fences” option.  The advantage to this was a very high probability of success in one iteration; the disadvantage was increased material cost (admittedly not large on an absolute scale) plus a potential for longer cycle times for sealing.

Again, going from memory, the heated sealing plates were so powerful in terms of pressure and temperature driving the sealing that this did not affect the sealing cycle time.


  1. When faced with a problem coming back from the field, working systematically through the process from start to finish can be a simple way to isolate the location where the failure is occurring, thus minimizing the number of possibilities that need to be investigated.
  2. When taking samples of manual work, don’t give warning; just take them.  This avoids workers taking extra care because they know it’ll be looked at more closely.
  3. When doing experiments to evaluate an improvement idea, be sure to include production pieces as a control.
  4. Sometimes a hard-hitting, fast-to-implement solution is called for – even if it’s more expensive than an optimized, minimal-cost one – especially if the problem is jeopardizing shipments, customer satisfaction, and market share, etc.

© 2013, David Hunt, PE

Fix the Problem I

In the movie Disclosure character Tom Sanders, played by Michael Douglas, is head of Manufacturing for a start-up company with a revolutionary new technology, which is about to be merged with a publishing company.  Without spoiling things too much, with the production line seemingly beset by production problems, he is clearly being blamed – and forced out – in part by the very aggressive new head of the company, played by Demi Moore, someone with whom he has worked before (and with whom he used to be romantically involved).

Alone and seemingly in a hopeless situation, he receives an email saying “Fix the problem.”  A clue to his course of action.

An excellent book I read some years ago introduced me to the concept of multi-var analysis; a production problem solving technique which I’ve used successfully (for example, here).  One of the things the book stressed was that – in the best of all possible worlds, of course – one can only say a problem is solved when you know what to do to turn it on and off.  That is to say, you’ve identified to root cause of the failure (using multi-var, Ishikawa diagrams, etc., as a part of a formal problem-solving process) and taken deliberate action to solve it and prevent its recurrence (i.e., more than tweaking things and then putting a sign up saying “For G-d’s sake don’t touch this dial!”).

An example of identifying the root cause and coming up with a solution comes from a glue pouring operation at Ford.  Hot-melt glue was being dispensed into the groove of a standard tongue-in-groove glue joint, with the mating part being pressed in at the next station downstream.  Unfortunately the glue was not cooperating, not always being dispensed into the bottom of the glue groove, but sometimes being poured on the sides of the groove, on the edge of the groove “cup”, and even oozing over to drip down the outside.  This was creating a high scrap rate as in the next station a lens assembly with a decorated bezel insert was pressed into the body with the glue.  With a bad glue pour the headlight would fail the leak test, requiring disassembly to recover the valuable components such as the reflector inside and hardware on the back… not to mention material scrapped and an assembly cycle wasted.

I was asked to look into an inspection system to on-the-fly inspect the glue pour to kick out a lamp body with a bad glue pour.  With the glue being black and the body also being black, but the glue being a hot melt, an infrared (IR) camera was the obvious choice.  We invited a couple of IR camera makers to come in, set up equipment to take live pictures, and quote a system.  But this would only contain the problem, not solve it – and with the quotations coming in, do so quite expensively.

The logical question which I then asked was “Why are we getting bad glue pours?”  In a theoretically perfect world, everything should be fine.  The answer was “Variation”; the next Why? was “Why are we getting variation?” followed quickly by “Where is the variation?”

There were three places where we could be seeing variation.  The first was in the glue dispensing system (robot + dispensing machine).  But the robot had a manufacturer-stated repeatability in its path to within a fraction of a millimeter.  The dispensing system was likewise very repeatable in shot size and flow rate.  Verification of these was not just a matter of taking the manufacturers’ word for it, but was a part of the machine acceptance protocol.  Any variation in this part of the system was miniscule.  The one program (remember this) was spot-on repeatable.

The next place to look was at the fixtures, of which there were somewhere around 40.  There, too, the fixture acceptance protocol required that we examine fixture-to-fixture variation as a part of final approval of fixtures.  ANOVA testing of the fixtures during acceptance verified that the fixtures varied within a few percent of each other; fantastic Gage R&R numbers.

Last up were the parts themselves.  Since this was a high-volume molded body, how many mold cavities were there?  Three.  It’s a given that any product with multiple cavities will have cavity-to-cavity variations.  On top of that the bodies were made from polypropylene, which among molders is often nicknamed “polywarpylene” for its propensity to warp and distort from its ideal, molded shape.  Some dimensional checks verified that the positions of the grooves from cavity to cavity floated by up to half of the groove width itself.

Recall from above there was one robot program?  This one program was a compromise program pouring into three different grooves whose position varied from piece to piece.  Since the variation was the result of an inherent property of the material, it couldn’t be eliminated.

So I proposed the following:

  1. Develop custom programs for each cavity (requiring the purchase of extra memory for the robot).
  2. Put a bar code into each cavity identifying it, and an optical scanner in the station upstream from the dispensing area to pass the cavity information to the robot.
  3. The robot, using a cavity-custom program, would adapt to the variation between the cavities and eliminate the problems arising from the compromise program.

Unfortunately my plan was not implemented, and instead the plant elected to live with the scrap rate and troubleshoot it as they had always done before – reducing, but never truly fixing, the problem.  But this could have been fixed for a NPV cost far, far less than the scrap costs the plant incurred through the life of the product.

Lesson: When faced with variability that is inherent in the system and which cannot be engineered out, think about how you can adapt to it to neutralize its effect.

Part II will look at a package perforation issue I was asked to investigate as a part of a contract position I had at a medical device company.

© 2013, David Hunt, PE