Tag Archives: fix the problem

Fix the Problem XIII: What’s Different? What changed?

This is the latest of a series of case studies and examples from my career, where I attempt to summarize a problem solved, how I did it, all with an eye of passing along useful information… while still, of course, making a good faith effort to protect any confidential information. I hope this, and the others in the series – found HERE – will prove useful and education, illustrative of my abilities, and inspirational as to how I might fit into a new employer.  I am, after all, on a job search!

And a brief disclaimer: These cases are “a while ago” so it’s possible I am slightly off on some of the exact details – but the broad sweep of each case is correct.

Introduction

As a part of teaching at U Mass / Lowell’s Plastic Engineering Department, one of the tacks I’ve tried to stress is that the students’ program and thoughts need to be aimed – ultimately – at solving real-world problems.  I’ve told my students that when solving a problem in production, or returns from the field, two critical questions often assist in delving quickly to the root cause(s) of the issue.  Specifically, What’s different? and What changed?.

To that end, I’d like to put forth three examples where these two questions were instrumental in helping to find the issues involved.

Retaining Rings Wouldn’t Retain

In one position in my career I worked as a floor-level Manufacturing Process Engineer. I was constantly challenged (read: hammered) with issues ranging from fixtures to field returns to, well, you get the picture.  Lots of “opportunities”, as my then-boss used to say.

One of these was a long-standing issue predating my arrival in the department. On the smaller, hand-held units there were several pieces in a family of similar products.  Some would be sent out of our department and rarely come back from retaining ring issues (essentially, a female piece installed over a mating male piece and held on by that retaining ring).  Others would come back having “spontaneously disassembled”, sometimes even before having left our factory.  This was a constant sore spot in my area’s weekly quality issues, not to mention warrantee report.  Thus it became a priority to solve and get one high-profile issue off the list.

I sat down with a sample of each product: the male, the female, and the retaining ring, laid out to compare and contrast. Visually, on a first-pass look, there seemed to be no differences aside from subtle variations in overall lengths.  More to the point, arranging the pieces from small to large, and putting either an OK or a NOT OK post-it below the pieces, based on their problem frequency, there wasn’t an obvious pattern (e.g., if it had been the two largest ones, or the two smallest ones, that could have been a clue – but there wasn’t).  Since some worked, and some didn’t… “What’s different?”

Being a fan of the Value Engineering discipline, which drives me to think functionally, I asked “What holds these retaining rings on?”  Answer: The groove geometry.  The groove depth, the central diameter, the width… in point of fact, each assembly used the same retaining ring.  Clearly, the ring was not the issue.

So I pulled prints for everything. One possibility was that there was not enough clearance, or the tolerances were wrong, and somehow the retaining rings were being pushed off.  But looking at the nominal and “worst case” dimensions showed that was not the case.  Also note that the company had good machinists, and a strong SPC program; things were in control on that score… and each groove had the exact same dimensions and tolerances.

But doing the “sit and stare” at the drawings laid out side by side I realized that some of the drawings had a callout for the external corners of the groove: SHARP. Some didn’t.  And the ones that didn’t have that callout were the ones with the issue.  Aha, a clue.

The company had a default callout requiring that all corners be broken by – going from memory – a chamfer of .010-.015 inch unless otherwise specified. In looking at the retaining ring groove design recommendations from the supplier, they stated that the edge at the outer diameter of the groove needed to be called out SHARP and could not have a break, whether radius or chamfer.  And the parts showed it; parts without that callout did, indeed, have that edge broken.

I showed this to the Design Engineer who acknowledged the issue, concurred with doing an ECO, and I wrote it up to put through.

Result: The problem went away… because I laid the good and bad parts out, l considered multiple potential factors, but the thing that was different was a design detail missed on the problem assemblies.

It Was Good, Now It’s Not

At that same company, in the same position, another product had a significant percentage of leak test failures. A far more complex assembly than the one above, it had multiple potential leak path failure locations.  Again, an inherited sore spot.

My path was to systematically take failed torches and block off one possible leak path after another, attempting to isolate which of the multiple possible potential leak locations it could have been was the culprit. My goal was to systematically examine the leak location(s) once I’d identified them.

But in one meeting in discussing the larger area, of which mine was a part, one of the managers said “We used to have no leak failures. Let’s find out what changed and change it back.”  A detail I had not known at the time.

It turns out that a design change had been made, with the best of intentions, that resulted in moving several o-rings axially by – IIRC – about 20 thousandths of an inch, which created the potential for them to move under pressure and thus lose their sealing ability.

However, ECOs are not done for no reason.  As I recall, a more careful re-examination of the initial ECO and its reason for being done found that the change could be made with a reduction  in the positioning and seating of the seals – leaks not being considered the first time – and thus maintain the ECO but eliminate the ripple effect that created the leak test failure issue.

Result: The problem went away. And the key lesson learned is to identify the time frame when things change for the worse, and ask “What changed?”  Not just materials, personnel, processes, etc., but consider Design changes too.

Cracked Handles… Sometimes

In one company, the plastic we sold was injection molded into large trash cans – the kind that are used for homes and often picked up by an arm to be dumped into a truck. The company that made them was receiving complaints from customers that the handle used by the truck lifter was cracking.  This was creating quite a problem for them and we were asked to investigate.

My initial role was to do a stress analysis of the handle. I obtained load forces, etc., and built several finite element models of the handle simulating both centered and eccentric loadings.  None showed stresses high enough to create cracks although the stress hot spots were where the cracks were forming.  I also considered impacts, not just static loads – again, while the stresses were in the right places for the cracks they weren’t high enough to exceed the yield strength.  I increased the mesh density – refining the model as sometimes that can affect the stress levels, but the numbers held.  Based on the force and impact load cases I was given, there was no reason for the handles to be cracking.

Our initial thought was that some kind of chemical might be attacking the material, and we started to inquire about possible chemical contacts, but then a clue arrived through our sales group. Only one color of the several the company offered to the end users was having the issue.

Aha! “What’s different?”  We supplied the base resin; at the molding location the customer would blend in colorant masterbatches to create the color variations.  Pursuing this further, we learned that the base resin used by the masterback colorant provider was, for this one color, significantly lower in average molecular weight.  I.e., by blending in this color masterbatch the molder was introducing a weaker and less impact-resistant material to be blended into the base resin.  (Material note: plastic strength is directly related, as a general correlation, to molecular weight of the polymer chains.)

Looking back with more experience under my belt, I’d ask two questions: 1: “Can the failure be reproduced?” And 2: If the answer to the first question is YES, “What happens to products molded from uncolored resin?”  Assuming YES was the answer to the first question, and the failure didn’t happen with uncolored resin, that would at least have eliminated the base plastic as the sole cause of the problem.

When a different masterbatch blend with a material molecular weight like the other masterbatch colors was tried, the problem went away.

Again, the key clue was learning that one color didn’t work while the others did… “What’s different?”  (And in the back of your mind you can add the potential for different colors to be questioned.)

Adding to the Toolbox

Multiple tools exist to aid in systematically looking for the root causes of a problem, e.g., Ishikawa/fishbone diagrams, the Five Whys (and Two Hows) questioning, etc.  Adding “What’s different?” and “What changed?” as appropriate can add an important new tool to your problem-solving toolbox.

 

© 2016, David Hunt, PE

 

Fix the Problem XII: When Chemicals Attack

Please see the end for a “good-faith” disclaimer.

Resistance to chemical attack and galvanic corrosion are big factors in many designs, especially at elevated temperatures.   Even at ambient temperatures corrosion can be an issue: the lowly chain-link fence has its steel protected by a zinc coating which, over time, corrodes instead of the steel – but given enough time, that protection disappears and then the steel itself starts to rust.

In this essay I will outline three examples I have seen where chemicals attacked solid materials, and for each highlight some lessons for people to consider – whether in a design or failure-analysis mode.

Plastics and Acids

At one point in my career I was asked to do some thermal stress analysis because of a series of returned field failures. This was a plastic container made of a relatively new material chemistry whose operation cycle involved being immersed in hot water to heat the contents, and then dunked in cold water to shock-cool them. Over time, cracks would develop in the corners of this injection-molded container. In some instances, catastrophic failure would occur resulting in ruptures of the vessel during the cold-water dunk. To the best of my knowledge nobody was injured in any such event.

The plastic material was believed to be immune to the mildly-acidic contents, so the prevailing theory was that the thermal shock was creating the cracks. This conclusion was logical considering that these failures occurred during the cold dunk operation. My job was to verify this presumed failure mode.

But my first analyses showed something very different than the observed failures. Both partially-failed returns, as well as analysis of sectioned pieces, showed the cracks initiating on the inside of the vessel’s corners. In contrast, every analysis I did showed that any thermal stress cracks should initiate on the outside, where the outer layers – exposed to cold first from the cold dunk – should contract first resulting in crack-opening tensile stresses on the outside preferentially.

Eventually, more detailed and at-temperature chemical compatibility testing showed that the acids inside were attacking the insides of the corners of the vessels.

Result: The material was pulled from this application.

Lessons:

  • Chemical compatibility charts and theoretical assumptions are good guides, but verification testing needs to be done on the actual application with real-world conditions.
  • Molded-in stresses at the corners likely contributed to the problem; corners in injection-molded pieces tend to have higher stresses than the plastic material of the part in general.
  • The thermal shock cycles, while not the root cause, doubtless did contribute. Rapid thermal shocks should be avoided where possible, or included in any factor-of-safety calculations if unavoidable.

 

Stainless Steel Stress Corrosion

A heavily-welded assembly was found, in the field, to have a pin-hole leak at a weld joint resulting in its hot and pressurized contents leaking out (an obvious safety issue; as above, to the best of my knowledge nobody was harmed). This piece was removed from service and sent for cleaning prior to repair. The cleaning process involved, among other things, the assembly being repeatedly soaked and washed in an agent recommended by the cleaning facility for this type of application and assembly material. A spot-weld repair was then made over the pin-hole leak.

Not long after being returned to service the assembly, which had multiple independent channels only one of which was to be “live” at any time by design and operating procedure, was found to have channel cross-talk – leakage of the working fluid from one channel to another. It was again pulled from service; I was put in charge of the failure analysis.

Even a visual analysis of the piece showed an obvious crack on an internal surface. Pulling prints of this assembly, plus comparable assemblies, showed several differences – only one of which is specifically relevant. To wit: the design (unique to this one variation) resulted in a thin-wall condition in certain locations. A worse-case tolerance stack up would exacerbate the situation significantly.

Sending the assembly out for metallurgical examination, including with it the process history of the cleaning and spot-weld repair including the cleaning agents used, resulted in several key findings.

  1. Sectioning of the cracked location and subsequent microscope examination showed clear evidence of stress-corrosion cracking, with cracks fully penetrating from one channel to another. The pictures of the cracked area were “stereotypical” of this failure mode.
  2. The primary cleaning agent, though recommended for the base material, was discovered in a literature search to create stress-corrosion cracking in this material at high temperatures and applied stresses. The suspicion – believed without explicit proof – was that trace amounts of the cleaning agent remained in the relatively-rough surface finish of the machined surfaces.
  3. Welded structures in general, especially ones with significant machining and welding such as this one, tend to have high residual stresses which are known to play a large factor in stress-corrosion cracking.
  4. In operation, tightening of threaded fasteners close to the thin-wall area introduced further stresses in that region from the torques required.
  5. The design, with its thin-wall condition even at nominal design dimensions, was at the root of the problem.

Result: All assemblies of this design were pulled from service and scrapped. Additionally, this particular design concept was ruled out from consideration in the future.

Lessons:

  • Check designs for thin-wall areas.
  • High temperatures exacerbate any stress-corrosion situation.
  • Stresses, whether residual from processing (e.g., machining or welding) or from other operations (e.g., threaded fastener tightening) can also introduce stresses which increase chemical-cracking susceptibility.
  • When using any agent to clean, not only vet it with both the base material supplier and chemical agent manufacturer’s recommendations, but check with a literature search to see if similar situations have been reported as causing failures.
  • In parallel with the above, go above-and-beyond when it comes to rinsing, in particular if the surfaces being cleaned have a rough surface finish. (And if there is a thin-wall area and it is visible and/or directly accessible, spot-rinse the area to beat heck.)
  • This investigation relied heavily on the expertise of a skilled consultant; in this case, a metallurgist. When specialized expertise is needed, don’t hesitate to ask an expert once you realize you do not have the requisite knowledge.

 

Galvanic Action Surprise

In one application where a cermet component was used in an oil-based environment, the assembly where these small cermet components were used was initially had a carbon steel base part. For machining convenience and better machining-process yield, a switch was made to a stainless steel base.

Not long after this change a new failure pattern of the cermet pieces was observed in parts returned from the field. Initially thought to be a defect in the cermet piece, both the cermet piece supplier and an independent metallurgist provided links and other information pointing to galvanic corrosion as the likely suspect after being informed of the base material change.

This seemed strange, as galvanic corrosion requires water; this was immersed in oil. However, chemical analysis of the oil showed not only measurable amounts of water, but of materials (salts mostly) in the oil that – when exposed to the entrained water and then dissolved in it – would form a conductive path which is essential for galvanic corrosion to occur.

The switch from carbon steel to stainless moved the situation from one where the carbon steel had been the material degrading across a large surface (and therefore going unnoticed) to one where the much-smaller cermet was preferentially attacked – as stainless steel has a much higher resistance to galvanic corrosion than carbon steel.

Result: The application was switched back to a carbon steel, though a different one than the original for better and easier machining. No further failures of this type were observed.

Lessons:

  • Even oil environments can have enough water and trace elements to create a pathway for galvanic action to occur.
  • If changing materials for machining considerations, consider a material of a similar type (e.g., in this case merely a different carbon steel), rather than a major change of material type.
  • If necessary to change to a material more resistant to galvanic action, keep an eye on materials in the system that have not changed as they may now be attacked by galvanic corrosion when they were not before.
  • Most suppliers want the application to succeed; tap them as knowledge experts before making such a change. If they’ve been in the business for a while, odds are they’ve seen similar situations before.
  • Again, don’t be afraid to consult with outside subject-matter experts on your dime to be sure there is not a conflict of interest (i.e., the supplier had their own metallurgist; the one we hired was a second opinion without that potential influence of their working for the supplier).

===

Disclaimer: I was very opaque about details like when and where these occurred, and the specific applications – and deliberately so. I am attempting to tread the fine line separating writing with just enough detail to be educational, and not disclosing anything proprietary (or damaging, though note: there were no dangers from the third example, and in the first two – to the best of my knowledge – nobody was harmed and in all instances situations have been remedied and are no longer a danger going on multiple years now). No company names are mentioned, nor are process parameters, design details, or specific material callouts given except in an “arm waving” mode to set the stage. Should there be a desire to change anything on the part of the companies involved, please contact me directly and make some suggestions as to rewording to remove any information that might currently be creating a material-damage situation – a situation which I have labored in good faith to avoid with my deliberate vagueness.

 

© 2015, David Hunt, PE

Fix the Problem XI: The Measles Chart

A very useful tool in examining production problems of discrete items is the “Measles Chart” – marking the location of defects on a drawing or schematic of a product. In this essay I’ll discuss my first, instinctive, use of this technique, and then briefly show how I used it on the three radiators discussed in a prior Fix the Problem posting (link will be given below, close to the analysis), asking the readers to coordinate the images here with the text in that other essay.

Bins of Scrapped Parts

When I first started at Ford Motor Company in Sandusky, Ohio, my new boss told me to go out into the plant, explore, and start to learn my way around the 1.1 million square foot facility.  One day I was out wandering and came across the newly-launched Ford/Taurus headlight line, and noticed that every few minutes a shiny component would arc out from the central repair area inside the line and land in a large basket, something like 5 feet square, by 4 high; at that moment in the mid-morning the basket was half-full.  So I went into the repair area, introduced myself as a new engineer out exploring, and asked about the basket full of scrap parts.

I learned that the repair loop’s primary job, as expressed by-the-numbers, was to take assemblies that had failed the leak test, remove the reflector which formed the back of the headlight, replace it, and send it back into the loop to go through the leak test again. The presumption, therefore, was that the leak was the reflector’s fault.  At something like $15 a piece, with baskets being filled almost daily for return to the supplier, this was adding up to big bucks fast.

I asked if there were a way to determine the location of the leak, as I realized that if the reflector were defective the leak would be on the inside of the rubber molded “boot” around the edge of the reflector, while if it were a seating problem it would be on the outside. Sure; just spray soapy-water “snoop” and watch for bubbles while the lamp was under pressure.

Getting Buy-In

Because I was new, and this line was not my responsibility, I sought out the Line Engineer and – again – introduced myself. I brought him back to the line, explained what I wanted to do, and got his assent which he communicated to the repair operator.  Here’s what I wanted:

  1. I drew an oval representing the perimeter of the reflector, and asked the repair operator to make a mark where the leak was found
  2. Explain to the next shift repair operator to do the same thing
  3. I’d come by, collect the sheet, and give another one every morning.

What Did The Data Say?

 dn101 oval

This first image was typical. Almost all the marks were on the outside, indicating some kind of a seating issue.  With the cooperation of the Line Engineer, we instituted a new rework procedure:

  1. Use “snoop” to determine if the leak was inside or outside.
  2. If inside, remove and replace the reflector.
  3. If outside, make a mark with a wax pencil on the bottom of the lamp, back the screws holding the reflector off, jiggle and reseat the reflector, and redrive the screws.
  4. If a lamp with a mark came back, remove the reflector, replace it, and make another mark on the bottom of the lamp to form an “X”. If a lamp came back in even with a new reflector, scrap the whole thing because driving and redriving the self-tapping screws – should the lamp come back yet again – would weaken the plastic where they were driven if they were cycled too many times.

The Results

The reseating operation almost always worked. Instead of shipping a full basket back to the supplier virtually every day, returns fell to – going from memory – something like one basket every 2-3 weeks.  Furthermore, almost all returns were due to issues with the glue joining the hard plastic reflector to the soft rubber boot… which became the subject of an improvement project to find a replacement glue and more reliable / repeatable dispensing system compatible with that to-be-identified glue.  (But that’s a tale for another essay.)

To my recollection no full assembly was ever scrapped.

Radiator Measles

In my essay Fix the Problem IX I discussed problem solving on three different radiators.  In this case I added a new wrinkle: color coding for each shift (as this department was a round-the-clock operation).  Specifically, black for first shift, red for second, and green for third.  Please look at the images, below, and then read the essay about these radiator trouble-shooting cases.  (Note that these drawings are representative, and are instructional only.)

For the first radiator, the measles chart looked like this:

case 1

There was no clear pattern in either location or shift.

The second radiator had this chart:

case 2

Definitely a location dependency, but no shift dependency.  All high fins were adjacent to the Z-notch.

And the last one had a chart looking like this:

case 3

Both a location and shift dependency.  All high fins were right next to the notches for the braze bars.

Summary:

When faced with a production defect issue a Measles Chart, especially with the added shift color differences, can be enormously useful – yet very simple – in ferreting out patterns in time, personnel, and location. I strongly recommend its use as a standard, first-pass tool in collecting data before more advanced, time-and-resource-intensive data collection is pursued (again, assuming the product being produced is amenable to such charts, of course).

 

© David Hunt, PE

 

Fix the Problem X: Learning from Interview Problems

Interviews are where companies and candidates learn about each other to see if there is a fit between a candidate’s background and personality, and the company’s needs and culture.

One of the things candidates, in particular technical candidates, get asked on interviews is to solve problems.  Often these are company problems that have been already solved, in an attempt to see how the candidate approaches problems.  Sometimes these are questions about extant issues.

Plusses, Minuses, and Danger Zones

On the positive side, this is a chance for candidates to show their abilities – ideally referencing other accomplishments as well to build a case for their skills.  On the negative side, such questions are often about problems that took multiple people hundreds, if not thousands, of man hours to identify and resolve.  Thus they can be very difficult to actually solve, in particular as a candidate is not as familiar with the details of the product and the processes by which it is made.

And if the issue is one currently extant, there is a danger that the line of inquiry is being used to obtain free consulting.  I’m not accusing all companies of doing this, but the possibility does exist.  So in the process of answering a candidate does need to be on guard.

A Two-Way Street

These problems, however, are an opportunity for candidates to learn as well.  I will discuss three situations where I was presented with problems in interviews in the hopes that these experiences will be of use to my readers as well.

Fiber Optic Bubbles

Back in 2001 after being in a massive RIF from Visteon, I interviewed with a company that made fiber optic cables.  The process was fascinating!  They would take raw, uncoated optical fibers off reels and pass them through a coating bath, lining seven fibers next to each other; this bath material would coat the fibers and adhere them together side by side into a ribbon.  Then, two of these seven-fiber ribbons would be lined up edge to edge, passed through another coating bath, and stuck together.  These ribbons would then be stacked up, one upon another, to form a rectangular bundle, and run through a machine that extruded a multi-layer protective sheath around the whole thing.

The issue they were having was bubbles in the coating around the fiber; bubbles distorted the coating and exerted a force on the fiber which, apparently, would allow light leakage out of the fiber – degrading the signal.  They handed me one of the 14-fiber ribbons to look at.

I have discussed looking for patterns in prior essays, for example here and here; I immediately noticed two things.

First, the bubbles were only present at the centerline bond between the two smaller optic-fiber ribbons.  This was an important clue as to where in the process from raw fiber to end cable this defect was being introduced.

Second, in looking at the bubbles, the spacing between them was remarkably consistent.  These bubbles, then, were not randomly formed, but something in the process was doing something that created a periodic “hiccup” creating these bubbles.  Using a ruler I got the distance between the bubbles, IIRC an inch-and-change, and in asking for the speed of the ribbon I was able to calculate the time between bubble formation.

My guess was that something was oscillating in the system, but probably not smoothly; there was likely “something” sticking slightly, with the sticking creating jerking motions as that “something” stuck and unstuck.

Since the serpentine path the ribbons took was visible, my approach would have been to use a high-speed camera and/or strobe light to identify whatever parts of the system were oscillating at that identified frequency as the place to focus attention.  (E.g., if the frequency was 10 Hz, light the strobe off with a frequency of 20 Hz.  The “something” – my a priori guess was one of the slack take-up reels – would appear to be shifting back and forth between two positions after some experimentation to synchronize the strobe with the “something’s” cycle.)

They were excited by my idea.  Unfortunately, I never did hear anything further about whether the idea contributed to the solution of the problem (I asked a couple of times over the next year).  I suspect my idea contributed to improving someone’s performance review.

Lesson: Strobe lights and/or high speed photography can be enormously useful if a frequency-dependence of a problem can be identified.

Cavity Numbering

Another interview was with a company whose supplier – in China – was molding their components and shipping them to the US for assembly.  There were eight cavities in each of two molds; these two components would then be assembled to form the product.  Some of the pieces, A and B, would not fit together.  Measurements indicated that there were dimensional issues causing a problem with some cavities of part A not fitting into part B.

Here was the problem: they didn’t know which cavity was which – translating to an inability to determine which cavities worked interchangeably and which ones didn’t.  They already were pressuring their supplier to take the molds offline for a day or so to put in some kind of cavity marking.

Lesson: If you have multiple cavities in an injection mold, number them to aid in troubleshooting when confronted with situations like this.

Engineering Fundamentals Always Apply

In one very thorny problem presented to me, a glass-encased thermocouple was placed into a blind hole machined into an aluminum piece and potted in place with an elastomeric material.  These units were subject to thermal cycling as part of the unit’s operation.  Failure analysis showed the thermocouple’s glass casing was cracking.

This is where engineering fundamentals comes in.  What happens when things get hot?  They expand; each material has its own expansion coefficient.  Resins – from which the potting material was made – typically have a higher expansion rate than metals, in this case aluminum.  With the resin trying to expand faster than the material around it, its attempt to grow was constrained by the hole walls – introducing a compressive strain from the difference in expansion rates.

Stress – force per unit area – is strain times Young’s Modulus.  The expanding resin, being constrained from growing outward, squeezed inward on the glass and fractured it.  The solution was to change to a softer resin with a lower Young’s Modulus.  Even though the thermal expansion strains were the same, because the resin was softer, the forces were lowered and the glass didn’t crack.

Lesson: Sometimes you really need to get back to basics.  In this case, mechanics of materials.

Interview Problems: An Opportunity

Not all interviews work out (alas).  But the presentation of a problem to solve can be an opportunity – not just to prove your mettle, but to take advantage of the experiences of others to improve your own problem-solving toolbox by being walked through a real-life case study.

© 2014, David Hunt, PE

Fix the Problem IX: Three High-Scrap Issues on a Product Family

In Fix the Problem VIII I introduced a simple concept, requoted here:

You can only claim to have truly solved the problem when you understand it well enough to turn it on and then back off again.

I want to discuss three production problems, all on the same product family – but with three different root causes.

Background

While working on a cost-reduction team at Ford Motor Company that focused on material and labor savings on existing products, we would occasionally be called upon to act as extra eyes on specific problems in the plant. I was asked to look at three different radiator lines and their “high fin” scrap rate.

Let me define “high fin”. In automotive radiators there are thin, flat tubes that carry the radiator coolant from one header tank to the other; heat gets conducted through the tube walls and into the corrugations made from thin aluminum sheet. This sheet – really an aluminum ribbon – is transformed into the corrugated fins by use of a star-shaped wheel that acts as a mandrel to form the up-and-down corrugations. Functionally, these fins wick heat from the tubes of hot coolant and provide a large surface area for moving air to remove heat from them, thus providing a heat sink.

These fins, in turn, are placed between the flat tubes; the headers that formed part of the coolant reservoir and manifold (which evens the flow distribution between tubes) are placed, the assembly compressed, and steel clamps put on to maintain the tension until the radiator is brazed. As a part of the brazing operation, there is a wash with soapy water that sprayed down onto the radiator face to remove grease and other contamination prior to the braze heat being applied.

In each of these products, fins were being knocked loose from between the tubes and then brazed in that out-of-position position. This was not repairable, resulting in a piece of scrap and a brazing cycle wasted.

First Steps

In all three cases I got the last three months’ worth of scrap data to look for trends, and had scrap items put aside at the end of the line for me to look at, with the time and date of the exit from the braze oven written on them. I also had an intern temporarily assigned to me, and had them – on one shift per day (ideally this should have been for all three shifts, but it wasn’t practical) – watch the radiators going into the braze oven to see if we could determine where in the process stream from raw material to end product the defects were occurring. (Note: no high fins were seen going into the braze oven; this point, first made in Fix the Problem II: Medical Blister-Pack Punctures, is important – knowing where a problem starts to be seen can be a critical clue in identifying the root cause.)

Radiator One

This particular instance is shown on a portfolio page, here.

The most important thing was that over the last three months the fallout rate was getting worse. Something was clearly going in the wrong direction, so I started here. The next thing was that I put the radiators in a row, by time and date. I also graphed the data; my hope was to see if there were any shift dependencies or pattern in the locations of the high fins. They appeared to be random in both time and location.

Time for basics. What holds the fins in place until they’re brazed?

Looking at the prints there was an interference between the nominal fin height and the gap formed by the tubes. In other words, after the fixture compressed everything, the fins were compressed like springs, pushing against the tubes with friction holding them in place (thus the need for the braze bars to hold the compression). Clearly, something was not holding them in place, and not doing so in an apparent random fashion.

I polled some colleagues about the problem to get outside perspectives. What could be causing this? We did an Ishikawa fishbone diagram, went through a few Five Why exercises, and the dominant factor implied something was awry dimensionally (something could have changed in the material, too, but this seemed the best first-guess approach especially as there were similar products with no issue); so what dimensions would be critical for this function? I had a pretty good idea, of course, but I wanted other minds’ input to make sure I didn’t miss anything. The consensus was that three dimensions were the critical ones:

  1. The center-to-center distance on the aluminum headers into which the tubes were inserted prior to brazing.  This was an inspected dimension.
  2. The outside-surface-to-outside-surface of the flat tubes.  Again, this was an inspected dimension.
  3. The top-to-bottom distance of the corrugated fins.  Once more, this was an inspected dimension.

But what did the data say about these? For the header dimensions and the flat tubes, these were spectacularly consistent, with high process capabilities as indicated by their CpKs. But the fins were another matter.

We had a laser system that would measure the peak-to-peak dimensions of the corrugations. The way it worked was that a fin would be put in, and the data would be aggregated into an average and recorded. I wanted the raw data, and measured several fins and then graphed them: height as a function of position along the length of the fin (which comprised, perhaps, 80-100 peak-to-peak dimensions). And then I stared at those graphs.

Sometimes, all that’s necessary to solve a problem is “sit and stare” and let intuition and your subconscious work. In a EUREKA! moment I said “There’s a cycle in there.” So I took the data and – manually – did a best-fit sine wave, finding that the height of the fins was going up and down with a cycle every 26 corrugations.

Clearly the flat ribbon coming into the fin-making machine didn’t have a pattern in corrugations; there no were corrugations at that point. Similar to my example, here, I “walked the line” to find where the corrugations were formed by the fin machine. Then I went to the maintenance guys, those people who knew the equipment, showed them my sine wave, and asked what might be causing the 26 cycle. Immediately they replied “the star wheel”. The star wheel was what formed the corrugations, having 26 teeth.

Fortunately, the maintenance foreman was a sharp cookie; he had the machine shut down and taken apart. What they discovered was that the bearing on which this star wheel rotated was deteriorating, and said deterioration was introducing an eccentric motion as the wheel spun while it formed the corrugations. The bearing was replaced, everything aligned, and we started up again.

I redid the measurements – no cycle at all. CpKs were much better. Most importantly, high fin scrap fell to zero. To prevent this from happening again, the fin measurement procedure had an alert callout added for when the CpK started to drift above a given limit; standard SPC practice to alert the technician that the process was going out of control. The machine maintenance plan was also modified to include periodic alignment and bearing checks.

Lessons:

  1. Averages have their uses, but lots of information can get washed out by relying on them without examining the raw data.
  2. Using the five-why process, in this case to drill down to dimensional variation (a basic engineering fundamental), was instrumental in identifying the root cause.
  3. Intuition again played a key role, as did leveraging the different perspectives and experience sets of other people.
  4. Identification of where the problem was happening – somewhere inside the brazing process – was also a key datum.

Radiator Two

Unlike the first radiator, there was a definite and specific location to the high fin: it always happened right next to the side-rail Z-notch that was in the top and bottom rail forming the outside of the radiator, and it did it across all shifts; nor was there any change over the months indicating this was a chronic problem. Let me discuss the Z-notch.

The side rails were aluminum C-channel rails whose long center flat was brazed to the fins. In each rail was a stamped cutout, called the Z-notch. It created enough structure for handling and processing, but at the end of the line, the remaining material would be cut so that as the radiator heated and cooled there was a place in the rail allowing it to expand and contract freely – like heat expansion joints in a bridge. The high fins in this case were always, always, always at this location.

Since this problem was not observed across other lines, the question to answer was “What’s different?”

This ended up being very simple; I went to the prints – usually my first stop in any problem-solving exercise. Was the material different? No. What about other parameters? The length, one of the things that defines the stiffness of a beam, was “in the pack” of other side rail lengths. Nothing sticking out there. And the cross-sectional dimensions of the C-channel were identical to the others as this raw material was used across virtually every line. But as it turned out, the Z-notch was a different design than the others.

Looking at the geometry, the stiffness of the remaining material at this joint – the moment of inertia that quantified the stiffness of that rail at that point – was less than half what it was in any other design. In essence, that lack of stiffness formed a hinge point, allowing the forces from the compressed fins pushing outward to deflect it more, thus lessening the retention force on the fins – so that the force of the cleaning spray could knock it out.

I checked downstream with the operation to see if changing the Z-notch would impact them; no. I then approached our Design group, pointed out the differences, and asked if we could do a trial run with a standard Z-notch design. A week’s worth of test pieces were made, the appropriate trial forms signed, and a run was made. High fins disappeared with no impacts downstream – as checked for beforehand. Based on my findings and the successful trial, an ECO was written.

Lessons:

  1. In looking at a problem occurring on one product of a multiple-product family, the very first question while working through the possibilities should be “What’s different?”
  2. As with the dimensional analyses above, engineering fundamentals – in this case, an application of my Strength of Materials course – were essential to understanding the issue.

Radiator Three:

As with radiator two, the high fins would always occur right next to the side rails. However, there was one key datum that stuck out: they only happened on one shift out of three. One compression-assembly machine, but – obviously – three different operators. Therefore, in Ishikawa parlance, the likelihood was that it was a man (of man, machine, material, method, measurement, mother nature) that was the first thing to be investigated.

Under the guise of “just poking around” I stayed late on the second shift to watch the operator out of the corner of my eye. He, like the others, worked diligently… but there was one key difference in how he worked. (“What’s different?”) All the other operators would place the radiators on the conveyor belt, while he dropped them. But although he was dropping them, it was only from a few inches.

Getting permission from the foreman on the first shift, I did a few tests to drop the same radiator onto the belt. And nothing. This was interesting. I did some drop tests with other radiators, and nothing there as well. Clearly the drop was a difference, but not the determining difference.

So I talked with the operators; asking “What’s different about this radiator?” I was thinking that there might be some subtle difference, but all the operators I spoke with, on two shifts, had no comments. A review of the prints didn’t show anything obvious. And a key question, which had to be handled very diplomatically, was “What was different about this operator?”

As it turned out, he was a new hire, having been brought on within the past year; all the other operators were veterans with years of experience. Why did this make a difference?

As I’d mentioned in the introduction, steel clamps were put on to hold the tubes and fins compressed until the brazing operation could fuse them together into a solid piece. In talking with the operators, one casually mentioned that the clamps, called braze bars, had to be installed through small positioning notches in the C-section side rail, and had to go through both sides. If they didn’t go through, placing them on the conveyor could create a force that could distort/damage one of the C-section sides “especially if the radiator were dropped” since they were always put on a conveyor with the braze bars underneath the radiator. AHA!

I spoke with the second shift foreman, who had a discussion with the operator. He was not aware that the braze bars had to go through both sides of the side rail notch. Most of the time he got it right, but sometimes he didn’t – and not knowing how critical it was – would then drop the radiator on the conveyor. (Likely he had been told but merely forgotten.) The impact of the drop would be passed through the misaligned braze bars, distorting the side rails and allowing decompression of the fin.

This was the missing element. After being made aware (again, more likely reminded) of how critical this was, along with an explicit instruction to place, not drop, the radiator, the issue went away. I did design plastic lead-ins and got them mounted on the machine as a trial. The theory was that an operator could put the braze bars into the lead-ins and literally drop them in; the lead-ins would align everything so that the operator could not do it incorrectly. It worked, but the operators didn’t like them so they removed them.

  1. When faced with a shift-dependent problem the very first thing to be considered should be operator error.
  2. Had this not been the root cause, some kind of environmental issue – would have been the next thing to be considered; since things like temperature, light level, and being aware/awake can be affected by shift schedules, even when people are accustomed to their off shift schedule.
  3.  The ideal solution, when faced with an operator-dependent problem, is to engineer it out.  But all the best efforts are naught if the operators don’t like the solution and remove it.

© 2014, David Hunt, PE

Fix the Problem XIII – The Problem Toggle

(Author’s note: The “L” key on my computer – old one, new one finally ordered – was not working, and I missed it while posting this.  Apologies for the multiple revisions getting all the Ls into place in the title and link…)

While at Ford Motor Company, our group’s manager found a book that he thought was so good he recommended it to everyone, and even paid for our individual copies.  This book, Manufacturing Solutions for Consistent Quality and Reliability, was excellent.  But the book made one fundamental point about problem solving – whether in performance of a product or a process – that has stuck with me ever since (paraphrased):

You can only claim to have truly solved the problem when you understand it well enough to turn it on and then back off again.

With this in mind I’m going to review two instances from my career where this happened.  (I will revisit this theme in future essays to discuss more case studies.)

Plasma Cutting Torch: Mysterious Leaks in the Field

This particular torch was part of a plasma cutting system; I was the Manufacturing Engineer in charge of the torch area, and working on identifying the root cause of field failures was one of my responsibilities.  This particular torch had a significant rate of return; these returns, responded to by sending a new torch, was a large part of my department’s total warrantee cost.

The issue was that the problem could not be duplicated. We would receive the torch which, according to the customers, would have coolant leaking out the front “business end” of the torch.  No matter what we did, we couldn’t get returned items to do it in our lab.

Lesson One: Unless you can duplicate the failure, in order to experiment with what does and does not trigger it, you are flailing in the dark.

So one day we get a torch back where we’ve been told the coolant is gushing out in a waterfall.  We go to our own test machine, connect everything up, turn the coolant flow on… and nothing (like always).  We exchange parts of the system for new ones.  Nothing.  Go back to the original, returned pieces.  Nothing.  Try as we might with permutations of new parts and returned parts, we cannot duplicate the reported failure.

Let me take a moment to say that I trust intuition.  Something was nagging at the back of my mind.  I couldn’t even quantify it, but something was bothering me.  Just as the technician was about to turn off the coolant flow, I said “Hold it, I want to try something.”  I reached over to the torch, grabbed the retaining cap that threaded on and which held everything inside, and started to turn and loosen it.  I had barely touched it when coolant started to jet out of the opening.

I tightened it.  Nothing.  I started to turn it, and again, coolant flowed copiously.  AHA!  I tightened it to its hard-stop and made a mark.  Then I slowly started to loosen it until, like a floodgate opening, the flow started again.  I marked that too.  It took, maybe, a 0.25” of distance, as measured on the outside diameter, to make this difference.

Armed with this information I went to the prints and calculated that – going from memory – the axial translation of the cap being unscrewed was on the order of 0.020”.  This information was passed to the Design group, which found that the issue was design-related (details omitted for confidentiality reasons).  Designs of a few, key components were tweaked and prototypes made.

The result: The redesigned torch wouldn’t leak despite backing the cap off double what I had done.  After the change, warrantee returns started to drop dramatically as the redesigned torch was propagated into the customer base.

Lesson two: Intuition and hunches are often based on a subconscious stew of disparate facts coming together.  While you shouldn’t just go with them – a systematic approach like an 8D Problem Solving Process is needed – don’t ignore those tickles at the back of your mind.

One other thing to note.  In retrospect these symptoms and the “no problem found” status made sense.  Plasma cutting is often a dirty environment with grit, metal chips, etc., around.  Likely what happened is that people took off the cap while changing the consumables – the “razor blades” – putting it down in such a way that grit got onto the surface that was supposed to be flush with the surface that provided the hard stop when installing the cap.  This created an inadvertent shim that coupled with the design issue to create the leak.  In the process of being shipped to us the grit would fall off, removing that inadvertent shim and resulting in a torch that would function as intended with no problem.

Lesson three: When troubleshooting, think about the environment where the failure is occurring.  Ideally, go and watch.  There’s nothing like seeing the precise situation for generating data, even if that data only goes into the aforementioned subconscious.

O-ring Rollout: Leak Failures in Manual Assembly Area

At the same plant where I first was given this book, one of the products was a carbon canister assembly that fitted into the fuel cell.  Functioning to absorb gasoline vapors coming off the fuel tank for emissions control, some of them had several hoses with male attachments that would be inserted into a female port.  The work time standard was strict and people pushed hard to meet it.  (Note that I have a portfolio page about this problem.)

At issue was the fact that the O-rings forming the seal at these ports would roll out of the groove they were in, creating a leak path.  This assembly defect was internal to the female port and so was not visible.  The first indication there was an O-ring rollout was that the unit would fail the leak test.  The unit would then be methodically disassembled until the rollout was found.  Then it was reassembled after reseating the O-ring, and retested.  As you might imagine, this was quite time-consuming (not to mention not value-added).

Having been asked to look into the problem, one of my first actions was to look at the Design Guidelines.  Since my Master’s Research was in Design for Manufacturing and Assembly, I had – stated immodestly! – a pretty good grasp of the dynamics of how things go together.  One thing I noticed was the design of the lead-in.  Although “perfect” from a molding standpoint, having a radius as a lead-in was not so great from an assembly standpoint.  The reason being is that the O-ring needed to slide along the surface without being “grabbed” by friction.  The governing equation is:

Arctangent(angle) < coefficient of static friction

With a visual:

angle image

Note that in no case will an angle greater than 45 work.

What I found, in a detailed examination, was that it was possible to misalign the male insert sufficiently so that the O-ring would hit on the part of the radius where static friction would dominate.  This would then “grab” the O-ring and, as the insertion progressed, it would roll out of the groove.

In a Design for Assembly analysis I’d written before on my blog I referenced Fitt’s Law.  I applied it in this case and found that it was a difficult task for a person to do reliably – which explained the high rework rate.  If I redesigned the lead-in to be a 30 degree chamfer, as shown in the portfolio page (referenced again for convenience), I essentially made it impossible to NOT get the O-ring on a sliding surface.  (NB: a 15 degree angle is my “perfect” recommendation for this situation.)

Lesson Four: Very often there is a Design issue at the root cause of a production problem.  Not always, of course – but in my experience the probability has been very high that Design is a contributor to the issue.  Note that Design is the foundation: Reliability, Functionality, and Quality all start with Design… one can have production issues even with a good design, but one cannot have good production with a bad design.

Based on my write-up we made an insert for the mold of the female port (fortunately the mold boss forming the core of the port was an insert that could be easily changed!) and tried it.  Leak failures from O-ring rollouts fell to nothing.  But there’s one more lesson… I got a call from the Design group asking me why I was proposing changing the design, including altering the Design Guidelines.  When I asked if he’d seen my write-up, he said yes.  When I asked if there was a problem with it, or with the results showing it worked, he snarled – literally snarled – through the phone: “You’re just a Manufacturing Engineer, what can you know?”.  Needless to say I was tempted to retort, but again returned to the successful results and appealed to his “We’re all one company, right?” spirit.

Lesson Five: People can get very protective of “their turf”; keep that in mind as you propose changes, especially if the changes are in someone else’s department.  (In retrospect I should have involved the Design group from the beginning to have them on the team and involved once I figured out this was a Design issue.  Thus I would have avoided turf battles, toe-stepping, and bruised egos.)

The Problem Toggle

In both instances changes were made that turned the problem off.  In both instances we understood the root cause well enough that if we had gone back to the old design, the problem would have returned… and why it would have returned.  In these two cases there was just one true “root cause”, Design, but in other cases I’ve experienced there were multiple factors that worked together to create the problem.

Only by systematically working through a formal process, often including tools like an Ishikawa Diagram which can be very useful, testing each identified possibility by duplicating the failure conditions to see if the change affected failure rates, can problems be declared solved.

Otherwise, solutions become a variant of “I’ve got everything just right; don’t touch anything!”  And that’s not a way to design and produce in today’s hypercompetitive world.

 

© 2014, David Hunt, PE

Fix the Problem VII: Product Launch Crisis

At Ford Motor Company in Sandusky, Ohio, the flagship product of the plant was exterior lighting: headlights, taillights, turn indicators, and high-mount stop lights.  During launch there would inevitably be problems – sometimes in molding, or decorating, or assembly.  Things would, of course, get solved, but sometimes things were so bad that it became an all-hands-on-deck affair.

One product launch was going particularly badly.  Despite every shift in every department related to this product – a high-end luxury-car headlight – going 24/7, we literally could not get enough product out the door.  With every department scrambling, with the assembly plant screaming, the group I was in was also asked to contribute.

At the time the methodology called Theory of Constraints, developed by the late Eli Goldratt, was in vogue at the plant (I believed it then, I still believe it).  I believed there had to be one or two critical points in the process of going from resin pellets to finished headlights on pallets that were the prime constraints.  My task was to study the process from a 10,000 foot view and attempt to identify what those critical points were, so as to permit a focused and systematic attack on the problem rather than everyone milling around everywhere.  (Please keep in mind that I’m “making up” most numbers – they’re close, but after a decade and a half since I was in the situation, my memory is rather fuzzy.  But the overall analysis technique will hold.)

Let me be clear in saying my goal was simple: identify (and prove with data) the one or two process steps where the fallout was the most significant, and which was preventing meeting shipment quantities and deliveries.  In other words, identify the “worst of the worst” for more focused attention.

A Group Effort

One other thing: I must give credit to the team.  Although I gathered a lot of information that was useful to focusing attention, and chimed in with many suggestions, many people contributed to attacking the problem.  This is critical – the aggregation of multiple minds, with different perspectives and experiences, was the “magic factor” in making such enormous progress.  Vendor participation and vesting in the success of this effort were also critical, as in many instances they had expertise we did not have in-house.

Beginning at the Beginning

I started with our customer’s demand.  We had a volume of X lights that had to go out the door every day.  I worked backwards from there.  I quickly found that we easily had enough purchased-part hardware for the back of the light.  We also had enough of the internal reflectors that handled the headlight beams; bulbs were, of course, standard and plentiful.  Leak testing was the primary failure point for a light once it started going through the assembly cell – leaks coming from bad glue pours.  The failure rate through the assembly line, almost always at the leak tester, meant that for every light that got packed ready to go, we had to try to assemble 1.2 lights.  This was well within the capacity of the assembly line to handle – although the failure rate was not sustainable in the long-term as the gluing operation, once done, could not be reworked or repaired… thus creating an expensive piece of scrap.

There were two main components to the light: the lens and the body.  I had a hunch where the problem was, and anecdotal feedback from the floor, but hunches and anecdotes are just that; I needed data.

The Body

The body consisted of injection molded polycarbonate.  The body was metalized to reflect the light from the main lamp and the two integrated signal lamps.  Masking was tricky on metalizing for a complex, curved piece like this one… in order to get 1.2 bodies ready for assembly, we had to attempt to metalize 1.4 pieces.  This was not a problem for the metalizing department’s capacity.

Swimming further upstream, the fallout from the molding process meant that we needed 1.5 molding cycles for every light being shipped.  Again, this was well within the capacity of the molding machine.  Clearly the body was not the issue.

body

The Lens

The lens was also a complex, 3D curved piece that wrapped around the front corners of the car.  Arriving at the assembly line, it required two secondary operations after molding.  Again swimming upstream towards raw materials, the next process was the painting operation.

In this instance there was a thin strip of paint applied to the back of the flange that wrapped around almost the whole exterior of the lamp.  An after-thought, it was not believed to be necessary initially as the back of that flange had some texture which was thought, during initial design, to be sufficient.  However, the prototype and first-run pieces showed that paint was necessary; thus it was kluged into the production sequence (at a vendor, as we didn’t do painting in-house).  Here was the first bottleneck – the fallout was huge.  To have 1.2 lenses ready-to-assembly, we needed 5 lenses shipped to the painting vendor.

Proceeding further upstream, the polycarbonate lens needed a clearcoat protection on the outside.  A scratch-and-UV protection, this too ended up with a huge fallout.  To have 5 lenses ready to get painted, we needed to have 7 lenses delivered to the clearcoat process.

Lastly, the injection molding process.  This lens was a multi-shot lens; what this means is that a large, multi-station mold was used to overmold one material over another to form a single piece with the amber turn signal plastic integrated into the lens.  Looking at the bins full of scrap at the machine, and gathering data, I found that to have 7 lenses going to clearcoat, we had to try to mold 11 lenses.  Fallout was primarily due to bleeding of one material, the amber, into the space which should have been clear.  Given the cycle time of the mold to produce a lens and the volumes required, this was not possible in a 24 hour day.

Clearly, the lens was the bottleneck component.  This confirmed both my hunch and the anecdotes – but now I had evidence.

lens

Where to Begin?

My first visit was to the painting vendor seeing as they had a 5:1.2 needed-in to coming-out ratio.  I found that there were two primary scrap issues.

The first issue was that overspray was creating scrap.  Since this was a kluged-in-at-the-last-minute process step, masks had been hastily fabricated in order to meet deadlines.

This highlighted the need to invest in new “best available” masks.

The second issue was, interestingly enough, also related to masking – in clearcoat.  The clearcoat overspray would wrap around from the front of the lens to spatter on the back side of the flange where the paint would be applied.  The chemistry of the clearcoat was such that it had a very low surface energy – surface energy being critical in good adhesion – and the paint would not stick to it; this failure mode comprised, per the vendor’s data, about 40% of the fallout.

New masking for the clearcoat operation was also ordered.

Handling Damage

Another area of fallout was between the molding and clearcoat operations.  The lenses were placed in large baskets, layered one on top of the other with cardboard in between the layers.  These baskets would then be conveyed from molding to clearcoat; movement of the lenses, even just a few dozen yards across the plant, resulted in scratches and scuff marks making lenses unusable.  In a sharp-pencil look at this, we were having a 10% fallout from this short travel distance alone.

Stackable soft-foam-lined packing boxes, customized to this specific lens, were immediately ordered.

Low Hanging Fruit

The new masks for both painting and clearcoating took a week or two, as did the new protective containers for transporting the lenses from molding to clearcoating.  The results between these changes were nothing short of amazing.  The paint fallout fell from 5:1.2 to 2:1.2 from the combination of fighting paint overspray as well as better protecting the paint surface from clearcoat overspray.  Fallout from transportation damage fell to zero.  These simple changes resulted in a sea-change situation: the molding process, still strained, was now able to keep up with demand.

Additional process changes to the molding process – IIRC slowing down the injection process slightly – lowered pressures and reduced the bleeding occurrence enough so that even with a slightly longer cycle time, it more than made up for that with significantly-reduced scrap.  (After all, it’s not how fast you can crank out parts – it’s how fast you can crank out good parts.)

Further Steps

According to Theory of Constraints, the process with the longest cycle time is the process constraint – it is the drum that defines the pace of production; a defect in the constraint operation can never be made up by anything done in the downstream operations.  Further, any defect downstream resulting in a scrap piece can be considered a wasted constraint cycle.  In this case the molding of the lens was that constraint process.  Thus it was critical to address the molding defects.

With the improvements downstream there was an ability to build up an inventory of lenses to allow work on the mold – and fix the issues that resulted in scrap lenses.  This had the benefit of being able to revisit the injection rates, speeding them up again now that the mold sealing against leaks was more robust.

After this intense effort production was still not in great shape; we still had unacceptable levels of scrap fallout at multiple places in the pellets-to-pallets flow.  However, this concentrated focus – facilitated by my fallout map – was able to move this product from burning crisis to a more standard launch-issues mode.

Lessons Learned:

  1. Starting at the end of the production line and working backwards through the various components’ fallouts can show where the critical issues lie.  A graphical approach like the one I used can help.
  2. Last-minute additions to the required processes – in this case painting – should be scrutinized in particular as they are often (based on my experience here and in other such situations) a huge contributor to launch problems.
  3. Masking for any paint or coating operation is critical: don’t be cheap, get the best masks possible even if they come at a premium cost.  (We also eventually invested in better masks for metalizing the body – another area of significant, though not critical-path, fallout.)
  4. Handling can create scrap; in this case, polycarbonate is great for impacts but has poor scratch and scuff resistance without a coating.  Better to overprotect than underprotect.
  5. A few low-hanging fruit improvements were the seed crystals in a progression of changes that allowed major progress.  Often simple things can make huge differences.
  6. In the case of the painting operation, we leveraged the supplier’s expertise and involvement in the project’s success.  Good suppliers are not only worth a premium in terms of quality and time, but in terms of having very specific knowledge about their core business.
  7. A critical piece of information was that we needed to anticipate – in both design and production planning – painting the backs of such lamp flanges in the future.  Part of the masking issue was that this had not been anticipated, and designing in a small lip/other feature for better mask seating would have helped enormously.  This information was passed back to our Design group.

Addendum: I realized that most of these in this series are manufacturing-related.  I would love to write about design aspects of my career, but in most instances these fall under NDAs I signed.

© 2014, David Hunt, PE

Fix the Problem VI: The Role of Material Microstructure in an Overmolding Application

While working at Ford Motor Company in Sandusky, Ohio – aside: home of the incredible amusement park Cedar Point – I worked with two other engineers on “Integrated Molding and Assembly” (IMA) as a new technology to replace using glue in automotive lighting.  Our first application was on the Ford Econoline headlight, as highlighted here on a portfolio page (alongside a small test piece, which I reference below).  This process was patented; my first patent!

Essentially, this process molded a seal bead of plastic, in the case of the Econoline made from polycarbonate, to join two other pieces – the lens and the body, also polycarbonate – and form a bond that was strong and leak-tight.  The process also worked with using acrylic and ABS bodies and either acrylic or ABS as the seal bead.  In almost all cases, burst testing cracked the substrate parts before the seal joint broke.  Descriptively, this was an overmolding process – a seal was overmolded onto the two substrate pieces.

Giddy with success including a successful launch on a real product (the Econoline), and as the plant was casting around for other potential product lines to bring in to diversify the plant’s product mix, we took a long look at nylon intake manifolds.  The idea was that two halves of the manifold would be molded, and in either a rotary or shuttle mold the two parts would then be joined, and a seal bead overmolded; resin pellets in, complete manifold out.  The patent space was already crowded for this product and similar process technologies, but we and the Ford attorneys believed we had something unique that could be done in a patentable way.

So we molded a bunch of lenses and bodies in our little test mold (again, see my portfolio page, link is above) made out of nylon 6/6 to test the concept; this being the standard material for the application.  We put them in the mold, overmolded a seal bead out of the same material… and we could pop the pieces apart with our hand.  There literally was no adhesion between the seal bead and either of the base parts.

Panic set in.  Everyone was wondering what was going on; it had worked so well before.  And then… I had an insight.  All of the materials we had molded and sealed with enormous success were amorphous plastics.  Nylon, in contrast, is a crystalline plastic.  There is a huge difference between these in several dimensions, and the one that mattered was heat capacity; crystalline materials typically have a higher heat capacity because of the material’s microstructure.

For a strong bond in an overmolding application, the overmolding material needs to remelt a microlayer of the substrate to which it can then bond.  Because crystalline materials have a higher heat capacity, more energy was needed to form that remelted microlayer when we had nylon parts; more energy than the molten seal bead of the same material had available.

To test this theory I arranged to use Amodel, a material that’s similar to nylons chemically but with a higher processing temperature, as the seal bead on our nylon 6/6 substrates.  This time it worked; the assembly didn’t fall apart.  Clearly I was on the right track; it was the crystalline nature of the substrates that was the issue.  When we switched to nylon 6 for the substrates, a nylon also used – though not as commonly – for intake manifolds, it worked even better… because nylon 6 has a lower melt temperature and heat capacity. This allowed for much better formation of that necessary remelted microlayer.

The project was now a “GO!”, but we found out that in the course of talking with one supplier, we had not obtained a non-disclosure agreement (NDA) first.  Ford’s lawyers said this ruined any patent possibility we might have had.  Without the ability to legally protect the technology, the project died.

So here are the key points to take away from this essay:

  1. When overmolding, amorphous materials are generally easier to overmold onto than crystalline ones.
  2. If overmolding onto a crystalline material, make sure the overmolding material is molded hot enough to overcome the higher heat capacity inherent in the substrate’s crystalline structure (even if amorphous, mold the lower melt temperature material first, and use the higher temperature material as the overmolded material).
  3. When overmolding, the more similar the overmold and substrate materials are chemically, the better; here’s a good Rule of Thumb: If they can be alloyed (e.g., PC/ABS alloys), they’re similar enough for a good overmold application.
  4. I strongly recommend adding features to physically “back up” the ability of the two materials to engage with each other from overmolding and thus resist whatever forces will be applied. I.e., use keyway features, flanges, and grooves to form a physical interlock.
  5. When you have a concept that you even think might be patentable, make sure you protect yourself with NDAs before you talk with anyone outside.  A few days of delay while getting things signed can save a new technology from being ruled not patentable.

© 2014, David Hunt, PE

Fix the Problem V: Cultural Awareness is Critical

A preface: This essay is only peripherally about the technical details of solving a particular problem. As I’d mentioned in my lead-in to Fix the Problem IV, anything I write about engineering work I have done for a past employer has to be generic to respect my ongoing confidentiality obligations.  Those hoping for an inside scoop as to engineering details are bound to be disappointed.

In one of my engineering jobs, I had developed a custom test protocol to quantitatively evaluate – and thus prove anecdotal evidence – that a redesigned component was a step-change improvement in resisting erosion, dramatically reducing process drift during a production run.

Needless to say, with this data in hand there was a strong impetus to push plants globally use these new components.  One plant, in Japan, resisted.  They said they’d tried them and they didn’t work.  Now this didn’t square with every other plant that had tried them – something was amiss.  What I discovered was that the plant had not followed the correct procedure in installing these components, damaging them in the process of trialing them.  Here’s where things get interesting.

Japan’s culture, like most Asian cultures, sets great store is person’s “face”, or reputation – how they are viewed by those around them.  (Interestingly, the miniseries Shogun, based on the novel by James Clavell, accurately depicts many examples of this – I’ve been re-watching the miniseries; Parts 1, 2, 3, and 4.)

As an American, directness and candor is part of my background; however… my first viewing of the miniseries gave me a life-long interest in Japan and Japanese culture.  In graduate school I took a year of Japanese, though the rule “if you don’t use it, you lose it” sadly applies.  (To those Japanese who compliment my ability, I always reply “Yappari Nihongo wa dekimasen.”*)  So I understood that a “typical” American approach of broadcasting this finding to all involved would cost several people at the Japan plant face.  I might solve the problem, but at the price of weakening my ability to interact with them in the future.  (An important disclaimer: While I have some familiarity with Japanese culture and language, I am in no way an expert; in this case I just happened to get lucky with the intersection of an interest and a work situation.)

So I went to my boss and proposed an alternative.  I would pass the information to our Technical Services person, who had a strong relationship with the plant manager.  Then, one-on-one, he would communicate to that plant manager what the issue with the prior trial of the components had been.  That plant manager could then “discover” the root cause of the previous attempt’s failure, and retry it using the correct procedure understanding how to do so based on his life-long background in Japan.

And this is what happened.  The problem was solved.  Did the “line” people know it was I who found the root cause of the problem?  Probably not – but that didn’t matter.  My boss – the most important person – knew.  The Technical Services person knew.  The plant manager knew.  And the components were tried again, this time successfully; this improvement being the most important thing, of course.

By having had an understanding the culture of Japan, I avoided an incident where I could have cost multiple people face, and thus damaged their reputations and ability to continue to do business.  I also avoided compromising my ability to interact with them in the future, as they would be exceedingly nervous about my costing them face again having experienced it once before.

Culture is important.  Language and details of it are also important.  When dealing with other countries and cultures, it’s critical to know things like this.  (I remember after Shogun was on TV; people I knew went around saying “Hai!” (yes) and “Wakarimasu ka?” (Do you understand?).  But as anyone who has dealt with the Japanese know, they’ll be in a business meeting repeatedly saying “Wakarimasu.” – I understand – but at the end say no.  Because understanding is not agreeing, it’s just a way to acknowledge what you said.  There is another word, if I recall correctly its “kashikomarimasu”; that’s “I understand and agree.”)

For a nice overview of some common blunders and things to avoid, across many nationalities, here’s a google search with a whole bunch of links showing up.  This one looks quite good.  And not only are there behavioral/cultural mores to consider, but gift-giving can also be tricky.

So until an asteroid hits, WW IV starts, or the world economy collapses we’re going to be doing business globally, and understanding how not to offend people of different cultures will be a critical part of that.  Even a casual attempt to learn about different cultures and mores, even in passing, will give people – whether working or seeking work – a step up in this global economy.

Take that step up.  Make sure to learn about different cultures on a continuous basis.  You never know when some tidbit might be useful.  And will put you a step up in your boss’ eyes, as your knowledge will make him look good.

* Loosely translated: “When everything is considered, I don’t speak Japanese.”

© 2014, David Hunt, PE

Fix the Problem IV: Water Ingestion into a Headlight

A note.  A couple of people have asked why I am not posting case studies from my six years at Cabot Corporation but, instead, am reaching back earlier in my career.  This is not because I have nothing to post; rather, because Cabot’s technology is highly proprietary and confidential, I am extremely constrained in what I can say.  (One report I wrote detailing a root-cause analysis of a field failure was, by necessity, so laced with the design details of a critical system component that people were ordered to only read it online on the company’s intranet and not even print it, lest a copy somehow fall into a competitor’s hands!  Needless to say, while a fascinating experience from which I’m sure people could learn, I don’t think I could write about it in a meaningful way without revealing things I can’t reveal because of an ongoing confidentiality agreement).  At some point in the future I will try to write up some things from there, but will need to be extremely careful in not disclosing anything proprietary.

When I joined Ford Motor Company in the picturesque city of Sandusky, Ohio, the best boss I’ve ever had told me to go out to the floor and “get lost”.  His idea was to have me become familiar with the plant and the product lines it manufactured: automotive exterior lighting, air cleaners, and carbon canisters for the fuel system.  Over time and several projects I became very familiar with leak testing and water ingress into lighting systems.  I became one of the few people outside the Quality department involved in examining the lights returned from dealers that had water in them.  (One of the recommendations that came out of that effort was passing back information to the dealers that if the water was clearly just condensate from air coming through the vent, cooling overnight, and precipitating dew inside the lamp, just turn it on and drive off the water with the bulb’s heat; nothing was wrong with the light itself.)

It was not long after the launch of the Ford Escort’s new look, which including a new headlight, that we started to get reports from the assembly plant where – once or twice a week – a headlight was filling with water during the post-assembly car wash.  So, logically, I got tapped with the problem.

First things first, I asked what we’d done with the returned items as we’d had at least a dozen of them.  These headlights had been dried out and leak-tested on our production line.  They passed the leak test.  We also had a set of testing protocols, done on every light as a part of the pre-launch design validation process, consisting of water spray testing while the light was on, while off, cycled on-off, humidity and dust testing, etc.  Every single light we’d gotten back was comprehensively run through these protocols again; they all passed.

The Design Engineer was stumped, as was the New Model Launch Engineer.  My immediate conclusion was that it was not the light – it was the light in the environment of the car.  I told the “troubleshooting manager at large”, a manager late in his career whose job it was to handle customer issues like this, that we needed to be called the next time it happened and drive up to take a look at the light in the vehicle.  We only had to wait a couple of days and we got the call; we drove up immediately.  Sure enough, standing water in the headlight.

We drove the car into a side spray booth used for additional spot washing, and I took up a hose.  I started spraying a jet of water around the perimeter of the light’s interface with the sheet metal around it.  When I got to one point underneath the integral side turn signal, I saw water coming in through the vent and into the lamp.  A little fine-tuning of the position and angle and I could reliably get a good stream of water coming in.  What was happening was that the water was ricocheting off the sheet metal underneath the light, and into the vent which was shaped like a piece of curved macaroni.  Since the opening of the vent faced forward, it acted like a scoop to catch the water.  It took me no more than ten minutes to find it, and once I knew how I could reliably do it on any Escort.

The first step, now that we knew how the water was coming in, was to block the flow of water from getting to the vent.  I originated the containment action: using a foam strip stuck to the light just in front of the vent, and which would be compressed against the sheet metal when installed and act as a dam.  We installed several such lights into a couple of different Escorts in rotation and I tried to spray water in – I couldn’t do it despite a good hour of trying.  This was a good containment action to patch the problem.

Our Design counterparts then designed molded-in ribs that created a housing around the vent; this design was mocked up on several more lights.  These ribs were intended to shield the vent from any flow of water impinging on the opening to get scooped in.  Again, I couldn’t duplicate the failure.  With the customer’s approval we changed the mold and did one more proof test series on real cars.  Once more, success: I couldn’t get water into the lights with the new molded ribs.

Lessons:

  1. When presented with a component failure where the component itself, even after the system failure, passes the test in stand-alone tests – look at the systemic environment.
  2. Only by testing a real failure with the failing component in situ can the failure mode truly be investigated.
  3. Refine or modify the testing protocols to include the surrounding environment/system when doing validation testing of new, similar components.
  4. Solicit other opinions early, especially on highly-visible problems.  I was only called in when the incidents reached double digits and the assembly plant manager was screaming at our Design team.
  5. Any failure mode discovered that can be designed out in the next product must be.  DFMEA is a great tool for this, as is creating an explicit and formal method for passing information from Manufacturing back upstream.  To wit, our specific recommendations to our Design group:
    1. Obtain or kluge a mock-up of the sheet metal and mounting environment of the light to use in the spray, moisture, and dust testing protocols.
    2. If using open-ended vent tubes, either have the opening pointed away from any possible water infringement, or proactively shield it to avoid this situation from occurring again.

© 2013, David Hunt, PE