Early on in the development of FieldKit, we did a lot of whiteboard exercises, the kind where a question would be posed to the team and we would then put our answers on sticky notes which were collected on a large board to provoke discussion. I very clearly recall the exercise about what “keeps us up at night,” the worst fears we each had with respect to the FieldKit project.
I wrote “Supply Chain Collapse.”
And here we are, years later, still negotiating the global chip shortage that began in 2020. As prescient as it might sound now, no precognitive talents were required to smell trouble coming. During that exercise, I was really just giving voice to something a lot of hardware people had been noticing for some time. We’d lived through the MLCC Capacitor Shortage in 2017 without too many major catastrophes. When supply of those came back, some were bold enough to make snarky t-shirts about it but this was gallows humor; we knew it wouldn’t be the last time.
Very often in technology development, you will start out amazed that something can work because you’re ignorant of the whole chain of engineering that lies beneath the surface. Over time, though, you will come to be even more amazed that something ever works (or goes on working), precisely because you do understand how long the chain of technology is underneath, and the fragility of the individual links in that chain. Such is the electronics supply chain. It’s an incredibly sophisticated globe-spanning miracle in motion, but it is not elastic.
The supply chain crisis in a nutshell
Semiconductor manufacturing requires space, fresh water, energy, specialized equipment, and skilled labor. All of these are expensive, so unless demand is growing fast, it’s uncommon to maintain a much more manufacturing capacity than is needed to fill projected demand. Manufacturers will instead keep a cushion of standing inventory to absorb temporary variations. A modest over-capacity and modest over-stock handily absorb modest variations in the market conditions. Then there’s a global pandemic.
The pandemic strikes first in parts of the world where lots of electronics are assembled, which drives down demand for parts. The market panics, and companies looking down the barrel of something between a recession and a depression get skittish and want to sit on their cash, so they reduce purchasing. Consumers follow suit, suppressing demand further. For the people running chip companies, orders were slashed in half (with no clear end in sight), and the difficulty and expense of maintaining production during the early pandemic surely felt like bleeding. The clear solution: stop or slow production and fill orders from your standing inventory.
As the economy began to restart, however, demand for chips didn’t just come back to normal. Companies needed to make up for lost time, so demand ramped up from half of typical to more than typical, to even more than could possibly be produced. Meanwhile, all the standing stock has been sold. Orders pile up which cannot be filled. In the first industries to be affected, assembly lines shut down for want of parts. This makes the news, and every middle manager reads there is a supply chain crisis. Not wanting to be caught up short, they give the only instruction they can think of: Buy up everything. Stockpile. This, of course, made everything worse.
If this sounds like the toilet paper shortage writ large, you’re not wrong, but unlike toilet paper, semiconductor fabrication facilities take years to construct and billions of dollars, and with this demand increase in chips being, most likely, temporary, it doesn’t make sense to build a new multi-billion-dollar factory to open in two or three years. So for the time being, we’ve had to adjust expectations for new computers, automobiles, consumer electronics, and, well, FieldKit stations.
It happened to us
To paraphrase Hemingway, the chip shortage hit FieldKit in two ways: gradually and then suddenly. Early in the crisis, we did better than many. I’m a cautious designer by nature, and there are many manufacturers whose parts I won’t even consider due to their histories of shortages, especially in the retail quantities that small players like us have access to. The first big shortages were in parts popular in the automotive industry, and we didn’t catch much of that. Then it was GPS modules, which we had already adjusted for once. We had no sooner finished redesigning our Weather cluster to accommodate our prior barometric pressure sensor being discontinued by the manufacturer when the designated replacement became completely unavailable virtually overnight. Regular lockdowns in China started making it difficult to get printed circuit boards. It was clear we were in for trouble, and we started doing some modest stockpiling of our own.
Once the “chip shortage” hit the press, it was all over. Tens of thousands of parts would sell between the time you put them in your cart and tried to check out. We escaped major redesigns through 2021 by drastically reducing our output. Some of the parts we ordered late last summer for first quarter 2023 production have only recently arrived. To be the “Hardware Guy” in an organization has meant to be the bad news guy in every meeting since the beginning of the pandemic.
If the effects of all this on production were the injury, the effects it had on attempts to develop anything new for FieldKit were definitely the added insult. R&D became a dark comedy sketch where you design a circuit board and by the time you received it, the parts to populate it were no longer available. Start over, buy the parts for your prototype and then order the circuit board, assemble the prototype and find that production quantities of the parts will not be available for a year or longer. Can you afford to buy production quantities of a part before you know if it does the job in the circuit? Maybe so, but do you bring a product to market with one production batch in hand and no estimate of when you can do another? These questions are becoming typical.
And for existing products, when does a delay in part availability become too long to tolerate, forcing you to redesign around some replacement part? How do you know the replacement part will not eventually suffer the same fate? We haven’t yet had to redesign any of our redesigns, but it will happen sooner or later.
We at FieldKit have had to develop many tools for taking the temperature of our stock availability situation, and every one of them indicates the situation is worse now than it has been yet. If I had to pick the single question I’ve been asked the most, it would be, “when do we think things will be back to normal?” This is tricky; what do we even mean by Normal?
Well, Normal in the Before Times (i.e. pre-pandemic) meant you could design around a part by a reputable manufacturer, still in production, and rest assured that you could almost certainly get your hands on hundreds or even thousands of them within 30 days of deciding you wanted them. That level of stock availability is easily three years away from being something you can stake a business on.
The environmental sensing technology we build at FieldKit is a tiny piece of the larger conservation technology landscape, and there are countless others like us who are doing our best to keep up in these interesting times. Many conservation projects have been delayed, heavily modified, canceled, or reduced in scope because the technology necessary for them has simply been unavailable. We’ve seen this happen with more than a few projects using FieldKit technology, and multiplying those delays and losses by all the other conservation technology efforts out there is, frankly, staggering. It is tough to think about all the amount of precious work that could have been but was not.
As conservation technologists, we can close some of these gaps by building designs with more resiliency, having more variations on a design than we might have considered normal a few years ago, and resigning ourselves to re-order our priority list around part availability, rather than what customers and partners might want more. On the hardware side, we’re working much harder, with greater risks, and a lot more waste, but we are working. Because the problems we are dealing with can’t wait for an end to the crisis; we have to push forward with what we have.