The Infamous "We're Hosed" Incident

My first "real job" upon graduating college was, among other things, a lot of fun and more than a little educational. This was in 1997, which was probably one of the best years in history to have graduated with a 3.0 GPA and a BS degree in computer science. They were hiring programmers off the street back then! I'm not even kidding. I had an interview and accepted an offer to work as a contractor at General Motors before I even took my last final exam. This was the job that brought me from western New York to Michigan, and as far as life-shaping decisions go, this one was pretty big.

A Complete Tech 2 Kit

Back then, I was working on a vehicle diagnostic tool called the Tech 2. It was a small (for its time) hand-held embedded computing device that could connect to any GM vehicle's communications bus, read, analyze, and store diagnostic information in real time, as well as reprogram all of the on-board computers. We frequently tested the tool's ability to test and modify a given vehicle's fuel-to-air mixture, and detect when a sensor had gone bad (or we'd just disconnected its power), for examples.

My assignment was with the team that developed and maintained the core operating system for this tool. Our code was the first bit of code that ran in the system, bringing it from power-up to functional, and providing all the real-time scheduling and communications protocols for the rest of the more vehicle-specific stuff. It was pretty challenging stuff for a newbie fresh out of college, but I ran with it and enjoyed it a lot.

One of the challenges we faced was that we basically had a 10MB flash "hard drive" that we could only access 1MB at a time, and that flash drive was filling up fast.  Mind you, there was no file system, so we addressed the flash directly by address and used a card page register to pick which 1MB we wanted to see at any given time.  This made for some .. interesting .. code constructs, and made it necessary to copy certain  parts of the software into RAM at boot time - things like the OS's API layer, which was contained on the flash drive at an addressed specified in a look-up table that started at address 0.  You still with me?  Good.

I had written the code that would locate this API code and transfer it from the flash drive into it's designated spot in the RAM during the tool's boot-up sequence.  In the name of defensive coding, I had written a block of code to detect the absence of this file and react accordingly - by disabling all interrupts and entering an infinite loop after printing an error message in the center of the screen. My downfall was having the error message read, simply, "WE'RE HOSED."  I figured it was ok, though .. the API binary was an integral part of every release .. surely this error message would NEVER be displayed, right?  Right??

Wrong.  In fact, during the very next release cycle, the CDs that were shipped to every registered GM dealership in the WORLD were pressed without including that particular file.  As soon as a few dealers updated their Tech 2s with the new software, the calls started pouring in from around the world - Australia, South Africa, Sweden, and all over the USA, to name a few.  Everyone wanted to know what the heck does this "we're hosed" message mean, and why had their expensive diagnostic tool become nothing more than an expensive paperweight with a display proclaiming this ambiguous message to the entire world?

Needless to say, I was mortified. We quickly rushed a new release into production that not only replaced "we're hosed" with a more appropriate error message, but also made sure the API binary was included this time.  As far as I know, the more appropriate error message was never seen .. which is a good thing.

The moral of the story?  When you're coding error messages, never ever assume they won't be seen, because if you make that assumption even once, chances are that's the one time you'll be ... hosed.