Friday, November 01, 2013

'We're going to have congressional hearings about this project' - the Healthcare.gov problems trigger bad memories of a fucked up time

Lately the problems of the Healthcare.gov site have been in the news. It is pretty much a disaster. A very large software project apparently was rushed out the door without adequate testing, and it didn't work as well as hoped. Also, Autumn followed Summer this year and at night the sun went down with commendable reliability. At any rate, the uproar about it and the tense hearings remind me of my first job out of school, where one of my co-workers, a grizzled (although, in retrospect, probably in his mid-30s at the oldest) veteran assured me our doomed Bad News Bears debacle of a software project would most likely end with Congressional Hearings.

Most people's first jobs out of school don't come with the threat of ending up testifying before Congress if you screw up. In this case, I had somehow miraculously landed a job despite having majored in the least (at the time) marketable major possible, pure math. This was the 90s, so jobs were in abundance. Democrats said this was because of Clinton. Republicans credited a magical lag effect from Reagan. This magical lag effect was not cited in the case of George W. Bush. Anyhow, it was also true that this was a government job with the Navy that paid peanuts. The pay was so bad, in fact, that my Dad got pretty mad at me for accepting the job. But I was by this time pretty much under Bloomington's spell, and didn't want to leave, so I accepted the crappy job with a defense contractor (I didn't get the government employee benefits) and gave it a go.

Things were pretty messed up right out of the gate. The idea behind the project was to develop software for a 'minehunting sonar system' for the Navy, so at least there'd be no nuclear disasters if we screwed up. The defense department had long used Ada as their standard programming language, but here they were going to give C++ a go. So they hired somebody who just finished a Master's in Pure Math and had taken one Object Oriented programming course and paid him $20K a year, and mission accomplished as far as having a C++ guru on board. They put a guy in his early 30s who was a software developer who mainly just wanted to be liked in charge of his first ever project. The cast of characters included other science grad school sad-sack contractors like myself, aforementioned grizzled veteran, whose face would turn a cartoony shade of red when he railed against whatever he was railing at that day, a kind of cool hippie-ish dude who was the classic mad hacker and listened to Christian Metal bands, and a whole bunch of other government lifers.

For whatever reason, the 18 months I was there, we were somehow able to get by with only producing some really uninspiring design diagrams using 'Rational Rose', a couple hundred lines of C++ code, and some Perl scripts. We attended a couple of review meetings with Navy higher ups which mostly consisted of us battling with our arch-enemies, a base in Panama City that was duking it out with us for the project. The meetings would end with the Navy officials disgusted enough with both groups that neither one got to go home with the prize. So nobody got fired. Hooray!

Meanwhile the spectre of Raytheon, the contractor who had developed the original version of the software, loomed in the background. They were the scary, well-oiled private government contractor that today is still a champion of getting a hold of as much taxpayers' money as possible for their shareholders. They seemed a little less scary when you talked to the old timers I worked with, and they told you what an utter abortion Raytheon's early versions of the software were. Horrible as in if the software was still running a minute after it started, you knew it was going to be a great day. It sounded even more horrible to me in my naive 25-year-old youth, when I had no idea how widespread failure and dysfunction was in the software world.

At one point Raytheon called several of us who were working as contractors in for interviews. This made for an OK free trip to Rhode Island, which I can take or leave, and I purposely blew the interview by first pretending I didn't know the difference between a VAX (operating system) machine and a FAX machine, and then picking up an inertial unit somebody was clearly testing and moving and rotating it around. The place seemed like a miserable engine of soul destruction and everybody who wasn't grumpy was kind of robotic. By the end of the day I was ready to go Charles Bukowski if this was all the professional world had to offer. Anyhow, whatever info Raytheon was after, they didn't get it from me, but I suppose if they were trying to determine if the contractors currently working on the project were pretty fully mentally disengaged, they got their answer.

Oh yeah, also the HR glad-handing fake smile in a suit from Raytheon I spoke to at the end of the day said 'Ah yes. The 'Clemson Tide'' when he saw I had gone to Clemson. I want a name when I lose. Call me Pastor Blues.

Eventually I escaped the project for a job testing software for a less dysfunctional German medical device company (although there was drama and tension in abundance there too, I would find). Years later I heard from a former co-worker who'd kept in touch that the project was still going and they were still using our code (presumably with my comments like '/* update all this crap */' edited out). Nobody had to testify before Congress. But the project was a miserable episode in my life I was happy to leave behind, and I have all kinds of empathy and sympathy for the poor folks dealing with the Healthcare.gov site now.