This was in my Draft folder...from 2008. Let's review:
EveryBlock - no longer exists, got swallowed up by another company.
Amazon - well known to be a phenomenally shitty workplace.
Metaweb - acquired by Google 2010
SAS - I don't really think about them much anymore
Lego - nice and all but I'm not obsessed with Lego. You gotta be obsessed with that shit to get a job there.
Things are suboptimal in the career arena right now. I'd rather not indulge in livejournal-esque expansion on that topic now. Instead, in a true power-of-positive thinking inspirational move, here are some places I think would actually be great places to work. Maybe I'm naive or deluded. Maybe I'm over-reaching. Regardless, here we go....
EveryBlock
A Chicago-based startup that makes raw, often unstructured data meaningful via mapping. Find out where all the crimes are going down. Find out which restaurants have good inspection scores. Cool things about EveryBlock:
Chicago
Python
Django (plus the guy that created Django)
Lots and lots of data manipulation, scraping, processing, munging, mangling, and so on
Amazon
Google gets all the 'changing the world' press, but Amazon put EC2, S3, and SimpleDB out there before Google App Engine came along. At a recent data warehousing conference, a big-shot in the industry sang the praises of the incredible Oracle DBAs that work at Amazon. They'd have to be incredible. It'd be a place where I could tolerate being the (relatively) dumb new guy in exchange for the incredible learning opportunities. Or so the theory goes.
The SAS Institue
Because of my math background, my whole life (it seems) people have told me: 'You know what you should do? You should work for the SAS Institute.' Actually it makes sense for other reasons as well. SAS always shows up in the 'best employers to work for' articles in the business magazines. It's located in North Carolina, and I love North Carolina - even more now that Jesse Helms left the building.
Metaweb (the creators of Freebase)
Wikipedia is cool, but Metaweb takes the idea a step further, giving the data more structure and creating the cool MQL (Metaweb Query Language). Here's an idea that's in the rich early stages - the possibilities are endless. That and the tweet from the MetaWeb dude saying 'look for me at the conference, I'll be in purple hair and a Freebase t-shirt'. It'd be great to have co-workers like that.
Lego
It's Lego! It's located in the alleged happiest place on Earth (although this article cooled my jets on that idea in a big way).
Saturday, August 24, 2024
You Can't Force STEM pt I
I was at Bloomington's Makevention Maker Convention (hence the name, think Maker Faire but in a more independent way) last weekend and I spent some time talking to Joey who collects old calculators, and modifies them or makes and programs new ones, usually using the MSP430 microcontroller from TI. He's a member of Bloominglabs so I've followed his work for some time, but it was only recently I got a chance to look at it in greater depth after it was featured on Hackaday, and then in some of the presentation materials he put together for his display.
As a person who's worked in the IT world for some years I have some sense of what constitutes a difficult problem and good code to solve such a problem, and I'm very impressed with what he's done. He doesn't study computer science though, he studies Eastern European languages and cultures. I asked him the obvious question: have you considered doing this professionally?
The answer is he has given it consideration, but doing it for work is not appealing at all. He has no interest in working on projects with narrow constraints and very little about them that he finds interesting. In high school he had done some programming projects for school helping out outside of the regular class day, and even just 2 hour stretches of being in an office by himself coding were unpleasant.
It's great to find these things out before saddling yourself with crazy student debt. Before you know it you're there in your cube having a panic attack while looking at a photo of your kid, which makes it even worse. David Foster Wallace memorably described this scenario in his posthumously published novel about IRS employees struggling with boredom, The Pale King.
There is a whole lot of talk about the importance of STEM education for kids (sometimes they add in art, and it's STEAM. But not Alyssa Milano's early 80s TEEN STEAM). Suggestions include teaching kids to code or setting up makerspaces where kids can learn 3D printing and such.
I recently heard of a project where kids are required to wear 'business casual' and work in a simulation of a software development project coordinated by some well-meaning but ham-fisted 'the children are our corporate future' types. Business casual? Really? Everybody knows programmers, especially the really good ones, get to have crazy hair and tattoos, and wear flip-flops to work if they want to. We want to put kids on the treadmill to death before they've even had fun college experiences? We want Dilbert to be relevant to another generation?
Additionally, it's bad enough hearing adults talk about 'full stack web developers' or 'rock stars that are crushing it' or regurgitating some mangled Agile Argot they've patched together from blog posts and talking to other people fumbling their way thru 'Scrum Mastery' during the work day. Coming home to get the same noise from the kids would be pretty soul-deflating, so no thanks.
This is one area where Minecraft has the advantage. It appears to be a shared generational experience on the order of Woodstock, from what I can tell, only with billions of people instead of a couple hundred thousand tripping in a muddy field. Minecraft was made with, from what I can tell, exactly zero interest in promoting STEM education, or any kind of education, for young people. Most parents are probably ambivalent at best about letting kids on there, although at this point parents who don't let their kids play Minecraft may as well be keeping them in a bunker with a year's worth of canned goods as far as isolating them from their peers goes.
As a person who's worked in the IT world for some years I have some sense of what constitutes a difficult problem and good code to solve such a problem, and I'm very impressed with what he's done. He doesn't study computer science though, he studies Eastern European languages and cultures. I asked him the obvious question: have you considered doing this professionally?
The answer is he has given it consideration, but doing it for work is not appealing at all. He has no interest in working on projects with narrow constraints and very little about them that he finds interesting. In high school he had done some programming projects for school helping out outside of the regular class day, and even just 2 hour stretches of being in an office by himself coding were unpleasant.
It's great to find these things out before saddling yourself with crazy student debt. Before you know it you're there in your cube having a panic attack while looking at a photo of your kid, which makes it even worse. David Foster Wallace memorably described this scenario in his posthumously published novel about IRS employees struggling with boredom, The Pale King.
There is a whole lot of talk about the importance of STEM education for kids (sometimes they add in art, and it's STEAM. But not Alyssa Milano's early 80s TEEN STEAM). Suggestions include teaching kids to code or setting up makerspaces where kids can learn 3D printing and such.
I recently heard of a project where kids are required to wear 'business casual' and work in a simulation of a software development project coordinated by some well-meaning but ham-fisted 'the children are our corporate future' types. Business casual? Really? Everybody knows programmers, especially the really good ones, get to have crazy hair and tattoos, and wear flip-flops to work if they want to. We want to put kids on the treadmill to death before they've even had fun college experiences? We want Dilbert to be relevant to another generation?
Additionally, it's bad enough hearing adults talk about 'full stack web developers' or 'rock stars that are crushing it' or regurgitating some mangled Agile Argot they've patched together from blog posts and talking to other people fumbling their way thru 'Scrum Mastery' during the work day. Coming home to get the same noise from the kids would be pretty soul-deflating, so no thanks.
This is one area where Minecraft has the advantage. It appears to be a shared generational experience on the order of Woodstock, from what I can tell, only with billions of people instead of a couple hundred thousand tripping in a muddy field. Minecraft was made with, from what I can tell, exactly zero interest in promoting STEM education, or any kind of education, for young people. Most parents are probably ambivalent at best about letting kids on there, although at this point parents who don't let their kids play Minecraft may as well be keeping them in a bunker with a year's worth of canned goods as far as isolating them from their peers goes.
Thursday, January 16, 2014
SSIS: 'to find this programmatically needs some circus'
In 2005 Microsoft introduced SSIS, which is a nice friendly GUI interface for defining dataflows between systems. It replaced DTS, which was also a GUI interface, albeit quite a limited one that was a pain to get to do simple things (secure copy instead of FTP? GTFO.)
Like a lot of friendly GUI things, it's nice for the first 20 minutes you use it until the cracks start to show. Still, when dealing with moving data between multiple servers, it's fast to develop something and it performs well. HOWEVER.
Something I have hated about it from day one is this: when a row errors out, you can redirect it. You will be provided with a cryptic error code (you can Google it, no worries), a brief description, and a 'Column ID'. Not the name, the ID.
Here's the fun part. That ID is buried deep in the GUI. click-click-click. If you are a programmer type who likes to be able to get your info quick-like via a search of a quick command line command, well, go fuck yourself.
The ID USED to be searchable if you switched from the pretty picture of your flow to the underlying XML. XML is never fun to look at, but if you can get vital info from it, it's like how a dog can get away with eating its own shit because it has a nice personality.
So now it's 2014. If my calculations are correct, 2005 was 9 years ago. Today, though, it still only shows the numeric ID, and YOU CAN'T GET IT IN THE XML.
Even Google is little help. Sure, you can do it, but you have to go through a ludicrous number of hoops to get a goddamn descriptive piece of text a human might like to read. As this guy puts it, 'to find this programmatically needs some circus'.
http://www.codeproject.com/Questions/440653/how-to-Get-ErrorColumn-Name-in-SSIS
Thanks, Microsoft. There's just no reason for this.
Like a lot of friendly GUI things, it's nice for the first 20 minutes you use it until the cracks start to show. Still, when dealing with moving data between multiple servers, it's fast to develop something and it performs well. HOWEVER.
Something I have hated about it from day one is this: when a row errors out, you can redirect it. You will be provided with a cryptic error code (you can Google it, no worries), a brief description, and a 'Column ID'. Not the name, the ID.
Here's the fun part. That ID is buried deep in the GUI. click-click-click. If you are a programmer type who likes to be able to get your info quick-like via a search of a quick command line command, well, go fuck yourself.
The ID USED to be searchable if you switched from the pretty picture of your flow to the underlying XML. XML is never fun to look at, but if you can get vital info from it, it's like how a dog can get away with eating its own shit because it has a nice personality.
So now it's 2014. If my calculations are correct, 2005 was 9 years ago. Today, though, it still only shows the numeric ID, and YOU CAN'T GET IT IN THE XML.
Even Google is little help. Sure, you can do it, but you have to go through a ludicrous number of hoops to get a goddamn descriptive piece of text a human might like to read. As this guy puts it, 'to find this programmatically needs some circus'.
http://www.codeproject.com/Questions/440653/how-to-Get-ErrorColumn-Name-in-SSIS
Thanks, Microsoft. There's just no reason for this.
Friday, November 01, 2013
'We're going to have congressional hearings about this project' - the Healthcare.gov problems trigger bad memories of a fucked up time
Lately the problems of the Healthcare.gov site have been in the news. It is pretty much a disaster. A very large software project apparently was rushed out the door without adequate testing, and it didn't work as well as hoped. Also, Autumn followed Summer this year and at night the sun went down with commendable reliability. At any rate, the uproar about it and the tense hearings remind me of my first job out of school, where one of my co-workers, a grizzled (although, in retrospect, probably in his mid-30s at the oldest) veteran assured me our doomed Bad News Bears debacle of a software project would most likely end with Congressional Hearings.
Most people's first jobs out of school don't come with the threat of ending up testifying before Congress if you screw up. In this case, I had somehow miraculously landed a job despite having majored in the least (at the time) marketable major possible, pure math. This was the 90s, so jobs were in abundance. Democrats said this was because of Clinton. Republicans credited a magical lag effect from Reagan. This magical lag effect was not cited in the case of George W. Bush. Anyhow, it was also true that this was a government job with the Navy that paid peanuts. The pay was so bad, in fact, that my Dad got pretty mad at me for accepting the job. But I was by this time pretty much under Bloomington's spell, and didn't want to leave, so I accepted the crappy job with a defense contractor (I didn't get the government employee benefits) and gave it a go.
Things were pretty messed up right out of the gate. The idea behind the project was to develop software for a 'minehunting sonar system' for the Navy, so at least there'd be no nuclear disasters if we screwed up. The defense department had long used Ada as their standard programming language, but here they were going to give C++ a go. So they hired somebody who just finished a Master's in Pure Math and had taken one Object Oriented programming course and paid him $20K a year, and mission accomplished as far as having a C++ guru on board. They put a guy in his early 30s who was a software developer who mainly just wanted to be liked in charge of his first ever project. The cast of characters included other science grad school sad-sack contractors like myself, aforementioned grizzled veteran, whose face would turn a cartoony shade of red when he railed against whatever he was railing at that day, a kind of cool hippie-ish dude who was the classic mad hacker and listened to Christian Metal bands, and a whole bunch of other government lifers.
For whatever reason, the 18 months I was there, we were somehow able to get by with only producing some really uninspiring design diagrams using 'Rational Rose', a couple hundred lines of C++ code, and some Perl scripts. We attended a couple of review meetings with Navy higher ups which mostly consisted of us battling with our arch-enemies, a base in Panama City that was duking it out with us for the project. The meetings would end with the Navy officials disgusted enough with both groups that neither one got to go home with the prize. So nobody got fired. Hooray!
Meanwhile the spectre of Raytheon, the contractor who had developed the original version of the software, loomed in the background. They were the scary, well-oiled private government contractor that today is still a champion of getting a hold of as much taxpayers' money as possible for their shareholders. They seemed a little less scary when you talked to the old timers I worked with, and they told you what an utter abortion Raytheon's early versions of the software were. Horrible as in if the software was still running a minute after it started, you knew it was going to be a great day. It sounded even more horrible to me in my naive 25-year-old youth, when I had no idea how widespread failure and dysfunction was in the software world.
At one point Raytheon called several of us who were working as contractors in for interviews. This made for an OK free trip to Rhode Island, which I can take or leave, and I purposely blew the interview by first pretending I didn't know the difference between a VAX (operating system) machine and a FAX machine, and then picking up an inertial unit somebody was clearly testing and moving and rotating it around. The place seemed like a miserable engine of soul destruction and everybody who wasn't grumpy was kind of robotic. By the end of the day I was ready to go Charles Bukowski if this was all the professional world had to offer. Anyhow, whatever info Raytheon was after, they didn't get it from me, but I suppose if they were trying to determine if the contractors currently working on the project were pretty fully mentally disengaged, they got their answer.
Oh yeah, also the HR glad-handing fake smile in a suit from Raytheon I spoke to at the end of the day said 'Ah yes. The 'Clemson Tide'' when he saw I had gone to Clemson. I want a name when I lose. Call me Pastor Blues.
Eventually I escaped the project for a job testing software for a less dysfunctional German medical device company (although there was drama and tension in abundance there too, I would find). Years later I heard from a former co-worker who'd kept in touch that the project was still going and they were still using our code (presumably with my comments like '/* update all this crap */' edited out). Nobody had to testify before Congress. But the project was a miserable episode in my life I was happy to leave behind, and I have all kinds of empathy and sympathy for the poor folks dealing with the Healthcare.gov site now.
Most people's first jobs out of school don't come with the threat of ending up testifying before Congress if you screw up. In this case, I had somehow miraculously landed a job despite having majored in the least (at the time) marketable major possible, pure math. This was the 90s, so jobs were in abundance. Democrats said this was because of Clinton. Republicans credited a magical lag effect from Reagan. This magical lag effect was not cited in the case of George W. Bush. Anyhow, it was also true that this was a government job with the Navy that paid peanuts. The pay was so bad, in fact, that my Dad got pretty mad at me for accepting the job. But I was by this time pretty much under Bloomington's spell, and didn't want to leave, so I accepted the crappy job with a defense contractor (I didn't get the government employee benefits) and gave it a go.
Things were pretty messed up right out of the gate. The idea behind the project was to develop software for a 'minehunting sonar system' for the Navy, so at least there'd be no nuclear disasters if we screwed up. The defense department had long used Ada as their standard programming language, but here they were going to give C++ a go. So they hired somebody who just finished a Master's in Pure Math and had taken one Object Oriented programming course and paid him $20K a year, and mission accomplished as far as having a C++ guru on board. They put a guy in his early 30s who was a software developer who mainly just wanted to be liked in charge of his first ever project. The cast of characters included other science grad school sad-sack contractors like myself, aforementioned grizzled veteran, whose face would turn a cartoony shade of red when he railed against whatever he was railing at that day, a kind of cool hippie-ish dude who was the classic mad hacker and listened to Christian Metal bands, and a whole bunch of other government lifers.
For whatever reason, the 18 months I was there, we were somehow able to get by with only producing some really uninspiring design diagrams using 'Rational Rose', a couple hundred lines of C++ code, and some Perl scripts. We attended a couple of review meetings with Navy higher ups which mostly consisted of us battling with our arch-enemies, a base in Panama City that was duking it out with us for the project. The meetings would end with the Navy officials disgusted enough with both groups that neither one got to go home with the prize. So nobody got fired. Hooray!
Meanwhile the spectre of Raytheon, the contractor who had developed the original version of the software, loomed in the background. They were the scary, well-oiled private government contractor that today is still a champion of getting a hold of as much taxpayers' money as possible for their shareholders. They seemed a little less scary when you talked to the old timers I worked with, and they told you what an utter abortion Raytheon's early versions of the software were. Horrible as in if the software was still running a minute after it started, you knew it was going to be a great day. It sounded even more horrible to me in my naive 25-year-old youth, when I had no idea how widespread failure and dysfunction was in the software world.
At one point Raytheon called several of us who were working as contractors in for interviews. This made for an OK free trip to Rhode Island, which I can take or leave, and I purposely blew the interview by first pretending I didn't know the difference between a VAX (operating system) machine and a FAX machine, and then picking up an inertial unit somebody was clearly testing and moving and rotating it around. The place seemed like a miserable engine of soul destruction and everybody who wasn't grumpy was kind of robotic. By the end of the day I was ready to go Charles Bukowski if this was all the professional world had to offer. Anyhow, whatever info Raytheon was after, they didn't get it from me, but I suppose if they were trying to determine if the contractors currently working on the project were pretty fully mentally disengaged, they got their answer.
Oh yeah, also the HR glad-handing fake smile in a suit from Raytheon I spoke to at the end of the day said 'Ah yes. The 'Clemson Tide'' when he saw I had gone to Clemson. I want a name when I lose. Call me Pastor Blues.
Eventually I escaped the project for a job testing software for a less dysfunctional German medical device company (although there was drama and tension in abundance there too, I would find). Years later I heard from a former co-worker who'd kept in touch that the project was still going and they were still using our code (presumably with my comments like '/* update all this crap */' edited out). Nobody had to testify before Congress. But the project was a miserable episode in my life I was happy to leave behind, and I have all kinds of empathy and sympathy for the poor folks dealing with the Healthcare.gov site now.
Monday, July 08, 2013
Free as in Beer: My Recent MOOC experience
I last sat in a conventional classroom in December of 1994. I was taking a Topology Final as a Ph.D. student (Mathematics, if you don't recognize 'Topology') at IU. I had already decided to quit the program and go find a job outside of the academic world, so why I was there is anybody's guess. I think I worried that an especially crappy grade would haunt me for the rest of my life.
So, for nearly 20 years, I've been educated either through my own initiative or via training courses for work. I've attended the occasional workshop, like a recent one at Bloominglabs about using a lathe and a mill. I've been completely outside of the conventional college class and tests world. Eventually I stopped having bad dreams about being late for a Final Exam, not knowing exactly where it was being held, and not having studied or even attended classes. These were replaced by anxiety dreams about showing up for a race late once I started running in races.
2 months ago I decided to try out a MOOC, specifically, 'Intro to Data Science' on Coursera, taught by professor Bill Howe of the University of Washington. As somebody with a (very distant) math past who works with databases all day, learning about technologies I don't get to use during the workday like MapReduce, Hadoop, and the machine learning toolkit for Python, scikit-learn was appealing. It was also a chance to venture into a classroom setting without paying big money like some friends who've decided to go for MBAs (personally, I have zero interest in pursuing an MBA, but I did admire their dedication and devotion to keeping up with the challenge).
Even though the class was free, and I was one of 70,000 people who signed up, the idea of not doing well for whatever reason did provoke some anxiety. For one assignment, students submitted code which was then run by the auto-grader, nicknamed 'Darth Grader' by students on the forum. It suffered under the load, and there was often a long wait before getting results like 'you didn't calculate a value for @JonasBrothers' (because I removed punctuation including the very meaningful '@' symbol from Twitter data). One night I had a new anxiety dream for the MOOC era, where I refreshed my browser to find my scores had all been accidentally converted to zero by the autograder.
One Sunday I realized in the middle of the day that a quiz about MapReduce was due that afternoon. This stress was compounded by the fact that the service we were supposed to use, JSMapReduce, was suffering under the load much as Darth Grader had a few weeks previously. The discussion forums were life savers, and some suggested just running the job on your own machine using a Python library that simulated MapReduce (I say simulated because everything was being run in one process, pretty much missing the whole point of MapReduce, which is to split a load over a crudload of servers).
In general I found the forums to be the most worthwhile and surprisingly beneficial part of the experience. I am as skeptical as anyone of crowdsourcing, and you'd expect with 70,000 students signed up for a course, the forums would be chock full of noise and cluelessness. This was not really the case at all. People shared knowledge and experience (but, for the most part, followed the rules and did not share code). This helped a bunch with the optional AWS project (run a MapReduce job to crunch a TB of data). I found out that there's a $100 grant available to students, so I didn't have to pay for the services out of my own pocket (keeping the course truly free), and in a thread people compared notes about how many nodes they used and how to tweak settings when setting up your job. There were also helpful discussions about setting up and running Pig (a high level query language for Hadoop MapReduce jobs, sort of like SQL, but only sort of) on your own machine, so I was able to debug my pig scripts locally without having to pay for time on Amazon Elastic MapReduce (on AWS). (In the end, I racked up only $8 worth of charges against my $100 credit - we were warned it could cost up to $20).
Some critics say the forums are no match for the rapid fire face to face discussions you can have at a University. That is, if you're not as introverted as I was in my University days. I was lucky enough to have some accessible professors, although in retrospect what that offered was a mixed bag. When people get jazzed about the fact that you're hanging on their every word, they can veer off into weird political or racist directions. They can give you really horrible advice, like the advisor who told me not to take a graph theory class. I will probably expound on this in a future post.
I did enjoy the assignments, although several other students hated the open-ended requirements in some cases (for example: 'participate in a kaggle competition'). I thought the openness was kind of fitting, given the subject matter. Data Scientists have to figure out what the data is telling them without a set of hard fast consultant-friendly requirements.
The Kaggle assignment was fun and humbling. There is a tutorial-like competition on Kaggle (a website where Data Scientists and wannabe Data Scientists compete for money and glory solving problems in scientific or business domains) about 'Predict Based On These Variables If A Person Survived the Sinking Of The Titanic', which walks a person through examples with Excel, Python, and scikit-learn (also Python - it's a Machine Learning toolkit). Why I say it was humbling is that a kind of hokey and hackey Python example in the tutorial did a better job making predictions than the more impressive sounding 'Random Forest', unless you the competitor applied a whole lot of what's called 'Feature Engineering' to the problem to figure out how to deal with missing data and to identify how best to use the info provided. The assignment was due too soon - I would have liked to have dug into that more. As it was it was something of a 'here's a firehose of new tools, good luck!' experience. The Chief Data Scientist at Kaggle, Jeremy P. Howard, made an appearance in an 'Ask Me Anything' in the forums, and this thread was as valuable as any lecture.
The course ended a couple of weeks ago. I completed all the required assignments, but I still don't know if I got a certificate of completion yet. I'm not sure what I will do with such a certificate, but as I've never been a Mayor Of Starbucks on 4square I would like to have some sort of virtual recognition or mark of greatness. The ending did feel a bit like a fizzling out, really.
Since the course finished I've resumed tinkering on some Bloominglabs related things, like my open-ended Arduino based bike computer and playing with our new(ish) laser cutter some more. I find that since I started doing that stuff, a purely academic exercise like completing an assignment is not as rewarding, because ultimately these often feel like meals you clear away when you're done, and move on. On the other hand, w/out more structure, peer pressure, and etc, there is a tendency for my hacking projects to be very open-ended and not really result in some point of completion (I tend to be best at 'completing things' if there's an upcoming opportunity to show the thing off). It's a trade off.
Ultimately I give Coursera a passing grade because I'm going back for more. I'm taking 'Maps And the Geospatial Revolution', starting July 17. Join me if you want to make some cool maps.
Some handy references:
http://www.katyjordan.com/MOOCproject.html
http://www.educause.edu/ero/article/retention-and-intention-massive-open-online-courses-depth-0
http://www.nytimes.com/2013/04/21/opinion/sunday/grading-the-mooc-university.html?pagewanted=all&_r=1&
So, for nearly 20 years, I've been educated either through my own initiative or via training courses for work. I've attended the occasional workshop, like a recent one at Bloominglabs about using a lathe and a mill. I've been completely outside of the conventional college class and tests world. Eventually I stopped having bad dreams about being late for a Final Exam, not knowing exactly where it was being held, and not having studied or even attended classes. These were replaced by anxiety dreams about showing up for a race late once I started running in races.
2 months ago I decided to try out a MOOC, specifically, 'Intro to Data Science' on Coursera, taught by professor Bill Howe of the University of Washington. As somebody with a (very distant) math past who works with databases all day, learning about technologies I don't get to use during the workday like MapReduce, Hadoop, and the machine learning toolkit for Python, scikit-learn was appealing. It was also a chance to venture into a classroom setting without paying big money like some friends who've decided to go for MBAs (personally, I have zero interest in pursuing an MBA, but I did admire their dedication and devotion to keeping up with the challenge).
Even though the class was free, and I was one of 70,000 people who signed up, the idea of not doing well for whatever reason did provoke some anxiety. For one assignment, students submitted code which was then run by the auto-grader, nicknamed 'Darth Grader' by students on the forum. It suffered under the load, and there was often a long wait before getting results like 'you didn't calculate a value for @JonasBrothers' (because I removed punctuation including the very meaningful '@' symbol from Twitter data). One night I had a new anxiety dream for the MOOC era, where I refreshed my browser to find my scores had all been accidentally converted to zero by the autograder.
One Sunday I realized in the middle of the day that a quiz about MapReduce was due that afternoon. This stress was compounded by the fact that the service we were supposed to use, JSMapReduce, was suffering under the load much as Darth Grader had a few weeks previously. The discussion forums were life savers, and some suggested just running the job on your own machine using a Python library that simulated MapReduce (I say simulated because everything was being run in one process, pretty much missing the whole point of MapReduce, which is to split a load over a crudload of servers).
In general I found the forums to be the most worthwhile and surprisingly beneficial part of the experience. I am as skeptical as anyone of crowdsourcing, and you'd expect with 70,000 students signed up for a course, the forums would be chock full of noise and cluelessness. This was not really the case at all. People shared knowledge and experience (but, for the most part, followed the rules and did not share code). This helped a bunch with the optional AWS project (run a MapReduce job to crunch a TB of data). I found out that there's a $100 grant available to students, so I didn't have to pay for the services out of my own pocket (keeping the course truly free), and in a thread people compared notes about how many nodes they used and how to tweak settings when setting up your job. There were also helpful discussions about setting up and running Pig (a high level query language for Hadoop MapReduce jobs, sort of like SQL, but only sort of) on your own machine, so I was able to debug my pig scripts locally without having to pay for time on Amazon Elastic MapReduce (on AWS). (In the end, I racked up only $8 worth of charges against my $100 credit - we were warned it could cost up to $20).
Some critics say the forums are no match for the rapid fire face to face discussions you can have at a University. That is, if you're not as introverted as I was in my University days. I was lucky enough to have some accessible professors, although in retrospect what that offered was a mixed bag. When people get jazzed about the fact that you're hanging on their every word, they can veer off into weird political or racist directions. They can give you really horrible advice, like the advisor who told me not to take a graph theory class. I will probably expound on this in a future post.
I did enjoy the assignments, although several other students hated the open-ended requirements in some cases (for example: 'participate in a kaggle competition'). I thought the openness was kind of fitting, given the subject matter. Data Scientists have to figure out what the data is telling them without a set of hard fast consultant-friendly requirements.
The Kaggle assignment was fun and humbling. There is a tutorial-like competition on Kaggle (a website where Data Scientists and wannabe Data Scientists compete for money and glory solving problems in scientific or business domains) about 'Predict Based On These Variables If A Person Survived the Sinking Of The Titanic', which walks a person through examples with Excel, Python, and scikit-learn (also Python - it's a Machine Learning toolkit). Why I say it was humbling is that a kind of hokey and hackey Python example in the tutorial did a better job making predictions than the more impressive sounding 'Random Forest', unless you the competitor applied a whole lot of what's called 'Feature Engineering' to the problem to figure out how to deal with missing data and to identify how best to use the info provided. The assignment was due too soon - I would have liked to have dug into that more. As it was it was something of a 'here's a firehose of new tools, good luck!' experience. The Chief Data Scientist at Kaggle, Jeremy P. Howard, made an appearance in an 'Ask Me Anything' in the forums, and this thread was as valuable as any lecture.
The course ended a couple of weeks ago. I completed all the required assignments, but I still don't know if I got a certificate of completion yet. I'm not sure what I will do with such a certificate, but as I've never been a Mayor Of Starbucks on 4square I would like to have some sort of virtual recognition or mark of greatness. The ending did feel a bit like a fizzling out, really.
Since the course finished I've resumed tinkering on some Bloominglabs related things, like my open-ended Arduino based bike computer and playing with our new(ish) laser cutter some more. I find that since I started doing that stuff, a purely academic exercise like completing an assignment is not as rewarding, because ultimately these often feel like meals you clear away when you're done, and move on. On the other hand, w/out more structure, peer pressure, and etc, there is a tendency for my hacking projects to be very open-ended and not really result in some point of completion (I tend to be best at 'completing things' if there's an upcoming opportunity to show the thing off). It's a trade off.
Ultimately I give Coursera a passing grade because I'm going back for more. I'm taking 'Maps And the Geospatial Revolution', starting July 17. Join me if you want to make some cool maps.
Some handy references:
http://www.katyjordan.com/MOOCproject.html
http://www.educause.edu/ero/article/retention-and-intention-massive-open-online-courses-depth-0
http://www.nytimes.com/2013/04/21/opinion/sunday/grading-the-mooc-university.html?pagewanted=all&_r=1&
Monday, March 18, 2013
Keeping Louisville Retro: Louisville Arcade Expo 2013
The Louisville Arcade Expo has been happening since at least 2011. Last year it was on my radar, but I didn't get it together to go, and I felt much regret after missing it. We recently set up a MAME cabinet at Bloominglabs, so I've been on a classic video games kick lately and this year was not going to miss it. I rounded up a couple other people to make the trip from Bloomington to Louisville to check it out. The day was a magical event that exceeded all expectations, with only a couple of negatives that we'll talk about later.
It was held in the Ramada Plaza, well outside of downtown Louisville in a sort of non-descript industrial zone. Of course, we were planning to spend the day in a retro 80s-90s bubble, so the setting didn't matter too much. There was a lot of floor space featuring 100s of pinball games and 100s of classic arcade games (I may be exaggerating a bit here. My point is, the space was huge.) There were also competitions with the action projected on the wall, an '80s Living Room', a vendor area, a food area for people wanting a $7 rum and coke or $3 hot dog, and finally my favorite spot, the classic computers and consoles room.
This room was laid out in chronological order, starting with the classic Atari 2600 (yes, there were games before then, but here we started with the 2600). That was my first home video game, one I spent many, many hours playing. I've since revisited the games in emulated form, both on PCs and in the form of a cheap 'Jaxx Pacific' joystick I have featuring some old games, but this was the first time in 25+ years that I picked up the cartridges and popped them in, which is a different feeling. I played Space Invaders, Choplifter, and Missile Command, opting to steer clear of the horrible, terrible 2600 port of Pac Man (although I saw quite a few people playing this over the course of the day. It was as horrible as I remembered!) One minor problem was that one of the joysticks was sticky, so I swapped it out for the other one, and then marveled that people would subject their old video games and arcade cabinets to a weekend of wear and tear (thank you, people who did).
 |
| The Atari 2600 and some cartridges |
 |
| Virtual Boy, played by Real Boy |
The aforementioned Vectrex was a totally different story. Some of my favorite classic games (Tempest and Asteroids for example) use vector graphics displays, as opposed to the raster-scan approach that other games (and TVs) use. With Vector graphics, rather than drawing pictures line-by-line, the lines in the picture are drawn with the beam. The resulting look is very bright and sharp, 'jaggy' free, and emulators totally fail to convey via software what this kind of hardware looks like. Vector displays fell by the wayside, so you can't really go out and buy a new one, but fortunately people keep the old arcade cabinets and the Vectrex consoles running, and I may have to get one from eBay sometime soon.
 |
| Vector Pilot on the Vectrex. The colors are from a transparent overlay. |
 |
| This captures the intensity of the display better. |
 |
| It's dark. Arcades are dark. |
 |
| What would have been a fleeting moment of glory in the 80s, now captured for as long as blogspot exists. |
 |
| Not endorsed by 'Grizzly People' |
A guy who has been on my radar somewhat was there, but his chair was empty. Sonny Rae Tempest had his Atari 2600 game poem 'Nothing is 0k' on display. Honestly this one required a bit of deciphering, and my eyes weren't really up to making out the binary codes on the screen. I am a fan of his 'stuck in traffic' Atari poem Calm, Mute, Moving, but as the controller is a simulated cigarette, it probably didn't lend itself well to sharing it with hundreds of people and their germs.
Food-wise, things were not so great. 'Wick's Pizza' is attached to the hotel, so we all thought, hey, great, let's eat there. However, two of the people I came with waited there 20 minutes to be served before bailing out to go to Dairy Queen. Later, I went at around 2pm, thinking I was being clever and wouldn't have to wait because of a lunch rush, but a horde of bikers showed up right before me. They were benevolent Harley-Davidson bikers, who have a median age of my age + 15, but still, it was annoying to have to wait a long time for a passable stromboli. Our other friend tried to go there and arrived just as a fight was breaking out, with two guys in ties (at the opposite end of the dress code spectrum from our benevolent bikers) pushing each other saying threatening things, and he was told by the manager that he couldn't serve him as he had to deal with the situation. He ended up making due with a hot dog, because by that time the Dairy Queen's grill wasn't working.
I had one of the great $7 Rum and Cokes later in the day, and while it was pretty strong, it was still $7, and of course I had to tip the server. I'm not an animal.
The crowd was all-ages, and not everybody was as nerdy as myself. People did tend to gravitate toward the games of their youth, which was not surprising, but I did see some kids playing the ancient games and having fun. It certainly appeared to be a huge success. I will be coming back to this one, and I recommend it to anybody in the area with an interest in pinball or arcade games. It was great fun.
 |
| Mario Brothers celebrate a successful Arcade Expo |
Sunday, December 16, 2012
Harry Potter and the Outdated Piece Of Shit Technology With Management Support
We were having this discussion Friday, which is kind of sad. There is a politically favored technology in house that sucks ass, and we feel a great deal of motivation and satisfaction in demonstrating how our little team can blow it and the army of devs working with it out of the water using SQL Server and a bit of knowledge and expertise. For example, some doofus will spend 3 days trying to extract the answer to a question from this sorry Paleolithic system with its BASIC code, and then out of desperation they turn to us, and ta-da, answer in seconds.
However this is always followed by the profoundly demotivational realization that basically the political situation is here to stay. We more or less get told to go back into our boxes. It is kind of a shame and I wonder how many places there are around the world where this is how it goes. Generally my way of dealing with it is trying to keep building my own knowledge and expertise so as not to get left behind or complacent, and our little team keeps trying to do good things, maybe just to amuse each other at this point. But like I say, it's quite demoralizing sometimes.
However this is always followed by the profoundly demotivational realization that basically the political situation is here to stay. We more or less get told to go back into our boxes. It is kind of a shame and I wonder how many places there are around the world where this is how it goes. Generally my way of dealing with it is trying to keep building my own knowledge and expertise so as not to get left behind or complacent, and our little team keeps trying to do good things, maybe just to amuse each other at this point. But like I say, it's quite demoralizing sometimes.
Thursday, December 06, 2012
Ground Kontrol Arcade, a Great Place for Video Games and a Bad Place for Comedy
Occasionally I get to travel for my job. I like that it's occasional, but on the other hand I would hate to never get a chance to travel. I've had jobs where I went nowhere both figuratively and literally. In this case, we're upgrading to SQL Server 2012 (from 2005), so some training was in order. Fortunately one of the cities where training was taking place was Portland, OR, a place I've wanted to visit ever since I learned that the Dream Of The 90s Is Alive In Portland via Portlandia, a show I have a strange relationship with. What I mean by that is though it is a comedy and funny and it's supposed to be making fun of Portland and its residents and what absurd liberal hippies they are, in my case it makes Portland look like a really great place to live, possibly like Bloomington, but bigger. Everything that's supposed to be one of Portland's foibles is actually endearing to me. Even the dumpster divers are kind of charming. So I had to check it out now that I had the chance.
I knew I had to see Powell's, the multi-level bookstore the size of a city block (plus a science and technology building across the street), and I did. I went there twice, even, and may post about it more later. A friend of mine, Bloominglabs' Jenett T., told me about a place called Ground Kontrol Arcade. It turns out Ground Kontrol is an 80s style arcade, full of 80s games, mostly classic 79-84 era games. There are mostly pinball games upstairs, but I'm not too into that.
For the most part I think the 80s really sucked: Cold War paranoia, that doddering old fool Reagan, really bad drum sounds and dated production, and I had really out-of-control acne. I did love the video games, though, so when I found out about this place, I had to go. I had some fool idea that I'd hang out for just 2 hours or so and have dinner after, like 2 hours would be enough at what would probably be my one and only chance to re-live the glory of Flynn's Arcade in the movie Tron.
I got there about 7, which was cool because that's when they switched to being 21 and over, and yes they serve beer and a couple goofily named arcade themed cocktails (e.g., Vodka Tronic). I had a local beer (Black Butte Porter) and made change. Having a pocket full of quarters when you are over 40 ordinarily would be a damned weighing down your pants in a lopsided way nuisance, but this night it felt great. I started out with Dig Dug and made the rounds, spending more time with Frogger, Tempest, and Asteroids than with the other games, and spending considerable time playing 1986's Rampage, the game where you play a monster destroying cities in the Chicagoland area. In 1986, I was starting college and video games had gone off the radar. I missed Rampage altogether, so it was new to me and I liked it a lot.
A couple observations:
I should also mention the sink in the bathroom is outlined with a glowing line, like it's Tron's restroom.
My plan to stay for 2 hours went out the window pretty quickly. Screw it, I could (and did) pick up something at Burger King on the way home. Hours passed, then I heard an announcement from a woman who turned out to be an emcee for an open mic comedy thing they were doing. They don't have a stage, so she was standing in the middle of a few tables near the bar where people drink local beers and eat nachos and other video game arcade bar food. It seemed a very awkward setting for a stand-up performance.
I wasn't exactly there for comedy, and by the crowd's reaction neither was anybody else. Of course, it didn't help that the comic was doing jokes about elderly relatives dying (Too soon! Even if grandma is 90.). She was followed by a guy who did some 'I'm reading a journal entry' conceptual thing who went over about as well, who in turn was followed by a loud 'wacky' guy with a hipster mustache who at least tailored his material to the venue, starting off with a rant about 'Burger Time' which I was receptive to. Apparently they also hold 'Rock Band Karaoke' nights, which I imagine are a bit more fitting and go over well.
Eventually I did go home and eat cheap fast food. Just like the old days and all that. It was a very fun place, highly recommended to anybody wanting to indulge in a couple of hours of nostalgic quarter-wasting.
Oh yeah. The link. Ground Kontrol Classic Arcade.
I knew I had to see Powell's, the multi-level bookstore the size of a city block (plus a science and technology building across the street), and I did. I went there twice, even, and may post about it more later. A friend of mine, Bloominglabs' Jenett T., told me about a place called Ground Kontrol Arcade. It turns out Ground Kontrol is an 80s style arcade, full of 80s games, mostly classic 79-84 era games. There are mostly pinball games upstairs, but I'm not too into that.
For the most part I think the 80s really sucked: Cold War paranoia, that doddering old fool Reagan, really bad drum sounds and dated production, and I had really out-of-control acne. I did love the video games, though, so when I found out about this place, I had to go. I had some fool idea that I'd hang out for just 2 hours or so and have dinner after, like 2 hours would be enough at what would probably be my one and only chance to re-live the glory of Flynn's Arcade in the movie Tron.
I got there about 7, which was cool because that's when they switched to being 21 and over, and yes they serve beer and a couple goofily named arcade themed cocktails (e.g., Vodka Tronic). I had a local beer (Black Butte Porter) and made change. Having a pocket full of quarters when you are over 40 ordinarily would be a damned weighing down your pants in a lopsided way nuisance, but this night it felt great. I started out with Dig Dug and made the rounds, spending more time with Frogger, Tempest, and Asteroids than with the other games, and spending considerable time playing 1986's Rampage, the game where you play a monster destroying cities in the Chicagoland area. In 1986, I was starting college and video games had gone off the radar. I missed Rampage altogether, so it was new to me and I liked it a lot.
A couple observations:
- Burger Time, while whimsical, is so slow it's painful.
- For a game from 1979, Asteroids holds up. One thing I'd forgotten was how satisfying the rumble is when you turn on the thrust. In general, like other good games from the era, it makes the most of the technology available.
- I miss vector graphics (Tempest, Asteroids, Star Wars). The really sharp bright lines are missing in emulators like MAME. Sometimes emulation just doesn't cut it.
- These games were hazardous after all. I hurt my shoulder with some overzealous joystick work.
I should also mention the sink in the bathroom is outlined with a glowing line, like it's Tron's restroom.
My plan to stay for 2 hours went out the window pretty quickly. Screw it, I could (and did) pick up something at Burger King on the way home. Hours passed, then I heard an announcement from a woman who turned out to be an emcee for an open mic comedy thing they were doing. They don't have a stage, so she was standing in the middle of a few tables near the bar where people drink local beers and eat nachos and other video game arcade bar food. It seemed a very awkward setting for a stand-up performance.
I wasn't exactly there for comedy, and by the crowd's reaction neither was anybody else. Of course, it didn't help that the comic was doing jokes about elderly relatives dying (Too soon! Even if grandma is 90.). She was followed by a guy who did some 'I'm reading a journal entry' conceptual thing who went over about as well, who in turn was followed by a loud 'wacky' guy with a hipster mustache who at least tailored his material to the venue, starting off with a rant about 'Burger Time' which I was receptive to. Apparently they also hold 'Rock Band Karaoke' nights, which I imagine are a bit more fitting and go over well.
Eventually I did go home and eat cheap fast food. Just like the old days and all that. It was a very fun place, highly recommended to anybody wanting to indulge in a couple of hours of nostalgic quarter-wasting.
Oh yeah. The link. Ground Kontrol Classic Arcade.
Subscribe to:
Posts (Atom)