Statistics tells us what this blog is about

Hidden in the Google App Engine App Gallery among the me-too twitter-based apps and variants on 'save something so I can read it later' (I use Firefox plug-in Readeroo) is the glamorously named app 'Statistically Improbable Phrases'. Give it a URL, play with some sliders, and you're rewarded for your minutes of effort with a nifty tagcloud:



It seems a pretty accurate representation of what this blog's about. Notice there's love in there, but not love's opposite. I'm so positive and upbeat. Nice work Anand, whoever you are.

Radical Novelty - Edsger Dijkstra and what might have been

Edsger Dijkstra is known for his many contributions to computer science, including but not limited to the algorithm named after him, structured programming, and the shunting yard algorithm. He's also famous for his ninny-eviscerating essays like 'Go To Statement Considered Harmful' (1968) and 1975's even more wonderfully titled 'How Do We Tell The Truths That Hurt?' (there are likely many country and western songs that share this title), which contains such deathless truths as these:

• Programming is one of the most difficult branches of applied mathematics; the poorer mathematicians had better remain pure mathematicians.
• It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.
• The use of COBOL cripples the mind; its teaching should, therefore, be regarded as a criminal offence.
• Projects promoting programming in "natural language" are intrinsically doomed to fail.

Sadly, apparently nobody listened to him at all. Neither did anyone apparently listen to what he had to say in the equally brilliant 'On the cruelty of really teaching computer science', a 29-page manifesto whose words resonate so strongly with me 20 years later that I'm simultaneously thrilled and nauseated. Thrilled that somebody so brilliant would express ideas and attitudes similar to what's been boiling beneath my own impassive surface for years, nauseated by my tendency to go with the flow and firmly entrench myself as part of the problem. So what does he say? Well, for starters, he discusses the idea of a 'radical novelty', something that turns the world upside down and destroys the established order. In this case he's talking about computers. Dijkstra says when confronted with radical novelties we try to come up with analogies and metaphors to describe them in terms of our experience, but that in the case of a sharp discontinuity (a radical novelty) this approach is useless. The comparison he draws is to learning about quantum mechanics, a mind-fuck of an experience where thinking about Bobby and Suzy on a see-saw isn't going to do you a damn bit of good anymore. He then goes on to recount the suppression of and resistance to radical novelties throughout history: math's Michael Jordan, C.F. Gauss, is labeled a coward for keeping his discovery of non-Euclidean geometry to himself, fearing he'd get the Galileo treatment. He also recalls the story of King Ferdinand's visit to the conservative University of Cervena, where the King is assured by the Rector:

"Far be from us, Sire, the dangerous novelty of thinking."

Software Engineering comes under fire shortly after this. Though I've used the term (go with the flow), I've known real Engineers and what they do, and have considered it a bit of a joke myself (like calling a janitor a 'Sanitation Engineer'). Steve Ballmer and Microsoft side-step the controversy, preferring 'Developers'. Dijkstra's take:

Software engineering, of course, presents itself as another worthy cause, but that is eyewash: if you carefully read its literature and analyse what its devotees actually do, you will discover that software engineering has accepted as its charter "How to program if you cannot."

(It's funny because it's true.) Dijkstra then tells the truths that hurt about Artificial Intelligence:

Finally, to correct the possible impression that the inability to face radical novelty is confined to the industrial world, let me offer you an explanation of the --at least American-- popularity of Artificial Intelligence. One would expect people to feel threatened by the "giant brains or machines that think". In fact, the frightening computer becomes less frightening if it is used only to simulate a familiar noncomputer. I am sure that this explanation will remain controversial for quite some time, for Artificial Intelligence as mimicking the human mind prefers to view itself as at the front line, whereas my explanation relegates it to the rearguard. (The effort of using machines to mimic the human mind has always struck me as rather silly: I'd rather use them to mimic something better.)

So by this point Dijkstra has laid out that we have something completely new and incredibly powerful at hand, but by refusing to recognize it as such, we're blowing it, big time. Further, our efforts to get our heads around it by relying on analogies and metaphors are misguided at best, damaging at worst. As I look at my 'Desktop' and see icons representing 'Folders', I can see what he's talking about (but maybe I'm just too much of a die-hard command line UNIX guy). Since the essay promises something about teaching computer science being cruel, we are by this time wondering when Dijkstra will get around to that, and finally he does. He blasts traditional academia for buckling under the business elite, who dismiss traditional academics and intellectuals as largely irrelevant and powerless. He again rejects 'this guy talks to that guy' anthropomorphism of computers and presents a cool proof of the theorem that if you remove the two corner squares of a chessboard, it's impossible to cover what remains with dominoes that each cover two squares. Most would be tempted to use the fast and powerful computer to stupidly run thru all the permutations, Dijkstra shows that by thinking in terms of a set's definition (mathematical type thinking) instead of in terms of individual elements (brute force) - you can derive the answer much more quickly, just using your little brain. More importantly, the idea is if you think that way and try to approach the radical novelty that is the computer using ideas and thought processes from formal mathematics, you will see much better results than if you try to program a computer by connecting cartoonish machine icons to each other inside your IDE. What's funny to me (and I guess I'm alone here, I've really run on long I know, but this essay has been running through my mind all week now) is Dijkstra's proposed 'appropriate way to teach computer science' essentially involves teaching formal mathematics and providing a programming language that 'has not been implemented on campus so that students are protected from the temptation to test their programs'. Radical indeed, but familiar in concept to the math geeks out there. Sadly, today the focus and method is as far from Dijkstra's vision as it could possibly be. The focus is on teaching languages used in industry, with non-industry languages like Scheme used in some programs regarded as hoops to jump through and hazing to suffer through. Throughout the industry there's a profound dearth of imagination, a slog-through-it mentality, misplaced pride in taking what little one has bothered to learn to absurd extremes rather than to step back and think of a better, maybe even simpler, way. It's too bad, really, because in the words of the man himself:

Teaching to unsuspecting youngsters the effective use of formal methods is one of the joys of life because it is so extremely rewarding. Within a few months, they find their way in a new world with a justified degree of confidence that is radically novel for them; within a few months, their concept of intellectual culture has acquired a radically novel dimension. To my taste and style, that is what education is about. Universities should not be afraid of teaching radical novelties; on the contrary, it is their calling to welcome the opportunity to do so. Their willingness to do so is our main safeguard against dictatorships, be they of the proletariat, of the scientific establishment, or of the corporate elite.

So there you have it. Edsger W. Dijkstra is my lunatic fringe preacher, and I'm going to keep going to his church, consequences be damned!