Dreaming in Code

Time has been too short for me to comment on a couple of articles I've saw last week, so I'll just write something brief.

The first story was Software is Hard, which is an interview with Scott Rosenberg, the author off Dreaming in Code. Some of this story seems so familiar, and yet some of it seems unaware of modern software production techniques. Not that I can blame anyone for this. No technique works in all situations, and few are taught in schools or universities around the world. The most effective become a "fad" for a while, as those whose projects are most suited to them discover these techniques, but the resulting hype sets unrealistic expectations for people with less compatible project requirements.

Of course, the techniques I'm referring to are the various types of Agile programming, such as XP (the one that suffered the most from hype), Scrumm, and my own company's COSM. It is not a part of my duties to work directly with COSM, but I've still spent some time with it, and I agree that it is impressive, for the project planning as much as for the techniques of integrating business component with code components. But Agile is not the only useful technique out there.

Having defended these techniques, I will also say that they are not a panacea. Most people who use them don't really use them. They pick up on some ideas, but for reasons of practicality of preference, they decline to implement the entire paradigm. Another problem is that programmers are often at fault. It's one thing to have a great plan, but it is completely different to follow it. The type of plan you have is also going to suit different programmers differently. Highly skilled and creative programmers may be better with agile techniques, while those who pass a postgraduate course in programming will often be better suited to more structured environments like the waterfall model. (Note: I've never heard of, let alone seen, the waterfall model actually working).

It is this last part about asking talented (or any) coders to conform to management techniques that aligns most with what Scott has to say on the topic. One point of note is that he points out how good programmers enjoy re-inventing the wheel as it may often be faster than learning the intricacies of how "someone else" did it. The problem with this being that the resulting code is not tested as well, and has not been iteratively improved by running in the real world.

While this comment is true, I think Scott is looking at the problem from a traditional "commercial" viewpoint. Many programmers today, especially those in agile environments, will choose to use an open source project which implements some important functionality. The "challenge" that programmers enjoy is then sated by the integration of the many tools and libraries typical of most projects. Have a look at the library directory in Mulgara for a common example. There are 75 separate Jar files in there, all of which are from external projects.

In summary, he's sort of right, but he's also missed the ability of agile programming to work in some places. He has also seemed to have missed open source software, both as a project paradigm, and as a resource that modern programmers are absolutely ready to use.

By the way, I liked the title of the book. In my early days of programming I had several dreams in C: not about C, they were actually in C. The plot was a set of actions which was represented by functions, with people, places and events being structs. I've heard the experience is common.

A Stratified Profession

A different tone was set by Prof. Neil McBride in his article The Death of Computing. He sets a very cynical tone criticizing those still adhering to computationally pure programming techniques, and advocating the pragmatic approach. In particular, Neil talks about an inter-disciplinary approach for computers.

I like the idea of the interdisciplinary approach. After all, computers were always tools which we use to do jobs. They don't exist for their own benefit (though this has almost changed now with the advent of the internet). When I was younger, I noticed that all the successful approaches to computing relied on mixing skills with computers with expertise in engineering, finances, medicine, and so on. This made sense to me then, and it makes more sense now. This is particularly evident in the field of computational biology, where unbelievably complex systems (like genomes) are finally starting to be managed.

I also agree that in the past universities considered the computational purity above all else, and that rigidly sticking to this path is ignoring the pragmatism that the commercial world today wants. However, there seemed to be no acknowledgment of the continued need for the purists as well.

The idea seems to be that where once there were purists, today we need only pragmatists. I disagree with this. I think we need the purists as much as we ever did, but now there are also roles for the pragmatists. Whereas the profession was once restricted to the academic mindset, today we have a stratification, with different people needed at all levels.

We really do need people who don't understand a lot of the underlying system but who are prepared to be the resources needed for large financial institutions to build their enterprise systems. We need the talented developers who can take an idea, and in a few months turn it into a prototype that can set a new standard on the internet. We also need the theoreticians who analyze logic and modeling systems for completeness, or create new mathematical techniques for representing and transforming data.

I agree with the need to take a pragmatic approach. However, it can be taken too far, as is evident in many learning institutions today. Favoring pragmatism over everything else is throwing the baby out with the bathwater.