This is post tries to explain some of what I was talking about in my talk at weactuallybuildstuff.com, it’s series of two posts, I’ve tried write it as one post but it got too long and icky, so I’ve split it into two.
I believe at the heart of Software Development culture there is an identity crisis. I say this because I believe software development to be an engineering discipline, based on science, but many of fellow developers do not believe this. They say it is an art form or a craft.
To support my argument I think first it would be useful to look at what engineering is. Wikipedia currently defines engineering as:
Engineering is the application of scientific, economic, social, and practical knowledge in order to design, build, and maintain structures, machines, devices, systems, materials and processes
In software, we’re trying build stuff so, it seems clear to me that following engineering practices is necessary. But what are good engineering practices? This is where the science bit comes in. Wikipedia currently defines science as:
Science (from Latin scientia, meaning "knowledge") is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe.
So science is how we organize knowledge, so we can use science and particularly the scientific method to investigate what are good software engineering practices. Accept, I’m left with the overwhelming feeling that we don’t do this. With open source we have become very good at sharing building blocks for software, this is a good thing and I think it’s something our industry can be proud of, but it isn’t enough. To again more insight into best practices we need industry and write more papers and reports on successful and unsuccessful projects (understand the reasons for failure is very import too). Industry also needs to get better about publish statics about projects, and knowing which statistics to collect. We also need a better relations between academia and industry so academics can take up the challenge of performing meta-analysis once this data is available. I not knocking academia, I believe there’s some good working been done out there, but what also needed is a different kind of work. I think more people need to do the for software what epidemiologist do for medicine. In medicine an epidemiologist is someone how looks for health best practices by studying the available data, think computer science desperately needs someone to pull together the available data on software engineering and help tease out what the best practices of our industry really are. (Apologies if you are an academic working in the way I describe, but I don’t feel your voice is being heard)
When I talk about this there are two arguments I often hear people use against software needing science and engineering, so I’d like to deal with them briefly here. The first is that software can’t be engineering as it is an art/craft. I think this is wrong as it false dichotomy. Many artists and crafts folk rely on science and engineering, they often spend years learning the details of how things look and techniques to reproduce them in there chosen medium. If you want to produce giant sculptures as Louise Bourgeois did, then you’ll need to know a thing or two about engineering, though if your Damien Hirst you may outsource this. Interestingly, for perhaps the most famous artist, Leonardo da Vinci, there was no difference between art and science. Leonardo often drew the world in an attempt to understand it better and also to express his ideas about science and technology. The second I often hear against the use of science/engineering in software is that software engineering is simply too complex to be study by science. While it’s true than there are many factors that effect what makes a successful software project and this makes it difficult for scientist to pull part how to build software successful project and give it back to us in easy to understand chunks, not to even try seems a little defeatist. After all, where people have applied the scientific method with rigor we have learn a great deal on many complex subjects.
For what it’s worth I’m not saying current trends in software development, like agile and craftsmanship, are wrong, I merely saying we’re not doing enough to check they are taking us in the right directions. As Scott Hanselman pointed out there’s clearly room for improvement. So, summary, don’t be afraid to follow intuition and draw inspirations form many other unrelated domains, but always fall back on scientific method to check these intuitions are leading in the right direction.
The follow will “How We Fool Ourselves: Why Intuition Isn't Enough” will be published shortly (at my current blogging rate “shortly” means before end of 2015).
(Interesting aside: I found this article from Joe Duffy “Bluring the line between research and development” had and interesting and not dissimilar view of the relationship between industry and academia)
Feedback:
Feedback was imported from my only blog engine, it's no longer possible to post feedback here.
re: The Software Developer Identity Crisis - gogmsa
Excellent post! It's good to read that some people at least think out of the craftmanship box.
It seems to me agile/craft was a (good) reaction to failures of project using upfront design.
Yet nobody would have build the Taj Mahal with an agile method.