The Vingean Programmer Archaeologist
First, if you have never read Vernor Vinge’s A Fire Upon the Deep, you are missing out on a truly outstanding science fiction novel. Go ahead and read it. I’ll wait.
Secondly, if you still haven’t read it, the “Programmer Archaeologist” is but one of a great many cool science fiction concepts that occur in this book. The basic premise is that faster-than-light travel and information exchange set the stage for an intragalactic Internet that is so vast, and so old, that it has literally outlived every civilization currently using it. This is an internet that is literally hundreds of billions of years old, cobbled together from a billion spacefaring civilizations using at least as many layers of virtual machines, emulators, and natural language translators. History records extend back just as far, if you can find the right combination of indexing engines and language translators to turn the records into something you can use.
In such a setting, essentially all useful software really has been written already. In order to do useful work, it follows then that you do not really need to write code anymore: you have to find code that already does what you want, and stitch it together with other code to make it do what you need. The sheer vastness and complexity of the computer systems make it virtually impossible to manage these systems any other way. (imagine trying to reimplement a 10-billion-line codebase!)
Thus is born the concept of a Programmer Archaeologist: a person whose vocation is to find and patch bits of code together to adapt software to new requirements.
It’s a wonderful concept, but it’s more than that: it’s a direct extrapolation of today’s state of the art.
To pick an example I know well, IMVU’s service is a combination of a large number of independently authored technologies: Apache for HTTP, MySQL for relational storage, Perl, memcached, Mogile, PHP, Python, Flash, Win32, PIL, and on and on and on.
Large, uniform, designed ivory tower projects still exist, of course. Mozilla, to name one, has reached somewhere around 30 million lines of code, and it is all highly internally consistent. For the most part, though, we are the Programmer Archaeologists, today. (at least, we probably are if we’re getting useful work done!)