A Discussion of Open Source Software
By Stephen Adler - Nov 1998
I have been watching the evolution of the Open Source Software (OSS) phenomenon over the past 10 years and with the posting of several commentaries about it, notably Eric Raymond's "The Cathedral and the Bazaar" and his leaked/annotated version of the "Halloween Memos", I thought I would write down some thoughts which have been knocking around my head on this subject for a while.
First off, the OSS phenomenon is a reflection of another, more fundamental phenomenon which I don't have a word for. I'll call it the "Total Freedom of Information Exchange" phenomenon, which has been aided by the communications technology that has created an "Information Ecosystem", also referred to as the Internet. A lot of focus has been given to the crown jewel of the OSS phenomenon, GNU/Linux OS, but there is clearly a larger force behind this. This "Total Freedom of Information Exchange" is this larger force. And this Information Exchange is just another manifestation of the manner in which our collective society has grown since the dawn of history. It has reached different levels of "Freedom" or "Totality" depending on the tools we have available at the time to aid in this process of "Information Exchange".
The basic premise of this idea is that knowledge is something which cannot exist on its own. No single human being has the mental capacity to discover or know everything. What humans can do is understand someone else's ideas and elaborate on them. This is done through communication, specifically through language, memory and writing. These tools of information exchanges thus give rise to knowledge.
One can now follow the growth of knowledge and our civilization in this context. Go back to the stone age, and one would imagine very small clusters of humans, trying to survive with the knowledge they had then. They could speak to each other, but not write anything down. So knowledge is stored in memory and passed on by word of mouth. It seems rather clear to me that there is a limit to the amount of collective knowledge which can be passed on by word of mouth. Since the amount of knowledge which can be referenced at any one time is limited, then the extent of social evolution will in turn be limited.
There are only a hand full of elders in this small community which the young students could go to for advice. Once writing was invented - perhaps out of necessity - the knowledge base can be expanded, and done so on a large scale. I would imagine that one evolves from primitive social conditions - small clusters or tribes trying to survive the natural elements - to empires like the Roman, Egyptian, Mayan, and other empires all over the world. These civilizations, for the most part, tamed the natural elements for it citizens and have grown in size, by several orders of magnitude, compared to their primitive counterpart.
Part of this concept includes the ability for a diverse set of people to have access to the knowledge base and contribute to it. Being able to preserve knowledge by writing it down in books and storing books in libraries, allows for a larger number of people to gain access to these books, learn from them, pick up from where the authors left off and advance their ideas. These are then written down for the next generation to repeat the process. The key point here is that there are many people who are involved in this process. The more people involved, the greater the chance that someone will come up with a good idea or new discovery which is then recorded for others to learn from. One could postulate that a few "smart" people made major advances in the knowledge base possible. If this is the case, the more people who are able to take part in the process of learning, understanding, and recording new thoughts, the greater the probability that one of those engaged in this process will make a big advance, such as the invention of writing and a numerical system, the discovery of medicine etc.
Another important aspect of this development process is that the people who are engaged in it be able to pick up where others left off. This maximizes the rate of expansion of the knowledge base and forms the basis of the concept termed "pushing the envelope". Once an idea is recorded and published, then it is up to the next person in line to come up with a new idea that builds on the previous one. There is no need to try and come up with the same idea again. If this were done, it would indicate inefficiency in the system.
One can focus on two aspects, the knowledge base and the rate of growth of the knowledge base. I would then postulate that the degree of social advance or the state of any one civilization is proportional to the rate at which the knowledge base is being advanced. Societies have institutionalized the knowledge base and methods of advancing it (i.e., maximizing the growth rate) by setting up schools and universities, each having as its centerpiece the library, or what I call the knowledge base. In a society with a large population, these institutions will attract a few "smart ones," or rather, those who are interested in learning what is in the knowledge base. They study it (go to class), expand on it (do research), and contribute to it (publish). The larger the population base, the more people are attracted to these universities, and the greater the rate at which the knowledge base is expanded. From the expanded knowledge base, society can then tap into it, and use it to make it possible to enlarge the size of its civilization. The larger the civilization, the more people contribute to the knowledge base which leads to an exponential growth model.
But there is a limit to this imposed by physical distances and the ability for someone to go to a university or for the universities to interact with each other. A barrier to societal growth is the inability for a single person to go easily from one place to another. Or at least for his or her idea to travel from one place to another. Time passes; these problems are solved by applications reaped from the growing knowledge base. Crude oar-powered boats gave way to wind-powered boats, which in turn gave way to steam-powered boats. Propeller-powered airplanes developed into jet-powered airplanes.
A message travels by the written word being transported by humans using human transport, (boats, horses, trains, etc.) to messages being transported by telegraph, radio signals, light pulses etc. The physical means of getting information from one place to another plays a critical role in the expansion of the knowledge base because the more people who have access to it and the greater the chance that there is someone who understands the more complex issues and is able to expand on them, and record his or her new ideas and pass them on to others.
This brings us to the present day. Our institutionalized form of maintaining the knowledge base and expanding upon it are our schools and the students who study there. Now all students have access to a PC, which in turn have access to the Internet. This now forms the infrastructure for "Total Freedom of Information Exchange." Anyone in the world can communicate with anyone else anywhere in the world. With the population of the globe now standing on the order of 10 billion, one only needs a very small probability, (one part in a billion?), for two people with a good idea to exchange information on it and advance that idea. This in turn could lead to vast social changes. This idea could be some new technology, social reform, or art form. With the advent of the Internet and the methods it has established to store and communicate ideas, (e-mail, news groups, and now web pages), this probability is now greater. With the expanded accessibility to the Internet enjoyed by the general public, this raises this probability even higher. So, with everyone connected to the Internet through PCs, and thus able to communicate with anyone else, the environment of "Total Information of Freedom Exchange" is established. This exchange takes place within the context of an "Information Ecosystem", giving rise to the highest rate of growth of the knowledge base ever recorded in history.
So what are some of the consequences of this environment? To start with, we have the open source development paradigm, which exemplifies this evolution of knowledge. Linus, having early access to the Internet back in 1991, posted his ideas about an operating system to a news group. At the time, a very small percentage of the total number of people who had access to the Internet responded to Linus' ideas by trying them out, elaborating on them, and replying with ideas of their own, fixing bugs and adding their own enhancements. The processes grew, very much like some kind of organic life form, and today we have a living, evolving, and maturing Linux kernel, with all of its drivers and enhancements. One thing to remember is that Linus did not do it on his own. He planted the seed of the Linux OS in the "information ecosystem" where it grew through this process of "Total Freedom of Information Exchange" into what we know today and run on our PCs. The explosive growth of the OSS phenomenon attests to the fecundity of this information ecosystem.
For a moment I would like to digress to some of the issues which are of concern to the Open Source community. First of all, as I have been trying to elaborate, the Open Source process is a manifestation of our societal method for expanding the knowledge base. It is being accelerated by the fact that it 'lives' in this environment of "Total Freedom of Information Exchange".
The mere fact that the source is available and freely distributable is equivalent to publishing your ideas in a publicly available journal for your peers to read. Einstein published his ideas on Relativity in a peer review journal and thus ended up teaching all of humanity about the very weird behavior of space and time, first by getting his peers engaged in discussing relativity, then enlisting their aid in developing the theory. Finally the word got out to the general public via the press. In the same way, the Apache development team worked on an idea, the httpd server, published their ideas, and, by making the source code available to all, made possible further development of the idea in the public arena.
The OSS paradigm, in time, will deepen the knowledge of computer science and its applications at a rate unknown to humankind. In general, the rate of growth of the "whole knowledge base" - everything from physics, chemistry, biology, and economics to literature, art and language - will now expand at unprecedented rates.
There is one other issue that makes this "Total Freedom of Information Exchange" possible, and that is the language we use to communicate these ideas. The fact is, the need for a common language plays a key roll in the "Information Ecosystem" and our ability to maintain this "Total Freedom of Information Exchange." For the simple fact that the fewer the languages involved, the more people can communicate with each other and the more the knowledge base can expand. As it stands now, English seems to have become this default language for people engaged in the physical sciences, international business, and now the Internet. More important are the technical languages that make up the syntax of the computer languages we use to write software and the protocols we use to communicate over the Internet. The fact that we have open standards, which are 'owned' by no one, makes it possible for this information ecosystem to exist. One cannot communicate without them.
Let me now proceed with a simple analysis of this "Information Ecosystem." Is it a fragile system comparable to a physical ecosystem like a rain forest that can be damaged by improper care? Does it exist without context, or within the context of the evolution of humanity? I postulate that this "Information Ecosystem," which has given rise to the GNU/Linux OS and the open source phenomenon, can be affected by improper care, but I'm not sure to what extent. Is it possible that the "Information Ecosystem" might abruptly cease to exist through misuse?
For example, there are tendencies due to economic forces to try and capitalize, segment and own portions of what has evolved from the knowledge base. Patents are one institutionalized form. The robber barons of the turn of the century are another. Control and ownership of large-scale industries, which in turn, for profit motive, misuse the environmental ecosystem, is in existence today. Many times in history there has been the tendency to hoard knowledge or destroy the knowledge base for the gain of a select few who are in power. Restricting access to the knowledge base either by economic means or through some kind of social standing has been practiced. Religious institutions and "royalty" (i.e. an aristocratic social structure: kings, czars, lords, barons, or what have you) have done so through out history. I am sure that there are countless examples, but two in our recent history come to mind. The Nazi's burning of books written by Jews and Stalin altering historical facts to advance his political power. Both Stalin and Hitler profoundly altered the development of our global society in recent times. The question is whether misuse of the information ecosystem could do the same.
I don't think so, at least to the degree which Stalin and Hitler achieved. One could poison the information ecosystem by introducing proprietary protocols or by decommoditizing them. Another form of info-ecosystem poison would be to somehow restrict the flow of information, by say disallowing use of anonymous ftp sites or shutting down the Usenet news group system. Another form would be to somehow limit access to the Internet by introducing exorbitant access fees such as those charged by telcos. Another form would be to compartmentalize the Internet, much like building the Berlin wall, segregating regions of the Internet and making it hard for people to communicate between some kind of net barrier. There are many more examples.
What about looking at the other end of the scale? Should those who use the information ecosystem play an active role in preserving it and developing OSS projects? Or are the forces that brought the information ecosystem into being strong enough to keep it growing such that its rate of expansion remains maximized? Sometimes I lie awake at night worrying about the state of the Linux kernel or the GNU compilers, wondering whether the development will suddenly stop due to key players being absorbed into the private sector or simply becoming interested in other projects. This could be paranoia on my part, akin to worrying about the sun rising tomorrow morning. The problem is, as far as I can recall, the sun has risen every morning since I was born. But the Internet wasn't around when I was born. The fact is, this information ecosystem is so new, one really has no idea how it will evolve or if it will even evolve beyond its present state.
I assume it will evolve further and to a much greater extent than we can imagine now. Think back to 1990. Did anyone expect that something as simple as hypertext mark up language would unite the world? I would sleep better, and maybe all the businesses which are starting to accept the OSS paradigm as a viable option to turn a profit, if they would take care of the this information ecosystem. Or maybe the educational and research institutions which brought this information ecosystem into existence as we know it today will be its keeper. Keep in mind that its primary use is to expand the knowledge base. Should the Internet's maintenance be entrusted to the private sector or public sector? I could imagine a non-profit organization whose sole purpose would be to oversee that the information ecosystem stay clean by promoting access to it and further developing protocols and languages which are now open.
It could also be an institution where OSS projects could be based: the CVS trees, master mirror sites, etc. This non-profit organization would be overseen by a board of directors made up of CEO's of a handful of leading edge companies who promote and make a profit from OSS projects. Or maybe an institution like the National Science Foundation or the Library of Congress could setup an institution devoted to promoting OSS projects and advancing open protocols. Besides setting up an IT infrastructure to house OSS development projects, it could fund grants to OSS developers and development teams for the purpose of advancing ideas or methods which could be used by others. In other words, to expand the knowledge base of IT and software development. Maybe it is too much of a utopian vision. Maybe not.
To conclude, we are living in very exciting times. The final "communications frontier" has been attained through the establishment of the Internet throughout the world and the foundation of open standards through which we can communicate with each other in this information ecosystem. There is a healthy ongoing debate within the OSS community focused on this phenomenon. Web sites such as slashdot.org will play a key role in helping people participate and keep the debate alive, for this phenomenon is something we should never take for granted.
Copyright (c) Stephen Adler, 1998-2010
Authors notes:
Last modified Nov 19, 1998.