 |
 |
 |
||||
 |
 |
 |
 |
 |
||
|
|||||||||||||||||||||
OPEN SOURCE INTELLIGENCE
Felix Stalder & Jesse Hirsh
In the world of spies and spooks, Open Source Intelligence (OSI) signifies useful information gleaned from public sources, such as newspapers, phone books and price lists. We use the term differently. For us, OSI is the application of collaborative principles developed by the Open Source Software movement to the gathering and analysis of information. These principles include: peer review, reputation- rather than sanctions-based authority, the free sharing of products, and flexible levels of involvement and responsibility.
Like much on the Internet in general, including the Open Source Software movement, practice preceded theory also in the case of OSI. Many of the Internet's core technologies were created to facilitate free information sharing between peers. This included two-way communication so that information could not only be distributed efficiently, but also evaluated collaboratively. E-mail lists - the most simple of all OSI platforms - have been around since the mid 1970s. In the 1980s, bulletin boards, FidoNet and Usenet provided user-driven OSI platforms with more sophisticated and specialized functionality. In the 1990s, many of these platforms were overshadowed by the emergence of the WordWideWeb. Tim Berners-Lee's foundational work on web standards was guided by a vision of peer collaboration among scientists distributed across the globe. While OSI's precedents reach back through the history of the Internet - and if one were to include peer-reviewed academic publishing, much beyond that - a series of recent events warrant that it be considered a distinct phenomenon that is slowly finding its own identity, maturing from a practice "in itself" to one "for itself." Projects like the Nettime e-mail list, Wikipedia and the NoLogo.org website each have distinct history that led them to develop different technical and social strategies, and to realize some or all of the open source collaborative principles.
The culture of the Internet as a whole has been changing. The spirit of free sharing that characterized the early days is increasingly being challenged by commodity-oriented control structures which have traditionally dominated the content industries. At this point, rather than being the norm, free sharing of information is becoming the exception, in part because the regulatory landscape is changing. The extension of copyrights and increasingly harsh prosecution of violations are attempts to criminalize early net culture in order to shore up the commodity model, which is encountering serious difficulties in the digital environment.
Years of experience with the rise and fall of "proto-OSI" forums has accumulated to become a kind of connective social-learning process. Uncounted e-mail lists went through boom and bust cycles, large numbers of newsgroups flourished and then fell apart due to pressures from anti-social behavior. Spam became a problem. Endless discussions raged about censorship imposed by forum moderators, controversial debates erupted about ownership of forums (is it the users or the providers?), difficulties were encountered when attempting to reach any binding consensus in fluctuating, loosely integrated groups. The condensed outcome of these experiences is a realization that a sustainable OSI practice is difficult to achieve and that new specialized approaches must be developed in order to sustain the fine balance between openness and a healthy signal/noise ratio. In other words, self-organization needs some help.
< open source collaborative principles >
One of the early precedents of open source intelligence is the process of academic peer review. As academia established a long time ago, in the absence of fixed and absolute authorities, knowledge has to be established through the tentative process of consensus building. At the core of this process is peer review, the practice of peers evaluating each other's work, rather than relying on external judges. The specifics of the reviewing process are variable, depending on the discipline, but the basic principle is universal. Consensus cannot be imposed, it has to be reached. Dissenting voices cannot be silenced, except through the arduous process of social stigmatization. Of course, not all peers are really equal, not all voices carry the same weight. The opinions of those people to whom high reputation has been assigned by their peers carry more weight. Since reputation must be accumulated over time, these authoritative voices tend to come from established members of the group. This gives the practice of peer review an inherently conservative tendency, particularly when access to the peer group is strictly policed, as it is the case in academia, where diplomas and appointments are necessary to enter the elite circle.
The point is that the authority held by some members of the group- which can, at times, distort the consensus-building process - is attributed to them by the group, therefore it cannot be maintained against the will of the other group members. If we follow Max Weber's theory that power is the ability to "impose one's will upon the behavior of other persons," this significantly limits the degree to which established members can yield power. Eric Raymond had the same limitations in mind when he noted that open source projects are often run as "benevolent dictatorships." They are not benevolent because the people are somehow better, but because the dictatorship is based almost exclusively on the people's ability to convince others to follow their lead. This means that coercion is almost non-existent. Hence, a dictator who is no longer benevolent and alienates his or her followers loses the ability to dictate.
The ability to coerce is limited, not only because authority is reputation-based, but also because the products that are built through a collaborative process are available to all members of the group. Resources do not accumulate with the elite. Therefore, abandoning the dictator and developing in a different direction - known as "forking" in the Open Source Software movement - is relatively easy and always a threat to the established players. The free sharing of the products produced by the collaboration among all collaborators - both in their intermediary and final forms -ensures that that there are no "monopolies of knowledge" that would increase the possibility of coercion.
The free sharing of information - in this case code as opposed to software development - has nothing to do with altruism or a specific anti-authoritarian social vision. It is motivated by the fact that in a complex collaborative process, it is effectively impossible to differentiate between the "raw material" that goes into a creative process and the "product" that comes out. Even the greatest innovators stand on the shoulders of giants. All new creations are built on previous creations and themselves provide inspiration for future ones. The ability to freely use and refine those previous creations increases the possibilities for future creativity. Lawrence Lessig calls this an "innovation commons," and cites it as one of the major reasons why the Internet as whole developed so rapidly and unexpectedly (The Future of Ideas, 2001).
It is also important to note that an often overlooked characteristic of open source collaboration is the flexible degree of involvement in and responsibility for the process that can be accommodated. The hurdle to participating in a project is extremely low. Valuable contributions can be as small as a single, one-time effort - a bug report, a penetrating comment in a discussion. Equally important, though, is the fact that contributions are not limited to just that. Many projects also have dedicated, full-time, often paid contributors who maintain core aspects of the system - such as maintainers of the kernel, editors of a slash site. Between these two extremes - one-time contribution and full-time dedication - all degrees of involvement are possible and useful. It is also easy to slide up or down the scale of commitment. Consequently, dedicated people assume responsibility when they invest time in the project, and lose it when they cease to be fully immersed. Hierarchies are fluid and merit-based, whatever merit means to the peers. This also makes it difficult for established members to continue to hold onto their positions when they stop making valuable contributions. In volunteer organizations, this is often a major problem, as early contributors sometimes try to base their influence on old contributions, rather than letting the organizations change and develop. None of these principles were "invented" by the open source movement. However, they were updated to work on the Internet and fused into a coherent whole in which each principle reinforces the other in a positive manner. The conservative tendencies of peer review are counter-balanced with relatively open access to the peer group: a major difference from academia, for instance.
Most importantly, the practice of open source has proved that these principles are a sound basis for the development of high-end content that can compete with the products produced by commodity-oriented control structures.
< examples of open source intelligence >
| > nettime
Nettime is an email list founded in the summer of 1995 by a group of cultural producers and media activists during a meeting at the Venice Biennale. As its homepage states, the list focuses on "networked cultures, politics, and tactics"(www.nettime.org). Its actual content is almost entirely driven by submissions from members. It is a good example of true many-to-many communication. Nettime calls its own practice "collaborative text filtering." The filter is the list itself - or to be more precise, the cognitive capacities of the people on the list. The list consists of peers with equal ability - though not necessarily interest - to read and write. The practice of peer review takes place on the list and in real time.
The list serves as an early warning system for the community, a discussion board for forwarded texts as well as a sizeable amount of original writing, and, equally importantly, an alternative media channel. This last function became most prominent during the war against Yugoslavia, when many of members living in the region published their experiences of being on the receiving end of not-so-smart, not-so-precise bombs.
By March 2002, the number of subscribers grew to 2500. The number of people who read nettime posts, however, is higher than the number of subscribers to the list. Nettime maintains a public web-based archive that is viewed extensively, and some of the subscriber addresses are lists themselves. Also, as a high-reputation list, many of the posts get forwarded by individual subscribers to more specialized lists (another kind of collaborative text filtering).
The majority of subscribers come from Western Europe and North America, but the number of members from other regions is quite sizeable. Over the years, autonomous lists have been spun off in other languages: Dutch, Romanian, Spanish/Portuguese, French and Manadarin. Despite its growth and diversity, nettime has retained a high degree of coherent culture discussion by a technology-savvy but critical European-style political left, who stress the importance of culture and social aspects of technology, as well as the importance of art, experimentation and hands-on involvement. This flexible coherence has been strengthened through a series of real-life projects, such as paper publications including a full-scale anthology, Readme! Ascii Culture and the Revenge of Knowledge (1999), and a string of conferences and "nettime-meetings" in Europe during the 1990s.
Since its inception, the list has been running on majordomo, a popular open source e-mail list package, and assorted web-based archives. Technically, the list has undergone little development. Initially, for almost three years, the list was open and unmoderated, reflecting the close-knit relationships of its still small circle of subscribers and the "clubby" atmosphere of general netculture. However, after spam and flame wars became rampant, and the deteriorating signal/noise ratio began to threaten the list's viability, moderation was introduced. In majordomo, a moderated list means that all posts go into a queue and the moderators - called "list-owners," an unfortunate terminology - decide which posts get put though to the list, and which are deleted. This technological set-up makes the moderation process opaque and centralized. The many list members cannot see which posts have not been approved by the few moderators. Understandably, in the case of nettime, this has led to a great deal of discussion about censorship and the "power grabbing" moderators. The discussion was particularly acrimonious in the case of traffic-heavy ASCII-art and spam-art that can either be seen as creative experimentation with the medium, or as destructive flooding of a discursive space. Deleting commercial spam, however, was universally favored.
In order to make the process of moderation more transparent, an additional list was introduced in February 2000, nettime-bold. This channel has been carrying all posts that go into the queue prior to moderators' evaluation. Because this list is also archived on the web, members can view for themselves the difference between what was sent to the list and what was approved by the moderators. In addition to increasing the list's transparency, having access to the entire feed of posts created the option for members to implement parallel but alternative moderation criteria. In practice, however, this has not yet occurred. Nevertheless, giving members this option has transformed the status of the moderators from being the exclusive decision makers to "trusted filters." It has also provided the possibility for forking (i.e. the list splitting into two differently moderated forums).
Nettime is entirely run by volunteers. Time and resources are donated. The products of nettime are freely available to members and non-members alike. Even the paper publications are available in their entirety in the nettime archives. Reflecting its history and also the diversity of its contributors and submissions, nettime has maintained the rule that "you own your own words." Authors decide how to handle redistribution of their own texts, though to be frank, it is hard to have control over a text's after-life once it has been distributed to 2,500 addresses and archived on the web.
Despite its many advantages - ease of use, low technical requirements for participating, direct delivery of the messages into members' inboxes - the format of the email list is clearly limited when it comes to collaborative knowledge creation. Moderation is essential once a list reaches a certain diversity and recognition, but the options for how to effect this moderation are highly constrained. Nettime's solution - establishing an additional unmoderated channel - has not essentially changed the fact that there is a very strict hierarchy between moderators and subscribers. While involvement is flexible (ranging from lurkers to frequent contributors) the responsibility is inflexibly restricted to the two fixed social roles enabled by the software (subscriber and moderator). The additional channel has also not changed the binary moderation options: approval or deletion. The social capacities built into the email list software remain relatively primitive, and so are the options for OSI projects.
| > wikipedia.com
Wikipedia is a spin-off of Nupedia. Nupedia, a combination of GNU and encyclopedia as the name indicates, is a project to create an authoritative encyclopedia inspired and morally supported by Richard Stallman (www.gnu.org/encyclopedia/free-encyclopedia.html). However, apart from being published under an Open Content license, Nupedia's structure is similar to the traditional editorial process. Experts write articles that are reviewed by a board of expert editors (with some public input via the "article in progress" section) before being finalized, approved, and published. Once published, the articles are finished. Given the extensive process, it's not surprising that the project has been developing at a glacial pace.
Wikipedia was started in early 2001 as an attempt to create something similar - a free encyclopedia that would ultimately be able to compete with the Encyclopedia Britannica - but it was developed via a very different, much more open process. The two projects are related but independent - Nupedia links to articles on Wikipedia if it has no entries for a keyword, and some people contribute to both projects, but most don't.
The project's technological platform is called Wikiweb, named for the Hawaiian word wikiwiki, which means fast (www.wiki.org). The software was originally written in 1994 and recently rewritten to better handle the rapidly growing size and volume of Wikipedia. The Wiki platform incorporates one of Berners-Lee's original concepts for the Web: to let people not only see the source code, but also freely edit the content of pages they view. In the footer of each Wikipage is the option to "Edit this page," which gives the user access to a simple form that allows them to change the displayed page's content. The changes become effective immediately, without being reviewed by a board or even the original author. Each page also has a "history" function that allows users to review the changes and, if necessary, revert to an older version of the page.
In this system, writing and editing are collective and cumulative. A reader who sees a mistake or omission in an article can immediately correct it or add the missing information. Following the open source peer-review maxim, formulated by Eric Raymond as "given enough eyeballs, all bugs are shallow," this allows the project to grow not only in number of articles, but also in terms of the articles' depth, which improves over time through the collective input of knowledgeable readers. Since the review and improvement process is public and ongoing, there is no difference between beta and release versions of the information (as there is in Nupedia). Texts continuously change. Peer-review becomes peer-editing, resulting in what Larry Sanger hailed as the "most promiscuous form of publishing."
At least as far as its growth is concerned, the project has been very successful. It passed 1,000 pages around February 12, 2001, and 10,000 articles around September 7, 2001. In its first year of existence, over 20,000 encyclopedia entries were created - that's a rate of over 1,500 articles per month. By the end of March 2002, the number of articles had grown to over 27,000. The quality of the articles is a different matter and difficult to judge in a general manner. Casual searching brings up some articles that are in very good shape and many that aren't. Of course, this is not surprising given the given the fact that the project is still very young. Many of the articles function more as invitations for input than as useful reference sources. For the moment, many texts have an "undergraduate" feel to them, which may be appropriate, since the project just finished its "first year." However, it remains to be seen if the project will ever graduate.
Both Nupedia and Wikipedia have been supported by Jimbo Wales, CEO of the San Diego-based search engine company Bomis, who has donated server space and bandwidth to the project. The code-base was rewritten by a student at the University of Cologne, Germany, and for a bit more than one year, Larry Sanger held a full-time position (via Bomis) as editor-in-chief of Nupedia and chief organizer at Wikipedia. In January 2002, funding ran out and Larry resigned. He now contributes as a volunteer. There are currently close to 1,200 registered users, but since it's possible to contribute anonymously, and quite a few people do, the actual number of contributors is most likely higher.
Wikipedia has not suffered from the resignation of its only paid contributor. It seems that it has reached, at least for the moment, the critical mass necessary to remain vibrant. Since anyone can read and write, the paid editor did not have any special status. His contributions were primarily cognitive, because he had more time than anyone else did to edit articles and write initial editing rules and FAQ files. His influence was entirely reputation-based. He could, and did, motivate people, but he could not force anyone to do anything against their will.
The products of this encyclopedia are freely available to anyone. The texts are published under the Open Content License (www.opencontent.org). This states that the texts can be copied and modified for any purpose, as long as the original source is credited and the resulting text is published under the same license. Not only the individual texts are available, the entire project - including its platform - can be downloaded as a single file for mirroring, viewing offline, or any other use. Effectively, not even the system administrator can control the project.
The scale of people's involvement in the project is highly flexible, ranging from the simple reader who corrects a minor mistake, to the author who maintains a lengthy entry, to the editor who continuously improves other people's entries. These roles depend entirely on each contributor's commitment, and are not pre-configured in the software. Everyone has the same editing capabilities.
So far, the project has suffered little from the kind of vandalism that one might expect to occur given its open editing capabilities. There are several reasons for this. On the one hand, authors and contributors who have put effort into creating an entry have a vested interest in maintaining and improving the resource, and due to the "change history" function, individual pages can be restored relatively easily. The latest version of the platform has an added feature that can send out alerts to people who request them whenever a specific page has been changed. The other reason is that the project still has a "community" character to it, so there seems to be a certain shared feeling that it is a valuable resource and needs to be maintained properly. Finally, in case of read differences over content, it's often easier to create a new entry rather than to fight over an existing one. This is one of the great advantages of having infinite space. So far, self-regulation works quite well. It remains to be seen how long the current rate of growth can be sustained, and if it really translates into an improvement over the quality of individually-written encyclopedia entries. So far, the prospects look good, but there are very few examples of the long-term dynamics of such open projects. Given the fact that the Encyclopedia Britannica has been publishing since 1768, long term development is clearly essential to such a project.
| > nologo.org
NoLogo.org is perhaps the most prominent second-generation slash site. This makes it a good example of how the OSI experience, embodied by a specific code, is now at a stage where it can be replicated across different contexts with relative ease. NoLogo.org is based on the current, stable release of Slashcode, an open source software platform released under the GPL, and developed for and by the Slashdot community. Slashdot is the most well-known and obvious example of OSI, since it is one of the main news and discussion sites for the open source movement (www.slashdot.org).
Of particular importance for OSI is the collaborative moderation process supported by the code. Users who contribute good stories or comments on stories are rewarded with "karma," which is essentially a point system that enables people to build up their reputation. Once a user has accumulated a certain number of points, she can assume more responsibilities, and is even trusted to moderate other people's comments. Karma points have a half-life of about 72 hours. If a user stops contributing, their privileges expire. Each comment can be assigned points by several different moderators, and the final grade (from -1 to +5) is an average of all the moderators' judgments. A good contribution is one that receives high grades from multiple moderators. This creates a kind of double peer-review process. The first is the content of the discussion itself where people respond to one another, and the second is the unique ranking of each contribution.
This approach to moderation addresses several problems that bedevil e-mail lists very elegantly. First, the moderation process is collaborative. No individual moderator can impose his or her preferences. Second, moderation means ranking, rather than deleting. Even comments ranked -1 can still be read. Third, users set their preferences individually, rather than allowing a moderator to set them for everyone. Some might enjoy the strange worlds of -1 comments, whereas others might only want to read the select few that garnered +5 rankings. Finally, involvement is reputation- (i.e. karma-) based and flexible. Since moderation is collaborative, it's possible to give out moderation privileges automatically. Moderators have very limited control over the system. As an additional layer of feedback, moderators who have accumulated even more points through consistently good work can "meta-moderate," or rank the other moderators.
The social potential embodied in Slashcode was available when Naomi Klein's January 2000 book No Logo: Taking Aim at the Brand Bullies became a surprising best-seller. In the wake of the anti-globalization protests in Seattle in November 1999, and after, the book began to sell in the 10,000s and later 100,000s. She found herself caught in a clash of old and new media and facing a peculiar problem. A book is a highly hierarchical and centralized form of communication - there is only one single author, and a very large number of readers. It is centralized because users form a relationship with the author, while typically remaining isolated from one another. This imbalance of the broadcast model is usually not a problem, since readers lack efficient feedback channels.
However, today many readers have e-mail and began to find Naomi's e-mail address on the web. She started receiving e-mails en masse, asking for comments, advice, and information. There was no way she could take all these emails seriously and respond to them properly. The imbalance between the needs of the audience and the capacities of the author were just too great, particularly since Naomi had no interest in styling herself as the leader or guru of the anti-globalization movement. (Of course that didn't stop the mass media from doing so without her consent.) As she explains the idea behind the NoLogo.org: "Mostly, we wanted a place where readers and researchers interested in these issues could talk directly to one another, rather than going through me. We also wanted to challenge the absurd media perception that I am "the voice of the movement," and instead provide a small glimpse of the range of campaigns, issues and organizations that make up this powerful activist network - powerful precisely because it insistently repels all attempts to force it into a traditional hierarchy" (nologo.org/letter.shtml).
The book, which touched a nerve for many people, created a global, distributed "community" of isolated readers. The book provided a focus, but nowhere to go except to the author. The Slashcode-based web site provided a readily available platform for the readers to become visible to one another and break through the isolation created by the book. The book and the OSI platform are complementary. The book is a momentary and personal solidification of a very fluid and heterogeneous movement. The coherent analysis that the traditional author can produce still has a lot of value. The OSI platform, on the other hand, is a reflection of the dynamic multiplicity of the movement, a way to give back something to the readers (and others) and a connective learning process. More than the book, Nologo.org fuses action with reflection.
Of course, all the problems that are traditionally associated with public forums are still there, dissent - at times vitriolic and destructive - is voiced, but the moderating system allows members of the group to deal with differences in opinion in ways that do not impede the vitality of the forum. The learning process of Slashdot, in terms of to how to deal with these issues, benefited NoLogo significantly. Within the first year, 3000 users registered on the site which serves requests of some 1500 individual visitors per day.
< the future of OSI >
As a distinct practice, Open Source Intelligence is still quite young. However, there are at least three reasons to be optimistic about its future. First, the socio-technological learning process is deepening. The platforms and practices of OSI are becoming better understood, and consequently the hurdles for users as well as providers are getting lower. On the users' side, the experience of learning how to deal with participatory, rather than broadcast media is growing. Their distinct character is being developed, mastered and appreciated. For providers, the learning experience of OSI is embedded in sophisticated, freely available GPL software. The start-up costs for new projects are minimal, and possibilities for adapting the platform to the idiosyncratic needs of each project are maximized. The resulting diversity, in turn, enriches the connective learning process. Second, as the mass media converges into an ever smaller number of (cross-industrial) conglomerates, which relentlessly promote and control their multitude of media products, the need for alternative information channels rises, at least among people who invest time and cognitive energy into being critically informed.
Given the economics of advertisement-driven mass media, it is clear that the value of an "alternative newspaper" is rather limited. OSI platforms, by distributing labor throughout the community, offer the possibility of reaching a wider audience without being subject to the same economic pressures that broadcast and print media face to deliver those audiences to advertisers, particularly considering the fact that paid subscriptions allow access to advertisement-free content.
The more homogenous the mainstream media becomes, the more room opens up for alternatives. And if these alternatives are to be viable, then they must not be limited to alternative content, but must also explore the structure of their production. This is the promise and potential of OSI. Finally, the field itself is becoming more professional. A new middle layer of organizations is emerging, which focuses specifically on the development of OSI, particularly on the mixing of technological platform and social community.
The range of technologies are as wide as the range of communities, and a close relationship exists between the two. Technologies open and close possibilities in the same sense that social communities do. As Lawrence Lessig pointed out, what code is to the online world, architecture is to the physical world (Code and Other Laws of Cyberspace, 1999). The way we live and the structures in which we live are deeply related. The culture of technology increasingly becomes the culture of our society.
Making one's potential and one's goals congruent is the goal of this emerging middle layer of professional OSI service providers. The company we founded in December 2000, Openflows (www.openflows.org), is one of those (small) companies providing nodes for the OSI networks. This contributes to the growth of OSI beyond the geeky worlds of Slashdot and many of its knock-offs. What Openflows seeks to provide, and what others will have to address, is the role and presence of social facilitation in the culture of a society immersed in technology.
April, 2002. Written for Subsol Reader, forthcoming Autonomedia Press.