Sunday, November 24, 2013

Free Communication is Disrupting Hierarchies

Jeff Jarvis recently blogged:
I doubt the net’s creators realized how subversive it was to connect anyone to anyone, bypassing the institutions that mediated those connections: from media to government, universities to retailers. These institutions are now circling wagons to protect their prerogatives: copyright for media, secrecy for government.

Somewhat related discussions are taking place at the Defrag conference and on blogs talking about corporate organizations and government surveillance.

However this is nothing new, over centuries there has been a gradual reduction in the cost of communication between people, which has caused social changes, disrupted old industries, created new industries and led to political revolutions. The end point that we have reached in the last few years is that the cost of communication is now zero. There is no incremental cost for me to send a tweet that is delivered to thousands of people a few seconds later. If the tweet contains novel ideas or information that people want to share, then in a few seconds, millions of people could see it. With about 10,000 direct followers my own twitter mention reach was in the millions last week.



When I see an industry or organization that was built on assumptions that are no longer true, I expect to see disruption follow, and opportunities to create new industries and organizations. The assumption that communication is expensive is baked into our world. The new industries and organizations that take advantage of this change are emerging now.

To see how subversive this is, we need to look at some history. Every step in the reduction of communication cost has enabled changes that have been resisted by the hierarchy, led to revolutions, and in most cases have been co-opted and controlled by the elite.

The ability to control, coordinate and influence a large group of people depends upon effective communication. Early examples include the creation of sea-going ships and roads to sustain empires, from the Romans to the British Empire as examples. The leaders of these empires could gather information via messengers and send out decrees to manage at a distance, by using a coordinated hierarchical communication network. The cost of communication for the workers was prohibitive, so all information trickled down through the local authorities, whether civil, military or religious. The only low cost communication took place within the local village.

Most workers were illiterate, and taking England in the middle ages as an example, a series of disruptions were caused by the invention of the printing press. The loss of control that resulted caused major problems for authorities, and allowed ideas to spread in new ways.

As cities got bigger, local communication got faster, newsletters and newspapers started to spread ideas more quickly and the creation of postal services brought the cost of peer to peer communication down far enough that everyone could communicate over larger distances. Governments co-opted these new mechanisms, regulating the news, running the postal services. The industrial revolution created commercial entities with their own hierarchies of control and communication. The ability of ideas to spread rapidly and reach many people allowed groups to self-organize, leading to political upheaval. Revolution was enabled by the ability to spread ideas outside the incumbent ruler's hierarchy of communication and control.

Today we live in societies where effectively everyone is literate, connected and mobile. However we still have the same hierarchical institutions that were developed hundreds of years ago. Governments to manage the hierarchies of society, Companies to manage the hierarchies of commerce, Religions to manage the hierarchies of belief, Universities to manage the hierarchies of education.

These hierarchies still have value, they continue to represent ownership and have responsibilities but they no longer have control. They were sustained in the past by imposing their will on people, and carefully managed what information people are exposed to. The hierarchies are now very large and deeply nested, so rather than a small ruling elite and a large peasant population, there are very many levels in the middle, which makes many of us have a vested interest in the continuation of parts of the hierarchies.

We are now all connected by a series of relatively flat peer to peer networks. Facebook for a curated network of family, friends and interest groups, twitter for real-time news and a much more dense ad-hoc interconnection of interests, email for direct communication, and many other services. The "Arab Spring", the "Occupy Movement" and other recent political revolutions were able to find and coordinate large numbers of people extremely rapidly (hours or days) without forming a hierarchy first.

The new world is a blend of hierarchies with many peer to peer links that reach across and between the old hierarchies. The problem for the elite leaders is that they are used to having control, owning the flow of information and managing deeply nested organizations. In this new world, the leaders have ownership, responsibility, influence but much less control, they control and manage a small part of the flow of information, and their organizations are flatter and less well defined. At the extreme there are organizations like Github, which are figuring out how to make an emergent self organizing structure work. To scale a flat organization there needs to be a very strong sense of culture and unity of purpose. I think Netflix has found a productive balance between a hierarchy of responsibility for aspects of the business and freedom at each level to self organize the delivery of what the business needs. Broad and open communication across the business, and many skip-level one-on-one meetings flattens the organization and minimizes "politics".

Looking beyond business, Religion only really works if you don't think too hard about any other variant of religion. Religions all claim to be true, but claim different incompatible and incoherent things, so that exposure to ideas such as the Outsider Test for Faith causes deep problems for the faithful. Religion in general is being disrupted by science and free communication, and polarization into atheism and reactionary fundamentalism is one of the outcomes in society.

Education is based on the communication of skills and information from teachers to students, and the big institutions of education are being disrupted. They no longer control the flow of education, and the costs are increasing beyond the ability of the students to repay them, so companies like Coursera and the Khan Academy are building alternatives. For software developers, the body of code you own and contributed to on github matters far more than where (or even if) you studied. The state of the art isn't being created and taught in University research departments any more, the commercial ecosystems that form around technologies are where the bleeding edge of innovation is being disseminated to new recruits.

These disruptions are on the way, but change tends to occur rapidly when a tipping point is reached rather than gradually. A point made by Nicholas Taleb in his discussion of Black Swan events, is that its easier to point out that a change could happen and plan for what to do next, than to predict when exactly it will come. So like a cartoon Wily Coyote who has run off a cliff, we ignore the change until we look down and realize that nothing is holding us up.