Sunday 26 November 2017

Tackling the Open Source dilemma

Here is the dilemma that you and your boss are faced with when considering Open Source:
Looked at through the lens of traditional management, Open Source collaboration is time consuming, imprecise, unreliable, hard to manage, rarely addresses short term objectives, and is hard to quantify in a business case. And yet, in a digital economy, collaborative communities regularly out-innovate and out-compete closed or centrally controlled initiatives.
So how do we justify following a more effective, sustainable, open and equitable strategy?

This is what we will be covering today:
  • The digital economy,
  • Complexity,
  • Trust,
  • Innovation and Obsolescence,
  • and what leads to Success or Failure.

The first thing to recognise is that the Digital Economy has fundamentally changed the rules of business. Ignore this at your own peril.
Zero Duplication Costs and the Connectivity of the Internet has led to Wicked Complexity, Rapid Innovation, and on the flip side, Rapid Obsolescence.

Let’s start by talking about Complexity.
Software systems have become huge, interdependent and complex.
It is no longer possible for one person to understand all of a system’s intricacies.
So decision makers need to assume, deduce and trust information provided by others.
It means that sourcing trustworthy advice has become a key criteria for success in the digital economy.
So how do we assess trustworthiness?

It turns out we all use a variant of this trustworthiness equation.

  • We trust people who are credible and who have have track record of providing reliable advice in the past.
  • We trust people who are open and transparent.
  • We trust ourselves, our family, our friends, because they look out for us, and we look out for them.
  • We are suspicious of people who stand to gain from advice they give us.

We also trust processes.
  • We trust that the democratic process leads to fair governance and management of resources.
  • We trust that the scientific method leads to reliable research that we should act upon.
  • We trust that the “survival of the fittest” competition of market economies leads to better products.

But we also know that all processes can be gamed.
And the more complex a system, the easier it is to bamboozle people and game the system.

Part of the reason Open Source has been so successful is that it’s characteristics lead to trustworthiness.
These characteristics include:
  • Freedom,
  • Altruism,
  • Openness,
  • Meritocracy,
  • and Do-ocracy.
Let’s break these down one by one.

Open source, by definition, provides the receivers of the software with the four freedoms:
  1. Freedom to use the software unencumbered; 
  2. Freedom to study the source code and find out how it works; 
  3. Freedom to modify, retask, and improve the code;
  4. Freedom to copy and share with others.
Providing such a valuable gift, which provides significantly more value to the receiver than to the giver, increases the trustworthiness of the giver.

Additionally, openness and transparency is almost universally applied to all Open Source development practices and communication.
  • Conversations are public; Everyone has the opportunity to join and contribute; 
  • Decisions are made openly; 
  • Issues and limitations are published and shared.
Being transparent and open to public critique reduces the potential for hidden agendas and creates trustworthiness.

In a meritocracy, the best ideas win, no matter who suggests them. It is the sign of an egalitarian community rather than a hierarchical or dysfunctional one.

With a do-ocracy the person motivated to do the work decides what gets done. In complex systems, the person closest to the problem will usually be best qualified to make the technical decisions.

A key strategy for managing complexity is to divide large systems into modular subsystems.
Using modular architectures, connected by open standards:
  • Reduces system complexity,
  • Enables interoperability,
  • Which reduces technical risk,
  • And facilitates sustained innovation.
It means you can improve one module, without impacting the rest of your system. This helps with maintenance, innovation, and keeping up with latest technologies.

Collaboration is a key focus of both Open Source and Open Standards narratives. Hence, successful Open Source applications usually provide exemplary support for standards.

By comparison, from the perspective of dominant proprietary companies, it makes business sense to apply vendor lock-in tactics, making cross-vendor integration difficult. Adoption of Open Standards threatens vendor lock-in tactics, and consequently dominant vendors are often reluctant and half-hearted in their support of Open Standards.

In the digital economy there are two dominant business models which work well.
  • You solve a generic problem by supplying an awesome "category killer" application which you distribute to the world; 
  • Or you provide personalised, specialised or localised services, typically using category killer applications.
There is a natural symbiotic relationship between the two.
If you are solving a generic problem, by yourself, you will be out-innovated!
There are simply more developers in the rest of the world than you can ever muster within your team.

Because software is so time consuming to create and so easy to copy, it is excessively prone to monopolies.
This holds true for both proprietary and open source products. A product that becomes a little better than its competitors will attracts users, developers and sponsors, which in turn allows that product to grow and improve quickly, allowing it to attract more users.
This highly sensitive, positive feedback leads to successful software projects becoming “category killers”.

This means that most of the software you own will be out-innovated within a year or two.
Your software is not an asset, it is a liability needing to be updated, maintained, and integrated with other systems. It is technical debt, and you should try to own as little of it as possible.
The question is: should you select Proprietary or Open Source as the alternative?

Openness democratises wealth and power, which is a good thing for all of us, even those with wealth and power.
Open Source and Proprietary business models differ in how their realised value is shared.
Open source licenses are structured such that multiple companies can use and support the same open source product, so the market self corrects any tendencies toward price-fixing.
Effectively, Open Licenses democratise information.
It enables everyone to share in the value created by technology.
As software markets mature, and components become generic commodity items, the collaborative practices of Open Source moves to becoming the most effective means for creating and managing functionality.
Collaboration trumps Competition for commodity items!

By comparison, the ruthless competition between proprietary companies results in “winner takes all” scenarios. Many of the richest people in the world are self made software entrepreneurs.
Jeff Bezos who started Amazon has recently been ranked as the richest man in the world, stealing the spot from Bill Gates who started Microsoft. Mark Zuckerberg who started Facebook comes in at number 5. Jack Dangermond from ESRI is down at #603, with a mere $3.2 billion dollars to his name.

Lets explain this another way, following the money trail. Proprietary business model favours multi-nationals who establish themselves in big markets such as in the US or Europe.
From our Australian software spend, a small commission is provided to the local sales guy and systems integrator, and the rest is funnelled into the multinational who often farms development into cheap development centres.

Open Source on the other hand favours local business. The software is free, so the majority of money spent is on support and integration type services, which is typically applied locally, keeping money and expertise local.

Let’s look into the characteristics which make projects successful or not.
Open Source projects are highly susceptible to being Loved to Death. This happens when a project attracts an engaging user base without attracting matching contributions. Volunteers become overwhelmed leaving insufficient capacity to cover essential business-as-usual tasks.
Don’t overload the community you depend upon. It is both bad karma and bad business.
Successful projects have worked out how to either:
  • Politely say NO to “gifts” of unsupported extra code and excessive requests for help;
  • Or how to help uses become contributors, either in kind, or financially.
If your organisation isn’t ready to act as a good community citizen, actively caring about the community’s long term sustainability, then you will probably have a disappointing Open Source experience. You will make self-centred, short term decisions, and you won’t get the support you need when you most need it. You will likely be better off with proprietary software. (And the community would be better off without you.)
The success criteria for Open Source projects was researched by Professor Charlie Schweik who studied thousands of projects. As you can see from this graph, most projects are abandoned. Of the remainder, most projects don’t attract more than one or two staff, and very few attract a large community.
Viewed another way, you can see that:
  • 4/5 projects are abandoned.
  • 1 in 7 remain with just 1 or 2 developers.
  • Only 1 in 100 manage to attract 10 core contributors.
On this graph we’ve drawn in the success rate for projects, and you can see that as you attract developers, your chance of long term success increases dramatically.
This is showing ruthless Darwinian evolution at work. Only projects of exceptional quality attract sustained growth and large communities. They fit in the “magic unicorn” category.

So how do you find these magic unicorn projects?
Charlie’s team distilled further insights from their research. They found that successful projects typically possess:
  • A clearly defined vision;
  • Clear utility;
  • And leaders who lead by doing.
Then projects which manage to attract a medium to large team tend to:
  • Provide fine scaled task granularity, making it easier for people to contribute;
  • And often have attracted financial backing.

To get insights into project health, you can look at Open Hub metrics.  This slide is from the OSGeo-Live project and shows the status of leading Desktop GIS applications.
And for QGIS you can see that it has a very healthy community with over 100 active contributors. 
Another strong indicator of a project’s success is whether it has completed an Open Source Foundation’s incubation process.
  • Quality
  • Openness
  • Community Health
  • Maturity
  • Sustainability

Bringing this all together into a concise elevator pitch for your boss:
  • The Digital Economy leads to High Complexity, Rapid Innovation and Rapid Obsolescence. Get with the program, or become obsolete.
  • Increased complexity requires us to trust more. So increase the value you place on trustworthiness, openness and transparency. 
  • Software is technical debt. It needs significant maintenance to remain current. Own as little of it as possible.
  • Collaboration and openness fast tracks innovation.
  • For the long term play, Collaboration trumps Competition. If you are solving a generic problem, by yourself, you will be out innovated! Value, recognise, select and apply collaborative practices.
  • Don’t be naive, most Open Source projects fail. Learn how to pick winners.
  • Openness and Collaboration leads to the democratisation of wealth and power. Learn how to be part of the community - it makes good business sense.

  • Slide deck is available online.
  • An earlier version of these slides was presented at QGIS Conference in Sydney, Australia, November 2017. It was restructured for print and published as an article in the August 2018 edition of Position Magazine.
  • The text behind these slides, by Cameron Shorter, is licensed under a Creative Commons Attribution 4.0 International License
  • If this presentation was of interest to you, then please let me know (use comments below or email address on the slide above). I enjoy hearing from people who share similar interests, or are facing similar challenges, and ideas people have on related topics. 
Related presentations:

No comments: