Saturday, 7 July 2018

Commenting on Australia's draft commitments to the Open Government National Action Plan

Australia has a draft of commitments for our Open Government National Action Plan, and have asked for comment. The plan has some wonderful goals, but I wish the authors had a more practical understanding of the workings of digital economies and open source communities. The commitments could be made so much more implementable and impactful. I've provided the following specific comments, along with a reference to my previous submission which is more comprehensive.

Improve the sharing, use and reuse of public sector data

Government's draft commitment: 
… consult across government, through the new National Data Advisory Council, with the Open Government Forum and with the public including businesses, civil society groups and research and non‑profit sectors. The National Data Advisory Council will be a multi-disciplinary expert panel drawn from public sector and civic society organisations.

My comment:
Sharing public data provides noble goals, but as written, government can claim this commitment complete merely by opening access to datasets without addressing the more challenging task of making sure the data is usable.
For this policy to be implemented effectively, it should tie back to measures of data use. This requires an understanding of the characteristics of data management, and should mention measures such as: quality, fitness for purpose, ability to integrate with other data sources, standards based data structures, relevance, usability, timeliness, sustainability of maintenance.

Enhance public engagement skills in the public service

Government's draft commitment:
  • Develop and implement an Open Dialogue Roadmap
  • The Establishment of an APS Engagement Hub
My comment:
In order to be impactful, this commitment needs to extend beyond dialogue and facilitating the sharing of ideas. It needs to extend to the more important issue of collaboration during acquisition, implementation and deployment.
We need to recognise that governments around the world are all solving very similar problems, and the most efficient way to address these problems is to work on them collaboratively. However, cross-agency and cross-jurisdiction collaborative implementation is the exception rather than the rule. This can usually be traced to the difficulty of explaining and applying collaborative techniques within government frameworks.
I recommend this commitment be extended to cover the providing of tools to measure and implement cross-agency collaboration. This commitment will require a lot of work, however many of the techniques needed can be derived from Open Source community processes.

Enhance state and territory participation in the Open Government Partnership

Government's draft commitment:
  • Establishing a framework and mechanisms to facilitate communication between state and territory governments and Australian Government officials responsible for Australia’s Open Government commitments to support collaboration and learning on open government matters, and highlight the opportunity for formal subnational cooperation and membership in the Open Government Partnership, and 
  • To demonstrate the benefits of enhanced collaboration on open government matters: engaging with state and territory Information Commissioners to seek agreement to conduct surveys to measure citizens’ awareness of the right to access government information, and their experiences and outcomes in exercising that right.
My comment:
Enhancing state and territory participation is a good start, but the scope of this commitment should be extended up to international collaboration, and down to local government and further. Note that there will always be more people with great ideas who are outside your organisation than can ever mustered from within. So with every policy and procedure you write, and every application you build, do it collaboratively. 
Use, Extend, or Create, in that order:
  • Use existing material if it exists;
  • Otherwise extend and give back;
  • As a last resort, create your own, share, and endeavour to attract collaborators.

Saturday, 30 June 2018

Technical Documentation Writing Principles

This internal document describes stakeholder requirements and derived writing guidelines for each technical documentation type used within Learnosity's communication strategy.

Cameron's Cover

When working at Learnosity, I co-authored this writing guide. Learnosity is a tech company that empowers education providers to build a breadth of assessment questions within a web page, run it at scale, and analyse results. The challenge for us technical writers was to explain the powerful technology clearly, concisely and intuitively to a broad range of stakeholders. When talking with others I’ve discovered that there are many tech companies facing similar documentation challenges. Learnosity has kindly allowed me to share this internal document to help others craft something similar. This version has been lightly edited to remove some Learnosity specific information and help readability for an external audience.

Positioning document types - today


Positioning document types - tomorrow


Key Messages to convey

  • Learnosity EdTech technology is world-leading.
  • Learnosity technology provides value for money.
  • Learnosity people are technical experts.
  • Learnosity and its people are trustworthy:
    • Trustworthiness = Credible * Reliable * Open * Intimate / Self Interest.
    • We provide convincing reasons for why Learnosity products, services, and value are superior.
    • We have a long track record of delivering.
    • Trusted organisations have selected us, recommend us after doing their due diligence.
    • Someone I trust has recommended Learnosity.

Writing principles

  • “Every page is page one”. Assume someone could have reached this page as a first page from google. Make sure there is sufficient navigation to orient yourself within the page.
  • Set the stage sufficiently so that the reader understands the purpose of this document, what they will gain from it and any assumed knowledge they will require.
  • Use hyperlinks to reference terms and other pages.
  • Write timelessly, so your content is still meaningful in a year or two. Avoid terms like “new feature” which is relative to the time it was written. (Release notes can and should be time-specific, as they are related to a point in time).
  • Respect the reader's time. Write clearly and concisely.
  • Use hyperlinks to reference terms and other pages.
Quality Control:
  • Check for typos and that the instructions make sense.
  • Ensure formatting is consistent throughout the demo.
  • Ensure that the images are compressed and of an appropriate size.
    • Images should be crisp and royalty-free.
    • Check for unnecessary scrolling - both horizontal and vertical.
    • Check that the content fits well into target screen sizes, i.e. nothing crucial is cut off. This is particularly relevant for graphs and charts.

Publishing principles

  • No broken URL links or broken HTML, or broken demos. (We should check this automatically.)
  • Examples should be available to be cut-and-pasted in the user's preferred programming language. (Ideally write a conversion script to automatically convert a server-side PHP to JSP, .net, etc.)

Exemplar Examples

We need to make use of example content which:
  • Is universally understandable.
  • Is timeless and doesn’t need to be updated.
  • Where possible, is simple enough for the reader to know answers to questions.
  • Is professionally constructed, and has content which is recognisably of high quality to an educational professional.
  • Demonstrates the power of the feature being demonstrated.
  • Ideally is applicable across multiple domains.
Universal topics include:
  • Science
    • Weather
    • Animals
    • Colours
    • Chemistry
    • Anatomy
    • Solar System
  • Maths
  • Food groups
    • Categorise fruit, vegetables, serials, protein
    • Healthy food categorisation
    • Food with allergies
  • Health
    • Body parts
    • First aid
  • Safety
Topics which are not universal are:
  • Geography (People are likely to only know about Cities close to where they live)
  • History (as history is usually tied to Geography or a specific culture or a time period)
  • Pop culture, or news items, as they become dated.
  • Questions using regional units, such as money or miles/kilometres
  • Culturally sensitive questions, such as based on religion.
Blooms Taxonomy Teacher Planning Kit is useful for constructing questions.

Document Types

Solutions

Description:
The aim of a Solutions page is to help the reader imagine how their domain-specific problems can be solved using Learnosity functionality. It is about describing and showing Learnosity features using use cases and terminology familiar to the reader's domain.
Length: 2 to 4 pages, including diagrams.
Target Persona(s):
  1. Non-technical, business savvy Product Owner who is unfamiliar with Learnosity.
  2. C-level decision maker.
User Stories:
  1. As a Product Owner, I want to understand that my business domain challenges can easily be solved with Learnosity functionality.
Doc Structure:
  • Title: Describe how Learnosity can be used to address a specific domain.
  • Introduction:
    • 1 to 2 paragraphs: Discuss the domain, and how it is possible to solve challenges in this domain.
    • Probably start with a sentence defining the domain - so that the reader and author both understand each other's domain definition.
    • Ideally, mention key business requirements associated with learning in this domain - maybe legal regulations, implications of not being trained, specific learning challenges relevant to this domain (such as requiring complex mathematical script).
    • Focus on the “Why” and “How” in this top part rather than “What”. I.e. we probably don’t mention Learnosity yet.
  • Multiple sections of Domain-specific concepts/challenges
    • Heading
    • Include a hero image
    • 1 to 2 paragraphs: Discuss how Learnosity addresses an important challenge within this domain.
    • Ideally include a “Find out more” link, which points to a Learnosity page, ideally a Features page, or maybe an example of this feature in action.
  • Example:
  • More info:
    • Provide links to relevant Learnosity documentation (Typically point to concept descriptions within Guides/Knowledge Bases, Case Studies, Demos, maybe Features, probably not Reference pages.)

Demos

Description:
Seeing a working demo is often the easiest way to visualise how a feature could be used, and gain confidence that it can be implemented.
Target Persona(s):
  1. Non-technical, business savvy Product Owner.
  2. C-level decision maker.
  3. Product owner, trusted author, or trusted developer.
Demo Guidelines:
When creating interactive assessment examples it should be the best representation of our tools and functionality. We want viewers to be impressed! Consider:
  • Is the content applicable to the particular domain of the solution?
  • Does content follow our exemplar guidelines?
  • Does the content showcase a variety of levels/standards?
  • Does the content include a wide variety of question types?
  • Would imagery and/or videos help make the content look more appealing?
  • Consider using various layout options e.g. split-screen, paginated passages, tabbed passages etc. as appropriate.
  • Are all the features listed on the Solutions page incorporated into a demo?
  • Demos should provide a compelling experience on mobile and non-mobile screens, i.e. on smaller screens from 768px wide and up. In some instances it may be appropriate to create two demos - one for mobile and one for desktop.
Quality:
  • Demos should have quality criteria, such as auto testing, with designated Learnosity maintainer.
  • Demos are likely to break when software is updated. Testing demos should become part of the software release process.
  • Demos probably should have a matching tutorial, and tutorials probably should have a matching demo.

Case Study

Description:
Provide a clear, concise, high level, compelling overview of how an organisation has implemented a Learnosity based solution to solve a specific problem (or problem set). The problem might be about increasing efficiencies, reducing costs, attracting sales, adding features, or solving a technical challenge. Our aim is to use domain-specific language to help the reader realise that Learnosity has experience addressing their specific use case successfully with other companies that they know and trust, and then forward the reader to the relevant Learnosity solutions/features pages.
The reader will typically reach this Case Study on recommendation of a Learnosity salesperson, or after reading a Learnosity Solutions page.
Length: Ideally limit to two to three pages, including diagram(s). More complex use cases can be longer, but be aware that “less words get read more”.
Target Persona(s):
  1. Non-technical, business savvy Product Owner who is unfamiliar with Learnosity.
  2. C-level decision-maker.
User Stories:
  1. As a Product Owner, I want to confirm that Learnosity has been used to implement something similar to what I want to achieve.
  2. As a Product Owner, I want to confirm that the implementation was successful, and within expectations of time/effort.
  3. As a Product Owner, I want to confirm that the implementation was from an organisation I trust - typically in a related industry.
  4. As a Product Owner, I want to be reassured that Learnosity is a company that I should be comfortable doing business with culturally; that they understand my problems and are here to help.
  5. As the Company in the case study, I want it to speak favourably of me such that I might reference the case study in my marketing pipeline.
Doc Structure:
  • Title:
    • The title should describe a concept that we are explaining. It should be short and to the point. Up to ~ 5 words.
  • In pull text font, describe the value of this case study in one sentence.
  • About
    • Put this into a right sidebar (or at the bottom for smaller screens)
    • Introduce company as not all readers may be familiar with the organization. Consider including: company specialities, location, size, industry.
    • Typically you can source much of the content from the company's marketing material.
    • Usually you won’t include everything the company does, but rather focus on the area of interest to this case study.
  • The Challenge:
    • Set scene: 1 to 2 paragraphs: <XXX Company> wished to <achieve some goal/functionality/solve some user need>.
  • <Image>:
    • Provide relevant hero image(s), covering ~ 1⁄3 of the page(s). Try to convey as much meaning as possible. Typically use a contextual photo, screenshot or flowchart or infographic. Don’t embed working demos here. We want to minimise our doc maintenance and draw the reader into more advanced pages.
    • Ideally have at least one image on each page.
  • The Solution:
    • 2 to 5 paragraphs.
    • Explain how this has been implemented. Sparingly link to Learnosity feature pages mentioned.
    • Explicitly state the value (time, money, reliability, new business, better student engagement) gained by using Learnosity. Where possible, source compelling stats or figures to support statements of value. These will ideally be provided by the customer if they are prepared to share.
    • Use domain-specific lexicon rather than Learnosity lexicon. We aim to use language a reader is already familiar with rather than confusing them with jargon.
  • Quote(s):
    • Ideally source quotes from the customer. This could be one hero quote at the end, and possibly one or two quotes in the body of the text.
    • “<big title> at <XXX Company> said that Learnosity's product made our project simpler/faster/more intuitive/more reliable/....”
  • More info:
    • Provide links to relevant Learnosity documentation (Typically point to concept descriptions within Guides/Knowledge Bases, Demos, maybe Features, probably not Reference pages.)
  • Output format:
    • HTML on www site (mandatory)
    • PDF (optional - for high value case studies). PDF will require extra effort to layout the content nicely on a paper page.

White Paper

Description: A white paper describes the business consideration (which might reference technical concepts from a business perspective) related to the complete implementation of a Learnosity based solution within a specific domain. Aim to help a potential customer understand all the technical complexity involved in implementing their solution, and by implication, have them appreciate the depth of knowledge we have in this domain. This aims to provide unbiased information, and should try not to be an overt sales pitch. It should have well-constructed business discussions and technical implementation considerations, backed by research where appropriate.
When to write:
  • Writing a White Paper will be a lot of work, and the paper likely will have a lifespan just a year or two (as we will be referencing versions of software and current research which will become dated).
  • A reason for writing a white paper will likely be triggered by wanting to pull in a specific big client. We tailor the white paper for the customer, but make it generic such that others can use it.
Length: 5 to 10 pages, including diagram(s)
Target Persona(s):
  1. Software Systems Architect.
  2. C-level decision maker.
User Stories:
  • TBD.
Doc Structure:
  • Title: The title should describe a concept that we are explaining.
  • Executive Summary:
  • Concepts:
  • Conclusion:
  • More info: Provide links to relevant Learnosity documentation

Features

Description:
Features on WWW allow us to define and display specific functionality in a product marketing voice. They should be short outlines that have more showing focus than reading.
Length: Less than 500 words (two to three paragraphs ).
Level of detail: Tell your mum how a given Learnosity feature works and/or its value
Target Persona(s): (in order of importance)
  1. Product Manager,
  2. C-Level/Decision maker,
  3. Developer,
  4. Author (of questions/assessments).
Assume prospective client can be from any of our verticals, and this is an entry point. So don’t speak to specific APIs or unique Learnosity terms. Keep language accessible to the first-timer.
User Stories:
  • As a generalist Product Owner or C-level executive (CEO, CTO, etc), I want to find out why using Learnosity would provide a better, more engaging experience for my learners and instructors. I’m also interested in the business advantages my organisation could gain by spending money on Learnosity. I’m likely researching a purchasing decision.
  • As an author or developer I’m wishing to quickly understand what a feature provides. (I might have come to this feature page from a link somewhere else in Learnosity docs).

Content Do's:
  • Do talk about meaningful products/features.
  • Every feature should be presented with the value it brings to potential customers and an example where appropriate. What does it enable users to do and why is this useful.
  • Reinforce Learnosity's expertise in the area if possible and appropriate.
Content Don'ts:
  • Don’t capitalise Learnosity terms, such as “Question”.
  • Don't talk about specific APIs or lower-level components.
  • Don't use unique "Learnosity-specific lexicon" here with its special meanings and interpretations. As far as possible, use general plain English to describe the subject matter.
  • Don't use specific industry jargon or buzzwords. No "web-scale", "MCQ", "KPIs", "Cloze", "RAID", "Modal", etc. Spell out acronyms, so “MCQ” becomes “multiple-choice questions”.
Demos:
  • Use live polished demos where appropriate with succinct and relevant context.

Site Map

We have a complicated site, with a lot of content.
  • We should create a site map page.
  • This task should also cover creating consistent header and footer information between sites.
  • We should also be considering the User Journey and restructuring as required.

Conceptual Overviews

Description: A group of overviews which bridge the gap between Marketing and Technical documentation by introducing technical concepts.

Overviews:
  1. Conceptual Overview - Learnosity Pillars: A high level overview which introduces the Authoring / Assessment / Analytics pillars.
  2. Conceptual Overview - Hierarchy of Learnosity components: A high-level overview which introduces the Barbusca doll of Questions, Features, Items, Assessment, Web Page.
  3. API Relationship Overview: Extend API Overview to show what API embeds in the other.
  4. Infrastructure Overview. What lives in the cloud, what is in the browser? How do you develop and then manage a Learnosity based site?
  5. Learnosity Web Development Overview: Explain functionality from an Developer's perspective, “Hello World” description. Introduce use of HTML, JS libraries, CSS. Leads into Quickstarts.
  6. Learnosity Author's Overview: Explain functionality from an Author's perspective.
  7. Learnosity Teacher's Overview: Explain functionality from an Teacher's perspective.

Quick Starts

Description:
Provide one very simple example, that it is easy to copy, understand, and get running.

Target Persona(s):
  • Product owner or trusted developer who has enough programming experience to read and understand code.
  • Might be an ex-programmer with dated programming experience.
  • Currently has very little exposure to Learnosity documentation or terminology.
User Stories:
  1. As a Product Owner or Trusted Techie, I want to Skim Read the API Quickstart Text so that I can quickly assess the effort required to implement the API and whether it might fit into my system.
  2. As a Trusted Techie, I want to implement the API Quickstart so that I can verify that it would be easy to implement the API and modify to my purposes.
Doc Structure:
  • Overview:
    • What this quickstart is covering.
    • Introduction to the technology and concepts.
    • Time to complete.
  • Prerequisites
  • Download
  • View example
    • Explain each section of the code, step by step.
  • How it works
  • Next steps / Related links
Provide:
  • Screenshots
  • Links to code that can be cut and pasted
  • Link to an environment which can be downloaded and deployed
Key Message(s) to convey:
  • How to get the simplest version of an API running
  • This API is easy to set up and run
  • High quality (of code and docs)
Content-Length: Run within 10 minutes

Reference Docs

Description:
Provide automatically generated function and parameter descriptions (presented in a similar format to javadocs). Reference material is product-centric. It aims to describe every public method and attribute of a public product.

Target Persona(s):
  • Software developers who are implementing Learnosity products, who already have an understanding of Learnosity product terminology.
User Stories:
  1. As a Software Developer who is implementing a Learnosity API, I want to understand the specific API options so that I can customise the Learnosity products I am integrating into my product.
Doc Structure:
Include tables of <param> : <description> : <options> for:
  • Initialization Options
  • Event Options
  • Public Methods

Tutorials

Description:
Provide a working description on how to implement a specific feature, sub feature or task.
Target Persona(s):
  1. Software developer, with at least a limited understanding of Learnosity products.
User Stories:
  1. As a Software Developer, I want to learn how to implement a specific Learnosity feature/sub-feature/task, so that I can easily implement it into my specific software system, and play with different configuration settings.
Doc Structure:
  • Title: A concept, such as “Styling” or feature such as “Paper based assessments”
    • The Title should help identify “why” a person will want to read this Tutorial.
  • Overview
    • Describe what this quickstart is covering in ~ two sentences.
    • Where appropriate, include one screenshot, or a collage, or diagram to help quickly convey meaning.
    • Time to complete
  • Example
    • Provide a reference to an example of this functionality working. This should be available within www/Features and/or Author Guide.
    • It is better to reference an example than to embed it. It helps set a common language for authors to talk to developers, and also reduces maintenance of documentation.
    • This example should also be available as a downloaded zip file.
  • Background
  • Introduction to the technology and concepts.
  • How it works
    • Provide link to example source code.
    • Explain concept and goal for each step.
    • Provide screenshots where appropriate.
    • Show code snippets and explanation.
  • Next steps / Related links

Training Course

Description:
At some point we will have a need for formal training courses. The material should draw heavily from Tutorial material. The training can be used for onboarding new employees initially.

Author Guide

TBD

Developer Guide

Description:
The developer guide consists of a series of documents, with each document explaining a specific “concept”, such as how to implement specific functionality. Documents should aim to build upon each other, starting simple and getting more complex. As required, documents should cross-reference each other.
What makes it a guide:
  1. Technically explains “how to” do something (as opposed to sales messages)
  2. Deals with core functionality as opposed to edge cases.
  3. Deals with concepts which are common to multiple APIs.
  4. Deals with multiple concepts rather than a single one.
  5. More than just “extended technical information”.
Target persona(s):
The target persona and expected prior knowledge will be different for each guide section. Typical personas will include:
  1. Competent software developer who is unfamiliar with Learnosity and needs to learn new Learnosity concepts.
  2. Software developer with Learnosity experience, wishing to implement an advanced Learnosity feature.
Doc Structure:
The Doc structure will be similar to Tutorials.

Knowledge Base

Description:
Describe details required to understand and implement an attribute/feature which is too complex to explain within Learnosity's reference documentation and too domain-specific to warrant the effort of writing a tutorial or solution.
Target persona(s):
  1. Software Developer who already has already set up a working Learnosity development environment and is familiar with Learnosity documentation.
User stories:
  1. As a Software Developer , I want to understand concepts behind a specific Learnosity feature so that I can implement the associated functionality.
Length: 1/2 - 2 pager (200 - 1000 words)
Doc Structure:
  • Topic: A concept or feature or attribute within a specific Learnosity API
  • Overview
    • Describe what this Knowledgebase is covering in 1 to 2 sentences.
    • Where appropriate, include 1 screenshot, or a collage, or diagram to help quickly convey meaning.
Guidelines:
  • Don’t explain something that should be assumed knowledge.
  • Focus on one concept. If you need to explain another feature or method etc., link to it.

Glossary

Description:
A table describing all the acronyms and lexicon used by Learnosity.
Learnosity ministry of Lexicon:
In moving forward, all new Learnosity terms should be put into the Glossary. For a term to be added to the Glossary, it should be considered by the “Ministry of Lexicon” and interested parties should be provided with the opportunity to refine lexicon selected.
Target Personas:
  1. Anyone involved in Learnosity.
User stories:
  1. As someone using Learnosity, I’d like to understand specific acronyms and terminology used by Learnosity.
Guidelines:
We should be hyperlinking to acronyms and terminology used by Learnosity, which should reduce replicated descriptions, and make content cleaner and easier to read.

Tooltips

Description:
Embedding tooltips enables information to be delivered “just-in-time”, enabling it to be available as it is required. Tooltip text would come from the Glossary.

FAQ Troubleshooting - Error Codes

Description:
Provide a table of error codes, descriptions, and steps to follow when receiving these codes.
Target personas:
  1. Partner developer, product developer, end-user (teacher/student)

Release Logs

Description:
Describe all the fixes and additional features added in the latest release.
Target Persona(s):
  1. Product owners and Techies with deep understanding about how their products are using Learnosity functionality.
User Stories:
  1. As a Product Owner making use of Learnosity Tools, I want to know what features have been updated/fixed, so that I can assess whether there is a business driver for me to update to the latest release of Learnosity.

Community Forums

Description:
Provide community forums, with the aim of having bi-directional conversations. Typically:
  • Developer asks question.
  • Learnosity polls community about future development direction.
  • Community/Learnosity provides multiple answers.
  • Community votes up best answers.
  • Where possible, answers point to existing docs. Popular topics should be “matured” to Knowledge Base articles or Guides.
Vision for community forum:
We care about our customers and they care about us. Mutual respect and reciprocity is part of what has made us successful. It is about sharing what is easy for you, and receiving what is easy for someone else. Our aim for the community is to scale our relationship of mutual caring and support those in our communities who are wanting to help each other.

Community Win Formula:
  1. Demonstrate commitment
  2. Everyone gets more than they put in
  3. Need help
  4. Expect help
  5. Appreciate help
  6. Simplify contributions
  7. Engage
  8. Care about each other
Value Goals:
  • Have our users feel a sense of ownership and loyalty for our Learnosity product:
    • By contributing to the direction of Learnosity.
    • By feeling part of a team.
    • By helping vote up priority features.
  • Ensure everyone in our community gets more out than they put in.
    • Help our user communities connect with and help each other.
  • Reduce support costs by:
    • Having users search for answers before asking questions.
    • Have user community answers other user questions.
  • Provide a low-cost sale which only has community support (no paid support)
  • Provide a location for our userbase to raise ideas, vote up the importance of ideas, which in turn can be used to collectively lobby Learnosity for feature development.
  • Provide a space to collate random ideas about Learnosity functionality without requiring professional quality control and review.
  • Provide a means for users to search for related problems.
  • Gain insight into our user's pain points from user usage patterns.
Implementation Goals:
  1. Keep it simple.
  2. Grow organically, starting with just one or two forums.
  3. Best practice suggests attract 5-10 posts/day to achieve a healthy community. Don’t split until you get 30 posts/day.
  4. Minimise extra work for Learnosity, by continuing to do what we are already doing - but moved to be within a community forum. In particular by moving many answers from the support team into community pages.
  5. Soft launch with “Beta” stamp.
  6. Only attempt to engage current customers/partners/authors (initially).
  7. We need to provide the opportunity for our community to engage and feel needed:
    1. If we answer every question straight away, we don’t create a space for others in the community to build a reputation and find a space to engage.
    2. Community engagement needs to result in positive action from Learnosity. It needs to be valued and valuable. We need to trust our community and implement their recommendations, otherwise don’t ask. Respect their time.

Wednesday, 28 March 2018

What could Open Government learn from us Open Technology folks?

Despite open government’s best intentions to prioritise collaboration, government bodies consistently duplicate each other’s effort. Collaborating as effectively as open communities is much harder than you’d think.

A number of us “open technologists” have drafted a paper describing the challenges government faces, along with our vision for how to address these. It is being presented as part of Australia’s updated Open Government National Action Plan.

Reading time: 20 minutes (8 pages).

Open letter

This letter is presented on behalf of the citizens, technologists and organisations signed below.

When addressing the updated Open Government National Action Plan, and actions from the plan, we request stakeholders:
  • Acknowledge that the indicators for success are more than just “value for money” and “mitigation of risk”. 
  • Measure and prioritise: 
    • “Effectiveness of collaboration”, 
    • “Sustainability in the face of rapid innovation”, and 
    • “Resilience to monopolistic behaviours”. 
  • Develop an “Open Government Maturity Model” which describes open government goals and the processes required to achieve them.
  • Measure effectiveness at realising open government goals.
  • Arm decision makers with accessible, evidence based research into what works, so they can trust, select and defend collaborative strategies which are often counter-intuitive within traditional hierarchically managed organisations.
  • Use, extend, or create open technologies, in that order:
    1. Use existing open material if it exists;
    2. Otherwise extend and give back;
    3. As a last resort, create your own system.
  • Embrace modular architectures backed by open standards.
  • Prioritise initiatives which can attract and sustain participation from multiple contributors and organisations.
  • Promote collaboration between all levels of government, and between nations.
  • Invest in the communities of the projects you depend upon. Ensure there is funding to maintain a core team. Reduce barriers to entry in order to attract a wide contributor base. Develop indicators for reporting on the success of these investment strategies.
  • Consider strategies to flatten government’s spending cycles, especially for community based projects. 
  • Prioritise agile, iterative development methodologies over “big bang”, “whole of government” purchases.

Background reasoning

Democracy is founded on collaboration

Democratic governments are based upon collaboration. They work on behalf of citizens, for their citizens’ benefit. Based on this social mandate, the Australian government committed to the principles of open government in 2010, and signed up to the international Open Government Declaration in 2015. This declaration emphasises how openness and technology is to be used to make governments more collaborative, transparent, accountable, responsive, effective, innovative, and empowering of citizens.

However, by 2018, government bodies are regularly not collaborating, even though individuals involved want to. Why? Old acquisition processes which prioritise "value for money" and "mitigation of risk" inadvertently cause agencies to duplicate effort. In the digital economy, success indicators additionally include “effectiveness of collaboration”, “sustainability in the face of rapid innovation” and “resilience to monopolistic behaviours”. If governments are to collaborate as effectively as “open” communities, like open source software or Wikipedia, we need to use these additional indicators.
Recommendation 1: Develop an “Open Government Maturity Model” which describes open government goals and the processes required to achieve them.
Such a model should draw from open community processes, such as the Apache Foundation’s open source incubation processes.

The allure of “open”

In the early days of the “open” movement, Eric Raymond prophesied in The Cathedral and the Bazaar:
“Perhaps in the end the open-source culture will triumph not because cooperation is morally right or software hoarding is morally wrong ... but simply because the closed-source world cannot win an evolutionary arms race with open-source communities that can put orders of magnitude more skilled time into a problem.”
Successful open communities have shown it is possible to attract more contributors from outside the organisation than can ever mustered from within.
In the digital economy, collaboration out-competes competition.

Principles of the digital economy

In the digital economy, free copying, free tools, and the interconnectivity of the Internet has made it possible to tap into the world’s collective intelligence. This has led to:
  • Exponential information growth;
  • Exceptionally complex systems;
  • Rapid innovation;
  • And on the flip side, rapid obsolescence.
While it will always be tempting to build your own system, any self-built system will likely be out-innovated and become obsolete.
Recommendation 2: Use, Extend, or Create open technologies, in that order:
  1. Use existing open material if it exists;
  2. Otherwise extend and give back;
  3. As a last resort, create your own, share, and try to attract collaborators.

“Open” is just the start

When adopting open technologies, we embrace “free access” to software, standards, and data. But there is more:
Openness is an enabler. It minimises legal and technical barriers to collaboration and sharing. Sharing ideas spawns more ideas and supercharges innovation. The reciprocity practices prevalent in collaborative communities inspire and empower individuals to contribute what they are passionate about, achieve their full potential, and collectively we all benefit.

The “community litmus test”

The Apache software foundation, like many open source foundations, emphasise the importance of diverse and sustainable communities for each of their projects. This applies the “wisdom of crowds” to validate the value and viability of each project. We believe governments should develop and apply similar criteria to validate the technical viability and community interest in projects they take on.
Recommendation 3: Prioritise initiatives which can attract and sustain participation from multiple contributors and organisations.

“Copying” is not “collaboration”

The Australian Digital Service Standard proudly states that it has been “adapted from the UK Government Design Principles”. This statement highlights a flaw in government’s approach to open principles. “Copying” instead of “collaborating” breaks a core principle of information management:
Retain a single point of truth.
Government employees should be reaching out to their counterparts to collaboratively harmonise policies, processes, guides, best practices, software and more.

“Government collaboration” is not as good as “open collaboration”

Collaboration in the “open” sense involves reciprocity, sharing, peer production, community building, trust, communication, inclusiveness, standards based interoperability, sustainability, and meritocracy.  Everyone involved is empowered to “scratch an itch”, develop an idea, and the community adopts the best ideas. This empowerment is a formula for rapid innovation.
However,
Traditional management views open collaboration as time consuming, imprecise, unreliable, hard to manage, rarely addresses short term objectives, hard to quantify in a business case, and rarely mentioned in acquisition guides. Yet, in a digital economy, collaborative communities are regularly out-innovating and out-competing closed or centrally controlled initiatives.
By contrast, Government’s interpretation of collaboration has typically been based on the International Association for Public Participation (IAP2) using a spectrum of:
  • Inform: You will be told;
  • Consult: Your concerns will be considered;
  • Involve: Your concerns will be options;
  • Collaborate: Your advice will be sought;
  • Empower: You will decide what we implement.
There is no mention of co-development. In all these cases, the sponsoring agency still controls the process; still controls the allocation of funds; and still controls the management of labour. Bureaucratic overhead typically hampers contributions from external individuals or agencies. And here we encounter one of the subtle differences between open communities and open government:
Governments currently organise labour through command and control hierarchies while open communities typically coordinate themselves loosely around principles of self-direction, co-development, volunteering and reciprocity.
When the Digital Transformation Agency was being launched, Malcolm Turnbull (who is now Prime Minister) stated,
"I'm a great believer in being much more global in our approach, ... we're all dealing with the same problems, pretty much. ...  We want to break down silos, break down all of the inertia that comes from empire building, so that citizens or businesses will have a seamless, straightforward way of dealing with government -- federal, state, or local."
The first Open Government National Action Plan focuses at the national level, without mentioning state or local government.
Recommendation 4: Promote collaboration between all levels of government, and between nations.

“Open” by itself is of little value

The Australia’s Digital Services Design Principle 10 states: “Make things open: it makes things better.” As such, agencies have been publishing open datasets and software but have not been evaluating if they are being used effectively.
Making things open and hoping they will be used is like talking into the void and hoping others will hear. It is hit-and-miss.
A digital asset only realises its value once it is discovered, and then integrated with other systems. The more widely it is used and extended, the more valuable it becomes.
Recommendation 5: Measure effectiveness at realising open government goals.

Loving a community to death

Collaborative projects are susceptible to being “loved to death”. This happens when a project attracts an active user base without attracting matching contributions. The core team becomes overwhelmed, leaving insufficient capacity to cover essential operation and maintenance tasks.
Organisations shouldn’t overload a community they depend upon. As well as being not nice, it is bad business. Successful open projects have worked out how to apply a combination of:
  1. Politely saying “no” to “gifts” of unsupported extra functionality;
  2. Helping users become contributors, either in kind or financially;
  3. Minimising the onboarding effort for both contributors and the project’s core team.
If a sponsoring organisation isn’t ready to act as a good community citizen, actively supporting the long term sustainability of a project, then the sponsor will probably have a disappointing experience. The sponsor will make self-centered, short-term decisions, and won’t get the support required when most needed. The sponsor will likely be better off with proprietary systems, and the open community would be better off without the sponsor.

Attracting community

A team from the University of Massachusetts researching the success characteristics of open source projects found that projects which were successful at startup typically possessed:
  1. A clearly defined vision;
  2. Clear utility;
  3. And leaders who led by doing.
The projects which grew tended to:
  1. Attract larger user communities;
  2. Attract external developers, with half attracting a developer from another country;
  3. Provide fine-scaled task granularity, making it easier for people to contribute;
  4. And often attracted financial backing.
There are two approaches to attracting co-contributors to complex systems:
  1. Design modular systems, with fine-scaled task granularity, minimal ramp-up effort, and attract many contributors.
  2. Cultivate and retain core contributors who contribute across multiple years of involvement.
There is an inverse relationship between episodic and core contributors. Making episodic contributions trivial creates work for the core team, and vise-versa. Key to success is sustaining a core team focused on attracting and simplifying episodic contributions.
Recommendation 6: Invest in the communities of the projects you depend upon. Ensure there is funding to maintain a core team. Reduce barriers to entry in order to attract a wide contributor base. Develop indicators for reporting on the success of these investment strategies.

Modularity and standards

A key strategy for managing complexity is to divide large systems into modular subsystems. It means you can improve one module, without impacting the rest of your system. This helps with maintenance, innovation, and keeping up with latest technologies.
Recommendation 7: Embrace modular architectures, backed by open standards.
Modular systems include the following advantages:
  • Enable interoperability;
  • Facilitate collaboration;
  • Reduce system complexity;
  • Mitigate risks of obsolescence and vendor lock-in;
  • Facilitate sustained innovation.

Destabilising effects of episodic spending

Open projects are vulnerable to the destabilising effects of episodic spending.

Organisations are often willing to pay a once-off fee to add extra features to a project, but are reluctant to pay for core project maintenance. Such investment results in high ramp-up costs as developers come on board, and then a loss of expertise when sponsorship ends. Technical debt is created for the new software without resources to maintain it.

Governments are prime culprits of episodic spending. Government budgets are managed around the financial year, with delayed budget approvals, resulting in discretionary spending centered around the last quarter of the year.

Proprietary business models are better structured to handle episodic funding. They can legally restrict software use unless a fee is paid, enabling spreading of development costs over time. Consequently, government’s propensity toward episodic spending inadvertently favours proprietary over open business models.
Recommendation 8: Consider strategies to flatten government’s spending cycles, especially for community based projects.

Fragmented spending

Government taxes are collected centrally then split between departments, then split between divisions, then split between teams, and so on, until a group is funded to address a requirement. This hierarchical breakdown of budgets is appropriate for funding physical tasks such as building roads or collecting garbage; however, for the implementation of generic software functionality, it is usually more efficient for agencies to pool resources and collaboratively work on a common code base.
Fragmented spending results in narrow, short term solutions instead of solving broad, holistic and long term problems.

Think agile instead of “big bang” purchasing

It’s tempting to address “fragmented spending” by aggregating budgets from multiple agencies into a central “whole of government” contract. From an accounting perspective, you’d think that the easiest way to acquire technology is by defining scope, acquiring budget, insource or outsource the work, and then manage the developers implementing the specification.

However due to the complexity of IT, developers and users will continuously provide ideas for improvement. Projects which adopt agile development methodologies, which continuously adjust direction to incorporate feedback, have a track record of producing better quality outcomes than traditional “waterfall” acquisition methodologies prevalent within government.  
Recommendation 9: Prioritise agile, iterative development methodologies over “big bang”, “whole of government” purchases.

Chasing funds instead of collaborators

Open source communities typically become sustainable and scale by attracting a growing pool of collaborators. Government projects typically become sustainable and gain prominence by attracting funding and “empire building”.  Collaborating and sharing credit with external organisations typically weakens the importance of the individuals and teams. We need to adjust recognition and incentives to reverse this.

Call to Action

We need to recognise that government agencies are consistently duplicating effort; that government approaches to collaboration have typically been sporadic and unsustainable; and that best practices in open communities exist but are not readily available to government decision-makers.
Recommendation 10: Arm decision makers with accessible, evidence based research into what works, so they can trust, select and defend collaborative strategies which are often counter-intuitive within traditional hierarchically managed organisations.
Research and guides should be developed collaboratively, between agencies, nations, organisations and citizens.

We need to be bold enough to challenge widely established practices; we need to be aware of established wisdom; we need to be opportunistic and pragmatic; and we need to have the insight to know when to choose one over the other.

Related Reading

  1. Shorter, Cameron (August 2017). Making GovHack (and Open Government) more impactful.
  2. Ward, Dan (October 2011), Lieutenant Colonel, US Air Force. Acquisition Lessons from a Galaxy Far, Far Away.
  3. United States Assistant Secretary of Defense (Networks & Information, Integration) / DoD Chief Information Officer and the Under Secretary of Defense for Acquisition, Technology, and Logistics (May 2011). Open Technology Development: Lessons Learned & Best Practices.
  4. U.S. Public Participation Playbook
  5. Australian Government Digital Transformation Agency. Digital Service Standard.
  6. Australian Government Digital Transformation Agency. Design Principles.
  7. Australian Government Prime Minister and Cabinet. Open Government National Action Plan 2016-18
  8. Waugh, Pia (January 2015). Collaborative innovation in the public service: Game of Thrones style.

Signed

If you are a technologist and agree with this vision, please add your technical credibility by signing. Add a comment below, or email <cameron . shorter AT g m a i l .c o M>. Diverse support will help sponsors wanting to back this vision.

Australia:
  1. Cameron Shorter, Technology Demystifier; spent over a decade consulting to government on implementing Gespatial Open Source Software, Open Standards and Open Data; ex board member of the Open Source Geospatial Foundation (OSGeo); mentor in OSGeo Incubation committee; co-author of OSGeo Incubation processes; co-founder of OSGeoLive Open Source project.
  2. Nicholas Gruen, Chair, Open Knowledge Foundation (Australia), CEO of Lateral Economics, Former Chair of the Australian Government 2.0 Taskforce (2009) and of Innovation Australia in 2013-14.
  3. Arjen Lentz, Exec. Director Open Query Pty Ltd; former Community Relations Manager, MySQL AB; co-founder, Open Source Industry Australia, Inc
  4. Lev Lafayette, President, Isocracy Network; former President, Linux Users of Victoria, 2011-2014
  5. Steven De Costa, Steering Group member, CKAN Association (Open Source Data Portal) & Executive Director of Link Digital
  6. Bruce Bannerman, Director, GeoInnovations Pty Ltd; ex IT Manager, Australian Federal Government; Charter Member Open Source Geospatial Foundation (OSGeo); Mentor in OSGeo incubation committee; Former voting member, Open Geospatial Consortium Technical Committee.
  7. Stuart Guthrie, Co-CEO, Polonious Pty Ltd, Former President Open Source Industry Australia, Open Source Software developer and business owner. Current business, based on Open Source Software with offices in Australia, the US and the UK. Verticals in Case Management, Industries: Banking, Insurance, Government, Universities and Schools.
  8. Luke Carbis, Director of Product & Innovation at XWP.
  9. Evan Leybourn, CEO, Business Agility Institute.
  10. John Bryant, Principal, Mammoth Geospatial & Co-Chair, Free and Open Source Software for Geospatial and State of the Map Oceania Conference, 2018.
  11. David Collins, Independent Developer, Trilobite Solutions
  12. Alex Leith, Principal Spatial Analyst, Auspatious and Co-Chair, Free and Open Source Software for Geospatial and State of the Map Oceania Conference, 2018
  13. Andrew Pam, President, Linux Users of Victoria 2014-present
  14. Tristan Gutsche, Innovation and Integration Architect.
Other nationalites:
  1. Brent Wood, Information Delivery Programme Leader, NIWA. Ex Council member, New Zealand Open Source Society (NZOSS), Charter Member Open Source Geospatial Foundation (OSGEO).
  2. Ivan Minčík, LINZ Spatial IT Solutions Architect. After many years of working for government in Slovakia and 2 yrs of work in New Zealand I am very happy to sign this document.
  3. Dirk Frigne, president Open Source Geospatial Foundation Europe (OSGeo Europe vzw), OSGeo Charter member, Former Vice president of  OSGeo, CEO Geosparc nv.
  4. Jo Cook, Astun Technology, UK, OSGeo advocate, founder and chairman of Open Source Geospatial Foundation (OSGeo) UK local chapter, vice chair FOSS4G 2013 international conference (Free and Open Source Software for Geospatial)
  5. Vasile Crăciunescu, researcher Romanian National Meteorological Administration, Open Source Geospatial Foundation board member.
  6. Suchith Anand, founder of Geo4All, the Open Source Geospatial Foundation's network of educational institutions.
  7. Patrick Hogan, NASA WorldWind Project Manager, NASA
  8. Charles Schweik, Professor, School of Public Policy, University of Massachusetts, Amherst
  9. Sascha Meinrath, founder of the Open Technology Institute in Washington DC; Director at X-Lab; Palmer Chair in Telecommunications, Penn State University. 
  10. Aaron Rodericks, Lead, Open Innovation, Open Government, Chief Information Officer Branch, Treasury Board of Canada Secretariat, Government of Canada

See Also



Text in this document is licensed under the Creative Commons Attribution 4.0 Licence.

Monday, 29 January 2018

Sustainability challenges with episodic contributions

Following on from an article by Ann Barcomb, I advised on managing the challenges of episodic contributions within open source projects.

Episodic volunteering challenges can be broken down into:
  • Attracting collaborators.
  • Maximising the value of each contribution.
  • Minimising collaboration overhead.
  • Managing sustainability.

Attracting collaborators

The OSGeoLive distribution of Geospatial Open Source software has successfully attracted and sustained over 100 episodic volunteers - developers, packagers, documentors, translators and occasional sponsors. When considering the community management of this project, we summarised the key characteristics for success as:
  • Start with a clear and compelling vision; inspiring enough that others want to adopt it and work to make it happen.
  • This should be followed by a practical and believable commitment to deliver on the vision. Typically this is demonstrated by delivering a “Minimum Viable Product”. 
  • Be in need of help, preferably accepting small modular tasks with a low barrier to entry, and ideally something which each person is uniquely qualified to provide. If anyone could fix a widget, then maybe someone else will do it. But if you are one of a few people with the skills to do the fixing, then your gift of fixing is so much more valuable, and there is a stronger moral obligation for you to step up.
  • Ensure that every participant gets more out of the project than they put in.
  • Avoid giving away free rides. If you are giving away something uniquely valuable; and it costs you time to provide that value for your volunteers; then it is ok to expect something of your volunteers if they wish to get something in return.
  • Use templates and processes to facilitate domain experts working together.
  • Reduce all barriers that may prevent people from contributing, in particular, by providing step-by-step instructions.
  • Set a schedule and work to it.
  • Talk with your community regularly, and promptly answer queries.
  • And most of all, have fun while you are doing it. Because believe you me, it is hugely rewarding to share the team camaraderie involved in building something that is much bigger and better than you could possibly create by yourself.
A team from the University of Massachusetts researched the success characteristics from thousands of Open Source projects and came to similar conclusions. They found that projects which were successful during initiation typically possessed:

found that projects which were successful at startup typically possessed: 
  1. A clearly defined vision;
  2. Clear utility;
  3. And leaders who led by doing.
The projects which grew tended to: 
  1. Attract larger user communities;
  2. Attract external developers, with half attracting a developer from another country;
  3. Provide fine-scaled task granularity, making it easier for people to contribute;
  4. And often attracted financial backing.

Software is a Liability

A naive misconception prevalent in the software industry is that “owning more software is better”. It’s not. The amount of information in the world doubles every two years or so, and the amount of software created is innovating at a similar rate. This means that most software will be out-innovated within a year or two unless it is continually maintained.
Your software is not an asset, it is a liability needing to be updated, maintained, and integrated with other systems. It is technical debt, and you should try to own as little of it as possible.
To ensure long term sustainability, it is increasingly important to spread the software maintenance cost, either by collaborating within an open source community, or by using a proprietary business model to aggregate license fees which are spent on a central product. Both models have their strengths and weaknesses, but for this essay we will focus on the open source model.

Cultivate your core team

Successful open source projects become large. With size comes complexity, which increases ramp up time and hinders casual contributions. The key strategies to address complexity are:
  1. Design modular systems, such that someone can quickly contribute to a specific task with trivial or minimal ramp-up effort.
  2. Cultivate and retain a core team who can spread ramp-up effort across multiple years of project involvement.
Established collaborative projects will have addressed both these use cases.

Episodic sponsorship

Open source projects are vulnerable to episodic sponsorship. Organisations are often willing to pay a once off fee to add extra features, but are reluctant to pay for core project maintenance. Such investment strategies are destabilising for a project’s core team as sponsored contributors tend to leave when sponsorship ends, leaving behind a technical debt of extra software without the acquired knowledge or resources to maintain the software.
Proprietary vendors are better structured for handling episodic funding. They can legally restrict software use unless a fee is paid, enabling Software-as-a-Service business models. License fees are then aggregated and spent sustainably over time.

Loved to death

Similarly, open source projects are susceptible to being “loved to death”. This happens when a project attracts an engaging user base without attracting matching contributions. The core team becomes overwhelmed leaving insufficient capacity to cover essential business-as-usual tasks.
Sponsoring organisations need to learn not to overload the community they depend upon. It is both bad karma and bad business. Successful open source projects have worked out how to apply a combination of:
  1. Politely saying NO to “gifts” of unsupported extra code and excessive requests for help;
  2. Help uses become contributors, either in kind or financially;
  3. Minimising the onboarding effort for both contributors and the project’s core team. 
If a sponsoring organisation isn’t ready to act as a good community citizen, actively caring about the long term sustainability of the communities they depend upon, then they will probably have a disappointing open source experience. They will make self-centred, short term decisions, and won’t get the support they need when they most need it. They will likely be better off with proprietary software. (And the community would be better off without them.)

Government agencies and episodic contributions

Governments deserve special mention. Governments all have a social mandate to provide services to their citizens. They are a prime candidate for engaging with the collaborative practices of open source software projects.
Drawing from these principles, governments around the world have been signing up to the Open Government Declaration which emphasises how openness and technology should be used to enable governments to be more collaborative, transparent, accountable, responsive, effective, innovative, and empowering of citizens. However, the episodic, fractured and short-term purchasing practices prevalent within government significantly undermines the collaborative potential between agencies and the communities that governments could be working with.
Let’s consider the key challenges.

Sporadic spending

Government budgets are managed around a financial year, with budget approvals often not being achieved until well into the year and not spent until the last quarter of the year. As explained earlier, such episodic spending is significantly destabilising to the people and companies who provide open source software services and favours proprietary business models instead.

Chasing funds instead of collaborating

Open source communities typically become sustainable and scale by collaborating rather than by chasing funding. Things are different within government agencies who are typically motivated to address short term goals with internal resourcing or by securing funding. The team’s importance is determined by the size of budget secured, not by the level of collaboration. In fact, collaboration and sharing the credit typically weakens the importance and need for the agency’s team.

Fragmented spending

Government taxes are collected centrally then split between departments, then split between divisions, then split between teams, and so on, until a group is funded to address a requirement. This hierarchical breakdown of budgets is appropriate for funding physical tasks such as building roads or collecting garbage; however, for the implementation of generic software functionality, it is usually more efficient for agencies to collaboratively work on a common code base. Fragmented spending results in solving narrow, short term problems instead of thinking holistically and long term.

Big Bang Purchasing

One of government’s solution to fragmented spending is “Big Bang” purchasing. Recognising a need, sourcing budget, going through an elaborate purchasing process, and engaging a primary contractor to provide proprietary software to implement the requirements. This methodology also has significant limitations, prone to project overruns, outdated and inflexible requirements, monopolistic practices which prevent information sharing and more.

Conclusion

There is a symbiotic relationship between episodic contributors and a project’s core team. A core team is needed to attract a community and simplify the contribution process. The community provides valuable contributions to the project. Everyone benefits from the product created.

Sunday, 26 November 2017

Tackling the Open Source dilemma




Here is the dilemma that you and your boss are faced with when considering Open Source:
Looked at through the lens of traditional management, Open Source collaboration is time consuming, imprecise, unreliable, hard to manage, rarely addresses short term objectives, and is hard to quantify in a business case. And yet, in a digital economy, collaborative communities regularly out-innovate and out-compete closed or centrally controlled initiatives.
So how do we justify following a more effective, sustainable, open and equitable strategy?



This is what we will be covering today:
  • The digital economy,
  • Complexity,
  • Trust,
  • Innovation and Obsolescence,
  • and what leads to Success or Failure.


The first thing to recognise is that the Digital Economy has fundamentally changed the rules of business. Ignore this at your own peril.
Zero Duplication Costs and the Connectivity of the Internet has led to Wicked Complexity, Rapid Innovation, and on the flip side, Rapid Obsolescence.

Let’s start by talking about Complexity.
Software systems have become huge, interdependent and complex.
It is no longer possible for one person to understand all of a system’s intricacies.
So decision makers need to assume, deduce and trust information provided by others.
It means that sourcing trustworthy advice has become a key criteria for success in the digital economy.
So how do we assess trustworthiness?

It turns out we all use a variant of this trustworthiness equation.

  • We trust people who are credible and who have have track record of providing reliable advice in the past.
  • We trust people who are open and transparent.
  • We trust ourselves, our family, our friends, because they look out for us, and we look out for them.
  • We are suspicious of people who stand to gain from advice they give us.

We also trust processes.
  • We trust that the democratic process leads to fair governance and management of resources.
  • We trust that the scientific method leads to reliable research that we should act upon.
  • We trust that the “survival of the fittest” competition of market economies leads to better products.

But we also know that all processes can be gamed.
And the more complex a system, the easier it is to bamboozle people and game the system.

Part of the reason Open Source has been so successful is that it’s characteristics lead to trustworthiness.
These characteristics include:
  • Freedom,
  • Altruism,
  • Openness,
  • Meritocracy,
  • and Do-ocracy.
Let’s break these down one by one.

Open source, by definition, provides the receivers of the software with the four freedoms:
  1. Freedom to use the software unencumbered; 
  2. Freedom to study the source code and find out how it works; 
  3. Freedom to modify, retask, and improve the code;
  4. Freedom to copy and share with others.
Providing such a valuable gift, which provides significantly more value to the receiver than to the giver, increases the trustworthiness of the giver.

Additionally, openness and transparency is almost universally applied to all Open Source development practices and communication.
  • Conversations are public; Everyone has the opportunity to join and contribute; 
  • Decisions are made openly; 
  • Issues and limitations are published and shared.
Being transparent and open to public critique reduces the potential for hidden agendas and creates trustworthiness.

In a meritocracy, the best ideas win, no matter who suggests them. It is the sign of an egalitarian community rather than a hierarchical or dysfunctional one.


With a do-ocracy the person motivated to do the work decides what gets done. In complex systems, the person closest to the problem will usually be best qualified to make the technical decisions.



A key strategy for managing complexity is to divide large systems into modular subsystems.
Using modular architectures, connected by open standards:
  • Reduces system complexity,
  • Enables interoperability,
  • Which reduces technical risk,
  • And facilitates sustained innovation.
It means you can improve one module, without impacting the rest of your system. This helps with maintenance, innovation, and keeping up with latest technologies.

Collaboration is a key focus of both Open Source and Open Standards narratives. Hence, successful Open Source applications usually provide exemplary support for standards.

By comparison, from the perspective of dominant proprietary companies, it makes business sense to apply vendor lock-in tactics, making cross-vendor integration difficult. Adoption of Open Standards threatens vendor lock-in tactics, and consequently dominant vendors are often reluctant and half-hearted in their support of Open Standards.

In the digital economy there are two dominant business models which work well.
Either:
  • You solve a generic problem by supplying an awesome "category killer" application which you distribute to the world; 
  • Or you provide personalised, specialised or localised services, typically using category killer applications.
There is a natural symbiotic relationship between the two.
If you are solving a generic problem, by yourself, you will be out-innovated!
There are simply more developers in the rest of the world than you can ever muster within your team.

Because software is so time consuming to create and so easy to copy, it is excessively prone to monopolies.
This holds true for both proprietary and open source products. A product that becomes a little better than its competitors will attracts users, developers and sponsors, which in turn allows that product to grow and improve quickly, allowing it to attract more users.
This highly sensitive, positive feedback leads to successful software projects becoming “category killers”.

This means that most of the software you own will be out-innovated within a year or two.
Your software is not an asset, it is a liability needing to be updated, maintained, and integrated with other systems. It is technical debt, and you should try to own as little of it as possible.
The question is: should you select Proprietary or Open Source as the alternative?


Openness democratises wealth and power, which is a good thing for all of us, even those with wealth and power.
Open Source and Proprietary business models differ in how their realised value is shared.
Open source licenses are structured such that multiple companies can use and support the same open source product, so the market self corrects any tendencies toward price-fixing.
Effectively, Open Licenses democratise information.
It enables everyone to share in the value created by technology.
As software markets mature, and components become generic commodity items, the collaborative practices of Open Source moves to becoming the most effective means for creating and managing functionality.
Collaboration trumps Competition for commodity items!

By comparison, the ruthless competition between proprietary companies results in “winner takes all” scenarios. Many of the richest people in the world are self made software entrepreneurs.
Jeff Bezos who started Amazon has recently been ranked as the richest man in the world, stealing the spot from Bill Gates who started Microsoft. Mark Zuckerberg who started Facebook comes in at number 5. Jack Dangermond from ESRI is down at #603, with a mere $3.2 billion dollars to his name.

Lets explain this another way, following the money trail. Proprietary business model favours multi-nationals who establish themselves in big markets such as in the US or Europe.
From our Australian software spend, a small commission is provided to the local sales guy and systems integrator, and the rest is funnelled into the multinational who often farms development into cheap development centres.


Open Source on the other hand favours local business. The software is free, so the majority of money spent is on support and integration type services, which is typically applied locally, keeping money and expertise local.



Let’s look into the characteristics which make projects successful or not.
Open Source projects are highly susceptible to being Loved to Death. This happens when a project attracts an engaging user base without attracting matching contributions. Volunteers become overwhelmed leaving insufficient capacity to cover essential business-as-usual tasks.
Don’t overload the community you depend upon. It is both bad karma and bad business.
Successful projects have worked out how to either:
  • Politely say NO to “gifts” of unsupported extra code and excessive requests for help;
  • Or how to help uses become contributors, either in kind, or financially.
If your organisation isn’t ready to act as a good community citizen, actively caring about the community’s long term sustainability, then you will probably have a disappointing Open Source experience. You will make self-centred, short term decisions, and you won’t get the support you need when you most need it. You will likely be better off with proprietary software. (And the community would be better off without you.)
The success criteria for Open Source projects was researched by Professor Charlie Schweik who studied thousands of projects. As you can see from this graph, most projects are abandoned. Of the remainder, most projects don’t attract more than one or two staff, and very few attract a large community.
Viewed another way, you can see that:
  • 4/5 projects are abandoned.
  • 1 in 7 remain with just 1 or 2 developers.
  • Only 1 in 100 manage to attract 10 core contributors.
On this graph we’ve drawn in the success rate for projects, and you can see that as you attract developers, your chance of long term success increases dramatically.
This is showing ruthless Darwinian evolution at work. Only projects of exceptional quality attract sustained growth and large communities. They fit in the “magic unicorn” category.



So how do you find these magic unicorn projects?
Charlie’s team distilled further insights from their research. They found that successful projects typically possess:
  • A clearly defined vision;
  • Clear utility;
  • And leaders who lead by doing.
Then projects which manage to attract a medium to large team tend to:
  • Provide fine scaled task granularity, making it easier for people to contribute;
  • And often have attracted financial backing.


To get insights into project health, you can look at Open Hub metrics.  This slide is from the OSGeo-Live project and shows the status of leading Desktop GIS applications.
And for QGIS you can see that it has a very healthy community with over 100 active contributors. 
Another strong indicator of a project’s success is whether it has completed an Open Source Foundation’s incubation process.
  • Quality
  • Openness
  • Community Health
  • Maturity
  • Sustainability


Bringing this all together into a concise elevator pitch for your boss:
  • The Digital Economy leads to High Complexity, Rapid Innovation and Rapid Obsolescence. Get with the program, or become obsolete.
  • Increased complexity requires us to trust more. So increase the value you place on trustworthiness, openness and transparency. 
  • Software is technical debt. It needs significant maintenance to remain current. Own as little of it as possible.
  • Collaboration and openness fast tracks innovation.
  • For the long term play, Collaboration trumps Competition. If you are solving a generic problem, by yourself, you will be out innovated! Value, recognise, select and apply collaborative practices.
  • Don’t be naive, most Open Source projects fail. Learn how to pick winners.
  • Openness and Collaboration leads to the democratisation of wealth and power. Learn how to be part of the community - it makes good business sense.



  • Slide deck is available online.
  • An earlier version of these slides was presented at QGIS Conference in Sydney, Australia, November 2017. It was restructured for print and published as an article in the August 2018 edition of Position Magazine.
  • The text behind these slides, by Cameron Shorter, is licensed under a Creative Commons Attribution 4.0 International License
  • If this presentation was of interest to you, then please let me know (use comments below or email address on the slide above). I enjoy hearing from people who share similar interests, or are facing similar challenges, and ideas people have on related topics. 
Related presentations: