Showing posts with label open government. Show all posts
Showing posts with label open government. Show all posts

Saturday, 2 November 2019

Scrappy Collaboration

Coordinate changes over time due to tectonic plate movement.
Confession: We broke government conventions, worked beyond our agency’s mandate, attended an international technical committee meeting on personal time, uncovered significant negative impacts for unsuspecting users in Australia’s multi-million dollar mapping program, and retrospectively received glowing praise and thanks.

Welcome to “scrappy collaboration”.

$225 million has been committed to improving Australia’s GPS positioning tenfold, to 5cm accuracy. However technical shortcuts previously embedded in web-mapping software, which ignore tectonic plate movements, are causing meter level map misalignments in web-mapping! This is an international problem which is being exposed by Australia’s advanced mapping program. Needless to say, misaligned maps are confusing for users and unacceptable for Australian mapping agencies.

When NSW Spatial Services discovered the project risks involved, and the minimal existing focus on the problem, we joined other Australian states, and worked beyond our agency’s mandate, to look for a solution.

An interrelated problem cluster
With investigation, we unravelled a cluster of interrelated problems:
  • Some of which had deep technical challenges;
  • With varying implications to diverse user groups and use cases;
  • Which were difficult to understand and explain;
  • With experts regularly talking past each other, each considering a different part of the problem;
  • With multiple holistic strategies possible (each strategy offering significantly different costs and benefits for each stakeholder group);
  • With high impact implications for software, data and users;
  • Requiring national and international collaboration between diverse stakeholders to resolve effectively.
Not my responsibility
The investigation turned out to be a significant time sink, distracting us from our core roles. If following a government playbook, we should have left the responsibility to others: Our software vendor, international standards foundations, national mapping organisations, ...

But we knew we had a moral obligation to step up. Our team understood the overarching problem; had access to geodetic surveyors and software engineers; and had personal connections with international experts and influencers from open-source and standards communities.

A scrappy approach
A complex, interdependent problem cluster has a high barrier to entry. Each stakeholder only understands and can influence a subset of the issues. No one has a business case to solve the greater problem for everyone. This is difficult to solve with traditional top-down management approaches.
Instead, we adopted a scrappy, very personal approach, trading our time, and collected knowledge with collaborators to achieve common goals.
  • We very publicly admitted the embarrassing problems we were facing in technical forums, and asked for help.
  • We shamelessly drew down favours from our friends in open source communities, government agencies, industry and standards organisations.
  • We spent extra effort coordinating contributions, answering questions, and addressing related problems relevant to our collaborators (because that is what friends do).
  • We concisely compiled descriptions of the problems, along with suggested solutions.
  • Joel Haasdyk, our datum modernisation manager and geodetic surveyor, then used personal leave to travel halfway around the world to present recommendations to the standards technical committee. 
When collaborating, we applied many of the hallmark characteristics prevalent in open source communities:
  • Respect the time of your collaborators; be concise in messaging.
  • Lead by example; put in the hard work.
  • Reach out to people; be vulnerable and ask for help.
  • Be clear about what you need and from whom.
  • Care about your collaborators; help solve their problems too.
Tap into your humanity - it has tangible business benefits.

Impact
The impact we achieved with collaborators was apparent from the reports coming from the standards community. Scott Simmons, Executive Director of the OGC Standards Program reported:
“I don’t know if I have ever seen a better and more compelling tag-team describe and lead discussion on a topic than Joel and Roger's presentation on the time-dependent limitations inherent in most OGC spatial standards. I received many emails and face-to-face comments from attendees afterwards stating that it was so valuable to understand the problem and see that there is a solution. Resulting from the presentation, the majority of OGC Standards Working Groups now understand the necessity to update their standards to account for time-dependence of coordinates, with some already initiating updates.”
Future standards development will consider tectonic plate movement and will start to address Australia’s map misalignment problems impacting our datum modernisation program. We are at the start of the journey, there is still plenty more to do, but we now are heading in the right direction.

Counterintuitive solution
So, when faced with complex problems, consider expanding scope rather than constraining it. Consider solving the problems of others as well as your own. Attract the collective wisdom and resources of collaborators and you'll likely find your solutions are more impactful, strategic and sustainable.

About the author
Cameron Shorter has been a Geospatial Business Analyst at NSW Spatial Services.


Friday, 12 October 2018

The Open Business - Business Case


Abstract:
Despite many attempts, large companies and governments’ rarely achieve the level of collaboration experienced by Open Source communities. Why?
Looked at through the lens of traditional management, Open Source collaboration is time consuming, imprecise, unreliable, hard to manage, rarely addresses short term objectives, and hard to quantify in a business case. And yet, in a digital economy, collaborative communities regularly out-innovate and out-compete closed or centrally controlled initiatives.
Backing successful collaboration within traditional business requires us to write compelling, counter-intuitive business cases which explain and justify the elusive practices of collaborative communities.
This presentation explains the subtle magic of open strategies in business terms, and will help you convince your boss to back them.

Presented at World Commons Week, November 2018. Youtube recording and slide deck are available online.



Fifty years ago Garrett Hardin described the tragedy of the commons,
where people acting in their own self-interest, inevitably will deplete or spoil a common resource, as each acts in their own self interest.
He argued that the tragedy can only be prevented by private property rights or government regulation.
And yet, within the last fifty years, we’ve discovered exemplar counter-examples where altruism trumps selfishness. Let’s look at the business case behind one of these examples - the Open Source movement.


Despite governments’ and businesses acknowledging the value of Open Source in policies and initiatives over the last decade. And despite numerous attempts. They rarely achieve the level of collaboration experienced by Open Source communities. Why?

Looked at through the lens of traditional management, Open Source collaboration is time consuming, imprecise, unreliable, hard to manage, rarely addresses short term objectives, and hard to quantify in a business case. And yet, in a digital economy, collaborative communities regularly out-innovate and out-compete closed or centrally controlled initiatives.
Backing successful collaboration within traditional business requires us to write compelling, counter-intuitive business cases which explain and justify the elusive practices of collaborative communities.
This presentation explains the subtle magic of open strategies in business terms, and will help you convince your boss to back them.
I’m going to focus on Open Source Software, but the principles translates across all types of Creative Commons - Open Data, Open Government, Open Standards, and other Open approaches.


This is what we will be covering today:
  • The digital economy,
  • Complexity,
  • Trust,
  • Motivations,
  • Innovation and Obsolescence.

The first thing to recognise is that the Digital Economy has fundamentally changed the rules of business. Ignore this at your own peril.
Zero Duplication Costs and the Connectivity of the Internet has led to Wicked Complexity, Rapid Innovation, and on the flip side, Rapid Obsolescence.

Let’s start by talking about Complexity.
Software systems have become huge, interdependent and complex.
It is no longer possible for one person to understand all of a system’s intricacies.
Solving problems requires the collective brain power of many people.

To understand this, we’ll introduce the Cynefin framework, developed by Dave Snowden when he worked for IBM Global Services. It describes how different decision methodologies should be applied at different levels of complexity.
The framework is broken into four decision-making domains.

The Obvious domain is the area of "known knowns".
The relationship between cause and effect is clear.
Establish the facts ("sense"), categorize, then respond, by following established rules and applying best practices.
This is the province of standard operating procedures and legal structures.
Beware of complacency, over-simplifying or creating volumes of processes and unwieldy red tape.

The Complicated domain is the "known unknowns".
The relationship between cause and effect can be deduced from analysis or expertise and there are a range of right answers.
Assess the facts, analyze, and apply the appropriate good operating practice.
This is the province of experts.
Be aware of the trustworthiness of advice, influence from vested interests, and balancing short versus long term goals.

The Complex domain is for the "unknown unknowns".
Cause and effect can only be deduced in retrospect, and there are no right answers.
Instructive patterns can emerge by conducting experiments that are safe to fail.
This is the provenance of hypotheses and the scientific method.

In the Chaotic domain, cause and effect are unclear, and events are too confusing to wait for a knowledge-based response.
Act to establish order; sense where stability lies; respond to turn the chaotic into one of the other domains.
This is the domain of firm leadership, tough decisions and action.

Open source collaboration has proven to be very effective within Complex and Complicated domains, which begs the question of “why”?
Why is an open approach so effective within complex domains, and conversely, why aren’t open approaches as dominant in Obvious and Chaotic domains?

Let’s start by looking at the characteristics of Open Source.

Firstly, most projects are abandoned, and of those that succeed, most only have a few developers, with the extra developers often coming from another country.

On this graph we’ve drawn in the success rate for the projects.
As you attract developers, your chance of long term success increases dramatically.
This is showing ruthless Darwinian evolution at work.

Effectively, Open Source is an environment where lots of competing ideas are tested.
Only projects of exceptional quality attract sustained growth and large communities.

And this is a key characteristic to notice. When you are giving away your software for free, success depends upon:
  • A compelling vision,
  • Clear utility,
  • And being so welcoming and caring that you attract and retain contributors.
The study also noticed that projects which grow:
  • Typically provide fine scaled task granularity, making it easier for people to contribute,
  • And often have attracted financial backing.
And here we start to uncover the magic of Open Source.
In the digital economy, there are more developers working on your problem, than you will ever have in your team. When projects can tap into this,
Collaboration out-competes competition!

Let’s look at one of the key factors in complicated systems - trust; and question what makes trustworthiness.

It turns out we all make use of a variant of this trustworthiness equation.
  • We trust people who are credible and have a track record of providing reliable advice in the past.
  • We trust people who are open and transparent.
  • We trust ourselves, our family, our friends, because they look out for us, and we look out for them.
  • And we are suspicious of people who stand to gain from advice they give us.

We also trust processes.
  • We trust that the scientific method leads to reliable research that we should act upon.
  • We trust that the competition of market economies leads to better products.
  • We trust that the democratic process leads to fair governance and management of resources.
But, we also know that all processes can be gamed.
And the more complex a system, the easier it is to bamboozle people, and game the system.

Part of the reason Open Source has been so successful is that it’s characteristics lead to trustworthiness.
These include:
  • Freedom and Altruism,
  • Openness,
  • Bottom up decision making,
  • Do-ocracy,
  • Meritocracy,
  • And modularity.
Let’s look at these in more detail.


… starting with Freedom and Altruism.
Open source, by definition, is given away for free, with the freedom to use and extend it as you see fit.
Why are open source developers so altruistic?
It turns out that it is wrong to assume that we humans are only driven by self interest.
As noted by Dan Pink in his book Drive, after our basic needs are met, we are also motivated by …

Autonomy. The desire to be self directed.

Mastery, the urge to get better at stuff.

And Purpose,
The desire to do something with meaning and importance.
Such altruistically motivated people, who provide significantly more value to the receiver than to the giver, increases the trustworthiness of the giver.

Then there is Openness.
Openness and transparency is almost universally applied to all Open Source development and communication.
  • Conversations are public; Everyone has the opportunity to join and contribute;
  • Decisions are made openly;
  • And issues and limitations are published and shared.
Being transparent and open to public critique reduces the potential for hidden agendas and creates trustworthiness.


And within Open Source communities, decisions tend to be made bottom-up rather than top-down.
When you can trust the motivations of your community, you are empowered use of bottom up decision making.

This is important, because in a complex system, the person closest to the problem is usually the best qualified to make decisions.

It creates a culture of “do-ocracy”.
Within a do-ocracy the person motivated to do the work decides what gets done. Their commitment is a better indicator of true value than a person at the top saying “someone should fix this”.

This leads into Meritocracy.
In a meritocracy, the best ideas win, no matter who suggests them. It is the sign of an egalitarian community rather than a hierarchical or dysfunctional one.

But we should be careful not to suggest that open practices easily solves all problems.
Open Source projects are highly susceptible to being Loved to Death. This happens when a project attracts an engaging user base without attracting matching contributions. Volunteer become overwhelmed leaving insufficient capacity to cover essential business-as-usual tasks.
Don’t to overload the community you depend upon. It is both bad karma and bad business.
Successful projects have worked out how to either:
  • Politely say NO to “gifts” of unsupported extra code and excessive requests for help;
  • Or how to help uses become contributors, either in kind, or financially.
If your organisation isn’t ready to act as a good community citizen, actively caring about the community’s long term sustainability, then you will probably have a disappointing Open Source experience. You will make self-centred, short term decisions, and you won’t get the support you need when you most need it. You will likely be better off with proprietary software. (And the community would be better off without you.)

Using modular architectures, connected by open standards:
  • Reduces system complexity,
  • Enables interoperability,
  • Which reduces technical risk,
  • it enables collaboration,
  • And facilitates sustained innovation.
It means you can improve one module, without impacting the rest of your system.This helps with maintenance, innovation, and keeping up with latest technologies.

Collaboration is a key focus of both Open Source and Open Standards narratives. Hence, successful Open Source applications usually provide exemplary support for standards.

By comparison, from the perspective of dominant proprietary companies, it makes business sense to apply vendor lock-in tactics, making cross-vendor integration difficult.
Adoption of Open Standards threatens vendor lock-in tactics, and consequently dominant vendors are often reluctant and half-hearted in their support of Open Standards.

Effectively, we are talking about monopolies.
Because software is so time consuming to create and so easy to copy, it is excessively prone to monopolies.
This holds true for both proprietary and open source products. A product that becomes a little better than its competitors will attracts users, developers and sponsors, which in turn allows that product to grow and improve quickly, allowing it to attract more users.
This highly sensitive, positive feedback leads to successful software projects becoming “category killers”.

This means that most of the software you own is likely to be out-innovated within a year or two.
Your software is not an asset, it is a liability needing to be updated, maintained, and integrated with other systems.
It is technical debt, and unless a product is part of your core business, you should try to own as little of it as possible.
The question is: should you select Proprietary or Open Source as the alternative?

Open Source and Proprietary business models differ in how their realised value is shared.

Open source licenses are structured such that multiple companies can use and support the same open source product, so the market self corrects any tendencies toward price-fixing.
It enables everyone to share in the value created by technology.

By comparison, the ruthless competition between proprietary companies results in “winner takes all” scenarios. Many of the richest people in the world are self made software entrepreneurs.
These are typically people and organisations who have mastered the Complex domain.

Let’s take a quick look into the indicators for successful open projects.

To get insights into project health, you can look at Open Hub metrics. In particular, look for signs of sustained collaboration and growth.

Another strong indicator of a project’s success is whether it has completed an Open Source Foundation’s incubation process.
I’ve been involved with the Open Source Geospatial Foundation’s incubation process where we look for indicators of:
  • Quality
  • Openness
  • Community Health
  • Maturity
  • And sustainability

Bringing this all together into a concise elevator pitch:
  • The Digital Economy leads to High Complexity, Rapid Innovation and Rapid Obsolescence. Get with the program, or become obsolete.
  • Increased complexity requires us to trust. So increase the value you place on trustworthiness, openness and transparency.
  • Software is technical debt. It needs significant maintenance to remain current. Own as little of it as possible.
  • For the long term play, Collaboration trumps Competition.
  • Truly care about your community, and they will care about you.
  • Learn how to describe the business case for good behaviour. It is counter-intuitive, but it is the foundation for long term successful business strategies.



© 2018 Cameron Shorter. The text behind these slides, is licensed under a Creative Commons Attribution 4.0 International License.

Saturday, 7 July 2018

Commenting on Australia's draft commitments to the Open Government National Action Plan

Australia has a draft of commitments for our Open Government National Action Plan, and have asked for comment. The plan has some wonderful goals, but I wish the authors had a more practical understanding of the workings of digital economies and open source communities. The commitments could be made so much more implementable and impactful. I've provided the following specific comments, along with a reference to my previous submission which is more comprehensive.

Improve the sharing, use and reuse of public sector data

Government's draft commitment: 
… consult across government, through the new National Data Advisory Council, with the Open Government Forum and with the public including businesses, civil society groups and research and non‑profit sectors. The National Data Advisory Council will be a multi-disciplinary expert panel drawn from public sector and civic society organisations.

My comment:
Sharing public data provides noble goals, but as written, government can claim this commitment complete merely by opening access to datasets without addressing the more challenging task of making sure the data is usable.
For this policy to be implemented effectively, it should tie back to measures of data use. This requires an understanding of the characteristics of data management, and should mention measures such as: quality, fitness for purpose, ability to integrate with other data sources, standards based data structures, relevance, usability, timeliness, sustainability of maintenance.

Enhance public engagement skills in the public service

Government's draft commitment:
  • Develop and implement an Open Dialogue Roadmap
  • The Establishment of an APS Engagement Hub
My comment:
In order to be impactful, this commitment needs to extend beyond dialogue and facilitating the sharing of ideas. It needs to extend to the more important issue of collaboration during acquisition, implementation and deployment.
We need to recognise that governments around the world are all solving very similar problems, and the most efficient way to address these problems is to work on them collaboratively. However, cross-agency and cross-jurisdiction collaborative implementation is the exception rather than the rule. This can usually be traced to the difficulty of explaining and applying collaborative techniques within government frameworks.
I recommend this commitment be extended to cover the providing of tools to measure and implement cross-agency collaboration. This commitment will require a lot of work, however many of the techniques needed can be derived from Open Source community processes.

Enhance state and territory participation in the Open Government Partnership

Government's draft commitment:
  • Establishing a framework and mechanisms to facilitate communication between state and territory governments and Australian Government officials responsible for Australia’s Open Government commitments to support collaboration and learning on open government matters, and highlight the opportunity for formal subnational cooperation and membership in the Open Government Partnership, and 
  • To demonstrate the benefits of enhanced collaboration on open government matters: engaging with state and territory Information Commissioners to seek agreement to conduct surveys to measure citizens’ awareness of the right to access government information, and their experiences and outcomes in exercising that right.
My comment:
Enhancing state and territory participation is a good start, but the scope of this commitment should be extended up to international collaboration, and down to local government and further. Note that there will always be more people with great ideas who are outside your organisation than can ever mustered from within. So with every policy and procedure you write, and every application you build, do it collaboratively. 
Use, Extend, or Create, in that order:
  • Use existing material if it exists;
  • Otherwise extend and give back;
  • As a last resort, create your own, share, and endeavour to attract collaborators.

Wednesday, 28 March 2018

What could Open Government learn from us Open Technology folks?

Despite open government’s best intentions to prioritise collaboration, government bodies consistently duplicate each other’s effort. Collaborating as effectively as open communities is much harder than you’d think.

A number of us “open technologists” have drafted a paper describing the challenges government faces, along with our vision for how to address these. It is being presented as part of Australia’s updated Open Government National Action Plan.

Reading time: 20 minutes (8 pages).

Open letter

This letter is presented on behalf of the citizens, technologists and organisations signed below.

When addressing the updated Open Government National Action Plan, and actions from the plan, we request stakeholders:
  • Acknowledge that the indicators for success are more than just “value for money” and “mitigation of risk”. 
  • Measure and prioritise: 
    • “Effectiveness of collaboration”, 
    • “Sustainability in the face of rapid innovation”, and 
    • “Resilience to monopolistic behaviours”. 
  • Develop an “Open Government Maturity Model” which describes open government goals and the processes required to achieve them.
  • Measure effectiveness at realising open government goals.
  • Arm decision makers with accessible, evidence based research into what works, so they can trust, select and defend collaborative strategies which are often counter-intuitive within traditional hierarchically managed organisations.
  • Use, extend, or create open technologies, in that order:
    1. Use existing open material if it exists;
    2. Otherwise extend and give back;
    3. As a last resort, create your own system.
  • Embrace modular architectures backed by open standards.
  • Prioritise initiatives which can attract and sustain participation from multiple contributors and organisations.
  • Promote collaboration between all levels of government, and between nations.
  • Invest in the communities of the projects you depend upon. Ensure there is funding to maintain a core team. Reduce barriers to entry in order to attract a wide contributor base. Develop indicators for reporting on the success of these investment strategies.
  • Consider strategies to flatten government’s spending cycles, especially for community based projects. 
  • Prioritise agile, iterative development methodologies over “big bang”, “whole of government” purchases.

Background reasoning

Democracy is founded on collaboration

Democratic governments are based upon collaboration. They work on behalf of citizens, for their citizens’ benefit. Based on this social mandate, the Australian government committed to the principles of open government in 2010, and signed up to the international Open Government Declaration in 2015. This declaration emphasises how openness and technology is to be used to make governments more collaborative, transparent, accountable, responsive, effective, innovative, and empowering of citizens.

However, by 2018, government bodies are regularly not collaborating, even though individuals involved want to. Why? Old acquisition processes which prioritise "value for money" and "mitigation of risk" inadvertently cause agencies to duplicate effort. In the digital economy, success indicators additionally include “effectiveness of collaboration”, “sustainability in the face of rapid innovation” and “resilience to monopolistic behaviours”. If governments are to collaborate as effectively as “open” communities, like open source software or Wikipedia, we need to use these additional indicators.
Recommendation 1: Develop an “Open Government Maturity Model” which describes open government goals and the processes required to achieve them.
Such a model should draw from open community processes, such as the Apache Foundation’s open source incubation processes.

The allure of “open”

In the early days of the “open” movement, Eric Raymond prophesied in The Cathedral and the Bazaar:
“Perhaps in the end the open-source culture will triumph not because cooperation is morally right or software hoarding is morally wrong ... but simply because the closed-source world cannot win an evolutionary arms race with open-source communities that can put orders of magnitude more skilled time into a problem.”
Successful open communities have shown it is possible to attract more contributors from outside the organisation than can ever mustered from within.
In the digital economy, collaboration out-competes competition.

Principles of the digital economy

In the digital economy, free copying, free tools, and the interconnectivity of the Internet has made it possible to tap into the world’s collective intelligence. This has led to:
  • Exponential information growth;
  • Exceptionally complex systems;
  • Rapid innovation;
  • And on the flip side, rapid obsolescence.
While it will always be tempting to build your own system, any self-built system will likely be out-innovated and become obsolete.
Recommendation 2: Use, Extend, or Create open technologies, in that order:
  1. Use existing open material if it exists;
  2. Otherwise extend and give back;
  3. As a last resort, create your own, share, and try to attract collaborators.

“Open” is just the start

When adopting open technologies, we embrace “free access” to software, standards, and data. But there is more:
Openness is an enabler. It minimises legal and technical barriers to collaboration and sharing. Sharing ideas spawns more ideas and supercharges innovation. The reciprocity practices prevalent in collaborative communities inspire and empower individuals to contribute what they are passionate about, achieve their full potential, and collectively we all benefit.

The “community litmus test”

The Apache software foundation, like many open source foundations, emphasise the importance of diverse and sustainable communities for each of their projects. This applies the “wisdom of crowds” to validate the value and viability of each project. We believe governments should develop and apply similar criteria to validate the technical viability and community interest in projects they take on.
Recommendation 3: Prioritise initiatives which can attract and sustain participation from multiple contributors and organisations.

“Copying” is not “collaboration”

The Australian Digital Service Standard proudly states that it has been “adapted from the UK Government Design Principles”. This statement highlights a flaw in government’s approach to open principles. “Copying” instead of “collaborating” breaks a core principle of information management:
Retain a single point of truth.
Government employees should be reaching out to their counterparts to collaboratively harmonise policies, processes, guides, best practices, software and more.

“Government collaboration” is not as good as “open collaboration”

Collaboration in the “open” sense involves reciprocity, sharing, peer production, community building, trust, communication, inclusiveness, standards based interoperability, sustainability, and meritocracy.  Everyone involved is empowered to “scratch an itch”, develop an idea, and the community adopts the best ideas. This empowerment is a formula for rapid innovation.
However,
Traditional management views open collaboration as time consuming, imprecise, unreliable, hard to manage, rarely addresses short term objectives, hard to quantify in a business case, and rarely mentioned in acquisition guides. Yet, in a digital economy, collaborative communities are regularly out-innovating and out-competing closed or centrally controlled initiatives.
By contrast, Government’s interpretation of collaboration has typically been based on the International Association for Public Participation (IAP2) using a spectrum of:
  • Inform: You will be told;
  • Consult: Your concerns will be considered;
  • Involve: Your concerns will be options;
  • Collaborate: Your advice will be sought;
  • Empower: You will decide what we implement.
There is no mention of co-development. In all these cases, the sponsoring agency still controls the process; still controls the allocation of funds; and still controls the management of labour. Bureaucratic overhead typically hampers contributions from external individuals or agencies. And here we encounter one of the subtle differences between open communities and open government:
Governments currently organise labour through command and control hierarchies while open communities typically coordinate themselves loosely around principles of self-direction, co-development, volunteering and reciprocity.
When the Digital Transformation Agency was being launched, Malcolm Turnbull (who is now Prime Minister) stated,
"I'm a great believer in being much more global in our approach, ... we're all dealing with the same problems, pretty much. ...  We want to break down silos, break down all of the inertia that comes from empire building, so that citizens or businesses will have a seamless, straightforward way of dealing with government -- federal, state, or local."
The first Open Government National Action Plan focuses at the national level, without mentioning state or local government.
Recommendation 4: Promote collaboration between all levels of government, and between nations.

“Open” by itself is of little value

The Australia’s Digital Services Design Principle 10 states: “Make things open: it makes things better.” As such, agencies have been publishing open datasets and software but have not been evaluating if they are being used effectively.
Making things open and hoping they will be used is like talking into the void and hoping others will hear. It is hit-and-miss.
A digital asset only realises its value once it is discovered, and then integrated with other systems. The more widely it is used and extended, the more valuable it becomes.
Recommendation 5: Measure effectiveness at realising open government goals.

Loving a community to death

Collaborative projects are susceptible to being “loved to death”. This happens when a project attracts an active user base without attracting matching contributions. The core team becomes overwhelmed, leaving insufficient capacity to cover essential operation and maintenance tasks.
Organisations shouldn’t overload a community they depend upon. As well as being not nice, it is bad business. Successful open projects have worked out how to apply a combination of:
  1. Politely saying “no” to “gifts” of unsupported extra functionality;
  2. Helping users become contributors, either in kind or financially;
  3. Minimising the onboarding effort for both contributors and the project’s core team.
If a sponsoring organisation isn’t ready to act as a good community citizen, actively supporting the long term sustainability of a project, then the sponsor will probably have a disappointing experience. The sponsor will make self-centered, short-term decisions, and won’t get the support required when most needed. The sponsor will likely be better off with proprietary systems, and the open community would be better off without the sponsor.

Attracting community

A team from the University of Massachusetts researching the success characteristics of open source projects found that projects which were successful at startup typically possessed:
  1. A clearly defined vision;
  2. Clear utility;
  3. And leaders who led by doing.
The projects which grew tended to:
  1. Attract larger user communities;
  2. Attract external developers, with half attracting a developer from another country;
  3. Provide fine-scaled task granularity, making it easier for people to contribute;
  4. And often attracted financial backing.
There are two approaches to attracting co-contributors to complex systems:
  1. Design modular systems, with fine-scaled task granularity, minimal ramp-up effort, and attract many contributors.
  2. Cultivate and retain core contributors who contribute across multiple years of involvement.
There is an inverse relationship between episodic and core contributors. Making episodic contributions trivial creates work for the core team, and vise-versa. Key to success is sustaining a core team focused on attracting and simplifying episodic contributions.
Recommendation 6: Invest in the communities of the projects you depend upon. Ensure there is funding to maintain a core team. Reduce barriers to entry in order to attract a wide contributor base. Develop indicators for reporting on the success of these investment strategies.

Modularity and standards

A key strategy for managing complexity is to divide large systems into modular subsystems. It means you can improve one module, without impacting the rest of your system. This helps with maintenance, innovation, and keeping up with latest technologies.
Recommendation 7: Embrace modular architectures, backed by open standards.
Modular systems include the following advantages:
  • Enable interoperability;
  • Facilitate collaboration;
  • Reduce system complexity;
  • Mitigate risks of obsolescence and vendor lock-in;
  • Facilitate sustained innovation.

Destabilising effects of episodic spending

Open projects are vulnerable to the destabilising effects of episodic spending.

Organisations are often willing to pay a once-off fee to add extra features to a project, but are reluctant to pay for core project maintenance. Such investment results in high ramp-up costs as developers come on board, and then a loss of expertise when sponsorship ends. Technical debt is created for the new software without resources to maintain it.

Governments are prime culprits of episodic spending. Government budgets are managed around the financial year, with delayed budget approvals, resulting in discretionary spending centered around the last quarter of the year.

Proprietary business models are better structured to handle episodic funding. They can legally restrict software use unless a fee is paid, enabling spreading of development costs over time. Consequently, government’s propensity toward episodic spending inadvertently favours proprietary over open business models.
Recommendation 8: Consider strategies to flatten government’s spending cycles, especially for community based projects.

Fragmented spending

Government taxes are collected centrally then split between departments, then split between divisions, then split between teams, and so on, until a group is funded to address a requirement. This hierarchical breakdown of budgets is appropriate for funding physical tasks such as building roads or collecting garbage; however, for the implementation of generic software functionality, it is usually more efficient for agencies to pool resources and collaboratively work on a common code base.
Fragmented spending results in narrow, short term solutions instead of solving broad, holistic and long term problems.

Think agile instead of “big bang” purchasing

It’s tempting to address “fragmented spending” by aggregating budgets from multiple agencies into a central “whole of government” contract. From an accounting perspective, you’d think that the easiest way to acquire technology is by defining scope, acquiring budget, insource or outsource the work, and then manage the developers implementing the specification.

However due to the complexity of IT, developers and users will continuously provide ideas for improvement. Projects which adopt agile development methodologies, which continuously adjust direction to incorporate feedback, have a track record of producing better quality outcomes than traditional “waterfall” acquisition methodologies prevalent within government.  
Recommendation 9: Prioritise agile, iterative development methodologies over “big bang”, “whole of government” purchases.

Chasing funds instead of collaborators

Open source communities typically become sustainable and scale by attracting a growing pool of collaborators. Government projects typically become sustainable and gain prominence by attracting funding and “empire building”.  Collaborating and sharing credit with external organisations typically weakens the importance of the individuals and teams. We need to adjust recognition and incentives to reverse this.

Call to Action

We need to recognise that government agencies are consistently duplicating effort; that government approaches to collaboration have typically been sporadic and unsustainable; and that best practices in open communities exist but are not readily available to government decision-makers.
Recommendation 10: Arm decision makers with accessible, evidence based research into what works, so they can trust, select and defend collaborative strategies which are often counter-intuitive within traditional hierarchically managed organisations.
Research and guides should be developed collaboratively, between agencies, nations, organisations and citizens.

We need to be bold enough to challenge widely established practices; we need to be aware of established wisdom; we need to be opportunistic and pragmatic; and we need to have the insight to know when to choose one over the other.

Related Reading

  1. Shorter, Cameron (August 2017). Making GovHack (and Open Government) more impactful.
  2. Ward, Dan (October 2011), Lieutenant Colonel, US Air Force. Acquisition Lessons from a Galaxy Far, Far Away.
  3. United States Assistant Secretary of Defense (Networks & Information, Integration) / DoD Chief Information Officer and the Under Secretary of Defense for Acquisition, Technology, and Logistics (May 2011). Open Technology Development: Lessons Learned & Best Practices.
  4. U.S. Public Participation Playbook
  5. Australian Government Digital Transformation Agency. Digital Service Standard.
  6. Australian Government Digital Transformation Agency. Design Principles.
  7. Australian Government Prime Minister and Cabinet. Open Government National Action Plan 2016-18
  8. Waugh, Pia (January 2015). Collaborative innovation in the public service: Game of Thrones style.

Signed

If you are a technologist and agree with this vision, please add your technical credibility by signing. Add a comment below, or email <cameron . shorter AT g m a i l .c o M>. Diverse support will help sponsors wanting to back this vision.

Australia:
  1. Cameron Shorter, Technology Demystifier; spent over a decade consulting to government on implementing Gespatial Open Source Software, Open Standards and Open Data; ex board member of the Open Source Geospatial Foundation (OSGeo); mentor in OSGeo Incubation committee; co-author of OSGeo Incubation processes; co-founder of OSGeoLive Open Source project.
  2. Nicholas Gruen, Chair, Open Knowledge Foundation (Australia), CEO of Lateral Economics, Former Chair of the Australian Government 2.0 Taskforce (2009) and of Innovation Australia in 2013-14.
  3. Arjen Lentz, Exec. Director Open Query Pty Ltd; former Community Relations Manager, MySQL AB; co-founder, Open Source Industry Australia, Inc
  4. Lev Lafayette, President, Isocracy Network; former President, Linux Users of Victoria, 2011-2014
  5. Steven De Costa, Steering Group member, CKAN Association (Open Source Data Portal) & Executive Director of Link Digital
  6. Bruce Bannerman, Director, GeoInnovations Pty Ltd; ex IT Manager, Australian Federal Government; Charter Member Open Source Geospatial Foundation (OSGeo); Mentor in OSGeo incubation committee; Former voting member, Open Geospatial Consortium Technical Committee.
  7. Stuart Guthrie, Co-CEO, Polonious Pty Ltd, Former President Open Source Industry Australia, Open Source Software developer and business owner. Current business, based on Open Source Software with offices in Australia, the US and the UK. Verticals in Case Management, Industries: Banking, Insurance, Government, Universities and Schools.
  8. Luke Carbis, Director of Product & Innovation at XWP.
  9. Evan Leybourn, CEO, Business Agility Institute.
  10. John Bryant, Principal, Mammoth Geospatial & Co-Chair, Free and Open Source Software for Geospatial and State of the Map Oceania Conference, 2018.
  11. David Collins, Independent Developer, Trilobite Solutions
  12. Alex Leith, Principal Spatial Analyst, Auspatious and Co-Chair, Free and Open Source Software for Geospatial and State of the Map Oceania Conference, 2018
  13. Andrew Pam, President, Linux Users of Victoria 2014-present
  14. Tristan Gutsche, Innovation and Integration Architect.
Other nationalites:
  1. Brent Wood, Information Delivery Programme Leader, NIWA. Ex Council member, New Zealand Open Source Society (NZOSS), Charter Member Open Source Geospatial Foundation (OSGEO).
  2. Ivan Minčík, LINZ Spatial IT Solutions Architect. After many years of working for government in Slovakia and 2 yrs of work in New Zealand I am very happy to sign this document.
  3. Dirk Frigne, president Open Source Geospatial Foundation Europe (OSGeo Europe vzw), OSGeo Charter member, Former Vice president of  OSGeo, CEO Geosparc nv.
  4. Jo Cook, Astun Technology, UK, OSGeo advocate, founder and chairman of Open Source Geospatial Foundation (OSGeo) UK local chapter, vice chair FOSS4G 2013 international conference (Free and Open Source Software for Geospatial)
  5. Vasile Crăciunescu, researcher Romanian National Meteorological Administration, Open Source Geospatial Foundation board member.
  6. Suchith Anand, founder of Geo4All, the Open Source Geospatial Foundation's network of educational institutions.
  7. Patrick Hogan, NASA WorldWind Project Manager, NASA
  8. Charles Schweik, Professor, School of Public Policy, University of Massachusetts, Amherst
  9. Sascha Meinrath, founder of the Open Technology Institute in Washington DC; Director at X-Lab; Palmer Chair in Telecommunications, Penn State University. 
  10. Aaron Rodericks, Lead, Open Innovation, Open Government, Chief Information Officer Branch, Treasury Board of Canada Secretariat, Government of Canada

See Also



Text in this document is licensed under the Creative Commons Attribution 4.0 Licence.

Friday, 18 August 2017

Making GovHack (and Open Government) more impactful

I've finally attended the GovHack weekend. As a dad, weekends used to be for taking kids to birthday parties and soccer games. But my boys have grown up, giving me the chance to see how GovHack compares to the Open Source communities I've been involved with for decades. I wanted to see what each can learn from the other and signed up as a coach.
GovHack is an annual event where volunteers band together for 48 hours to write applications with Open Government data. Participants compete for prizes for the most innovative and useful applications. It has grown every year since it started in 2009, attracting thousands of volunteers, running in 36 locations across Australia and New Zealand, and attracted numerous sponsors and an excessive list of open government datasets. Credit must go to the organisers for creating such a sustainable winning formula. But lets ask some tough questions and hopefully help GovHack become more impactful in future?

What is the point of GovHack?

What is the point of GovHack? It wasn't obvious from looking at the main website, but I found an answer buried in the GovHack 2016 Year in Review:
In his opening address, Craig Laundy, the Assistant Minister for Industry, Innovation and Science highlighted that open data was one of the keys to the Australian Government’s National Innovation and Science Agenda. He read a letter from Prime Minister Malcolm Turnbull which paid the following tribute to Govhack:
“Data without ingenuity is like a lamp without power – only when the two are connected do opportunities to innovate become clear. This is why GovHack is so important.”
Recommendation 1: We should be clear about the purpose and value of GovHack. We should prominently promote messages like "GovHack aims to contribute to the government's Innovation Agenda by encouraging and facilitating ingenuity with government's open data."

Is GovHack enabling Innovation?

So how successful has GovHack been at enabling innovation? It's hard to say really. The 2016 Year in Review provides plenty of details about numbers of participants, datasets used, awards, VIP presenters, red carpet events, but there is barely a mention of how successful GovHack has been at enabling innovation. The best I could find was a passing mention of an "IP Nova App" which started in GovHack 2015. I’ve since been told about a couple of others. But the point is that we are measuring how busy everyone is, and how much buzz is being created, but completely failing to report the impact on innovation.

Recommendation 2: Let's measure and report on the realised innovation resulting from GovHack. Let's then assess results and work out ways to improve GovHack's impact on innovation.

Maturing ideas is hard work

Why is it hard to find reports of GovHack ideas progressing into sustained initiatives? I can't say for sure, but suspect very few GovHack ideas actually grow into something. The simple truth is that good software takes substantial effort to design, write, test, deploy and maintain. While a 48 hour GovHack is useful for brainstorming ideas, it stills requires significant follow up if it is to mature into something useful. And here we notice the difference between Open Source Code Sprints and GovHack. On completion of Code Sprints, there are established and experienced communities committed to adopting and advancing worthy ideas. Who in the GovHack community is offering to help take good ideas through to maturity? I don't see such support mentioned in GovHack web pages.

Recommendation 3: GovHack sponsors' should aim to realise true value by helping to mature innovative ideas into reality. 

The majority of people I saw in the Sydney GovHack appeared to be University students or recent graduates. For these young people, GovHack provides a great practical learning experience, some mentoring, and an opportunity to network. However I couldn't help feeling there was an level of exploitation of these young volunteers. Government agencies are gaining significant value from volunteers testing their datasets, something that would cost orders of magnitude more if implemented internally. Morally, I feel these agencies should give more than a free meal and a chance to share in a prize. A good symbiotic relationship would hopefully consider providing more value for our young community.

Recommendation 4: Sponsors should consider formally setting up cadetships or project development opportunities as awards.

How good is the data?

Integrating data into innovative web or mobile applications typically should follow standard design patterns, with data published through a web service, then processed, integrated, and presented in innovative ways. Ideally government agencies should make data really easy to use, setting up data web services and providing clear documentation and examples. Instead teams were spending much of their GovHack time setting up the infrastructure to publish this data rather than spending their time being innovative.
It is worth being reminded of one of The Australian Digital Transformation Agency Design Principles:
Principle 4. Do the hard work to make it simple.
Making something look simple is easy. Making something simple to use is much harder - especially when the underlying systems are complex - but that’s what we should be doing. Don’t take “It’s always been that way” for an answer. It’s usually more and harder work to make things simple, but it’s the right thing to do.

Recommendation 5: Government should define a best practices guide for publishing data services, and then follow this guide.

How does government know if they are doing a good job? Ruthless survival of the fittest principles apply to Open Source and market economies. People don't buy substandard products. Only the best Open Source projects attract communities. Again, refer to the DTA design principles:


Principle 5. Iterate. Then iterate again.The best way to build good services is to start small and iterate wildly. Release minimum viable products early and test them with actual users; move from Alpha to Beta to Live adding features, deleting things that don’t work and making refinements based on feedback. Iteration reduces risk: it makes big failures unlikely and turns small failures into lessons. If a prototype isn’t working, don’t be afraid to scrap it and start again.

Recommendation 6: Agencies should measure the usability and usefulness of their datasets, assess and adjust accordingly. GovHack provides an opportunity to measure these metrics.

How good are we are implementing Open Government?

And so I come to my most pointed point, which was recorded as a video for my GovHack contribution:


Australia has embraced great policies around Open Government. These describe how openness and collaboration enable innovation. However, the practical implementation of these open principles have proven elusively difficult, with reported success stories coming from a few charismatic champions rather than being systemic across all government.
Why is that? Well, it’s complicated. There is a wealth of established wisdom, spread across the domains of Open Source Software, Open Standards, Open Data, Open Government, and more. However, we still lack clear and definitive guides which draws all this wisdom together into practical playbooks which can be easily applied by government agencies. Instead, current government practices and guidelines regularly hinder collaboration. Let’s fix that.

Recommendation 7: Let’s build an Open Government Playbook.

Let’s document the subtle magic which makes open and collaborative communities work.
This Playbook should cover technology, processes, governance, leadership, business paradigms, and ethics. It should be written in simple language, designed to support decision makers, architects, implementers and citizens to understand open principles.

Could GovHack be more impactful?

Acknowledging that GovHack runs impressively efficiently and has attracted a huge ground swell of interest and momentum, could we make it more impactful? I think we can. We should remind ourselves of the Open Government and GovHack goal of promoting innovation. We should measure innovation enabled and adjust accordingly. Adjustments will likely include aligning more closely with Open Source development practices.