Thursday 30 May 2013

OGC heed community pressure regarding "GeoServices REST API"

Following on from signification discussion and concern from OSGeo and OGC communities, members of the OGC have withdrawn the "Geoservices REST API" candidate OGC standard.
Thank you to all the people who have voiced their concerns, thank you to those who provided detailed reasoning and analysis, and thank you to the OGC and OGC members who have acknowledged and have started actioning this reasoning. You are contributing toward a better OGC for us all.

Carl Reed, CTO and Executive Director of OGC Standards Program made the following announcement:

On Tuesday of this week, the GeoServices REST API Standards Working Group (SWG) voted to approve the following recommendation to the TC Voting Members:
The GeoServices REST SWG recommends the Technical Committee (TC) approve withdrawal of the motion to approve the OGC GeoServices REST API documents as an official OGC standard.
Moved: Keith Ryden. Second: Clemens Portele. There was no objection to unanimous consent.
The following is the reason for requesting the motion to approve the candidate standard be withdrawn:
“Considering the breath of discussion both internal and external to the OGC process since the vote announcement, the SWG members feel that the vote cannot continue until the many questions raised have been addressed. Issues regarding OGC process, vendor advantage, duplication of capabilities, etc. have now overshadowed technical discussions of the merits of the specification. By withdrawing the OGC GeoServices REST API candidate standard, the necessary discussions regarding OGC process, policy, and position can continue separately.”
The SWG further discussed the need for OGC Members and staff to debate, clarify and potentially amend a number of policy and procedural issues before the SWG can decide to either disband or to resume technical work on the candidate standard.
Today or tomorrow the TC Chair (me) will be asking the TC Voting Members to consider the GeoServices REST API SWG recommendation. The motion is:
"The GeoServices REST API SWG recommends that the GeoServices adoption vote be withdrawn.  If there is no objection to unanimous consent in the next 10 calendar days, then the motion is approved. If there are objections, than a two week e-vote will be initiated."
Over the coming months there will be opportunities for OGC staff, members and the general public to engage in discussions related to policy and procedures, such as clear statements on openness and interoperability, overlapping standards, backwards compatibility and so forth. The idea is to begin a dialogue as part of the virtual meetings this and next month with a goal to address the issues raised and recommend changes at the upcoming September Frascati meetings.
I would like to thank the SWG for all of their hard work on the candidate standard. I would also like to thank everyone who contributed to the conversation.

Wednesday 22 May 2013

Will OGC’s standards meet government purchasing guidelines?

In what has become the Open Geospatial Consortium’s (OGC) most contentious vote to date, OGC members are being asked whether the proposed "Geoservices REST API" should be accepted as an OGC standard. A summary of concerns is listed in an Open Letter from the Open Source Geospatial Foundation (OSGeo) to the OGC. However, the crux of contentions hinges around the definition of an Open Standard and whether the "Geoservices REST API" qualifies as one.
When measured against government’s policy drivers of interoperability, fair competition, and economical use of government funds, the evidence is overwhelming. "Geoservices REST API" fails on all accounts. In fact, we should be questioning why our OGC processes haven’t identified and then addressed these issues much earlier.

Background

As background, the "Geoservices REST API" describes the interface to a dominant vendor’s web service (ESRI’s ArcGIS Server), and overlaps substantially with OGC’s existing suite of web service standards.

What is an Open Standard?

Most government purchasing guidelines, such as the United Kingdom Open Source, Open Standards and Re­Use: Government Action Plan, now include clauses such as:
The Government will use open standards in its procurement specifications and require solutions to comply with open standards. The Government will support the development of open standards and specifications.
Consequently, government contracts typically specify use of standards when purchasing systems. This places a responsibility on standards organisations to protect government policy when selecting and defining the standards baseline.
Superficially, the "Geoservices REST API" meets the European Interoperability Framework minimal definition of a standard:
  • The standard is adopted and will be maintained by a not-for-profit organisation, and its ongoing development occurs on the basis of an open decision-making procedure available to all interested parties (consensus or majority decision etc.).
  • The standard has been published and the standard specification document is available either freely or at a nominal charge. It must be permissible to all to copy, distribute and use it for no fee or at a nominal fee.
  • The intellectual property - i.e. patents possibly present - of (parts of) the standard is made irrevocably available on a royalty-free basis.
  • There are no constraints on the re-use of the standard.
However, the "Geoservices REST API" falls short of addressing government policy drivers for the creation of standards. These are summarised in the Guideline on Public Procurement of Open Source Software, written for the European Commission:
Public sector consumers of software have an obligation to support interoperability, transparency and flexibility, as well as economical use of public funds. When it comes to public procurement, the principles applied to the public sector require them to support (and certainly not to harm) competition through their procurement practices.
Good practice eGovernment services should provide access based on open standards, and in particular, never require citizens to purchase or use systems from specific vendors in order to access public services: this is equivalent to granting such vendors a state-sanctioned monopoly.
Let's address these issues point by point.

Costs and Interoperability

Regarding when to create new standards, the United States Open Technology Development (OTD): Lessons Learned & Best Practices for Military Software has the following advice:
... use/modify/create open standards, in that order.
Unfortunately, the "Geoservices REST API" would create new standards rather than use and/or extend existing OGC web services. Emphasis on reuse of standards is important for increasing interoperability, as duplication of standards typically results in:
  1. Implementation costs to support multiple standards increases.
  2. Consequently, organisations and/or applications may choose to only support one standard, or only support one standard fully.
  3. Sponsors (such as governments) who require compliance with standards will discover that applications don't communicate together, due to applications supporting different standards that essentially do the same thing.
  4. After a while, in order to solve interoperability issues, a respected international organisation or program will likely take the initiative to mandate one prefered standard for all agencies to follow. To date, the OGC has provided this leadership.
  5. One standard taking prominence over the other will likely lead to the other being neglected or deprecated, resulting in many OGC compliant systems becoming legacy systems in the process. This should be considered an undesirable outcome for a standards organisation.
Costs increase, interoperability decreases.

Fair competition

ESRI’s ArcGIS Server is currently the only server which provides a full implementation of the "Geoservices REST API", as you would expect when an API is derived directly from a product. As such, if the "Geoservices REST API" were to be included in the OGC baseline, and government contracts continue to reference the OGC baseline in contracts, then governments would be giving one vendor a significant market advantage while other vendors wear the cost of developing matching implementations for the proposed standard.
Further, ESRI may continue to use its market dominance to promote use of the "Geoservices REST API" at the expense of existing OGC web services. (As described in ArcGIS Server documentation, support for OGC’s W*S services are disabled by default while GeoServices REST and KML are enabled).

Where are the Open Source implementations?

Another test for identifying open standards is defined by the United States Open Technology Development (OTD): Lessons Learned & Best Practices for Military Software:
Verify that the standards used are open; a simple test for openness is to determine if the standard is implemented by open source software.
Currently, very little open source has been developed to support the "Geoservices REST API" and there is wide opposition to the proposed standard from the Open Source community.
Open Source implementations referenced by proponents of the "Geoservices REST API" include immature implementations, partial implementations and a library application. That is: a roadmap document for GeoServer, a sandbox implementation of an Openlayers client, a 52North SOS extension to ArcGIS Server and the GDAL translation library.
By comparison, there are multiple production-grade, client and server, open source implementations, which cover the full breadth of existing OGC standards, which have matured over the past decade, and there are open source reference implementations for most (all?) current OGC standards.
So by the Open Technology Development definition, The "Geoservices REST API" hasn’t yet reached the maturity of an Open Standard.

Where are OGC’s gatekeepers?

There are 481 OGC members, with close to 100 of them with voting privileges, yet regularly, less than 40% of these voting members actually vote on proposed standards. This is a concern if these members are being relied upon to uphold OGC values, and we should question why voting is so low. A key factor in low voter turnout is likely the complexity and volume of material voters need to understand in order to make an informed decision. Gatekeepers just don’t have the time to be abreast of all the issues, and current standards are hard to read. The increase in the breadth and application of OGC standards has led to a stronger need for integration of standards, architectural overviews, and clearer implementation guidelines.
Maintaining and verifying quality best addressed by defining and following development and validation processes, and OGC processes should be improved to match the complexity of the systems they represent. In particular, OGC should revisit goals and requirements for quality standards, then resource technical writers and reviewers to work against such requirements. Approving a standard is therefore simplified to verifying the process is valid and has been followed. This would require OGC sponsorship priorities changed to provide greater emphasis on quality over quantity of standards.

A blueprint for moving forward

Let's expand on the steps involved in deciding on the value of a standard:
  1. Governments policies should embody government best practices. Many countries have already taken this initiative.
  2. Standards organisations should embrace such government policies.
  3. A clear definition of an "open standard" is required, which addresses government policy requirements of interoperability, fair competition, free access to government services, and economical use of public funds. This should be expanded into clear guidelines to be applied by OGC Gatekeepers and standards developers. The OGC should revisit the "open standards" definition, and in particular, ensure the definition extends beyond the technical to include policy implications.
  4. Suitable training should be available to OGC Gatekeepers and implementers of OGC based solutions.
  5. The OGC and OGC sponsors should consider realigning priorities. In particular:
  • Place a greater emphasis on quality over quantity of standards. This includes: harmonising competing standards, improving quality of writing to support understandability and implementability, and extend testing to verify standards are implemented correctly.
  • Provide simple and clear descriptions of standards. The OSGeoLive project has addressed similar issues by providing a concise one-page project overview, plus a ten-minute quickstart, translated into 11 languages, for fifty of the best geospatial open source applications.

Summary

As the success of the OGC increases, the OGC will need to be mindful of business and policy implications associated with adopting established interfaces as standards. Specifically, accepting the currently proposed "Geoservices REST API" as a standard will have detrimental impacts on interoperability, fair competition, and economic use of public funds. Instead, the positive aspects of the "Geoservices REST API" should be harmonised and incorporated into the existing OGC baseline of standards. Also, as the breadth of technology covered by OGC standards increases, it is becoming more difficult for gatekeepers to monitor the quality of these standards and consequently it is becoming more important to focus on quality and understandability of these standards. In moving forward, the OGC membership should revisit OGC priorities, and consider placing a greater emphasis on quality over quantity.

Post Note

As a result of this, an open letter, and many other objections, the Geoservices Rest API was withdrawn from being considered as an OGC standard, as described here. The OGC then initiated an Ideas4OGC review, to rebaseline OGC priorities and processes in order to address weaknesses that had been identified in OGC processes.

Thursday 16 May 2013

Starting build cycle for OSGeo-Live 7.0

We are starting the build cycle for the 7.0 OSGeo-Live DVD/USB/VM which will be released in September 2013, ready for the global FOSS4G conference in Nottingham, UK. We would like to hear from anyone wishing to add new projects to the live DVD, anyone wishing to extend or add extra translations, or anyone who has ideas on how we should shape the upcoming release.
Also, could all projects please reply to us with which stable version of their software should be included in this release.

Key Milestones

17 Jun 2013 All new applications installed, most old applications updated
15 Jul 2013 Feature Freeze (all apps updated)
05 Aug 2013 User Acceptance Test (all apps installed and working)
26 Aug 2013 Final ISO sent to printers
... full schedule

OSGeoLive 7.0-alpha1 released

We have released the first alpha version of OSGeo-Live, which includes an update from Xubuntu 12.04 to 12.04.2 along with updated versions of applications from UbuntuGIS repository. Feel free to start testing your applications in the latest release. Download Alpha 1

About OSGeo-Live

The OSGeo Live demo DVD/VM/USB is an Xubuntu based distribution of Geospatial Open Source Software, available via a Live DVD, Virtual Machine and USB. You can use it to try a wide variety of open source geospatial software without installing anything.

Monday 13 May 2013

Last chance to reject "Geoservices REST API" standard

Next Wednesday, the OSGeo Board will deliver an Open Letter to the OGC and OGC voting members, with multiple signatures, demonstrating the large number of people concerned about the negative consequences associated with making the "Geoservices REST API" an OGC standard.
The "GeoServices REST API" was initially developed by ESRI and implemented on the ArcGIS Server platform before going through the OGC process. It significantly overlaps with OGC's existing W*S services, but isn't based upon these existing standards.  Consequently, we have grave concerns that the two competing sets of standards, which essentially cover the same use cases, will have far reaching, detrimental impacts on interoperability, complexity, and costs within the spatial industry, including being bad for Geospatial Open Source software.

Many people, including leaders of OSGeo and related communities, have already signed this letter. Thankyou. If you agree that "Geoservices REST API" will be bad for OSGeo and/or the greater spatial community, then please help us deliver this concern to OGC voters before they vote. Add your signature to the Open Letter before we deliver it on Wednesday 15 May 2013, and forward this message onto other OSGeo communities. (I notice that messages from this thread are being forwarded to the Spanish OSGeo email list. Thankyou.)

If you are looking for a deep analysis of the issues, I suggest reading the open letter which is now draft complete: http://wiki.osgeo.org/wiki/Geoservices_REST_API.