The international FOSS4G 2012 conference, which was scheduled to be held in Beijing in Sept 2012,
was cancelled.
This has been a disappointing setback for our OSGeo community, and here
I capture some of the key events which lead up to this cancellation,
and with our hind site perspective, identify areas we can change to make
future conferences more resilient and successful.
This conversation started on the
OSGeo Conference email list, then moved to the
FOSS4G 2012 Lessons Learned wiki page as ideas consolidated.
The following discussion ground rules applied:
- Avoid letting the discussion break into a witch hunt, or blame
game. Remember that almost all people involved in FOSS4G 2012 were
volunteers, giving of their precious time freely.
- Instead, identify an event or decision, discuss the
implications of the event, and ideally follow up with some
recommendations on what we can do in future.
Host City Selection
Prior to 2012, OSGeo's Conference Committee had agreed to a 3 year rotation for the location of FOSS4G conferences, which went:
- Europe (2010)
- North America (2011)
- Rest of the world (2012)
- Europe ...
The bid process involves cities providing a light, 2 page, "Letter of
Intent", followed by a comprehensive bid if the "Letter of Intent" was
approved. However by Letter of Intent deadline for FOSS4G 2012 there
were
no Letters of Intent. The deadline was extended, and Letters of intent were
received from Rome (Europe), Prague (Europe), Hanoi (Asia), and a
late entry from Beijing (Asia).
This was
summarised by OSGeo Conference chair,
- What happened is that we did not receive any submissions before
the initial deadline, and then we opened the bidding to all areas, and
then we received 1 submission from the desired region and 2 from Europe,
and then a second late submission from the desired region.
- My opinion is that the stated desired region is in fact still
the desired region, and that all OSGeo conference committee members
should keep this information in their head as they vote. (meaning: all 4
letters are an option for this voting stage, but the preferred region
is 'anywhere other than NA or Europe')
In the end, only Prague [Europe] and Beijing [desired region]
submitted a full FOSS4G bid, and when it came to a final vote, the OSGeo
Conference committee
was split
between a bid from a more experienced team in Prague, and following
OSGeo's established rotation with Beijing. In retrospect, we should have
put more emphasis on selecting the experienced FOSS4G team.
European and North American international FOSS4G events have
traditionally attracted more delegates and sponsors, which makes these
conferences:
- More financially profitable
- Less financially risky
- Reach more people (although not necessarily reaching more regions)
As we move forward, we may wish to favour selection of committees and
cities with prior experience of holding local or regional FOSS4G events
before being awarded an international event.
Competing regional conferences
In 2011, major regional conferences started in both Europe and North
America, which competed for international FOSS4G attendance, along with
some FOSS4G conferences from the region. It was debated whether OSGeo
should support and encourage these new regional conferences, knowing
that they would have an
impact on attendance at Beijing.
As explained in a
post by the Chair of the OSGeo Board:
- From all that I can tell, now FOSS4G Beijing will become a
local conference with support from "OSGeo international". This and no
more. It will not be the Global or World conference that FOSS4G was
before because we will have a FOSS4G CEE and FOSS4G North America event
(plus the regular local ones) in the same year. There is no chance at
all that Beijing can attract the same vibrant global participation that
we had at the last global FOSS4G conferences.
- The question is not whether we will have a FOSS4G in Beijing
or CEE or North America. From all that I can tell we will have them all.
There is no reason (and probably no way) to stop the North American or
CEE initiative or both. Instead it is great to see so much interest and
momentum - and we would be stupid to stifle it.
The competing regional conferences are listed
here and
here.
Local Organising Committee experience
Lack of Professional Conference Organiser
The Local Organising Committee (LOC) had teamed with a Professional
Conference Organiser (PCO), starting from the bidding for the FOSS4G
2011 conference. However, it seems that at some point the PCO stopped
helping with FOSS4G. The LOC were then unsuccessful in trying to sign up
a new
PCO.
This was a significant setback for FOSS4G 2011, as PCO's bring
significant experience in running a conference. They have experience
with local venues and business, and they manage many of the day-to-day
tasks which takes workload off the LOC.
From what I can gather, the PCO were not contractually engaged
with the PCO up front, which allowed them to disengage later. What are
the lessons? The LOC should contractually engage with the PCO very early
in the conference cycle, and OSGeo oversight should ensure this
happens.
Loosing key LOC members
One of the key Chinese OSGeo community members, Professor Yu, passed
away shortly after Beijing was awarded the conference. This was very
unfortunate, both on a personal level, and organisation level.
Loss of key committee members is reasonably common (although
usually people step down for various reasons, rather than pass away).
For instance, a key FOSS4G-Sydney evangelist, who promoted the Sydney
event at prior FOSS4G conferences, stepped back and didn't attend
Sydney's FOSS4G 2009. The original FOSS4G-Devar 2011 chair had to
stand down
for personal reasons shortly after the bid was accepted. These examples
highlight the need for organizing committees to have strength in depth,
and in particular to have a backup plan if the conference chair has to
step down. This was a question that was asked of the Nottingham FOSS4G
2013 contenders, who have two backups to the conference chair, as well
as a committee with strength in depth overall.
Decision Making
A conference chair is asked to make many decisions related to the
conference, and the majority of the time, there is no clear
understanding about the benefits or downsides of each option. Usually
the only sure thing is that not making a decision will be detrimental to
the conference. Consequently, it is important for LOCs to become quick
and efficient at analyzing possibilities and then making decisions.
From what I can gather, the Beijing LOC would have benefited from
being more efficient in making decisions. For instance, in mid-November
2012 the
OSGeo-Live community asked the LOC to commit to distributing OSGeo-Live DVDs at the Beijing conference. The LOC took almost 3 months to
confirm they would support this. Other conferences usually provide such confirmation within a week, often within a day or two.
I suspect delays related to decisions would have contributed to
schedule slipages. The lesson here is that LOC's should be structured
and resourced such that they can make decisions efficiently. A prior
conference chair extended this observation to note the importance of the
conference chair:
- [A key lesson is the] importance of an active LOC and even more
importantly an active CHAIR. Committees don't move, they can't
communicate, they can't move. People can, so an active CHAIR is the
single critical ingredient. And the more that person in invested in both
organizing and communicating the event, the better it will be.
Schedule slip
As the deadline for the FOSS4G conference approached, there was significant
schedule slip
on key milestones, such as the ability to accept conference papers.
This was providing a visible indication of some of the other issues
listed in this analysis.
I think the lesson here is quite simple. Make sure there is an
appropriately resourced project manager responsible for managing the
conference schedule. (This task is usually provided by a PCO).
OSGeo Oversight
A second issue is that although OSGeo had identified concerns with
FOSS4G Beijing's progress reasonably early, intervention from OSGeo was
late in coming. A prior FOSS4G chair noted:
- We need to put harder stops in place to short circuit failure.
If you don't have a call for workshops out by February, [serious
questions are asked, such as should the conference be cancelled?]. If
you don't have $30K in sponsorship in place by April, [serious
questions]. If you don't have a call for papers out by May, [serious
questions]. This [FOSS4G 2012 conference] dragged out longer than it
should of because there were no hard stop points.
During the build up to FOSS4G Beijing, one of the key volunteers on
OSGeo conference committee, who had previously been very active, was
showing signs of burnout and was not contributing to his prior levels.
This left a noticeable hole in the OSGeo conference committee which was
not filled by another volunteer. The OSGeo Conference committee had
previously provided checks on conferences, such as reviewing and
approving the conference's budget and submitting to the OSGeo board for
approval, however this didn't happen for the FOSS4G Beijing conference.
What are the lesson's here? It may be that the critical role of
approving finances should be covered by a paid position, funded by
profits of FOSS4G conferences. Something like this was considered as
described under the
No mentor section below.
Is Failure Acceptable?
There has been discussion over the level of OSGeo Oversight that should be applied to a LOC. An OSGeo board member
noted:
- It is still desirable to give the local organizers quite a bit
of freedom and that we should accept that occasional failure is not a
disaster.
Which a prior FOSS4G chair
responded with:
- On the contrary, I'd say that random failures are a disaster
and will actually contribute to more failures. ...The success of a
conference is tied to the perceived expectations. Throwing a conference
is like throwing a party. Do you want to go to a lame party? No, you
want to go to a rocking party. If PartyPete throws awesome parties every
Thursday, you'll clear your schedule as next Thursday rolls around. If
LameLou throws passable parties sometimes, and sometimes cancels them,
you'll start going to PartyPete's instead. Consistency is very
important.
- The same thing will go double for sponsors: are you going to
commit to early sponsorship and send a cheque to an event that was
cancelled last year? Or will you hold on to your cheque until the last
minute just in case? The uncertainty effect is going to make the
financial situation of future conferences more precarious as sponsors
and registrants hedge their bets until later in the calendar. This will
only get worse if we embrace failure as an occasionally acceptable mode.
I'd argue that while it may be ok to experiment with some fringe
elements of a FOSS4G program, occasional failure of the international
FOSS4G conference should be considered unacceptable.
No mentor
A
proposal
was put to the OSGeo board, which was eventually approved, to have an
experienced FOSS4G mentor support the Beijing Local Organising
Committee. (A funded mentor was not provided to previous conferences).
This proposal
fell through, and although some prior FOSS4G chairs were approached (and others?), a replacement mentor was not found.
This left the Beijing FOSS4G LOC committee without some key expertise which could have been very valuable.
What is the lesson here? I think this was a good idea which fell through, and is worth pursuing again in future.
Communication
Language barrier
From what I understand, Beijing LOC were most comfortable speaking in
Chinese, and had varying levels of experience with English. I observed
that finding the right English words to support a conversation and
convey important messages was a time consuming task, often involving
decisions being made in Chinese, then translated to English. This
communication overhead would have produced a significant workload on the
LOC, who were already working on the difficult and time consuming task
of running a FOSS4G conference.
I believe this communication gap also contributed to many of the
other symptoms discussed here. Slow communication between the LOC and
community would have:
- Contributed toward slow responses to community queries,
hindering the international community contributing prior experience
toward the LOC,
- Slowed decisions from the LOC resulting in schedule slip,
- Caused difficulties getting the quality control of the website correct,
- and reduced marketing and communication to potential international delegates.
Cultural Differences
I question whether cultural differences contributed to communication
shortfalls. From my observations, it seems Chinese are more circumspect
about sending public communication, often using face-to-face meetings,
and waiting for review from a superior before making a public statement.
This contrasts with open source communities I've observed, where many
opinions are discussed publicly, both amongst senior and junior
developers, until a rough consensus is reached.
The OSGeo board representative to FOSS4G 2011
noted:
- For a while I attempted to play [the FOSS4G board
representative] role ... Also, while I was nominally involved, in
practice there was never any discussion on any mailing list I was on ...
- To me, a lesson of the Beijing effort is that it is hard to be
involved remotely if the LOC won't communicate by email/irc/etc. I
presume all the discussions that did happen were done by private email
or in person but I was left with no visibility or ability to assist.
Collective Knowledge
I believe our experience with this conference highlights how much of
our collective FOSS4G knowledge is stored in volunteers' heads, and is
passed between different events through our various communication
channels. When we constrict information flow by introducing a language
barrier, we have also constricted access to our knowledge on how to run a
conference.
A few suggestions on ways to address this include:
- Collect our conference running knowledge in a central source,
that can be handed on without the high level of communication currently
being used. In particular, I'm suggesting starting to collect our
processes in a FOSS4G Cookbook or similar.
- Set up a permanent FOSS4G coordinator role (one person, or an
international PCO, or similar) who are responsible for coordinating
conferences and personally remembering lessons learned between
conferences. (Note the risk of this person resigning and loosing all
collected knowledge)
- Additionally, ensure key members in the LOC can communicate
fluently with the rest of the OSGeo community. In most cases at the
moment, this would mean speaking fluently in English.
Response to emails
There were a number of comments that I was privately CCed on which
indicated that the international community were not receiving responses
after emailing the LOC. Here are some examples:
- As I've told you before it has been frustrating to me to not
receive any feedback from the LOC on my offer to sponsor the event. I
basically had the plan to come with my whole team (5 people now), but
can't afford such investment considering the state the conference and
participation levels are at now. In fact we have moved focus to the
Nottingham event just after Beijing because it appears to be (1) better
organized (but that may just appear like it due to the lack of
communication from Beijing, (2) an audience that is of interest to
[company name] and (3) cheaper / closer to home.
Another from the
academic lead, who later stepped down:
- ... [regarding email responses] from two "important players" I
have had no feedback, namely from the local organizers and from OSGeo.
I think the lesson here is that the LOC and PCO should be suitably
motivated and resourced, and be provided with enough delegation to
respond to all community queries promptly. Every query should be
responded to within one working day, even if the response is "we will
have an answer to you after the LOC meets next week".
Website out of date
A conference's website is the primary form of communication with
potential delegates. For FOSS4G 2012, the website took an excessively
long time to be developed and brought online, and then when it was
brought online, it contained
incorrect information and broken links (mainly
cut and paste from the prior FOSS4G website). People were having significant issues with submitting papers and
registering to attend.
The FOSS4G LOC had hired an external web developer to create the
website, who had done a poor job of development. It seemed that there
was a lack of quality control from both the web developer, and LOC. In
the past, development of the website has either been managed by
technically experienced developers (as was the case in 2009), or by the
PCO.
The lesson here is that the website needs to be made a priority
and suitably resourced. There is the potential for website management
software to be passed on from one conference to the next. (We considered
this option in 2009 but found the Open Source conference management
software used by FOSS4G 2008 was not going to integrate easily with the
software our PCO was using). It would be worth future FOSS4G conferences
revisiting this question.
Minimal "buzz"
To a certain extent, a conference is successful because the LOC says
it is going to be successful (and potential attendees and sponsors
believe the statement). Presenters and sponsors attend the conference
because they believe there will be lots of delegates, and delegates
attend because they believe there will be lots of quality presenters and
sponsors. And one of the most effective ways for everyone to be
convinced of the conference's success is to create lots of "buzz". I.e.,
lots of press releases, articles, blogs, twitter discussion and more
talking about how good the conference is going to be.
FOSS4G 2009 possibly went a little too far by putting out
41 press releases. However, FOSS4G Beijing could certainly have benefited from more "Buzz", as the OSGeo Board Chair
noted:
- on the website at http://2012.foss4g.org/
there is still no option for submitting abstracts although the
submission has been opened - apparently without notice to any of the
regular OSGeo channels. Workshops submission ends in two weeks.
- No international speakers have been announced and there are
only Chinese sponsors listed (although interest by regulars was
documented as early as December 2012).
Engaging international organisers
Compared to prior international FOSS4G events, there was minimal
international involvement in organising the FOSS4G event. Of particular
concern was that the international academic track lead
resigned, saying:
- ... I regret [the LOC] did not fully support the setup I
proposed. Specifically, the LOC insists on using their own deadlines and
reviewing and publication plan. Of course they have every right to do
so, because it is in fact their conference...
There is a significant amount of work involved in organising a
conference, and it is very valuable to share tasks with the
international community. This has two key benefits:
- It allows the LOC to focus on the local issues (like sorting out the venue)
- It facilitates knowledge transfer between years, as roles like
the Academic track lead are often coordinated by the same core people
over a number of years.
So lesson here is look for opportunities to make use of the
international community to coordinate specific areas of the conference.
Weekly meetings
Less than 3 months before FOSS4G 2012 was due,
weekly meetings
were started between volunteers from the international community and
the LOC. I understand that the LOC were having meetings internally, but
there was little visibility of them from the international community.
The extra meetings facilitated transparency from the international
community into the progress of the LOC, which in turn provided
opportunities for the international community to volunteer to help.
Eventually, with the help of these weekly meetings it was assessed that
the level of effort required to bring the conference back on track,
along with the likely outcome, resulted in a decision to cancel the
conference.
In retrospect, these meetings should have started much earlier,
ideally from the start of the conference planning a year or so earlier
such that support from the international community could have made a
better impact in the earlier stages. So lesson hear is start having
periodic meetings from early in the planning cycle, and invite the
international community to participate if you can.
Who wrote this analysis?
The telling of history is always influenced by the perspective of the
narrator, and as such it is useful to know who the narrator was. This
narration was by myself,
Cameron Shorter.
I was the chair of FOSS4G 2009, I'm a member of the OSGeo Conference
Committee, and I voted for FOSS4G 2012 to be held in Beijing. I helped
volunteers add Chinese translations to the OSGeo-Live DVD, and commit to
handing the DVD out at FOSS4G Beijing. I later attended weekly meetings
with FOSS4G Beijing's LOC and other international volunteers, until it
was decided by the LOC that the conference should be cancelled.
Jáchym Čepický (Czech Republic) has been welcomed onto the board for a one year term to complete Jo Cook's term who stepped down just before the elections.
