Thursday, 28 May 2020

Tech writer patterns, anti-patterns and tasks

 This essay captures patterns, anti-patterns, and tasks needed by the tech writing community to create good docs. It aims to provide a focus point for ideas which people can collaborate around.

Reading time: 20 minutes

It has been used as founding concepts for The Good Docs Project, ideas for Season of Docs, and a few presentations and videos.

You can see the original essay, along with many review comments in a Google doc.

First draft: 28 May, 2020
Last updated: 19 March, 2023

[the dream] A vision for good docs

Open source is an amazing social success story. It creates huge technical value which is shared with the world. However incomplete, unpolished and outdated documentation is a pervasive problem suffered by most open source projects (and software projects in general).

A bunch of us technical writers have gathered under the banner of The Good Docs Project. We want to tackle these challenges. While our initial focus was on open source software documentation, we’ve realized that the same documentation patterns and anti-patterns keep repeating themselves across all sorts of domains. So by addressing systemic writing challenges, we hope to lift the effectiveness of the entire technical writing ecosystem. Would you like to join us?

Lessons so far

From our research, we’ve found pockets of awesomeness - spread across presentations, blog posts, forums, applications, and organizational specific practices. But there are holes as well, and the pockets are typically disconnected or working in silos. We’ve been building templates and writing instructions, and have been learning a lot about the challenges around creating good docs. What we discuss in this essay are highlights from what we’ve learned so far and what is left to do.

Why describe patterns?

Describing patterns helps us realize our challenges are universal, and as such, are best solved collaboratively.

It also helps us to break problems into their base components. Then different teams can solve the problems independently. For instance, the document structure used should be solved separately to the choice of markdown format.

[pattern] Modular design

The modular design pattern is often used in systems engineering. Components are defined with standard interfaces, allowing for multiple implementations, independent development, and rapid innovation, while keeping the overall system stable.

We shouldn’t try and build a mega documentation system to rule them all. Rather I see us defining a swarm of interconnected concepts, each of which is realized by multiple components, connected via standard interfaces.

This approach makes it possible for all of us to bite off a small, achievable part of the wider vision.

A component could be:

  • A style guide;
  • A doc storage format, such as markdown; or
  • A doc audit checklist.

[pattern] Top down / bottom up

What we are talking about is a simple top-down vision of lifting the community with good docs, with an ecosystem of bottom-up implementations, each being tackled by small groups independently of others. This pattern has many names.

There are already many great initiatives within the documentation space. The value we can bring is by helping stitch all these initiatives together.

[pattern] External collaboration

“Collaboration out-competes competition”

Looked at through the lens of traditional management, collaboration is hard to justify. It is:

  • Time consuming,
  • Imprecise,
  • Unreliable,
  • Hard to manage,
  • Rarely addresses short-term objectives, and
  • Hard to quantify in a business case.

And yet, in a digital economy, collaborative communities often out-innovate and out-compete closed or centrally controlled initiatives.

For example, the collaboration found in open source communities offers a compelling alternative to capitalism.

  • With capitalism, you pitch an idea, raise capital, hire staff, build, sell, and make a profit.
  • With open source, you paint an inspiring vision, attract collaborators, build, and share with everyone.

The secret:

  • There are more people in the rest of the world, working on your generic problem, than you can ever muster in your team.

Further information:

[pattern] Multidisciplinary collaboration

“It takes a village to raise good docs”

In researching the documentation needs of projects, we’ve discovered that crafting good docs requires multiple skill sets, typically coming from different communities. The breadth of skills required makes solving documentation challenges elusively difficult.

Ideally, documenters have access to:

  • A developer, with a deep understanding of the software being described.
  • A product manager, who has a holistic view of the application and it’s roadmap.
  • A software user, able to explain the application within the context of the application’s domain.
  • An “expert newbie”. Some of the best feedback comes from new users who get stuck.
  • An educator, who understands the principles of learning.
  • An information architect, who understands how to structure material.
  • A software support person, who learns all the pain points of users.
  • A researcher, who proactively identifies pain points and market opportunities.
  • A writer, who writes clearly and concisely with good grammar.
  • Translators, for translating to target languages.
  • A DevOps person, who can set up doc build pipelines.
  • A community builder, and coordinator, who can inspire collective action, capture offers of help, and help all these different personas to collaborate together.

[anti-pattern] Short-termism

Should you prioritize management’s short term objectives, or attempt to solve the difficult and mis-understood root causes?

With documentation decisions, short-termism is rampant. And there are reasons for this:

  • There is a very low barrier to entry to writing tick-box acceptable documentation.
  • However, there is a very high collaboration overhead associated with writing excellent documentation.
  • We find it hard to measure, explain and justify the value of good documentation.
  • Tech writers typically have low political capital within organizations.

[task] Business case for good docs

This task involves writing a template business case for investing in good docs within an organization. The business case should:

  • Be customizable for different use cases and organizations.
  • Cover pros and cons of different strategies, described in business terms.
  • Reference research.
  • Cover multi-levels of involvement:
    • Why write docs?
    • Why write good docs?
    • Why adopt best practices?
    • Why share and collaborate with external doc communities?

Further information:

[task] Docs fact pack

To support a business case for investing in documentation, it helps to be able to reference a Docs Fact Pack, with a list of compelling quotes, with links back to source research. This is especially useful when lobbying for investment in docs, especially for writers with limited access to decision makers within their organization.

I’ve seen the fact pack concept used effectively for a bicycle lobbying group back in the 1990s. News reporters would often copy a string of facts verbatim into published news articles. This added legitimacy and influence to the cause.

Further information:

[pattern] Templates

Templates provide a conduit for collaboration. They facilitate the transfer and refinement of knowledge between time periods, between domains, between job types, between projects, between organizations.

Templates enable significant efficiencies by allowing experts to focus only on the bits they do best.

  1. Writing professionals embed best practices into the template.
  2. Domain experts write better content, more efficiently, when they have a template to work against.
  3. Writers polish the expert’s content by reviewing for compliance against the template.

[task] Template for templates

Writing templates for key doc types is our initial focus. This should start with identifying common elements for all templates - consider it a template for templates.

What information and guidance should be captured in each template? What supporting information should be provided in a supporting document? Each template should identify a target audience. It should be backed up by reasoning and research, which will help an author trust the templates are based on best practices, and know if and when it would be appropriate to deviate from the template.

Further information:

[tasks] Content type templates

There are multiple templates needed for different content types. We will need to define target audiences, create the document structure, expected headings, along with writer guidance. This should be backed by evidence justifying why the template is best practice.

There is a lot of work to cover here, and can be addressed by multiple projects running in parallel.

A possible hierarchy of templates includes:

[pattern] Content design

“Content design is a way of thinking. It’s about using data and evidence to give the audience what they need, at the time they need it and in a way they expect.” Sarah Richards

It provides a systematic approach to structuring information.

Cartoon showing the many contributing factors to good content design
Content design

[task] Content design guide

Community developed documentation regularly becomes disorganized and difficult to navigate. This task involves researching how readers approach your documentation, and organizing your docs so that they are easy to maintain. In technical writing terms, this is called information architecture.

It should:

  • Help projects structure their documentation.
  • Help projects maintain their documentation.
  • Help projects move existing documentation to better information architectures.

It may include:

  • Templates for different information architecture needs.

Further Reading:

[task] Process to define audience and messaging

To write effectively, it is important to understand who your audience is, and what information you wish to convey.

This task involves describing the steps to work out the target audience and message.

[pattern] Quality definitions, checklists and audits

During Season of Docs 2019 we discovered that we didn’t have a good definition of “good documentation”. We wanted to assess quality, currency, completeness and fitness-for-purpose of a project's documentation set.

For open source software projects, you can find comprehensive incubation processes to assess the maturity of a project’s software, its community, and the community’s processes. Likewise, businesses have defined quality management auditing processes, such as ISO:9001 and CMMI.

Similarly, we want to describe a project’s documentation maturity, and associate that with business impacts of poor maturity. Projects will then have metrics to help prioritize what, where and how docs should be maintained.

Note: Quality needs differ for each doctype:

  • Reference docs should be accurate and current, but can tolerate poor grammar.
  • Tutorials should be unambiguous, but can reference older software.

Further information:

[task] Docs audit checklist

This task is about building a doc audit checklist (or set of checklists). It involves identifying, classifying and cataloging doc quality criteria. It will involve addressing from multiple perspectives, such as:

  • The target audience(s).
  • Each doc type's purpose.
  • The maturity of the project.
  • The short and long term capacity to write documentation.

Further information:

[task] Doc quality measurement strategy

This task involves defining the practical steps required for ongoing measurement of doc quality. Insights gained can help projects refine their doc strategy.

For instance:

GoalSignalMeasureHow?
Docs are currentDocs are up-to-date with softwareCheck timestamps and version variables of docs and software
Script which checks lookup table to find the correct software/docs and then checks timestamps.
Search docs for each API name and associated parameters.Software and doc passing script.

Consider:

  • How good is the measure at showing the signal?
  • If I gain insights from the signal, what business decision(s) will I be able to make, and how much business value can be realized or saved based on this?
  • What is the cost of measurement? Is it worth it?
  • How much lead time is required before I'll be getting meaningful business value from data collected?

[anti-pattern] Too much documentation

“Less words get read more.”

There is a balance to reach between providing your reader with enough information and overloading them with content. Large swaths of documentation is often indicative of deeper problems, such as a poor user interface.

[pattern] Reduce the need for documentation

Even better than good documentation is not needing the documentation in the first place.

  • Applications with an intuitive user interface reduce documentation requirements.
  • Similarly, automated tools can reduce manual steps required, which in turn reduces the amount of documentation that has to be written/read/maintained.

[anti-pattern] Outdated documentation

It is difficult to keep documentation up-to-date with the rapidly evolving software it describes. Doing so typically requires a multi-pronged strategy. It may cover:

  • Integrate doc refreshes into software release checklists and processes.
  • Auto-generation of docs, screenshots and other content.
  • Periodic reviews.
  • Minimizing the content footprint.
  • Writing timeless documentation, avoiding language that anchors to a point in time.
  • Tracking status.
  • Categorizing the priority of content to maintain.
  • Recruiting, coordinating and supporting contributors.
  • Empowering users to fix and improve documentation.
  • Reducing technical barriers to entry for updating docs.

[task] Strategy to keep docs current

This task involves describing options, feasibility, tools, tradeoffs and practical advice which projects can follow to implement strategies to keep documentation current.

This may extend to providing automated tools which systematically check and report on “stale” documentation.

[pattern] Content reuse

A documentation set can draw upon multiple sources. In presenting this information, there are multiple trade offs:

  • Using variables helps with re-purposing similar material, and re-releasing updates. However, it introduces a technical barrier-to-entry which can be off-putting to potential contributors. And new contributors might accidentally overwrite and break the logic. So content reuse can be both a pattern and anti-pattern.
  • Documentation can be customized to directly address the users’ profile. Or generic sections can be used, which are potentially ruggedized and maintained by a community.

Reuse sub-patterns include:

Find and replace:

An author applies a simple find-and-replace strategy for words like YOUR_PROJECT.

Variable substitution:

Variables are replaced at publishing time. These can be:

  • Standard terms, such as <table_of_contents>, <page_number>, or <last_updated>.
  • Author defined terms, such as <project_name>.

Section substitution:

For instance, a reader may be able to select the programming language used for code examples.

Cookbook sections:

A skeleton structural document specifies which subsections to include.

Inheritance:

A document inherits characteristics from a parent document. For instance:

  • Task template -spawns-> Tutorial template -spawns-> Hello World tutorial
  • The glossary for a document includes a subset of the glossary for the organization.

[task] Establish content reuse best practices

This task involves researching, cataloging and aligning best practices around content reuse. It might also involve:

  • Developing integration standards, or
  • Extending old systems, or
  • Developing new systems.

[pattern] Style guide

A documentation style guide captures guidance around writing style, language choice, grammar and related concepts. It helps communities build consistent documentation which align with best practices. Communities should adopt one of the established, widely used style guides and customizing as little as possible. The Good Docs Project has adopted the Google developer documentation style guide unchanged. However, there are legitimate business reasons for deviating from an established guide.

For instance, if your project is used solely within England, you may choose British English instead of American English. Or you may have domain specific terminology which should be added into the accepted word lists.

[task] Guide for adapting a style guide

This task involves helping an organization select a style guide, and how to select customizations for it. It should also explain the business consequences of deviating from a standard style guide.

Further information:

[task] Distill a common core style guide

There are a number of established documentation style guides, and there are legitimate reasons for the differences. However, there are many core concepts which are shared by all (or most) of them.

This task involves:

  1. Recognizing that writing communities would benefit from consistency in core concepts used by these guides.
  2. Distilling the core concepts from these guides.
  3. Adopting common language to describe these core concepts.
  4. Refactoring established guides into core concepts, and extension concepts.
  5. Ideally, each extension concept should be grouped and become a tick-box selection, such as “Should I use one word list or another”.
  6. An existing style guide could then be described (and selected) by a tickbox selection of concepts.

This task would be made significantly easier if the owners of leading style guides play a lead role in the process.

[task] Writing tools which support style guides

The quality of a writer’s work, and their productivity, can be significantly improved if they are provided with real-time feedback on their writing style as they write. Such tools already exist, in the form of spell checkers and grammar checkers.

This task involves updating writing tools to allow users to import a documentation style guide, and then have the tool check the user’s writing against the imported style.

Further information:

[task] Standard machine readable format for style guides

A machine readable format for style guides should be defined and adopted by the industry as a standard. This will facilitate interoperability between the ecosystem of documentation tools and the writers of style guides.

This task involves the development of the format, through to the industry and community adoption of it as a standard.

[task] Codify style guides into the machine readable format

In order to support style checking in writing tools, existing style guides will need to be codified into a machine readable format. Ideally, this format is one that has been widely adopted as a standard.

This task involves codifying existing style guides into a community standard machine readable format.

[anti-pattern] High technical barrier to entry

Within open source communities we have projects desperately needing docs, and grateful users keen to give back through documentation, but they are stumped by significant technical barriers to entry. The git/wiki tools used are difficult to learn, and clumsy to work with.

[anti-pattern] Compromising on doc toolchains

There are competing doc tool chains used to develop documentation, each with their pros and cons:

  1. Software developers tend to store docs in git next to their code, using wiki formats, such as markdown. This offers excellent version control, ability to add variables, and integration with doc generation pipelines. However, it has a high technical barrier to entry.
  2. Word processors, such as Microsoft Word and Google Docs, make it easy for reviewers, offering embedded track changes and comments. They also offer real-time syntax and grammar checkers which significantly helps writers.
  3. Content management systems tend to fall somewhere between these.

Compromising on the toolchain is a universal problem for technical writers. Considering the size of the problem, (number of technical docs) x (amount pain caused and time lost), it is surprising the problem hasn't been fixed yet. If a tool were to address this challenge it could become a category-killer.

Further Information:

[task] Integrate doc editors with git backend

A prototype integrated tech writing tool could be relatively easy - write a round trip converter which uses a WYSIWYG client like Google Docs, and git with a markdown variant for data storage.

This task involves integrating the best features from each of the tool chains and making the tool(s) accessible to all writers.

[task] Round trip conversion between formats

There are multiple doc formats and reasons to convert between these formats. It would be very valuable if tools could reliably round-trip convert from one format to another and back again, without changing the source documentation. Particularly valuable would be a tool chain for round-tripping from Google Docs to Markdown and back.

This task involves developing tools, conventions and processes which support round-tripping between documentation formats.

[anti-pattern] Multiple wiki formats

There are a multitude of wiki formats, each their own strengths and weaknesses, but the biggest weakness of all of them is the lack of standardization and resulting interoperability challenges. Even Markdown, one of the most popular variants, has multiple flavors.

[task] Standardize on markdown variant

For The Good Docs Project, we have decided to store our base documentation in Markdown, because it is widely used, and it is easier to convert docs to formats which offer more functionality rather than the other way around.

Unfortunately there are multiple variants of Markdown. We should select one variant as our standard, which will help with interoperability between implementations. CommonMark has defined an unambiguous version of markdown. Also Github Flavored Markdown (GFM) extends CommonMark with a few improvements, such as table support.

This task is to set up a git commit hook which checks for compliance with the Markdown variant we select, and then to apply compliance checks to existing documents.

[task] Markdown linting

We should make use of best practices in the formatting of Markdown documents.

This improves readability, and you avoid picking up false positive formatting changes.

This task is to select/define a format standard, provide a tool for linting, and check linting during git pull requests.

This task can be extended to other wiki formats supported in future.

[pattern] Traceability

Bi-directional traceability is a systems engineering practice, which enables systemic quality control within large, mission critical systems.

It involves tracing between business requirements, technical requirements, implementation, and various levels of testing.

Simple traceability can be managed within a spreadsheet, but traceability for complex systems requires supporting tools, processes and unique writing rules.

[task] Traceability tools and processes

This task involves identifying tools and processes that can be easily integrated with existing tools used by typical technical writers. In particular, this should address the use case of a writer working with an open source toolset.

[task] Consistency between documents

Guidance provided by documents should be aligned. For instance, recommendations within a writing template should align with the style guide.

This task involves auditing The Good Docs Project documents and bringing them into alignment. The dependencies should be traced to help future maintainers understand the impacts of proposed changes.

[pattern] Private/public division

Organisations regularly need to support mixed access restrictions to content, along with graduating content from one security domain to another. For instance, draft content might be restricted from public viewing until it has been reviewed, and the feature it is describing has been released.

This functionality is already supported by many tools.

[task] Guide for public/private management of documentation

This task involves updating tools, conventions and processes to access restrictions.

[pattern] Training

“Give a person a fish, and feed them for a day. Teach them to fish, and you feed them for a lifetime.”

Teaching others to write better is one of the most efficient ways to improve docs.

[task] Training

This task involves:

  • Identifying suitable training.
  • Updating or creating training material to align with best practices being adopted.
  • Making training material available.
  • Ensuring training material is equitable and inclusive.
  • Building a community around the maintenance and delivery of training material.

Further material:

[pattern] Glossary management

There is a cluster of lexicon management patterns, used with varying levels of uptake and maturity by projects. These include:

  • A glossary which defines terms and acronyms.
  • Preferred terms, terms to avoid, and alternative terms that writers should use.
  • Lists of equivalent terms for different languages, which help translators to translate terms consistently.
  • Cross domain lexicon management, determining which community gets to define a term, coordinating cross domain management of terms.

[task] Best practices for glossary management

This task involves defining the lexicon management problem space, and then starting to collate best practice processes in this space.

[task] Establish sharable glossaries

Develop standards, tools and processes to allow glossaries to share terms between organizations and projects.

Image shows how terms can be inherited from more authoritative glossaries, and terms suggested back to the authoritative glossaries.
Inheritance of terms between glossaries

Further information:

[pattern] Documentation publishing tools

The documentation publishing domain has received plenty of attention for the last few decades. I suggest this need not be a priority for us.

[pattern] Localization

Documentation translation, referred to as localization, helps reach a wider audience, who either don’t speak English, or don’t speak English well. Approaches to localization include:

According to wikipedia, only 23% of the world speaks English, and only 7% speak it as a first language.

[task] How-to for localization

This task involves writing guidance on considerations, processes and potential toolchains to consider when setting up a translation pipeline.

[pattern] Sharing knowledge promotes equity

“Vibrant communities are critical to project health, code quality, and adoption, but the lack of clear documentation about our project, its code, and the norms and processes of our community is a highly effective barrier to adoption and contribution.

If we do what we can to make sure that knowledge is accessible to everybody, regardless of disability, or language, or social capital, we promote equity.

Knowledge is power. Documentation puts that power in the hands of the people.”

Riona MacNamara at Write The Docs - Australia

[pattern] Education theory

There is a bucket load of research into learning theory:

  • Different learning styles.
  • What makes an idea “sticky” so that people remember it. We should reference that theory and update our templates to account for learning theory.

For instance, go listen to the Utilitarianism: Crash Course Philosophy #36 video which I think applies excellent learning strategies:

  • Fun
  • Engaging
  • Ties complex concepts back to stories you already know. (In this episode, they use the Batman and Joker characters which we already know to explain the behaviors of them).
  • Information density
  • Formula for presenting information: Into concept, examples, theory, summary.
  • Quick presentation pace.
  • Young presenter, the age of the target audience.

Further information:

[pattern] Reader/author ratio

Consider the cost of docs based on the reader/author ratio. The higher the ratio, the greater the importance to put on maturing the doc quality. For low ratios, as you may find for edge cases in community forums, there is typically a higher tolerance to lower quality docs.

[anti-pattern] Low political capital

According to Tom Johnson’s survey of 406 tech writers, 34% were lone writers, 31% only had 2 to 4 writers.

This low team size, along with peripheral involvement in business and engineering decisions results in tech writers typically having low political capital to lobby for improved documentation processes.

[call to action] Want to help?

While there is a barrier to entry for each of these problems, many will only require a few dedicated contributors to achieve significant benefits across the entire technical writing domain.

  • Do any of these problems resonate with you?
  • Are you trying to solve part of one of these problems already?
  • Do you want to collaborate in solving this?

If so, why not come join us at The Good Docs Project.

Posted by at No comments:
Labels: ,
Subscribe to: Posts (Atom)