The Cathedral and the Bazaar

The shift towards agile software development models, with continuous integration processes and publication of new features, forces us as technical writers to review our own working methods. Leaving behind the model imposed by the traditional CMS and integrating documentation tasks into the development cycle is now more necessary than ever.

The Cathedral and the Bazaar

In 1999, Eric S. Raymond, one of the leading representatives of the open source initiative, published his essay entitled The Cathedral and the Bazaar. In this popular essay, Raymond writes about his experiences during the development of open code software, at the same time as he encourages programmers and companies to embrace this work philosophy.

Raymond uses the concepts of cathedral and bazaar to exemplify two quite distinct development models. The cathedral represents the development of proprietary software, where the source code is only accessible to a small group of programmers. The bazaar, on the other hand, represents the development of open source software, where the code is accessible to collaborators. The rigidity of a cathedral stands as a metaphor for a closed, centralized system, whereas the dynamism of a bazaar represents an open, collaborative system. Raymond maintains that the possibility of incorporating improvements and eliminating errors is much more extensive in a collaborative model than in a centralized one. The conclusion is that the software resulting from a collaborative model has fewer errors and is necessarily of higher quality.


The thesis explored in The Cathedral and the Bazaar has given rise to lots of discussions among those defending and criticizing open source software. The collaborative model is not free from problems, but offers advantages that make it attractive to many software development companies. It is not unusual to see such companies looking nowadays for formulas enabling them to adopt their own open source methodologies without renouncing their control of the code and intellectual property over the same.

In parallel, software development tools are incorporating more and more features that enable the principles of collaborative development to be extended to teams of workers. Together with this, the process of continuous integration and continuous delivery (known by the initials CI/CD) enables new features to be rolled out and errors to be resolved relatively quickly. Suffice to think just about the frequency with which many of the apps on our mobile telephones are updated.

The problem with content management systems

It is strange to see how this change in development methodologies has had scant repercussion on software documentation. This field continues to be founded on tasks of drafting, review, and publication that, in essence, have changed little in recent years. For writing, XML is still the standard markup language. More often than not, reviewing is still done using PDF files or, even worse, over email. And publication, which has gradually shifted to the web, often requires hours of adjustments to generate the correct PDF or HTML file.

This workflow is today sustained by complex Content Management Systems (CMS) that aim to solve the documentation needs of many different organizations at one fell swoop, but end up not entirely solving the needs of any of them.


But the fundamental problem lies in the fact that a traditional CMS understands the drafting, review, and publication tasks as actions that follow each other in orderly fashion over time and not, as so often seen in experience, all at the same time. An added problem is that the possibility of incorporating improvements from collaborators is limited to the review phase, with the documentation effectively ring-fenced for most of its development cycle.

“Ditch your CMS”

In 2014, Patrick Arlt (@patrickarlt), in the context of the Write the Docs conferences, gave a presentation titled Ditch Your CMS With Git and Static Site Generators. In this presentation, Arlt advocates abandoning the use of content management systems in favour of open code tools that favour collaboration and reduce the complexity oy documentation project.

A year later, in April, 2015, Anne Gentle (@annegentle) described the documentation process adopted by the OpenStack software manufacturer in her article entitled Git and GitHub for open source documentation. The model proposed by Gentle (subsequently developed in greater detail in her book Docs like code) proposes that technical writers should work alongside the development teams, adopting their same tools and methodologies. Because, if a software’s documentation is an integral part of the product and is produced in parallel, isn’t it more consistent to use the same workflows? In this new paradigm, writing documentation is not that different from writing code.

In summary, the Docs like Code methodology proposes:

  • Ditching the use of traditional CMS systems in favour of collaborative tools like GitHub (a code repository very popular with developers).
  • Adopting lightweight markup languages, abandoning complex languages like XML. The best-known example of these languages is Markdown.
  • Incorporating continuous review processes where reviewers can record changes at any time using pull-request mechanisms offered by GitHub.
  • Publishing product documentation regularly through the use of static page generators that enable the texts in Markdown format to be transformed into HTML code.

GitHub

Millions of developers all over the world use GitHub as the repository for their application code. But what makes GitHub such an attractive environment for amateur coders, independent developers and companies? The commercial success of GitHub has been born out of its ability to simplify the use of the Git tool (a software utility originally created to manage the development of Linux) in an easy-to-use web interface. In addition, GitHub facilitates the collaboration of multiple people on the same project in a relatively straightforward way. The complexity and scalability needs of modern software have made this feature essential.

Yet this change in software development methods is not so far away from what is happening in the documentation arena. While a manual was the responsibility of a single technical writer in the past, nowadays, with growing technological complexity and the constant integration of changes, the final result requires collaboration and co-ordination involving multiple individuals. The cathedral model imposed by content management systems has not been able to respond to this change.

The time has come

Far from being a mere concept, Docs like Code is also a reality for software companies as important as Microsoft, Amazon, or Twitter. In May, 2016, Jeff Sandquist (@jeffsandannounced Microsoft’s new documentation platform docs.microsoft.com in the following terms:
All documentation is designed to allow community contributions. Every article has an Edit button that takes you to the source Markdown file in GitHub where you can easily submit a pull request to fix or improve content.
A large part of the manufacturer’s documentation is already located on this new documentation platform in a clear wager in favour of the adoption of open systems. Other companies such as Amazon have followed this trend and it is now possible to see an “Edit on GitHub” button on many of the articles intended for developers.



In an article on the I’d Rather Be Writing blog, the specialist in documentation and technical writer Tom Johnson wrote a comment that goes to the root of the problem of using CMS:

No more black boxes that handle your content. No more expensive, proprietary systems to submit to. No more impossible-to-adjust-outputs. You can integrate it all simply, easily, and inexpensively.

The enormous influence these companies have in the world of software will, sooner or later, bring about the adoption of open bazaar-type documentation models to the detriment of rigid cathedral-type models. If effective and agile software development is the result of a collaborative process of continuous integration and publication, then documentation projects must necessarily adapt to this new reality.

(This article was originally published in Spanish and translated into English)

Microsoft marca el camino a seguir en documentación técnica

El nuevo servicio documental de Microsoft es mucho más que un cambio estético en la plataforma de contenidos.

La pasada semana, Microsoft dio a conocer su nueva plataforma documental docs.microsoft.com. Este cambio, detallado en la primera entrada del blog del equipo, incluye elementos que no resultan novedosos en sí mismos, pero que sí lo son cuando se trasladan al ámbito de la documentación técnica. El aspecto de la documentación es ahora más parecido al de un blog que al del clásico documento técnico, con artículos más cortos y la incorporación de elementos como el tiempo de lectura de cada artículo, tipografías de mayor tamaño y diversas herramientas de comentario que recuerdan instantáneamente a Medium.


Las nuevas opciones de comentario y petición de cambios se añaden al clásico cuadro de sugerencias al final de cada artículo (que se mantiene). Con todas estas opciones y la posibilidad de añadir notas a cada párrafo no queda claro, sin embargo, qué respuesta pueden esperar los usuarios y cómo se integrarán las sugerencias enviadas en el documento final.

Pero la novedad más relevante no es estética. La gestión de cambios se realiza ahora desde GitHub, lo que permite solicitar revisiones del documento fuente mediante peticiones pull en la plataforma. Y todo ello aprovechando la sencillez de Markdown, lenguaje cuyo uso se está popularizando en los últimos años. Este uso de la plataforma GitHub no es nuevo -de hecho, Microsoft lo utilizaba ya en otras partes de su documentación-, pero por su generalización y la importancia de Microsoft, marca un punto de inflexión y un camino a seguir por otros fabricantes.

Jeff Sandquist, autor del artículo, señala también que los cambios incorporados no son sólo estéticos o de funcionalidad, sino también de contenido. Aunque no es mucho más lo que se dice sobre este aspecto, el comentario parece indicar que la documentación estará más orientada a la realización de tareas, algo que se puede ver ya en los artículos técnicos de Azure.

Aquellos que, de una u otra forma, trabajamos con la documentación del fabricante, hemos visto cómo se sucedían pequeños cambios en los últimos años, pero sin ninguna transformación significativa. Microsoft acomete así una transición necesaria, después de un tiempo en los que sus tres grandes áreas documentales -TechNet, MSDN y más recientemente, Azure- parecían evolucionar por caminos distintos.

El contenido de la nueva plataforma es aún escaso, pero los cambios introducidos apuntan a una documentación más adaptada a las necesidades del usuario, mejor estructurada y posiblemente con menos errores, gracias a las nuevas herramientas de comentario.


(este artículo fue publicado originalmente en LinkedIn el 11-05-2016)