Markup languages / Documentation

Before starting a description of Sphinx, we first start by introducing, markdown, quarto and reStructuredText files.

Disclaimer: this course is mainly adapted from the Quarto and Sphinx documentation.

Markdown

Markdown is a lightweight markup language developed by John Gruber and Aaron Swartz1 .

1 Aaron Swartz: hacktivist (1986-2013) who also helped develop Creative Commons licenses, RSS, Reddit, etc. For more on his epic life, see Brian Knappenberger’ documentary The Internet’s Own Boy: The Story of Aaron Swartz (2014)

Hence Markdown is an easy-to-read markup language, popular for simple text formatting, creating documentation for software projects, but also for writing nice emails (for Thunderbird or any browser, etc.).

The extension for a Mardkown file is .md. To render such a file in VSCode or Codium, and possibly export a .pdf or .html file, you need to install a package for that (for instance the Markdown All in One could be installed with Ctrl+Shift+p, and looking for the right name in Extensions: Install Extensions).

Left: Markdown code


# Title level 1

## Title level 2


# Another title level 1

## Another title level 2


This would be a paragraph. You can use **bold font** but also *italic* and or  ~~strikethrough~~.

Other elements are listed below :

- item 1
- item 2
  - sub-item 1
  - sub-item 2

Right: rendered

Title level 1

Title level 2

Title level 1

Title level 2

This would be a paragraph. You can use bold font but also italic or strikethrough.

Other elements are listed here :

  • item 1
  • item 2
    • sub-item 1
    • sub-item 2

A full description can be found here: https://www.markdownguide.org/basic-syntax/; a nice cheatsheet is also available here: https://www.markdownguide.org/cheat-sheet/.

Quarto

Quarto is an open-source scientific and technical publishing system. For instance, we have used it for creating the website you are reading right now. We describe here its main usage. Quarto allows rendering markdown elements and creating a website easily, that can run some code (R, Python, etc.) and display the results (tables, figures, etc.), possibly in an interactive way. The extension is .qmd for Quarto files.

As an example, you can see below how to create a code snippet in a Quarto, and how it is rendered, along with the code output.

Left: Quarto code

```{python}
import numpy as np
print(np.sum(np.ones(12)))
```

Right: Rendered

import numpy as np
print(np.sum(np.ones(12)))
12.0

Documentation for using python in Quarto: https://quarto.org/docs/computations/python.html

reStructuredText

reStructuredText (.RST, .ReST, or .reST) is a file format for textual data used primarily in the Python programming language community for technical documentation and is similar to the Markdown format.

It is part of the Docutils project of the Python Doc-SIG (Documentation Special Interest Group), aimed at creating a set of tools for Python similar to Javadoc for Java or Plain Old Documentation (pod) for Perl or vignette for R.

Docutils can extract comments and information from Python programs, and format them into various forms of program documentation.

In this sense, reStructuredText is a lightweight markup language designed to be both:

  1. Processable by documentation-processing software such as Docutils.
  2. Easily readable by human programmers who are reading and writing Python source code.

References:

Syntax

A ReST file is a plain text file with a .rst extension. Like Markdown, it allows you to easily write formatted text.

Headers

Section Header
==============

Subsection Header
-----------------

Lists

- A bullet list item
- Second item

  - A sub-item (indentation matters!)

- Spacing between items creates separate lists

- Third item

1) An enumerated list item

2) Second item

   a) Sub-item that goes on at length and thus needs
      to be wrapped. Note the indentation that must
      match the beginning of the text, not the
      enumerator.

      i) List items can even include

         paragraph breaks.

3) Third item

#) Another enumerated list item

#) Second item

Images


.. image:: /path/to/image.jpg
   :height: 100
   :width: 200
   :scale: 50
   :align: center
   :alt: ordinateur

   Caption text rendered below the image...

Literal blocks

::
  some literal text

This may also be used inline at the end of a paragraph, like so::

Some more literal text:

.. code:: python

   print("A literal block directive explicitly marked as python code")

Sphinx: a documentation generator

Sphinx is an extension of reStructuredText.

The documentation of a python package is usually located in a docs or doc folder located at the root of a project. For instance, in the biketrauma module we have considered in the courses Creating a python module, the structure could be as follows:

packaging_tutorial/
    ├── biketrauma/
    │     ├── __init__.py
    │     ├── io/
    │     ├── preprocess/
    │     └── vis/
    │     └── data/
    ├── doc/
    ├── setup.py
    ├── script.py
    └── .gitignore

In the Sphinx terminology, this doc folder is called the source directory. It contains:

  1. A configuration file conf.py with all the information needed to read the sources and build the doc. By building, it is meant the process of generating the doc (usually in html, pdf, etc.) from the ReST files.
  2. A directory structure containing .md or .rst files with the doc.

To help you, Sphinx comes with a script called sphinx-quickstart that sets up a source directory and creates a default conf.py with the most useful configuration values from a few questions it asks you. To use this, run:

$ sphinx-quickstart

Answer each question asked. Be sure to say yes to the autodoc extension, as we will use this later. There is also an automatic API documentation (API: Application Programming Interface) generator called sphinx-apidoc; see sphinx-apidoc for details.

EXERCISE: Setting up your documentation

Set up the documentation for the biketrauma Python module.

  1. Install the sphinx package with pip
  2. Create a doc folder and cd into it
  3. Launch sphinx-quickstart --sep.

Defining documentation structure

Let us assume you have run sphinx-quickstart. It has created a source directory with conf.py and a master document, index.rst (if you accepted the default parameters).

The main function of the master document is to serve as a welcome page and to contain the root of the “table of contents tree” (or toctree). This is one of the main things that Sphinx adds to reStructuredText, a way to connect multiple files to a single hierarchy of documents.

The toctree directive initially is empty and looks like so:

.. toctree::
   :maxdepth: 2

You add documents listing them in the content of the directive:

.. toctree::
   :maxdepth: 2

   usage/installation
   usage/quickstart
   ...

This is exactly how the toctree for this documentation looks. The documents to include are given as document names, which in short means that you leave off the file name extension and use forward slashes (/) as directory separators.

EXERCISE: installing a documentation
  1. Update the index.rst: by adding an image located here just below the title of the page
  2. Install the read_the_doc theme following details given here. Additional themes are available on sphinx-doc themes.
  3. Create the corresponding directory and files to add:
    • An Installation section with a few sentences and code snippets that explain how to install biketrauma
    • A Documentation section with subsections io and visu each one containing a title and a few lines of text.

Building the doc

During the configuration of Sphinx, a text file called MakeFile was created: In software development, Make is a build automation tool that automatically builds executable programs and libraries from source code by reading files called Makefiles which specify how to derive the target program.

References:

$ make html

Then to access the web pages created:

$ firefox _build/html/index.html
Note

There is also a sphinx-build tool that can help you to build without make.

EXERCISE: Makefiles
  1. List all the target defined in the Makefiles.
  2. Build your doc and visualize it with a web navigator.

API doc (autodoc)

When documenting Python code, it is common to put a lot of documentation in the source files, in documentation strings. Sphinx supports the inclusion of docstrings from your modules with an extension (an extension is a Python module that provides additional features for Sphinx projects) called autodoc.

In order to use autodoc, you need to activate it in conf.py by putting the string 'sphinx.ext.autodoc' into the list assigned to the extensions config value. Then, you have a few additional directives at your disposal.

For example, to document the function io.open(), reading its signature and docstring from the source file, you’d write this:

.. autofunction:: io.open

You can also document whole classes or even modules automatically, using member options for the auto directives, like:

.. automodule:: io
   :members:

autodoc needs to import your modules to extract the docstrings. Therefore, you must add the appropriate path to sys.path in your conf.py.

EXERCISE: docstring
  1. Write a docstring for the class biketrauma.io.Load_db and the function plot_location.
  2. Integrate this documentation in a section called API in the Sphinx toctree.
Back to top