ContextQMD
Back to Arrow

Building the Documentation

docs/source/developers/documentation.rst

latest7.9 KB
Original Source

.. Licensed to the Apache Software Foundation (ASF) under one .. or more contributor license agreements. See the NOTICE file .. distributed with this work for additional information .. regarding copyright ownership. The ASF licenses this file .. to you under the Apache License, Version 2.0 (the .. "License"); you may not use this file except in compliance .. with the License. You may obtain a copy of the License at

.. http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing, .. software distributed under the License is distributed on an .. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY .. KIND, either express or implied. See the License for the .. specific language governing permissions and limitations .. under the License.

.. _building-docs:

Building the Documentation

Prerequisites

The documentation build process uses Doxygen <http://www.doxygen.nl/>_ and Sphinx <http://www.sphinx-doc.org/>_ along with a few extensions.

If you're using Conda, the required software can be installed in a single line:

.. code-block:: shell

conda install -c conda-forge --file=arrow/ci/conda_env_sphinx.txt

.. note::

linuxdoc cannot be installed by Conda. It has to be installed via pip separately.

Otherwise, you'll first need to install Doxygen <http://www.doxygen.nl/>_ yourself (for example from your distribution's official repositories, if using Linux). Then you can install the Python-based requirements with the following command:

.. code-block:: shell

pip install -r arrow/docs/requirements.txt

Building

.. note::

If you are building the documentation on Windows, not all sections may build properly.

These two steps are mandatory and must be executed in order.

#. Process the C++ API using Doxygen

.. code-block:: shell

  pushd arrow/cpp/apidoc
  doxygen
  popd

#. Build the complete documentation using Sphinx.

.. note::

  If you are working on the Python bindings documentation then
  this step requires that ``pyarrow`` library is installed
  in your python environment.  One way to accomplish
  this is to follow the build instructions at :ref:`python-development`
  and then run ``pip install --no-build-isolation .`` in arrow/python
  (it is best to do this in a dedicated conda/virtual environment).
  Add ``-vv`` to the pip command to get output of the build process.

  You can still build the documentation without ``pyarrow``
  library installed but note that Python part of the documentation
  will be missing from the ``_build/html`` file structure and the
  links to the Python documentation will get broken.

.. code-block:: shell

  pushd arrow/docs
  make html
  popd

.. note::

Note that building the documentation may fail if your build of pyarrow is not sufficiently comprehensive. Portions of the Python API documentation will also not build without CUDA support having been built.

After these steps are completed, the documentation is rendered in HTML format in arrow/docs/_build/html. In particular, you can point your browser at arrow/docs/_build/html/index.html to read the docs and review any changes you made.

.. note::

If you are working on the Python documentation and are building the documentation with pyarrow build from source on macOS Monterey, the Python section of the documentation might not be included in the _build/html. In this case, try installing pyarrow in non-editable mode first before running the make html command.

.. code-block:: shell

 pushd arrow/docs
 python -m pip install ../python --quiet
 make html
 popd

Building with Docker

You can use :ref:Archery <archery> to build the documentation within a Docker container.

.. code-block:: shell

archery docker run -v "${PWD}/docs:/build/docs" debian-docs

The final output is located under the ${PWD}/docs directory.

.. seealso::

:ref:docker-builds.

.. _building-docs-pr-preview:

Building a docs preview in a Pull Request

You can build and preview the documentation within a GitHub pull request you are working on.

To do so, post the comment @github-actions crossbow submit preview-docs to the pull request. The rendered documentation will then be available within the GitHub Actions response, where you need to click on the Crossbow build badge:

.. figure:: ./images/docs_preview_1.jpeg :scale: 70 % :alt: GitHub Actions response with the crossbow build status.

Crossbow build status

and then in the summary of the workflow you can find the link to the Docs Preview summary at the bottom of the page:

.. figure:: ./images/docs_preview_2.jpeg :scale: 70 % :alt: Crossbow workflow page with the Docs Preview summary section.

Docs Preview summary section

Building for dev purposes

Building subsections ^^^^^^^^^^^^^^^^^^^^

To make it easier for developers to update parts of the documentation, one can build only a subset of it. You can build:

  • Specifications and protocol section (docs/source/format) with:

    .. code-block:: shell

    pushd arrow/docs make format popd

    Rendered HTML format can be found in arrow/docs/_build/html/format.

  • Development section (docs/source/developers) with:

    .. code-block:: shell

    pushd arrow/docs make dev popd

    Rendered HTML format can be found in arrow/docs/_build/html/developers.

  • C++ section (docs/source/cpp) with:

    .. code-block:: shell

    pushd arrow/docs make cpp popd

    Rendered HTML format can be found in arrow/docs/_build/html/cpp.

  • Python section (docs/source/python) with:

    .. code-block:: shell

    pushd arrow/docs make python popd

    Rendered HTML format can be found in arrow/docs/_build/html/python.

.. note::

When building only a part of the documentation the links will get broken!

For this reason building only a subset of the documentation should only be used in the initial work as it makes the building faster and easier.

To check for the correctness of the documentation overall one should build the whole documentation with make html or use :ref:GitHub Actions <building-docs-pr-preview>.

Building live ^^^^^^^^^^^^^

You can also build the documentation (or a part of it) live. This means the changes saved will automatically trigger the documentation to be rebuilt.

.. code-block:: shell

pushd arrow/docs make html-live

The same way one could use make format-live, make dev-live, make cpp-live or make python-live to auto-build part of the documentation.

Building a single directory for dev purposes without all the pre-requisites ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You can build documentation in a single directory without needing to install all of the pre-requisites by installing sphinx, setting up a temporary index in the directory you want to build and then building that directory.

The example below shows how to do this in the arrow/docs/source/developers directory.

Install sphinx:

.. code-block:: shell

pip install sphinx

Navigate to the arrow/docs directory:

.. code-block:: shell

cd arrow/docs

Create an temporary index file temp_index.rst file in the target directory:

.. code-block:: shell

echo $'.. toctree::\n\t:glob:\n\n\t*' > ./source/developers/temp_index.rst

Build the docs in the target directory:

.. code-block:: shell

sphinx-build ./source/developers ./source/developers/_build -c ./source -D master_doc=temp_index

This builds everything in the target directory to a folder inside of it called _build using the config file in the source directory.

Once you have verified the HTML documents, you can remove temporary index file:

.. code-block:: shell

rm ./source/developers/temp_index.rst