Documentation Automation

    While working on projects related to avionics, I needed to draw up several sets of documentation with a full description of the project. It was also necessary to take into account the requirements of many GOSTs on the design and content of documentation, such as ESPD, KT-178B and others.

    The description should include:
    • Software development plans
    • Software requirements
    • Description of software requirements implementation
    • Traceability (compliance) tables of software and implementation requirements
    • Description of software tests (Examples and software verification procedures)
    • Traceability (compliance) tables of software requirements and tests
    • Problem Report
    • Configuration pointer (description of software version and compatibility with third-party software and hardware)


    The volume of documentation is very large. The data in all documents are related to each other, so when you change the project (for example, adding a new requirement), you have to edit almost all the documents. Plus, you can make a mistake somewhere or forget to fix it, which leads to errors in the documentation.



    Later in the article I will describe how I solved this problem.



    Documentation Generator Architecture



    Therefore, it was decided to use automated tools that create all documents using data from primary documents - tables in CSV format, XML documents. With any changes in the project - you can restart the generation of the documentation set.

    CSV format tables are conveniently edited in a table processor. The project data (current version, name, compatible equipment) was stored in XML format.

    A description of the implementation of the requirements is already contained in the doxygen comments on the source code. Doxygen specifically for such cases can generate documentation in XML format.

    The documentation generator, based on document templates, creates LaTeX documents, which are already transferred to the customer in PDF format.


    Documentation Creation Chart
    Supervisor -> (Plans)
    Supervisor -> (Requirements)
    Supervisor -> (Description of tests)
    Supervisor -> (Detected problems)
    (Requirements) -> Programmers
    Programmers -> (Program code)
    (Program code) -> Doxygen
    Doxygen -> (Description of implementation)

    (Document templates) ->: Documentation generator :: LaTeX

    (Requirements) ->: Documentation generator :: CSV
    (Plans) ->: Documentation generator :: LaTeX
    (Description of implementation) - ->: Generator documents the XML ::
    (test Description) ->: Generator documents the XML ::
    (a problem is detected) ->: documentation Generator :: a CSV

    : documentation generator and -> (Set of documents): LaTeX


    Documentation Generator


    To implement such a system for creating documentation, I needed a utility for processing templates and a build script.

    I implemented the build script in the Makefile. The script performed the following actions:
    • I got the source
    • Generated XML Descriptions Using Doxygen
    • Collected all necessary documents from templates using pytemplate.py
    • Generated PDFs by LaTeX
    • Generated a folder tree and created a disk image for recording
    • formed the necessary supporting documentation (file with title pages, disc label)



    Documentation Generation Sequence Diagram
    GIT -> “Documentation Generator”: Source texts
    “Documentation Generator” -> Doxygen: Source texts of
    Doxygen -> “Documentation Generator”: XML description
    “Project Data” -> “Documentation Generator”: Templates
    “Documentation Generator” -> PyTemplate:
    Project Data Templates -> PyTemplate: CSV, XML
    PyTemplate-> LaTeX: LaTeX documents
    LaTeX -> Documentation Generator: PDF documents


    A key element of the system is a document template processing utility.

    Template Processing Utility


    Source codes can be obtained: github.com/krotos139/pytemplate

    Or you can install the utility using the command:
    sudo pip install pytemplateproc
    


    Utility usage: pytemplate.py [options]
    Options:
    • --version Display version
    • -h, --help Display startup key information
    • -t TEMPLATE, --template = TEMPLATE Specify the path to the template file
    • -o OUTPUT, --output = OUTPUT Specify the path to the output file
    • -f FORMAT, --format = FORMAT The format of the template file, can take values ​​(odt and text)
    • -a ARG, --arg = ARG Additional entity for the template


    The templates contain information - data from which external sources he needs. The utility during processing the template loads the necessary data, and uses it when filling out the template with data.

    Supported data sources:
    • CSV table (It can be edited in Exel subject to certain rules)
    • XML document
    • Text file
    • SQLite database
    • MD5 function from file
    • File Data Retrieval Function


    The template file and the path to the resulting file are transferred to the utility. Paths to data sources are not transferred to the program, since they are all defined in the template, and one template can use many different data sources.

    Template example:
    {%- set docs = load_csv("database2.csv")  %}
    \subsection{Список документов}
    \begin{longtable}{|m{2cm}|m{3cm}|m{3cm}|m{3cm}|m{3cm}|}
    \caption{Списки документов} \label{tab:reports}\\\hline
    {\centering Код} & {\centering Наименование} & {\centering Стандарт} & {\centering Децимальный номер} & {\centering Инвентарный} \\\hline
    \endfirsthead
    \caption*{\it{Продолжение таблицы} \ref{tab:reports}}\\\hline
    {\centering Код} & {\centering Наименование} & {\centering Стандарт} & {\centering Децимальный номер} & {\centering Инвентарный} \\\hline
    \endhead
    {%- for item in docs %}
    {{ item.id }} & {{ item.name }} & {{ item.ref }} & {{ item.sign }} & {{ item.inv }} \\\hline
    {%- endfor %}
    \end{longtable}
    \newpage
    


    Conclusion


    Using the pytemplate utility allows you to create documents and reports on templates using data to fill out templates. At the same time, data can be stored in a convenient format for spreadsheets or databases.

    Also popular now: