Build Status

1. Purpose

giblish is used to convert a source directory tree containing AsciiDoc files to a destination directory tree containing the corresponding html or pdf files and adds some handy tools for easier navigation of the resulting files.

An example of how giblish can generate the master branch of the official asciidoc.org documentation git repo can be found at http://www.rillbert.se/adoc/examples/adocorg/master/myindex.html

The added tools include:

  • An index page listing all rendered documents with clickable links.

  • A (stripped-down but nonetheless useful) text-search of your (html) documents (requires that you view your docs via a web-server.

  • If the source directory tree is part of a git repository, giblish can generate separate html/pdf trees for branches and/or tags that match a user specified regexp (see examples below).

  • Document ids - Note: the implementation of this is giblish-specific and thus you need to render your adoc files using giblish to make this work as intended. You can use document ids to:

    • Reference one doc in the source tree from another doc without depending on file names or relative paths. The referenced doc can thus be moved within the source tree or change its file name and the reference will still be valid.

    • Validate doc id references during document rendering and thus be alerted to any invalid doc id references.

    • Let giblish generate a clickable graph of all document references (requires graphviz and the 'dot' tool).

Note
 giblish shall be considered stable from a usage perspective, please report bugs to the issue tracker. It’s API however, is to be considered alpha, meaning that future versions may break the current usage of cmd line flags, generated artifacts, styling or other functionality.

2. Dependencies and credits

Giblish uses the awesome asciidoctor and asciidoctor-pdf projects under the hood. Thank you @mojavelinux and others for making these brilliant tools available!!

3. Installation

gem install giblish

Want to get started straight away? Go directly to the Usage Examples.

3.1. Some caveats

When using giblish for generating docs the following applies:

  • giblish will overwrite files with the same name in the destination directory.

  • giblish requires that the git working tree and index of the repo containing source documents are clean when generating documentation.

  • giblish will make explicit check-outs of all the branches or tags that matches the selection criteria. The working dir of the source git repo will thus have the last branch that giblish checked-out as the current branch after doc generation.

4. Text search implementation

The text search enables a user to search for a text string and receive matching sections in the documentation tree.

giblish ties together the following three pieces to enable the text search:

  1. the source text of all adoc files together with a JSON file that maps sections to their line numbers.

    • giblish collects this 'search data' when it generates the html files to the destination directory. The JSON file and all adoc source files are copied to a well-known place in the destination tree (see below).

  2. an html form somewhere on the rendered pages where the user can input search queries and initiate a search.

    • giblish injects such an html form in the generated index page when the user specifies the '-m' switch.

  3. a server side script that handles a user request to search the documents for a specific text string and presents the result to the user.

    • this gem contains an implementation of such a server side script. It is intended to be run as a cgi script and requires ruby and grep to be installed on the server where it runs.

4.1. Search data and html form parameters

giblish will copy all search data to a 'search_assets' dir just under the destination root. This is illustrated below.

When rendering documents from a git branch
dst_root_dir
|- branch_1_top_dir
|     |- index.html
|     |- file_1.html
|     |- dir_1
|     |   |- file2.html
|- branch_2_top_dir
|- branch_x_...
|- web_assets
|- search_assets
|     |- branch_1_top_dir
|           |- heading_index.json
|           |- file1.adoc
|           |- dir_1
|           |   |- file2.adoc
|           |- ...
|     |- branch_2_top_dir
|           | ...
When rendering documents not in a git branch
dst_root_dir
|- index.html
|- file_1.html
|- dir_1
|   |- file2.html
|...
|- web_assets (only if a custom stylesheet is used...)
|- search_assets
|     |- heading_index.json
|     |- file1.adoc
|     |- dir_1
|     |   |- file2.adoc
|     |- ...

5. Document ids and the reference graph

Note
 This is a non-standard extension of asciidoc. If you use this feature, you will need to generate your documents using giblish to make this work as intended.

giblish extends the cross reference concept in asciidoc with a document id mechanism. To use this, you need to:

  1. Add a :docid: entry in your document’s header section. The doc id can consist of up to 10 characters and must be unique within the set of documents generated by giblish.

  2. Refer to a document using the syntax <<:docid:#DOC_ID#>>.

  3. Run giblish with the -d switch when generating documents.

Using doc ids makes it possible for giblish to do two things:

  1. Make the reference from one document to another work even if one of the documents have been moved within the source tree.

  2. Produce a clickable 'map' of the generated documents where the different references are clearly seen (this feature require that the 'dot' tool, part of the graphwiz package is installed on the machine where giblish is run).

The use of the -d switch makes giblish parse the document twice, once to map up the doc ids and all references to them, once to actually generate the output documentation. Thus, you pay a performance penalty but this should not be a big inconvenience since the generation is quite fast in itself.

5.1. Example of using the docid feature

Consider that you have two documents located somewhere in the same folder tree, document one and document two. You could then use the docid feature of giblish to refer to one document from the other as in the example below.

Example document one
= Document one
:toc:
:numbered:
:docid: D-001

== Purpose

To illustrate the use of doc id.
Example document two
= Document two
:toc:
:numbered:
:docid: D-002

== Purpose

To illustrate the use of doc id. You can refer to document one as <<:docid:D-001>>.
This will display a clickable link with the doc id (D-001 in this case).

You can basically follow the same syntax as the normal asciidoc cross-ref, such as:

 * <<:docid:D-002#purpose>> to refer to a specific section or anchor.
 * <<:docid:D-002#purpose,The purpose section>> to refer to a specific section and
   display a specific text for the link.

The above reference will work even if either document changes location or file name as long as both documents are parsed by giblish in the same run.

6. Usage Examples

Here follows a number of usages for giblish in increasing order of complexity.

6.1. Get available options

giblish -h

6.2. Giblish html 'hello world'

giblish my_src_root my_dst_root

  • convert all .adoc or .ADOC files under the dir my_src_root to html and place the resulting files under the my_dst_root dir.

  • generate an index page named index.html that contains links and some info about the converted files. The file is placed in the my_dst_root dir.

The default asciidoctor css will be used in the html conversion.

6.3. Giblish pdf 'hello world'

giblish -f pdf my_src_root my_dst_root

  • convert all .adoc or .ADOC files under the dir my_src_root to pdf and place the resulting files under the my_dst_root dir.

  • generate an index page named index.pdf that contains links and some info about the converted files. The file is placed in the my_dst_root dir.

The default asciidoctor pdf theme will be used in the pdf conversion.

6.4. Using a custom css for the generated html

Generate html that can be browsed locally from file:://<my_dst_root>.

giblish -r path/to/my/resources -s mylayout my_src_root my_dst_root

  • convert all .adoc or .ADOC files under the dir my_src_root to html and place the resulting files under the my_dst_root dir.

  • generate an index page named index.html that contains links and some info about the converted files. The file is placed in the my_dst_root dir.

  • copy the css, fonts and images directories found under <working_dir>/path/to/my/resources to my_dst_root/web_assets

  • link all generated html files to the css found at /web_assets/css/mylayout.css

6.5. Using a custom pdf theme for the generated pdfs

giblish -f pdf -r path/to/my/resources -s mylayout my_src_root my_dst_root

  • convert all .adoc or .ADOC files under the dir my_src_root to pdf and place the resulting files under the my_dst_root dir.

  • generate an index page named index.pdf that contains links and some info about the converted files. The file is placed in the my_dst_root dir.

  • the generated pdf will use the theme found at <working_dir>/path/to/my/resources/themes/mylayout.yml

6.6. Generate html from multiple git branches

giblish -g "feature" my_src_root my_dst_root

  • check-out each branch matching the regexp "feature" in turn

  • for each checked-out branch,

    • convert the .adoc or .ADOC files under the dir my_src_root to html.

    • place the resulting files under the my_dst_root/<branch_name> dir.

    • generate an index page named index.html that contains links and some info about the converted files. The file is placed in the my_dst_root/<branch_name dir.

  • generate a summary page containing links to a all branches and place it in the my_dst_root dir.

6.7. Publish the asciidoctor.org documents with text search

giblish can be used to generate html docs suitable for serving via a web server (e.g. Apache). You can use the cgi script included in the giblish gem to provide text search capabilities.

Here is an example of how to publish the docs in the official asciidoctor.org git repo to a web server, including index pages and text search.

Note
 No consideration has been taken to how permissions are set up on the web server. Just running the below commands as-is on e.g.a standard apache set-up will bail out with 'permission denied' errors.

6.7.1. Generating the html documents

The example assumes that you have one machine where you generate the documents and another machine that runs a web server to which you have ssh access.

The generated docs shall be accessible via www.example.com/adocorg/with_search

You need to provide two pieces of deployment info to giblish when generating the documents:

  • the uri path where to access the deployed docs (/adocorg/with_search in this example)

  • the path in the local file system on the web server where the search data can be accessed (/var/www/html/site_1/adocorg/with_search in this example)

The following commands will generate the asciidoctor.org documentation and deploy the result to the web server.

  1. Clone the asciidoctor doc repo to your development machine

    git clone https://github.com/asciidoctor/asciidoctor.org.git

  2. Generate the html documentation with the correct deployment info

    giblish -j '^.*_include.*' -m -mp /var/www/html/site_1/adocorg/with_search -w /adocorg/with_search -g master --index-basename "myindex" asciidoctor.org/docs ./generated_docs
    Explanation of the parameters and arguments

    • -j '^.*_include.*' - exclude everything in the _include directory. (the asciidoctor.org repo stores partial docs here).

    • -m - assemble the necessary search data to support text search and include this data as part of the generated documents.

    • -mp /var/www/html/site_1/adocorg/with_search - the file system path on the deployment machine where the search data is located

    • -w /adocorg/with_search - the uri path to the deployed docs

    • -g master - publish all git branches that matches the regexp 'master' (i.e. only the 'master' branch).

    • --index-basename "myindex" - change the default name (index) that giblish uses for the generated index page. This is needed since asciidoctor.org contains an "index.adoc" file that would otherwise be overwritten by giblish.

    • asciidoctor.org/docs - the root of the source document tree.

    • ./generated_docs - a temporary storage for the generated html docs on the local system.

  3. Copy the generated files to the web server

    scp -r ./generated_docs [email protected]:/var/www/adocorg/with_search/.

6.7.2. Copy the text search script to the web server

This only needs to be done once (or if a new version of giblish breaks the currently used API).

Important
 The current version of giblish expects the script to be found at the URI path /cgi-bin/giblish-search.cgi. This might be customizable in future versions but is currently hard-coded. Thus, if your web-server serves pages at www.mywebsite.com, the search script must be accessible at www.mywebsite.com/cgi-bin/giblish-search.cgi

  1. Find the server side script that implements text search that is included with giblish

    gem which giblish

    In my case this returns /var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib/giblish.rb. This means that I will find the script in the same directory, i.e. /var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib.

  2. Copy the server side script to the /cgi-bin dir on the web server. In this example the cgi-bin dir is configured to be /var/www/cgi-bin

    scp /var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib/giblish-search.rb [email protected]:/var/www/cgi-bin/giblish-search.cgi