Writing Gamera toolkits
=======================

(c) 2004 Michael Droettboom

What is a toolkit?
------------------

A toolkit is a way to distribute code that uses Gamera but is not
included in the Gamera source tree.  This could be entire
applications that process images and return symbolic
results (eg. an OCR package), or simply a library of utility
functions (eg. for color image processing).

A toolkit is based on Python's generic package and module heirarchy,
which is described in the Modules chapter of the `Python tutorial`__.

.. __: http://www.python.org/doc/tut/

The Gamera toolkit framework actually provides very little beyond that:

- A special Python distutils-based framework for building Gamera
  plugins more conveniently.
- Support for adding a toolkit-specific drop-down menu to the Gamera
  GUI.

If neither of these features is necessary for your project, you may
decide to simply release your application or library as a standard
Python package.  `Distributing Python modules`__ in the Python
documentation is a good resource for how to do that.

.. __: http://www.python.org/doc/dist/

Creating a toolkit
------------------

The directory heirarchy
```````````````````````

Toolkits require a number of different files in a directory
heirarchy.  Here we assume the toolkit is called ``my_toolkit``.

   +----------+----------------------------------------------------------------+
   | ./       | Basic information files for building the toolkit               |
   |          +---------------+------------------------------------------------+
   |          | setup.py      | A Python ``distutils``-based build script.     |
   +----------+---------------+------------------------------------------------+
   | gamera/  | All the files needed by Gamera at runtime.                     |
   |          | Since Python is interpreted, these means                       |
   |          | Python source files.                                           |
   |          +---------------+------------------------------------------------+
   |          | toolkits/     | This is where the Python source code of the    |
   |          | my_toolkit    | toolkit goes.                                  |
   |          +---------------+------------------------------------------------+
   |          | toolkits/     | This is where the Gamera plugins for the       |
   |          | my_toolkit/   | toolkit go.                                    |
   |          | plugins/      |                                                |
   +----------+---------------+------------------------------------------------+
   | include/ | C++ header (``.hpp``) files.                                   |
   |          +---------------+------------------------------------------------+
   |          | plugins/      | Source code for the C++-based plugins.         |
   +----------+---------------+------------------------------------------------+
   | scripts/ | Command line scripts                                           |
   +----------+---------------+------------------------------------------------+
   | doc/     | Documentation                                                  |
   |          +---------------+------------------------------------------------+
   |          | gendoc.py     | A script to generate the documentation using   |
   |          |               | the Gamera documentation system.               |
   |          +---------------+------------------------------------------------+
   |          | src/          | The source files for the narrative             |
   |          |               | documentation.                                 |
   |          +---------------+------------------------------------------------+
   |          | html/         | The HTML output from the Gamera documentation  |
   |          |               | system.                                        |
   +----------+---------------+------------------------------------------------+

Some toolkits may go beyond this, of course, by including ``.cpp``
files in a ``src/`` directory or documentation in a ``doc/``
directory.

The skeleton toolkit
````````````````````

For convenience, a minimal skeleton of a toolkit is provided and
available from the files section of the `Gamera SourceForge site`__.

.. __: https://sourceforge.net/project/showfiles.php?group_id=99328&package_id=118473

This skeleton provides the very minimum needed to create a toolkit.
You will need to change all the references to the toolkit name
(Skeleton) throughout its source.  The ``rename.py`` script is
provided for this purpose. For example::

  python rename.py my_toolkit

will rename and edit all of the files to create a new toolkit called
``my_toolkit``.

Editing the files
`````````````````

The files included in the skeleton toolkit are self-documenting.  They
should require only minimal editing.  Mainly, toolkit authors will be
adding their own Python modules and Gamera plugins to the toolkit.

setup.py
''''''''

You only need to edit this file if you are doing anything more complex
than installing Python modules and building Gamera plugins.  For
instance, if you are building and linking to a third-party library.
Since this script is based on Python distutils, the distutils
documentation is the best resource for how to do that.

MANIFEST.in
'''''''''''

If you need to include more data files to your toolkit distrubution,
you will need to edit this file.  The format is described in the
distutils documentation.

gamera/toolkits/my_toolkit/__init__.py
''''''''''''''''''''''''''''''''''''''

If you want to add a drop-down menu to the Gamera GUI shell, you can
edit this file.  It is self-documenting.  You will probably want to
remove the example menu items that are included in the skeleton.

This file should also import any plugins that you want to have added
to the GUI's context menu (right-click menu).

Plugins
'''''''

`Writing plugins`__ is described in detail.  The Python metadata
files for a toolkit go in ``gamera/toolkits/my_toolkit/plugins/``, and
the C++ source code goes in ``include/plugins/``.

.. __: writing_plugins.html


Python modules
''''''''''''''

The Python modules in your toolkit should go in
``gamera/my_toolkit/skeleton``.

Documentation
'''''''''''''

Optionally, the toolkit may use the Gamera documentation system.  See
`Documenting toolkits`_ for more information.

Building and installing a toolkit
---------------------------------

Building and installing toolkits is very similar to building and
installing Gamera itself.  

**You must ensure that Gamera is installed and working before
attempting to build and install a Gamera toolkit. On Windows,
you should build and install gamera from the sources before
building a toolkit to avoid possible compiler incompatibilities.**

The complete instructions for building Gamera toolkits is included in
the skeleton example in the INSTALL file.  You should redistribute
this file with your toolkit.

Documenting toolkits
--------------------

Optionally, toolkit writers can use the Gamera documentation system.
Toolkits are also free to use another documentation
workflow if desired, of course.

To document a toolkit using the Gamera documentation system, the
author should perform four steps: 

  1) write narrative documentation,

  2) write plugin method documentation, 

  3) optionally write class and method documentation and 

  4) create a documentation generation script.

Writing narrative documentation
```````````````````````````````

The narrative (handwritten) documentation is intended for "how-to"
like documents or anything that is not simply documenting methods
one-by-one.

Put narrative documentation in ``doc/src/``, in ``.txt`` files in
reStructuredText_ format.  Each ``.txt`` file will be converted into a
corresponding ``.html`` when the documentation is generated.

.. _reStructuredText: http://docutils.sf.net/

Optionally, any inline images to be included can be placed in
``doc/src/images/``.

It is good practice to create an ``index.txt`` here which will act as
an entry-point to the documentation.

Plugin documentation
````````````````````

Document each plugin method in its Python docstring.  This docstring
should occur on the line following the ``class`` statement where each
``PluginFunction`` is defined.

Optionally, write a documentation example.  This is a snippet of code
that will be run to generate an image that demonstrates the
capabilities of the plugin function.  

These steps are described in greater detail in the `writing plugins`__
chapter.

.. __: writing_plugins.html#documenting-plugin-functions

By default, the documentation generator will document all imported
plugins.  Usually, this is the right thing to do.  However, your
toolkit may import plugins from other toolkits or from the Gamera core
that you do not wish to have documented.  In those cases, you can
specify the categories of plugins you want to be documented using the
``plugins`` keyword argument when calling the documentation
generator from your `documentation generation script`_.

For example, if the plugins in your toolkit were all in the category
"Foo", and you only want to document those, you would use:

.. code:: Python

   gendoc.gendoc(plugins=["Foo"])


Class documentation
```````````````````

Individual classes and their methods can be documented as well.  There
are two ways to do this: 1) inline with the narrative documentation,
or 2) one-class-per-file documentation.

Inline class documentation
''''''''''''''''''''''''''

The Gamera documentation system adds a new directive, ``docstring``, to
reStructuredText.  The directive looks up the docstring of a class,
function or method and inserts it inline into the document.  It takes
at least two arguments:

   *module*
     The module that the object is in.  This may be a
     dot-delimited package path, such as ``gamera.toolkits.skeleton``.
     
   *object*
      The Python object to document.

Additionally, any number of names may be added to the arugment list
which will be looked up in *object* and documented.

For example, to document the ``__init__`` and ``display`` methods on
the ``Image`` class in ``gamera.core``::

  .. docstring:: gamera.core Image
  .. docstring:: gamera.core Image __init__ display

This produces the following:

.. admonition:: Image

  .. docstring:: gamera.core Image
  .. docstring:: gamera.core Image __init__ display

One-class-per-file documentation
''''''''''''''''''''''''''''''''

To generate one-class-per-file documentation, use the
``classes`` keywoard argument when calling ``gendoc()`` (See below_).
This argument takes a list of 3-tuples, (*module*, *class*,
*methods*).  

   *module*
     The module that the object is in.  This may be a
     dot-delimited package path, such as ``gamera.toolkits.skeleton``.
     
   *class*
      The Python class to document.
  
   *members*
      The members to document, in order.  This may be either a list of
      strings (``['a', 'b', 'c']``) or a single string with method
      names separated by whitespace (``'a b c'``).

.. _below: #documentation-generation-script

For example, to document the ``__init__`` and ``display`` methods on
the ``Image`` class in ``gamera.core``.

.. code:: Python

   gendoc.gendoc(classes=[
                 ("gamera.core", "Image", "__init__ display")
	 	 ])

This will have the same effect as the example above, except it will
put the class documentation in its own file.

Documentation generation script
```````````````````````````````

A small script is required that loads the toolkits' plugins and then
calls out to the Gamera documentation system.  The example
documentation generation script included in the skeleton toolkit
example is as follows:

.. code:: Python

   #!/usr/bin/env python

   from gamera import gendoc

   if __name__ == '__main__':
      # Step 1:
      # Import all of the plugins to document.
      # Be careful not to load the core plugins, or they
      # will be documented here, too.
      # If the plugins are not already installed, we'll just ignore
      # them and generate the narrative documentation.
      try:
         from gamera.toolkits.skeleton.plugins import clear
      except ImportError:
         print "WARNING:"
         print "This `skeleton` toolkit must be installed before generating"
         print "the documentation.  For now, the system will skip generating"
         print "documentation for the plugins."
         print

      # Step 2:	 
      # Generate documentation for this toolkit
      # This will handle any commandline arguments if necessary

      # The optional classes argument can be used to document classes
      # See "Class documentation" in the "Writing toolkits" chapter.
      gendoc.gendoc()

Building the documentation
``````````````````````````

Once these three elements are in place, the documentation can be
generated and converted to HTML.

Prerequisites
'''''''''''''

Generating documentation requires two third-party Python
libraries:

  - docutils_ (version 0.3 or later) for handling reStructuredText documents.

  - SilverCity_ (version 0.9 or later) for colorizing source code.

.. _docutils: http://docutils.sourceforge.net/
.. _SilverCity: http://silvercity.sourceforge.net/

Generating
''''''''''

To generate the documentation, go to the ``doc`` directory in the
skeleton and run the ``gendoc.py`` script.

Alternatively, you can call the ``gendoc.py`` script with the
documentation directory as a commandline argument::

   gendoc.py -d /path/to/doc

The output will be placed in the ``doc/html/`` directory.  The contents of
this directory can be placed on a webserver for convenient viewing.

