..go quick through most CliMAF features

Read the Grand Tour

..install CliMAF

See Installing, configuring, using CliMAF, run an example

See Examples

..describe where my data is

Your data organization and datafiles maybe are already known to CliMAF; this is the case for a number of projects and re-analyses; so, see first projects

If your data is organized after one of the specific organization scheme known to CliMAF (i.e. for now CMPI5_DRS or EM) , but is located at some personal location, you just have to tell CliMAF about the root directories for your data. This could be as simple as:

>>> dataloc(project="GEOMIP", organization="CMIP5_DRS", url=['/tmp/data','~/data/CMIP5'])

If this is not the case, you will most probably be happy with the ‘generic’ type of organization, which allows to describe the file hierarchy for your data using templates with keyowrds, such as in:

  • for local data:

    >>> dataloc(project="MY_PROJECT", organization="generic", url=["/home/stephane/data/MY_PROJECT/${model}/${variable}_1m_YYYYMM_YYYYMM_${model}.nc"])
  • for remote data (yet only ftp):

    >>> cproject("MY_REMOTE_DATA", ('frequency', "monthly"), separator='|')
    >>> dataloc(project="MY_REMOTE_DATA", organization="generic",
    >>> ... url=["ftp:vignonl@hendrix:/home/vignonl/${model}/${variable}_1m_YYYYMM_YYYYMM_${model}.nc","beaufix:/home/gmgec/mrgu/vignonl/L/${simulation}"]),
    >>> calias('MY_REMOTE_DATA','tas','tas',filenameVar='2T')
    >>> tas=ds(project='MY_REMOTE_DATA', simulation='AMIPV6ALB2G', variable='tas', frequency='monthly', period='198101')

    where url is supposed to be in the format “protocol:user@ host:path”, but “protocol” and “user” are optional (that is to say part in italics is optional). So, url can also be “user@host:path” or “protocol:host:path” or “host:path”. ftp is default protocol (using $HOME/.netrc file).

See examples in the section on data access at Examples ; see function dataloc for reference.

..get on-line help

If you are reading this text, you have access to the prettier form of documentation; consider going back to the front page for getting quick links and a top-level table of content. If this does not fit, or if you prefer digging the hard way try that : from the Python shell, all relevant CliMAF functions and variables are auto-documented using stndard Python documentation features. Hence, you can :

  • list these functions and variables by:

    >>> dir(climaf.api)
  • get the general help on functions:

    >>> help(climaf.api)
  • get help on a given function (say cscript) by:

    >>> help(cscript)
  • get help on a CliMAF operator (say regrid) by:

    >>> help(regrid)

..know which CliMAF operators are available and what they do

While the set of basic CliMAF operators should be quite stable at some stage, it may evolve significantly in the initial development stage; and it actually varies among CliMAF versions; it can also be extended with your own operators (see ..declare my favorite script or binary as a CliMAF operator for my own use).

Thus, in addition to the operators described in section Standard operators and functions of the current version of the documentation, you can also look at the Python dictionary which serves as an index to the actual list of operators for your version; just type:

>>> cscripts

You may then ask for on-line help for any of the scripts , as e.g.

>>> help(plotmap)

You may know how the operator interfaces with Climaf by

>>> cscripts['plotmap'].command

..tune CliMAF verbosity level

CliMAF uses the Python logging package logging for informing about work done, at varied verbosity levels. There are two logging streams : one going to screen (or stderr), the other going to file climaf.log . See function clog() for setting the severity level for the former and function clog_file() for the latter

Note : at CliMAF startup, the severity level is set to the value of environment variable $CLIMAF_LOG (resp. $CLIMAF_FILE_LOG) an issue

For the time being, you may report an issue on the CliMAF issue page on GitHub . Just click on the green button “New issue” (you may have to create a login on GitHub for that )

..declare my favorite script or binary as a CliMAF operator for my own use

Using any script in CliMAF is very easy, and you can do so for your own use only. The basics of creating a new CliMAF operator based on a script or binary are explained at Operators : using external scripts, binaries and python functions. A simple example shows at Example for interfacing a diagnostic script with CliMAF. The detailed syntax is explained at Syntax for interfacing a script..The script can be located anywhere on the filesystem (you can quote an absolute path in the calling sequence pattern when declaring it ); at first, your script should echo each executed command, for debugging purpose; and you may have a look at its output in file ./scripts.out (in the working directory)

..contribute a diagnostic module

If you are willing to share as an Open Source sofwtare any diagnostic tool that can be integrated in CliMAF as a so-called ‘CliMAF standard operator’, you should first go through ..declare my favorite script or binary as a CliMAF operator for my own use for the basics of integrating it, and for testing the integration. At that stage, the actual binary or script will not be hosted in CliMAF installation directory. The next steps will be to :

  1. think about a name for the corresponding CliMAF standard operator : it should not collide with existing operators (see Operators : using external scripts, binaries and python functions), and should be both short and explicit; let us call it my_op for now; if the diagnostic module has more than one (main) output, also think twice about the names for the secondary outputs (details here) as they will also join the ‘CliMAF Reference Syntax’

  2. if you are working with a version of CliMAF that has been installed by somebody else, you should now install your own; see Installing, configuring, using

  3. if your module is a script, add its code in directory <climaf_install_dir>/climaf/scripts; the script filename is up to you, but should more or less ressemble or recall the name of the CliMAF operator choosen above

  4. if your module is a binary which needs some compilation, prepare a makefile for that, which ideally should be tested both with Intel and Gnu compilers (… TBD : think deeper on a tractable way to integrate binaries… please contribute to this analysis through email to climaf at meteo dot fr)

  5. check twice the Climaf function call that will allow to declare the operator in CliMAF, and adapt it to the new script location, as e.g.

    >>> cscript ("my_op", cpath+"/scripts/"+"<calling sequence pattern>")

    and edit file <climaf_install_dir>/climaf/ to add this call

  6. restart a CliMAF session and check that your test script for this module (where the cscript declaration of the module should now be discarded) still works after these changes

  7. prepare a text providing a description of the diagnostic; this text is intended to become part of CliMAF standard documentation; it should preferably be in REStructured Text format, and follow this template which will render e.g. as shows here , or for a real example like this , but this is not a firm pre-requisite. Save this text as <climaf_install_dir>/doc/operators/my_op.rst. The text may describe your diagnostic at length, and should provide all necessary bibliographic references for a sound sharing. It will automatically be included in the CliMAF operator Python on-line help (available as help(my_op)). It will be accessible from CliMAF doc if you add a reference in file doc/std_operators.rst

  8. submit your changes as described at Contributing changes for integration in CliMAF

..contribute to improve CliMAF documentation

CliMAF documentation is built using Sphinx and can easily be modified and improved, provided you are not afraid by looking at text files formated using the REStructured Text syntax (check that Sphinx is installed with which sphinx-build)

All doc files stand in directory <climaf_install_dir>/doc, with quite easy-to-understand filenames. You can modify any file and test the result by typing:

$ cd <climaf_install_dir>/doc
$ make -k html

and loading the resulting file <climaf_install_dir>/doc/_build/html/index.html in your browser.

Note : part of the doc (e.g. section Available functions ) is built from the Python docstrings (strings at the beginning of classes, modules, functions, …..).

  • This needs a Python module either sphinx.ext.napoleon or sphinxcontrib.napoleon. If the doc build fails complaining about one of this package, you can just comment it out in file doc/, near line 38, and un-comment the other one. If none of these two packages is available at your site, you will not be able to build the doc.
  • When wanting to improve the doc for one of the Python functions or classes: the full python object path let you know where to find the corresponding file and docstring, in directory <climaf_install_dir>/climaf

Once happy with the result, please contribute your work for a merge in next CliMAF release as described at Contributing changes for integration in CliMAF

..define a new data organization scheme

Please see the ‘generic’ data organization in ..describe where my data is . if this does not fit, please email to climaf at meteo dot fr

..record a CliMAF session

This is the same problem as recording any Python session. Just have a look at this thread , which proposes various solutions. The simplest is as simple as

import readline