HowTo…¶
..go quick through most CliMAF features¶
Read the Grand Tour
..install CliMAF¶
..describe where my data is¶
Your data organization and datafiles maybe are already known to CliMAF; this is
the case for a number of projects and re-analyses; so, see first projects
If your data is organized after one of the specific organization scheme known
to CliMAF (i.e. for now CMPI5_DRS
or EM
) , but is located at
some personal location, you just have to tell CliMAF about the root directories for your data. This could be as simple as:
>>> dataloc(project="GEOMIP", organization="CMIP5_DRS", url=['/tmp/data','~/data/CMIP5'])
If this is not the case, you will most probably be happy with the ‘generic’ type of organization, which allows to describe the file hierarchy for your data using templates with keyowrds, such as in:
for local data:
>>> dataloc(project="MY_PROJECT", organization="generic", url=["/home/stephane/data/MY_PROJECT/${model}/${variable}_1m_YYYYMM_YYYYMM_${model}.nc"])
for remote data (yet only ftp):
>>> cproject("MY_REMOTE_DATA", ('frequency', "monthly"), separator='|') >>> dataloc(project="MY_REMOTE_DATA", organization="generic", >>> ... url=["ftp:vignonl@hendrix:/home/vignonl/${model}/${variable}_1m_YYYYMM_YYYYMM_${model}.nc","beaufix:/home/gmgec/mrgu/vignonl/L/${simulation}SFXYYYY.nc"]), >>> calias('MY_REMOTE_DATA','tas','tas',filenameVar='2T') >>> tas=ds(project='MY_REMOTE_DATA', simulation='AMIPV6ALB2G', variable='tas', frequency='monthly', period='198101')
where url is supposed to be in the format “protocol:user@ host:path”, but “protocol” and “user” are optional (that is to say part in italics is optional). So, url can also be “user@host:path” or “protocol:host:path” or “host:path”. ftp is default protocol (using $HOME/.netrc file).
See examples in the section on data access at Examples ; see function dataloc
for reference.
..get on-line help¶
If you are reading this text, you have access to the prettier form of documentation; consider going back to the front page for getting quick links and a top-level table of content. If this does not fit, or if you prefer digging the hard way try that : from the Python shell, all relevant CliMAF functions and variables are auto-documented using stndard Python documentation features. Hence, you can :
list these functions and variables by:
>>> dir(climaf.api)
get the general help on functions:
>>> help(climaf.api)
get help on a given function (say
cscript
) by:>>> help(cscript)
get help on a CliMAF operator (say
regrid
) by:>>> help(regrid)
..know which CliMAF operators are available and what they do¶
While the set of basic CliMAF operators should be quite stable at some stage, it may evolve significantly in the initial development stage; and it actually varies among CliMAF versions; it can also be extended with your own operators (see ..declare my favorite script or binary as a CliMAF operator for my own use).
Thus, in addition to the operators described in section Standard operators and functions of the current version of the documentation, you can also look at the Python dictionary which serves as an index to the actual list of operators for your version; just type:
>>> cscripts
You may then ask for on-line help for any of the scripts , as e.g.
>>> help(plotmap)
You may know how the operator interfaces with Climaf by
>>> cscripts['plotmap'].command
..tune CliMAF verbosity level¶
CliMAF uses the Python logging package logging
for informing
about work done, at varied verbosity levels. There are two logging
streams : one going to screen (or stderr), the other going to file
climaf.log
. See function clog()
for
setting the severity level for the former and function
clog_file()
for the latter
Note : at CliMAF startup, the severity level is set to the value of environment variable $CLIMAF_LOG (resp. $CLIMAF_FILE_LOG)
..report an issue¶
For the time being, you may report an issue on the CliMAF issue page on GitHub . Just click on the green button “New issue” (you may have to create a login on GitHub for that )
..declare my favorite script or binary as a CliMAF operator for my own use¶
Using any script in CliMAF is very easy, and you can do so for your own use only. The basics of creating a new CliMAF operator based on a script or binary are explained at Operators : using external scripts, binaries and python functions. A simple example shows at Example for interfacing a diagnostic script with CliMAF. The detailed syntax is explained at Syntax for interfacing a script..The script can be located anywhere on the filesystem (you can quote an absolute path in the calling sequence pattern when declaring it ); at first, your script should echo each executed command, for debugging purpose; and you may have a look at its output in file ./scripts.out
(in the working directory)
..contribute a diagnostic module¶
If you are willing to share as an Open Source sofwtare any diagnostic tool that can be integrated in CliMAF as a so-called ‘CliMAF standard operator’, you should first go through ..declare my favorite script or binary as a CliMAF operator for my own use for the basics of integrating it, and for testing the integration. At that stage, the actual binary or script will not be hosted in CliMAF installation directory. The next steps will be to :
think about a name for the corresponding CliMAF standard operator : it should not collide with existing operators (see Operators : using external scripts, binaries and python functions), and should be both short and explicit; let us call it
my_op
for now; if the diagnostic module has more than one (main) output, also think twice about the names for the secondary outputs (details here) as they will also join the ‘CliMAF Reference Syntax’if you are working with a version of CliMAF that has been installed by somebody else, you should now install your own; see Installing, configuring, using
if your module is a script, add its code in directory
<climaf_install_dir>/climaf/scripts
; the script filename is up to you, but should more or less ressemble or recall the name of the CliMAF operator choosen aboveif your module is a binary which needs some compilation, prepare a makefile for that, which ideally should be tested both with Intel and Gnu compilers (… TBD : think deeper on a tractable way to integrate binaries… please contribute to this analysis through email to
climaf at meteo dot fr
)check twice the Climaf function call that will allow to declare the operator in CliMAF, and adapt it to the new script location, as e.g.
>>> cscript ("my_op", cpath+"/scripts/"+"<calling sequence pattern>")
and edit file
<climaf_install_dir>/climaf/standard_operators.py
to add this callrestart a CliMAF session and check that your test script for this module (where the
cscript
declaration of the module should now be discarded) still works after these changesprepare a text providing a description of the diagnostic; this text is intended to become part of CliMAF standard documentation; it should preferably be in REStructured Text format, and follow
this template
which will render e.g. as shows here , or for a real examplelike this
, but this is not a firm pre-requisite. Save this text as<climaf_install_dir>/doc/operators/my_op.rst
. The text may describe your diagnostic at length, and should provide all necessary bibliographic references for a sound sharing. It will automatically be included in the CliMAF operator Python on-line help (available ashelp(my_op)
). It will be accessible from CliMAF doc if you add a reference in filedoc/std_operators.rst
submit your changes as described at Contributing changes for integration in CliMAF
..contribute to improve CliMAF documentation¶
CliMAF documentation is
built using Sphinx and can easily be
modified and improved, provided you are not afraid by looking at text
files formated using the REStructured Text syntax (check
that Sphinx is installed with which sphinx-build
)
All doc files stand in directory <climaf_install_dir>/doc
, with
quite easy-to-understand filenames. You can modify any file and test
the result by typing:
$ cd <climaf_install_dir>/doc
$ make -k html
and loading the resulting file <climaf_install_dir>/doc/_build/html/index.html
in your browser.
Note : part of the doc (e.g. section Available functions ) is built from the Python docstrings (strings at the beginning of classes, modules, functions, …..).
- This needs a Python module either
sphinx.ext.napoleon
orsphinxcontrib.napoleon
. If the doc build fails complaining about one of this package, you can just comment it out in file doc/conf.py, near line 38, and un-comment the other one. If none of these two packages is available at your site, you will not be able to build the doc. - When wanting to improve the doc for one of the Python functions or classes: the full python object path let you know where to find the corresponding file and docstring, in directory <climaf_install_dir>/climaf
Once happy with the result, please contribute your work for a merge in next CliMAF release as described at Contributing changes for integration in CliMAF
..define a new data organization scheme¶
Please see the ‘generic’ data organization in ..describe where my data is . if this
does not fit, please email to climaf at meteo dot fr
..record a CliMAF session¶
This is the same problem as recording any Python session. Just have a look at this thread , which proposes various solutions. The simplest is as simple as
import readline
readline.write_history_file('/home/my_user/my_history')