CliMAF tour¶
CliMAF is a framework for Climate Model Assessment.
It allows to combine diagnostic binaries, scripts and Python functions as easily as possible
It is documented on ReadTheDocs
1 - Init CliMAF, and resets its cache¶
In [1]:
from climaf.api import *
craz()
Depending on the depth of the demo , one may tell CliMAF to be more or less verbose (both on stderr and on logging file). Levels are : critical < error < warning (stderr default) < info (log file default) < debug¶
In [2]:
#clog('info')
#clog_file('debug')
2 - Data location and datasets¶
A 'project' means a data organization and a number of specific facets.
It can have multiple data location roots.
Let us declare project 'example' with only one additional facet.
Note : All that is usually defined done in config files.
In [3]:
cproject("example" , ("frequency","monthly") )
print cprojects['example'].facets
For this project, we will use data in the distro.¶
In [4]:
dataloc(project="example",organization="generic",url=[\
cpath+"/../examples/data/${simulation}/L/${simulation}SFXYYYY.nc", \
cpath+"/../examples/data/${simulation}/A/${simulation}PLYYYY.nc"])
Out[4]:
Define a default value for some dataset facets¶
In [5]:
cdef("project","example")
cdef("period","1980-1981")
Define a dataset based on a given simulation, and one variable (this uses the facets default values)¶
In [6]:
tas_ds=ds(simulation="AMIPV6ALB2G", variable="tas")
You can also define a dataset restricted to a latlon box¶
In [7]:
tas_ds_r=ds(simulation="AMIPV6ALB2G", variable="tas", domain=[10,80,-50,40])
Have a look at dataset's symbolic expression (other patterns could apply , according to project's "DRS")¶
In [8]:
tas_ds_r
Out[8]:
CliMAF can identify all the files relevant to the dataset (see CMIP5 DRS case further below, § 10)¶
In [9]:
tas_ds.baseFiles()
Out[9]:
It can also hande other organizations : CMIP5 DRS (see below), EM, CAMIobs and OBS4MIPs at CNRM, OCMIP5 on IPSL's Ciclad¶
To TOC
3 - Using external scripts and binaries for implementing diagnostics¶
First diagnostic is basic climatology - Let us declare a 'CliMAF operator' for this diag, using an external binary and a few keywords conventions¶
In [10]:
cscript('time_mean' ,'cdo timmean ${in} ${out}' )
Out[10]:
Declaring a 'CliMAF operator' actually creates a Python function¶
In [11]:
time_mean
Out[11]:
A doc string is available for each 'operator function' as long a Rest syntax doc file exists with same name. Here, we just have the default doc.¶
In [12]:
help(time_mean)
Symbolically apply the script to a dataset, using the 'CliMAF operator', and have a look at object result by displaying its expression in CliMAF Reference Syntax (CRS)¶
In [13]:
tas_avg=time_mean(tas_ds)
tas_avg
Out[13]:
Define another operator, apply it on result above, and look at CRS¶
In [14]:
cscript('space_mean' ,'cdo fldavg ${in} ${out}' )
tas_ga=space_mean(tas_avg)
tas_ga
Out[14]:
To TOC
4 - From CliMAF objects to files or Masked Array¶
Computation actually occurs when wanting to 'export' objects in some way : here as a file using 'cfile'¶
In [15]:
cfile(tas_avg)
Out[15]:
The file cache will be used whenever the same object is requested (or even for an included objet). Here, in addition, we request a copy of the cache file, at some location¶
In [16]:
cfile(tas_ds,"~/tmp/tasavg.nc")
Out[16]:
You can display the header information of a netCDF representation of a CliMAF object¶
In [17]:
ncdump(tas_ds)
Out[17]:
Another export format : MaskedArrays (others could apply, as e.g. CDAT's MV)¶
In [18]:
tas_avg_MA=cMA(tas_avg)
Looking at the result¶
In [19]:
type(tas_avg_MA)
Out[19]:
In [20]:
tas_avg_MA.shape
Out[20]:
To TOC
5 - Basic and advanced displays¶
Plot the data, first way : let us declare a 'CliMAF operator', for using binary 'ncview', which has no managed output¶
In [21]:
cscript('my_ncview' ,'ncview ${in}')
Out[21]:
Using this operator is straightforward¶
In [22]:
my_ncview(tas_avg)
Out[22]:
2nd plot : declare a simplified script plot with PNG output. Here in NCL with a number of args for graphics settings¶
In [23]:
cscript('plotmap',\
"ncl "+ cpath +"/../scripts/plotmap.ncl infile=${in} plotname=${out} \
cmap=${color} vmin=${min} vmax=${max} vdelta=${delta} var=${var} \
title=${title} scale=${scale} offset=${offset} units=${units}",format="png")
Out[23]:
Applying the operator (actually, let us use a richer one: 'plot', a general purpose plot operator for plotting 1D and 2D datasets - maps, cross-sections and profiles -)
The only effect is to define a figure object
In [24]:
map=plot(tas_avg, min=260, max=300, delta=4)
map
Out[24]:
Require the figure as a file; this leads to actually compute it if needed (i.e. launch the script)¶
In [25]:
cfile(map)
Out[25]:
'cshow' will display the figure using the file cache [ Nothing will show on Web-only demo]¶
In [26]:
cshow(map)
To TOC
6 - Mixing CliMAF basics and Python (and/or IPython)¶
A more realistic way of configuring the NCL-based plots, using Python features for graphics settings on the basis of on plotted variable¶
First define an auxiliary function¶
In [27]:
def map_graph_attributes(var) :
rep=dict()
rep["offset"]=0.
rep["scale"]=1.
rep["color"]="BlueDarkRed18"
if var=='tas' :
rep["offset"]=-273.15 ; rep["scale"]=1.0 ; rep["units"]="C"
rep["min"]=-30 ; rep["max"]=30 ; rep["delta"]=2.
return rep
This allows to supply graphics attributes which are better tuned to the variable to display¶
In [28]:
map2=plot(tas_avg,title="The Title",**map_graph_attributes(varOf(tas_avg)))
map2
Out[28]:
Let launch the computation of the figure¶
In [29]:
figfile=cfile(map2) ; print(figfile)
Bonus : using Ipython Notebook features for inline display¶
In [30]:
from IPython.display import Image
Image(filename=figfile)
Out[30]:
To TOC
7 - Remapping data¶
For remapping a dataset to a named grid (known by cdo), let us use operator 'regridn'¶
In [31]:
dgr=regridn(tas_avg,cdogrid="r90x45")
mapgr=plot(dgr, title="2° Grid", min=260, max=300, delta=4)
Image(filename=cfile(mapgr))
Out[31]:
An other way is to regrid to the grid of another dataset, with 'regrid'; Let us use the dataset defined earlier on a latlon box¶
In [32]:
dgr2=regrid(tas_ds, tas_ds_r)
ncview(dgr2)
Out[32]:
To TOC
8 - Computing an annual cycle¶
Let us compute an annual cycle, using swiss knife operator 'ccdo' (which applies cdo on a single dataset or object, with a cdo operator as argument), and look at it (nothing will show on Web only demo)¶
In [33]:
anncycle=ccdo(tas_ds,operator='ymonavg')
ncview(anncycle)
Out[33]:
Define the average annual cycle on a latlon box using operator 'llbox', plot it (January will show), and display the plot¶
In [34]:
extract=llbox(anncycle, latmin=30, latmax=60, lonmin=-30, lonmax=30)
map_extract=plot(extract, title='extract', min=260, max=300, delta=4)
Image(filename=cfile(map_extract))
Out[34]:
Let's compute the space average of annual cycle¶
In [35]:
space_average=ccdo(extract,operator='fldavg')
Creating a figure with standard operator 'curves'¶
In [36]:
fig_avg=curves(space_average, title="Annual cycle")
Get the figure computed, and get its filename in CliMAF file cache¶
In [37]:
Image(filename=cfile(fig_avg))
Out[37]:
Computation of seasonal average and plot bias¶
In [38]:
# obs
if atCNRM : obs=ds(project='cruts3',variable='tas', period='1980-1981',grid='T127')
else :
if onCiclad : obs=ds(project='ref_ipsl', variable='tas', period='1980-1981', product='ERAI')
else :
print("I do not know how to find data on this machine")
exit(0)
anncycle_obs=ccdo(obs,operator='ymonavg')
In [39]:
# seasonal average
JFM=ccdo(anncycle,operator='selmon,1,2,3')
JAS=ccdo(anncycle,operator='selmon,7,8,9')
JFM_obs=ccdo(anncycle_obs,operator='selmon,1,2,3')
JAS_obs=ccdo(anncycle_obs,operator='selmon,7,8,9')
In [40]:
# bias
JFM_regrid=regrid(JFM,JFM_obs)
JFM_bias=minus(JFM_regrid,JFM_obs)
JAS_regrid=regrid(JAS,JAS_obs)
JAS_bias=minus(JAS_regrid,JAS_obs)
In [41]:
# plot
JFM_map=plot(JFM,title='JFM',contours=1)
JAS_map=plot(JAS,title='JAS', contours=1)
#JFM_obs_map=plot(JFM_obs,title='JFM')
#JAS_obs_map=plot(JAS_obs,title='JAS')
JFM_bias_map=plot(JFM_bias,title='Bias JFM')
JAS_bias_map=plot(JAS_bias,title='Bias JAS')
multiplot=cpage([[JFM_map, JAS_map],[JFM_bias_map,JAS_bias_map]],heights=[0.2, 0.2])
Image(filename=cfile(multiplot))
Out[41]:
To TOC
9 - Derived variables, the easy way¶
Define a dataset using a variable name that is not in data files - i.e. a 'derived' variable¶
In [42]:
creShortTop=ds(simulation="AMIPV6ALB2G",variable="crest")
Declare a new 'CliMAF operator', to be used for computing the derived variable¶
In [43]:
cscript('minus','cdo sub ${in_1} ${in_2} ${out}')
Out[43]:
Tell how the derived variable is derived, for project 'example', using operator 'minus' and other variables¶
In [44]:
derive('example','crest','minus','rst','rstcs')
You can use the 'derived' dataset in any expression; computation will occur when needed¶
In [45]:
my_ncview(creShortTop)
Out[45]:
To TOC
10 - CliMAF access to data organized according to CMIP5 DRS¶
This example will work as is on CNRM's Lustre and on IPSL's Ciclad, and can be easily tuned to other local CMIP5 datasets¶
First, let us declare a list of data for CNRM-CM CMIP5 data at theses sites (note : this is actually built-in , and not necessary)¶
In [46]:
urls_CMIP5_CNRM=["/cnrm/aster/data2/ESG/data1", "/cnrm/aster/data2/ESG/data2", "/cnrm/aster/data2/ESG/data5",
"/cnrm/aster/data4/ESG/data6", "/cnrm/aster/data4/ESG/data7", "/cnrm/aster/data4/ESG/data8"]
urls_CMIP5_Ciclad=["/prodigfs/esg"]
Declare locations for searching data for all projects, experiment and frequencies, and that data located in dirs listed in urls_CMIP5_CNRM and urls_CMIP5_Ciclad is organized as 'CMIP5_DRS'¶
In [47]:
dataloc(project="CMIP5", organization="CMIP5_DRS", url=urls_CMIP5_CNRM+urls_CMIP5_Ciclad)
Out[47]:
Define a default value for some dataset facets¶
In [48]:
cdef("frequency","monthly") ; cdef("project","CMIP5")
Define your dataset (a number of facets take default values)¶
In [49]:
tas1pc=ds(model="CNRM-CM5", experiment="1pctCO2", variable="tas", period="1860-1861")
Display the basic filenames involved in the dataset (all filenames in one single string). CliMAF will search them at the data location which is the most specific among all declared data locations¶
In [50]:
files=tas1pc.baseFiles()
print files
Let CliMAF generate a file with the exact dataset in its disk cache (select period and/or variables, aggregate files...)¶
In [51]:
my_file=cfile(tas1pc)
print my_file