[Go-essp-tech] grids in metafor, and the esg catalog

[Bryan Lawrence]

Hi Folks

Balaji and I had a conversation yesterday, prompted by my barrage of emails 
about grids which were themselves prompted by three things:

1) I had looked at what was in the spaceConfiguation of key properties in the
questionnaire, and
2) Phil had prompted me to compare a gridspec file and a CIM grid xml file, and
3) In the go-essp-tech call on information flow, we made some statements 
about code needed to handle grid descriptions ...

This email is by way of summary of that conversation and of a subsequent 
conversation with Phil (please both of you correct anything below which
is a bryan'ism).

Before takeoffi:

We had noted that in the previous version of the mindmaps (i.e.
the one in the alpha8 running now), there were some horizontal grid
definitions in the key properties of atmosphere and oceans, but
they differed in good and bad ways...

We expect gridspec files to be created at different times than the
questionnaire (many by Balaji himself); so we can't expect them to exist 
before q'naire input ... so we can't assume that we would create a CIM 
grid xml document  as output from the questionnaire by reading gridspec
to get (some) attributes.

In flight:

It became clear that we don't have any real use case (right now) for mixing
gridspec (which describes the grid the data is output on) and cim xml
which is primarily about describing (amongst other things, the resolution)
 of the model components for "understanding" and "discovery".
 E.g. "Which model ran with the highest atmopsheric resolution?
 (Never mind whether it wrote the data out at that resolution?) or
"Find me all the models using a cube-sphere ..."

The grid questions in atmopshere and ocean key properties were
different, and very different from the other realms.

We thought that the concept of resolution should be expanded
to include three subtypes:
 - a short name using "accepted notatations" eg N96L52
 - a numeric resolution in degrees for the mid-latitudes (to allow resolution ordering)
 - a textural comment on how the resolution varies (e.g. "higher resolution
in the tropics, with polar filtering").

(But note the mixing of horizontal and vertial resolution in the standard
short name, so we should be careful about layout of horizontal
and vertical resolution ... Phil and I had a chat about layout ...)

Landing: Conclusions

We should ask the same sort of grid questions in each realm, but
 - It should be acceptable to answer, say, in the case of aerosol,
"same as atmosphere"
 - there might be realm level specifications.

Currently, there is a risk of confusion between grids and components.

Complete CIM XML *as currently defined* could only be created
later, much later, and constructing them, then passing them to
ESG would be nightmare. Better to modifiy the logical model
for the CIM .... then we can export these to ESG, and then they can
(if they want) handle grid definitions (from "newCIMgrid") independently
of gridspec harvested (if they want) from the data archives.

Post flight: Actions Necessary:

1) Understand why the mindmap grid definitions have changed
again.

2) Build a grid page for each realm level entry, and subsume the
content from the mindmaps. (Plus some extra content from
Balaji). 

(Why build  this? Because then we can more easily say "same as 
the atmosphere "....)

Essentially come up with a new content model for a "simple" grid 
CIM.

(Step 2 will probably take about a day of coding).

3) Remove the grid config stuff from the mindmaps.

--- we can do stuff up to here before launch ----

4) Modify the ConCIM and ApCIM to conform to the new,
simpler Grid definition ...

5) Rupert can then export it.

--- We ought to be able to do that within a few weeks
of step 2 ---

6) Sylvia can then load it.

--- and Sylvia could tell us what the timescale implications of this are ---

Final question:

Should we begin down this path? Or is there something we didn't know?

Bryan

-- 
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848; 
Web: home.badc.rl.ac.uk/lawrence