[GO-ESSP] gridded data management systems -- converting to the Caching problem

[Benno Blumenthal]

Jon Blower wrote:

> The key features of this system
>would include:
>
>1) Data would be stored in a single format but can be extracted in a variety
>of formats
>2) Data could be sliced and subsetted in all possible ways (e.g. extraction
>of 1-D timeseries, 2-D areas, 3-D volumes/animations, 4-D data blocks) and
>extracted at different spatial and temporal resolutions
>3) Data could be stored on the original grid (including rotated grids) but
>extracted on the grid of the user's choice
>4) The necessary projection and interpolation would happen on the fly
>5) The system would allow complex queries to be made (e.g. "Give me all the
>times and locations at which the sea surface temperature was greater than 20
>degC in the North Atlantic in June 2003")
>
>  
>
The software we have developed (Ingrid), like the software Steve 
mentioned, provides functionality like 1-4.  5 is possible, though not 
how we usually make use of gridded data.  There are many other 
functions/transformations which are more appropriate for sampled 
continuous fields.

But more to the point, I think, of creating a broad-appeal software 
project, is to broaden the problem to include converting the data from 
its original source to the internal format.  Then you have a system that 
goes from many sources, (included standard web servers) to many formats, 
which is the standard server problem.   And then you are creating 
software that can be used for caching, a key part in making use of 
distributed data, which I believe converts the project into software 
that addresses a widespread need.

Benno