CEDAR email: Beta release of Madrigal 3 for community feedback

[Bill Rideout]

A fully functional beta release of Madrigal 3 is now available at http://madrigal3.haystack.mit.edu <http://madrigal3.haystack.mit.edu/> . It contains almost all data on the CEDAR Madrigal database, with the exception of very recent data and DMSP data.  The main feature of Madrigal 3 is the migration from the old CEDAR database format to an Hdf5 based file format.  Please give the site a try and email Bill Rideout at brideout at haystack.mit.edu <mailto:brideout at haystack.mit.edu> with any feedback.  For comparison, the present CEDAR Madrigal database (Version 2.6) can be found at http://cedar.openmadrigal.org <http://cedar.openmadrigal.org/> .  Below is a list of major changes in Madrigal 3:

Migration to Hdf5 file format - With Madrigal 3.0, the old 16 bit integer based file format has been replaced with Hdf5. However, the complete data model and well-defined parameters have been retained. The CEDAR Hdf5 format is designed to be completely self-contained. That is, any user with an Hdf5 reader can fully understand all data in the file without any reference to documentation. A full description of the format can be found at http://madrigal3.haystack.mit.edu/static/CEDARMadrigalHdf5Format.pdf <http://madrigal3.haystack.mit.edu/static/CEDARMadrigalHdf5Format.pdf> .

Web interface generates scripts to download any amount of data - With Madrigal 3, you can generate a script command to download a whole series of files with a few click. You can also create a script that will filter those same files and allow you to choose the parameters you want, again with just a few clicks. These scripts can be run with python, Matlab, or IDL.

Download files in new format - Prior to Madrigal 3, files could be downloaded only as ascii or Hdf5 (or the difficult to understand CEDAR format). Now they can be downloaded as ascii, Hdf5, or netCDF4.

Get data with selected/derived parameters and filters in any format - Prior to Madrigal 3, data with selected/derived parameters and filters was available only as ascii. Now it can be downloaded as ascii, Hdf5, or netCDF4.

Independent spatial parameters now built in to data model - Prior to Madrigal 3.0, there was no way to automatically tell what the independent spatial parameters were in vector data. With Madrigal 3.0, any data with vector data must define its independent spatial parameters, which can be found in the Hdf5 Metadata group. This allows Madrigal to automatically add array layouts of the data, making for easier plotting.

All new web interface - A much simplified web interface based on Django and bootstrap. Developed with much assistance from Jicamarca Observatory and Jose Antonio Sal y Rosas Celi.

Simple FTP-like web interface added - Designed for non-native English speakers. The url follows a very simple pattern that can be easily parsed if the user in unable to understand the Madrigal API's and automatically generated scripts.

Create CEDAR Hdf5 files with either Matlab or python - Prior to Madrigal 3, there was only a python API to easily create Madrigal files. Now both Matlab and python can be used to create CEDAR Hdf5 files.

All metadata easily accessible - The Access metadata menu item on the main navigation menu allows easy access to all Madrigal CEDAR metadata, including Madrigal sites, instruments, parameter definitions, and kind of data codes.

Please send all feedback to Bill Rideout at brideout at haystack.mit.edu <mailto:brideout at haystack.mit.edu> .  It is expected that a version will be available for Madrigal administrators to install in a few months.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/cedar_email/attachments/20160620/6e8990ea/attachment.html