[ncl-talk] Reading isccp revisted

Marston Ward marston.johnston at gu.se
Mon Oct 16 10:59:27 MDT 2017 

Hi Dennis,

Thanks a lot for the detailed response. Took a little time to get back to this issue.
Greatly appreciate the wealth of information and access to the data.
I also found some netcdf versions at the IPSL site: http://climserv.ipsl.polytechnique.fr/. But I think I trust the GISS data archive more. Note: the ftp site: ftp://isccp.giss.nasa.gov<ftp://isccp.giss.nasa.gov/> seems to be broken.
I understand message of supporting ISCCP. I’ve emailed several persons using the addresses listed on the site but the addresses no longer are valid. So I gave up.
I was just fishing for experience here among the NCL folks.

I’ll save this email for future reference.

Best,
/M


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Marston S. Ward, PhD
Department of Earth Sciences
University of Gothenburg, Sweden
Email: marston.johnston at gu.se<mailto:marston.johnston at gu.se>
SkypeID: marston.johnston
Phone: +46-31-7864901
Only the fruitful thing is true!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


From: Dennis Shea <shea at ucar.edu>
Date: Friday, 13 October 2017 at 16:23
To: Marston Ward <marston.johnston at gu.se>
Cc: "ncl-talk at ucar.edu" <ncl-talk at ucar.edu>
Subject: Re: [ncl-talk] Reading isccp revisted

THX for the sample ISCCP hdf file. It looks pretty sparse!

IMHO: as the hdf file is constructed, it is USELESS!
%>ncl_filedump

Variable: f
Type: file
filename:       ISCCP.D2.0.GLOBAL.2007.01.99.9999.GPC
path:   ISCCP.D2.0.GLOBAL.2007.01.99.9999.GPC.hdf
   file global attributes:
   dimensions:
      fakeDim0 = 6596
      fakeDim1 = 130
[SNIP]
   variables:
      ubyte Data_Set_6 ( fakeDim0, fakeDim1 )
         anno_label_1 : UTC  00
         long_name :    UTC  00
         units :        counts
         format :       I3
         hdf_name :     Data-Set-6
[SNIP]
Really, it looks like 'someone' had to do 'something' in hdf so they created this useless file.

---

Because this is on a public forum, I am sure that an 'ISCCP' search will contain this ncl-talk response. Hence, a disclaimer:

I don't want to support ISCCP-D2 (or D1 or DX etc, etc, etc).
Please send all ISCCP questions to ISCCP people.

The following deals with the netCDF file(s) I created derived from the original binary files. Maybe, the following will be of use.
---
ISCCP ended 12/2009
--
https://climatedataguide.ucar.edu/     <=== excellent site to explore   :-)
    Search:  ISCCP and click on ISCCP
    Please read the "Expert User Guidance"
---
Click: Get Data (External)
One link, which lists all the variables,  is here:

http://www.cgd.ucar.edu/cas/catalog/satellite/isccp/D2/means.html
The data set has been updated thru the end of the ISCCP project: 12/2009
ftp ftp.cgd.ucar.edu<http://ftp.cgd.ucar.edu>
anonymous
your_email
cd archive
ls
cd ISCCP
ls
get ncdump.isccp.1983-2009      <=== simple text file
get isccp.ps.gz                             <=== time series plots of ?all? variables
get isccp.1983-2009.nc.gz           <=== big file
quit
================ NCL D2 Stuff =================
NCL has been used on ISCCP binary data since 2004 (maybe earlier). As noted, ISCCP hdf is useless.

The following is provided as deep background information. I have forgotten everything I knew about the various scripts.
However, given NCL's backward compatibility, I speculate they would work.
They read the original binary and not hdf files.
---
d2bin2nc.ncl:
takes the  binary and converts each year-month binary file to a corresponding netCDF file. The script invokes a shared object that regrids the data to a 2.5 degree grid. These were later placed onto a single file via the netCDF operator: 'ncrcat'
---
d2read_NCL.f:
I converted the original ISCCP provided D2-f77 code into a subroutine such that a shared object (.so) could be created and invoked from an NCL script. This uses ISCCP methodology for regridding to a 2.5 degree grid.
The original D2READ.f is at the "official" ISCCP software URL:
   https://isccp.giss.nasa.gov/products/software.html
---
%> WRAPIT d2read_NCL.f
WRAPIT creates the shared object file: d2read_NCL.so
---
plot.ContourTimeS.ncl
 Read the final netCDF file and creates time series plots for many variables. I  suggest you examine any ISCCP variables for discontinuities. Remember ISCCP is a amalgamation of satellite inputs and these could change over time.
Good luck
D




On Wed, Oct 11, 2017 at 5:13 AM, Marston Ward <marston.johnston at gu.se<mailto:marston.johnston at gu.se>> wrote:
Hi,

I’m trying to compare ISCCP products from COSP with real time data, i.e., ISCCP D2 monthly means.
I’ve never worked with ISCCP before, and, while I can download a lot of data from Earthdata, they are in formats I do not recognise nor know how to handle.
Much of the isccp documents have email address that no longer work, so it is bit difficult to contact them.

While doing a search on the NCL webpage revealed: https://www.ncl.ucar.edu/Applications/isccp.shtml
But in this example the isccp data is already in netcdf format. I’m trying to get to this point.

Further, in the NCL archives I came across a similar problem back in 2005: http://www.ncl.ucar.edu/Support/talk_archives/all/1699.html
That seems to explain a bit how to read iSCCP D2 but much of this seem to me a black box.

Given the above information, how can I find information on how to read the D2 data onto a regular lat lon grid using NCL?
Anyone in the know, or with experience, out there and care to share?

I’ve attached the SDS hdf file but I don’t know what’s in the file and the data looks very strange.

Best,
/M

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Marston S. Ward, PhD
Department of Earth Sciences
University of Gothenburg, Sweden
Email: marston.johnston at gu.se<mailto:marston.johnston at gu.se>
SkypeID: marston.johnston
Phone: +46-31-7864901<tel:+46%2031%20786%2049%2001>
Only the fruitful thing is true!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171016/89cf2b7e/attachment.html>

More information about the ncl-talk mailing list