[ncl-talk] set_date_time
Dennis Shea
shea at ucar.edu
Mon Oct 9 07:25:19 MDT 2017
Please talk with the Panoply people.
---
The file has
double GridLat_grid1km(YDim_grid1km, XDim_grid1km)
double GridLon_grid1km(YDim_grid1km, XDim_grid1km)
---
short Optical_Depth_055_grid1km(time, YDim_grid1km, XDim_grid1km) ;
Optical_Depth_055_grid1km:*coordinates* = "GridLat_grid1km,
GridLon_grid1km" ;
Optical_Depth_055_grid1km:hdfeos_name = "Optical_Depth_055" ;
Optical_Depth_055_grid1km:projection = "Albers Conical Equal_Area" ;
Optical_Depth_055_grid1km:valid_range = -100s, 5000s ;
Optical_Depth_055_grid1km:_FillValue = -28672s ;
Optical_Depth_055_grid1km:unit = "None" ;
Optical_Depth_055_grid1km:add_offset = 0. ;
Optical_Depth_055_grid1km:scale_factor = 0.001 ;
Optical_Depth_055_grid1km:long_name = "AOT at 0.55 micron" ;
---
On Mon, Oct 9, 2017 at 3:22 AM, Kunal Bali <kunal.bali9 at gmail.com> wrote:
> Thanks for providing all these details Dennis Sir,
>
> After running the attached script, it creates the correct date in the
> netcdf file. But when I open the resultant netcdf file in Panoply software
> to visualize the grid structure, then the variable such as
> Optical_Depth_055_grid1km seems not georeferenced between latitude and
> longitude. The resultant netcdf file shows the georeferenced between *latitude
> and time* (please see the attached figure).
>
> So, that creates the one problem in extracting the particular latitude and
> longitude data (e.g. IISER Mohali (30.663 N, 76.732 E).
>
>
>
>
> Rehards
> Kunal Bali
>
>
>
>
>
>
> On Mon, Oct 9, 2017 at 10:11 AM, Dennis Shea <shea at ucar.edu> wrote:
>
>> Long answer BUT I suggest you read this carefully!
>> ----------------------------------------------------------------------
>> As is suggested (ad nauseum), always look at your data:
>>
>> %> ncl_filedump MAIACTAOT.h00v02.20003650620.*hdf * | less
>>
>> [snip]
>> filename: MAIACTAOT.h00v02.20003650620
>> path: MAIACTAOT.h00v02.20003650620.hdf
>> file global attributes:
>> *HDFEOSVersion : HDFEOS_V2.12 <=== HDF-EOS file*
>> StructMetadata_0 : GROUP=SwathStructure
>> END_GROUP=SwathStructure
>> [snip]
>>
>> ----
>> It is a HDF-EOS file!!!
>>
>> When a user sees HDFEOS [ because he/she looked at the file; :-) ], the
>> file extension *.he2* or equivalently, *.hdfeos* should be appended to
>> the file. This can be done programatically or on the command line. The
>> file need not be explicitly changed!
>>
>> %> ncl_filedump MAIACTAOT.h00v02.20003650620.hdf*.he2* | less
>>
>> NCL looks for the file MAIACTAOT.h00v02.20003650620.hdf*.he2* and*, *if
>> it does not find the file, it *'backs-off' *the last file extension [
>> here,* .he2 *] and reads the file
>>
>> *.*
>> *Why* should a user do this?
>>
>> %> ncl_filedump MAIACTAOT.h00v02.20003650620.hdf*.he2 *| less
>>
>> (a) If you look at the original file*, *you will see a rather
>> complicated file structure*. *It includes multiple *groups. *When the
>> .he2 extenson is used, NCL 'flattens' the file structure. It make it look
>> like a standard netCDF3 file. In this case the variables have appended
>> extensions named: _grid5km and _grid1km. A very nice feature.
>>
>> (b) An even nicer feature of the .he2/.hdfeos extension is that NCL *creates
>> value-addedarrays *containing the lat/lon arrays.
>>
>> double GridLat_grid5km ( YDim_grid5km, XDim_grid5km )
>> double GridLat_grid5km ( YDim_grid5km, XDim_grid5km )
>> double GridLat_grid1km ( YDim_grid1km, XDim_grid1km )
>> double GridLat_grid1km ( YDim_grid1km, XDim_grid1km )
>>
>> (c) NCL *creates *CF Convention attributes like 'coordinates' for
>> assorted variable.
>>
>> ------
>>
>> One of the less public NCL-developers (Dave Brown), deserves considerable
>> credit for NCL's IO capabilities. In particular, with respect to GRIB and
>> HDF files. Also, 'addfile' and 'addfiles'
>>
>> As I say ... "A user need-not-fear any grib/hdf/netCDF files if using
>> NCL!'
>>
>> ------
>>
>> The 2nd part of this answer is 'How to add a 'time' (record) dimension?'
>>
>> See attachment:
>>
>> %> mkdir MAIACTAOT_TIME <== will contain created file
>> %> ncl create_time.MAIACTAO.ncl
>>
>> -----
>> The keys are:
>>
>> ;---https://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml
>> f = addfiles(pthi, "r")
>>
>> *ListSetType (f, "join") ; add 'record' (time
>> dimension) *
>>
>> ;--- Create a time variable
>> refTime = "hours since 2000-01-01 00:00:00" ; arbitrary
>> time = *time_create_MAIACTAOT* (fili, refTime) ; local function
>>
>> An file dump of the created file in directory:
>>
>> %> cd /MAIACTAOT_TIME
>>
>> %> ncl_filedump maiactaot.h00v02.20003650620-20003660700.nc
>> or
>> %> ncdump -h maiactaot.h00v02.20003650620-20003660700.nc
>>
>>
>> -++++++++
>> Internal NCAR users, the files are at:
>>
>> /glade/p/ncldev/data/hdf
>>
>> MAIACTAOT.h00v02.20003650620.hdf
>> MAIACTAOT.h00v02.20003660525.hdf
>> MAIACTAOT.h00v02.20003660700.hdf
>>
>>
>> Cheers
>> D
>>
>> On Fri, Oct 6, 2017 at 9:18 PM, Kunal Bali <kunal.bali9 at gmail.com> wrote:
>>
>>> Thanks Mery for providing this information.
>>> I think the problem is with CDO.
>>>
>>> But is there any function to convert filenames to date-time. and then
>>> create time variable in the netcdf file.
>>> such as :
>>> MAIACTAOT.h00v02.*2000364**07**15*.nc --> 2000=year, 364=day, 07=hr,
>>> 15=min
>>> *2000-01-29 07:15:00*
>>>
>>> could you please give some information on that.
>>> thank you
>>>
>>> regards
>>> Kunal Bali
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Oct 6, 2017 at 2:16 AM, Guido Cioni <guidocioni at gmail.com>
>>> wrote:
>>>
>>>> I had to do something similar last week. Converting from hdf to netcdf
>>>> and then merge into a single file with time information. It was modis data.
>>>> I don't have time to explain the script so I'm copying here hoping that it
>>>> could help.
>>>>
>>>> for f in *.nc; do
>>>> year=${f:9:4}
>>>> day_of_year=${f:13:3}
>>>> month=`date -d "${year}-01-01 +${day_of_year} days -1 day" "+%m"`
>>>> day=`date -d "${year}-01-01 +${day_of_year} days -1 day" "+%d"`
>>>> echo ${year}-${month}-${day}
>>>> cdo -b F64 setdate,${year}-${month}-${day} ${f} ${f%.nc}_2.nc
>>>> done
>>>>
>>>>
>>>> Cheers
>>>>
>>>> Il 5 ott 2017 9:56 PM, "Marston Johnston" <shejo284 at gmail.com> ha
>>>> scritto:
>>>>
>>>>> It seems like you choose a fairly complicated way to solve the problem.
>>>>>
>>>>> May I suggest reading in the MAIACTAOT.h00v02.***.nc files in
>>>>> chronological order and then writing new netcdf files in ncl where you can
>>>>> create a time array with the correct dtg.
>>>>>
>>>>> It’s fairly quick and easy in ncl.
>>>>>
>>>>> I’ve had such problems before and found it easiest to with the above
>>>>> method.
>>>>>
>>>>>
>>>>>
>>>>> Hope this helps,
>>>>>
>>>>> /M
>>>>>
>>>>>
>>>>>
>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>
>>>>> Marston S. Ward, PhD
>>>>>
>>>>> Department of Earth Sciences
>>>>>
>>>>> University of Gothenburg, Sweden
>>>>>
>>>>> Email: marston.johnston at gu.se
>>>>>
>>>>> SkypeID: marston.johnston
>>>>>
>>>>> Phone: +46-31-7864901 <+46%2031%20786%2049%2001>
>>>>>
>>>>> Only the fruitful thing is true!
>>>>>
>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *From: *ncl-talk <ncl-talk-bounces at ucar.edu> on behalf of Kunal Bali <
>>>>> kunal.bali9 at gmail.com>
>>>>> *Date: *Thursday, 5 October 2017 at 20:40
>>>>> *To: *"ncl-talk at ucar.edu" <ncl-talk at ucar.edu>
>>>>> *Subject: *[ncl-talk] set_date_time
>>>>>
>>>>>
>>>>>
>>>>> Dear NCL users,
>>>>>
>>>>>
>>>>>
>>>>> I have one question regarding date-time setting from filename. This
>>>>> question is not actually related to NCL, sorry in advance.
>>>>>
>>>>>
>>>>>
>>>>> I have 4 .nc files named given below. The time is not created in these
>>>>> files during conversion from hdf to netcdf in NCL.
>>>>>
>>>>>
>>>>>
>>>>> MAIACTAOT.h00v02.*20003640715*.nc --> 2000=year, 364=day, 07=hr,
>>>>> 15=min
>>>>>
>>>>> MAIACTAOT.h00v02.20003650620.nc
>>>>>
>>>>> MAIACTAOT.h00v02.20003660525.nc
>>>>>
>>>>> MAIACTAOT.h00v02.20003660700.nc
>>>>>
>>>>>
>>>>>
>>>>> So, I did try to create the date-time with shell scripting (given
>>>>> below)
>>>>>
>>>>> but it is creating wrong date as the files are of December month, but
>>>>> it is generating January month date also. I don't know why.
>>>>>
>>>>>
>>>>>
>>>>> *2000-01-01 05:25:00*
>>>>>
>>>>> *2000-01-01 07:00:00*
>>>>>
>>>>> 2000-12-30 07:15:00
>>>>>
>>>>> 2000-12-31 06:20:00
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> the shell script, which I am using
>>>>>
>>>>>
>>>>>
>>>>> #######################################
>>>>>
>>>>>
>>>>>
>>>>> for n in MAIACTAOT.h00v02.20*.nc;do
>>>>>
>>>>> ls $n > text
>>>>>
>>>>> export xx=`cut -c18-28 text`
>>>>>
>>>>> export YYYY=`cut -c18-21 text`
>>>>>
>>>>> export JJJ=`cut -c22-24 text`
>>>>>
>>>>> export HH=`cut -c25-26 text`
>>>>>
>>>>> export MM=`cut -c27-28 text`
>>>>>
>>>>> mm=`date -d "$JJJ days $YYYY-01-01" +"%m"`
>>>>>
>>>>> dd=`date -d "$JJJ days $YYYY-01-01" +"%d"`
>>>>>
>>>>> mydate=${YYYY}-${mm}-${dd},${HH}:${MM}:00
>>>>>
>>>>> echo $mydate
>>>>>
>>>>>
>>>>>
>>>>> echo "cdo settaxis,$mydate $n re$n"
>>>>>
>>>>> cdo settaxis,$mydate $n re$n
>>>>>
>>>>>
>>>>>
>>>>> done
>>>>>
>>>>>
>>>>>
>>>>> cdo mergetime reMAI*.nc final.nc
>>>>>
>>>>> exit
>>>>>
>>>>> #################################
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Any suggestion regarding this issue would be very helpful.
>>>>>
>>>>>
>>>>>
>>>>> Thank You
>>>>>
>>>>>
>>>>> Kunal Bali
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________ ncl-talk mailing list
>>>>> ncl-talk at ucar.edu List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>
>>>>> _______________________________________________
>>>>> ncl-talk mailing list
>>>>> ncl-talk at ucar.edu
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>
>>>>>
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171009/c5070213/attachment.html>
More information about the ncl-talk
mailing list