[ncl-talk] Reading Multiple Files stored in Short Format

[Dennis Shea]

[1] *If necessary*, NCL's  *addfiles*
<http://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml>  will
rebase the 'time'  to the time units on the initial file.
[2] *addfiles* will not apply the scale_factor and add_offset. The short2flt
<http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>
function will use the initial scale_factor and add_offset only.

I had an old function (*addfiles_getVarShort*) that does just what Toni
suggested. Offline, I sent it  to Bappa. Luckily, it worked  :-)

*load* "./addfiles_GetVarShort.ncl"

   diri = "./"
   fili  =* systemfunc
<http://www.ncl.ucar.edu/Document/Functions/Built-in/systemfunc.shtml>*("cd
" + diri + ";  ls ERA_Interim_20*.nc")
   pthi = diri+fili

   print(pthi)
   print("=============================================================")

   varName = "WhatEverVariable"
   x = *addfiles_GetVarShort* (pthi, varName)
   printVarSummary(x)
   printMinMax(x,0)

   print("=============================================================")
   ymdh = *cd_calendar
<http://www.ncl.ucar.edu/Document/Functions/Built-in/cd_calendar.shtml>(*x&time,
-3)
   print(x&time+"   "+ymdh)                ; show times


On Sun, Jul 29, 2018 at 11:18 PM, Toni Klemm <toni-klemm at tamu.edu> wrote:

> Hi Bappaditya,
>
> Have you tried creating a new, empty array of the same size of all the
> files you want to concatenate and then create a loop in which you read in a
> single file, convert it to from short to float and then “add” it to your
> array?
>
> I’m not familiar with your data, so my idea might not work for you. Good
> luck though!
>
> Greetings from Texas,
> Toni
>
>
> *Toni Klemm**, Ph.D.*
> Postdoctoral Research Associate
> Department of Ecosystem Science and Management
> Texas A&M University, College Station, TX
> Contributor at the Early Career Climate Forum <https://www.eccforum.org/>
> www.toni-klemm.de | @toniklemm <https://twitter.com/ToniKlemm>
>
>
> On Jul 30, 2018, at 12:00 AM, Bappaditya Nag <bappaditya.nag82 at gmail.com>
> wrote:
>
> Hi,
>
> I have 6 hrly ERA-Interim data stored in chunks of monthly files in short
> format (i.e., each file have their own scale and offset factor). When I
> read such multiple files in ncl (i.e., using v->f[:]) and convert to float
> (i.e., using short2flt), I do not get the correct result since the scale
> and offset of the first file is used for all the files.
>
> Solution : I can unpack and repack the files in a single file, but that is
> against the purpose of storing the data in optimal compact format. I am
> looking for a solution to read every file, convert to float and
> then concatenate the data. Can this be done? Or any other more
> effcient solution. Any help is appreciated.
>
> Present Code is as follows:
>
> files = systemfunc("ls " + dir + "ERA_Interim/ERA_Interim_v_"+year+"*.nc")
> fil = addfiles(files,"r")
> ListSetType (fil, "cat")
> v6hrly = short2flt(fil[:]->v(iStrt:iLast,{lev1:lev2},:,:))
>
> Also attached is a figure generated from the code.
>
> Warm Regards,
> Bappaditya Nag
> Post-doctoral Scholar
> Florida State University
>
> <xt_Meridional Wind.png>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180801/7cc39bbd/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: addfiles_GetVarShort.ncl
Type: application/octet-stream
Size: 2373 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180801/7cc39bbd/attachment.obj>