[ncl-talk] Are recent versions of contributed.ncl backward-compatible?
David Brown
dbrown at ucar.edu
Fri Feb 3 11:25:39 MST 2017
It's not actually that there are different GRIB table versions (there
are, but that is not really what this is about). There were additions
to one GRIB table with respect to statistically processed variables.
NCL changed its naming scheme slightly in order to better account for
these additions to this table. We try not to change the naming scheme
as much as possible, but one in awhile it has been necessary to keep
the scheme viable in the face of new types of data. These statistical
variable types were one such addition, and it took a couple of
iterations to get it right. It would require going to deep into the
weeds to explain this more fully.
-dave
On Fri, Feb 3, 2017 at 9:46 AM, Daniel Barandiaran - NOAA Affiliate
<daniel.barandiaran at noaa.gov> wrote:
> Thanks David, your suggestion solved a couple of things: first, it took care
> of ncl_filedump, and also has my environment set up properly to run 6.3.0
> out of my local directory; it seems that aliasing things is not a substitute
> for setting paths correctly. One last little bit a fun, a direct example of
> what I was describing. Here is the same GRIB file header as reported by
> 6.1.2:
>
> [cpc-work2 ~]$ ncl_filedump
> /cpc/home/ebecker/ghcn_cams/ghcn_cams_1948_cur_t126.grb
>
> Copyright (C) 1995-2013 - All Rights Reserved
> University Corporation for Atmospheric Research
> NCAR Command Language Version 6.1.2
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
>
> Variable: f
> Type: file
> filename: ghcn_cams_1948_cur_t126
> path: /cpc/home/ebecker/ghcn_cams/ghcn_cams_1948_cur_t126.grb
> file global attributes:
> dimensions:
> initial_time0_hours = 828
> lat_126 = 190
> lon_126 = 384
> variables:
> float TMP_126_HTGL_123 ( initial_time0_hours, lat_126, lon_126 )
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>
> and here is 6.3.0, notice the different variable name:
>
> [cpc-work2 ~]$ ncl_filedump
> /cpc/home/ebecker/ghcn_cams/ghcn_cams_1948_cur_t126.grb
>
> Copyright (C) 1995-2015 - All Rights Reserved
> University Corporation for Atmospheric Research
> NCAR Command Language Version 6.3.0
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
>
> Variable: f
> Type: file
> filename: ghcn_cams_1948_cur_t126
> path: /cpc/home/ebecker/ghcn_cams/ghcn_cams_1948_cur_t126.grb
> file global attributes:
> dimensions:
> initial_time0_hours = 828
> lat_126 = 190
> lon_126 = 384
> variables:
> float TMP_126_HTGL_S123 ( initial_time0_hours, lat_126, lon_126 )
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>
> I am guessing this means that when NCL opens a GRIB file and you give it a
> variable name, it matches up what you asked for with a GRIB table and
> accesses the data that way, and that 6.1.2 and 6.3.0 have different GRIB
> table versions. Is this correct?
>
>
>
> On Thu, Feb 2, 2017 at 5:52 PM, David Brown <dbrown at ucar.edu> wrote:
>>
>> ncl_filedump is a csh script that runs NCL It finds NCL by looking in
>> the user's path. If you have your path set to use NCL 6.1.2 then it
>> doesn't matter which version of ncl_filedump you use. It will always
>> find the 6.1.2 version of NCL. So it makes sense that if you source
>> your .bashrc to use NCL 6.3.0, then ncl_filedump will use the 6.3.0
>> version of NCL and give you the new name of this variable with the
>> 'S'. The errors you are getting are caused by you running NCL 6.1.2
>> but trying to bring in 6.3.0 versions of the NCL source-code function
>> libraries. You can see this from your output.
>> Copyright (C) 1995-2013 - All Rights Reserved
>> University Corporation for Atmospheric Research
>> NCAR Command Language Version 6.1.2
>> ...
>> fatal:syntax error: line 631 in file
>> /cpc/home/dbarandiaran/ncl-6.3.0/lib/ncarg/nclscripts/utilities.ncl
>> before or near ; Draw box.
>>
>> -----
>> This is probably a result of having your PATH and NCARG_ROOT
>> environment variables out of sync.
>>
>> Try setting them as follows on the command line:
>>
>> export NCARG_ROOT=/cpc/home/dbarandiaran/ncl-6.3.0/l
>> export PATH=$NCARG_ROOT/bin:$PATH
>>
>> -dave
>>
>>
>>
>>
>>
>>
>> On Thu, Feb 2, 2017 at 2:34 PM, Daniel Barandiaran - NOAA Affiliate
>> <daniel.barandiaran at noaa.gov> wrote:
>> > Indeed. More strangeness: When I use ncl_filedump the variable is listed
>> > as
>> > TMP_126_HTGL_123, not..._S123. If I am running 6.1.2 then the script
>> > will
>> > read ..._123 but not _S123. When I source my .bashrc to use 6.3.0, then
>> > the
>> > script will read ..._S123 but not ..._123. Furthermore, even if I alias
>> > ncl_filedump to the 6.3.0/bin/ncl_filedump, it still executes using
>> > 6.1.2,
>> > which results in some errors, and still lists the variable as ..._123:
>> >
>> > Copyright (C) 1995-2013 - All Rights Reserved
>> > University Corporation for Atmospheric Research
>> > NCAR Command Language Version 6.1.2
>> > The use of this software is governed by a License Agreement.
>> > See http://www.ncl.ucar.edu/ for more details.
>> > fatal:syntax error: line 631 in file
>> > /cpc/home/dbarandiaran/ncl-6.3.0/lib/ncarg/nclscripts/utilities.ncl
>> > before
>> > or near ; Draw box.
>> >
>> > gsn_polygon_ndc(wks,xbox+xpos(i),ybox+ypos(i),gnres) ; Draw box.
>> > ^
>> >
>> >
>> > fatal:syntax error: possibly an undefined procedure
>> >
>> > fatal:syntax error: line 634 in file
>> > /cpc/home/dbarandiaran/ncl-6.3.0/lib/ncarg/nclscripts/utilities.ncl
>> > before
>> > or near \n
>> > gsn_polyline_ndc(wks,xbox+xpos(i),ybox+ypos(i),lnres)
>> > ----------------------------------------------------------^
>> >
>> >
>> > fatal:syntax error: possibly an undefined procedure
>> >
>> > fatal:syntax error: line 638 in file
>> > /cpc/home/dbarandiaran/ncl-6.3.0/lib/ncarg/nclscripts/utilities.ncl
>> > before
>> > or near \n
>> >
>> >
>> > gsn_text_ndc(wks,label_strings(i),font_space+xpos(i),ypos(i)+font_space,txres)
>> >
>> > -------------------------------------------------------------------------------------^
>> >
>> >
>> > fatal:syntax error: possibly an undefined procedure
>> >
>> > fatal:Syntax Error in block, block not executed
>> >
>> > fatal:error at line 645 in file
>> > /cpc/home/dbarandiaran/ncl-6.3.0/lib/ncarg/nclscripts/utilities.ncl
>> >
>> >
>> > Variable: f
>> > Type: file
>> > filename: ghcn_cams_1948_cur_t126
>> > path: /cpc/home/ebecker/ghcn_cams/ghcn_cams_1948_cur_t126.grb
>> > file global attributes:
>> > dimensions:
>> > initial_time0_hours = 828
>> > lat_126 = 190
>> > lon_126 = 384
>> > variables:
>> > float TMP_126_HTGL_123 ( initial_time0_hours, lat_126, lon_126 )
>> >
>> >
>> >
>> > On Thu, Feb 2, 2017 at 4:23 PM, Dennis Shea <shea at ucar.edu> wrote:
>> >>
>> >> Punch line:
>> >>
>> >> GRIB is a moving target :-(
>> >>
>> >> On Thu, Feb 2, 2017 at 2:18 PM, David Brown <dbrown at ucar.edu> wrote:
>> >>>
>> >>> Hi Daniel,
>> >>> I think the problem is simply that the name of this variable has been
>> >>> changed for more recent versions of NCL. I found the file you are
>> >>> trying to read at ftp.cpc.ncep.noaa.gov. ncl_filedump now reports the
>> >>> name as TMP_126_HTGL_S123. The name change was made in order to
>> >>> characterize the last number as representing a statistical processing
>> >>> method defined in this GRIB1 table:
>> >>> http://www.nco.ncep.noaa.gov/pmb/docs/on388/table5.html, where 123 is
>> >>> listed as "Average of N uninitialized analyses, starting at the
>> >>> reference time, at intervals of P2".
>> >>> Sorry for the confusion, but particularly with GRIB files, it is
>> >>> important to recognize that NCL creates these names based on
>> >>> information in the GRIB file. As the code evolves, occasionally names
>> >>> have needed to be modified slightly. Anytime you encounter the message
>> >>> that a variable is not in the file, it is wise to use ncl_filedump to
>> >>> see if the name actually appears in the file.
>> >>> -dave
>> >>>
>> >>>
>> >>>
>> >>> On Thu, Feb 2, 2017 at 1:46 PM, Daniel Barandiaran - NOAA Affiliate
>> >>> <daniel.barandiaran at noaa.gov> wrote:
>> >>> > Here is the relevant code for the problem with the GRIB read:
>> >>> >
>> >>> > ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>> >>> >
>> >>> > f
>> >>> >
>> >>> > =addfile("/cpc/home/ebecker/ghcn_cams/ghcn_cams_1948_cur_t126.grb","r")
>> >>> > TIME = cd_calendar(f->initial_time0_hours,0)
>> >>> > inntime = dimsizes(TIME(:,0))
>> >>> > inyrmin = 1981
>> >>> > inyrmax = toint(datmaxyr)
>> >>> > innyr = inyrmax - inyrmin + 3
>> >>> > ntime = innyr * 12
>> >>> > tind := ind(TIME(:,0).ge.inyrmin)
>> >>> >
>> >>> > year = fspan(inyrmin,inyrmax+2,innyr)
>> >>> > year!0 = "year"
>> >>> > year&year = year
>> >>> >
>> >>> > time = fspan(0,ntime-1,ntime)
>> >>> > time!0 = "time"
>> >>> > time&time = time
>> >>> > time at units = "months since 1981-01-01 00:00:00"
>> >>> > delete(TIME)
>> >>> > TIME = cd_calendar(time,0)
>> >>> >
>> >>> > lat = f->lat_126({minlat:maxlat})
>> >>> > nlat = dimsizes(lat)
>> >>> > lon = f->lon_126({minlon:maxlon})
>> >>> > nlon = dimsizes(lon)
>> >>> >
>> >>> > lat!0 = "lat"
>> >>> > lat&lat = lat
>> >>> > lat at units = "degrees_north"
>> >>> >
>> >>> > lon!0 = "lon"
>> >>> > lon&lon = lon
>> >>> > lon at units = "degrees_east"
>> >>> >
>> >>> > tin = f->TMP_126_HTGL_123(tind,{minlat:maxlat},{minlon:maxlon})
>> >>> >
>> >>> > ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>> >>> >
>> >>> > And the error code that comes out:
>> >>> >
>> >>> > fatal:["Execute.c":6332]:variable (TMP_126_HTGL_123) is not in file
>> >>> > (f)
>> >>> >
>> >>> > fatal:["Execute.c":8575]:Execute: Error occurred at or near line 163
>> >>> > in
>> >>> > file
>> >>> > tmp_cv_fcst_namer.ncl
>> >>> >
>> >>> > ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>> >>> >
>> >>> >
>> >>> > This script works fine in 6.1.2, and the variable name is given
>> >>> > explicitly
>> >>> > rather than using a GRIB table.
>> >>> >
>> >>> >
>> >>> > On Thu, Feb 2, 2017 at 3:23 PM, Mary Haley <haley at ucar.edu> wrote:
>> >>> >>
>> >>> >> Hi Daniel,
>> >>> >>
>> >>> >> Unfortunately you won't be able to point to a 6.3.0 contributed.ncl
>> >>> >> while
>> >>> >> using a 6.1.2 "ncl", because it is very likely that contributed.ncl
>> >>> >> is
>> >>> >> depending on a new built-in function that's only available in the
>> >>> >> 6.3.0 ncl
>> >>> >> executable.
>> >>> >>
>> >>> >> We should try to figure out why you can't open your GRIB file with
>> >>> >> 6.3.0.
>> >>> >> We try *very hard* to be backwards compatible, but sometimes with
>> >>> >> GRIB
>> >>> >> files, there are things we have to change, usually having to do
>> >>> >> with
>> >>> >> variable names. I don't know if that's what's happening in your
>> >>> >> case.
>> >>> >>
>> >>> >> Are you getting an error? It would be helpful if we could see it.
>> >>> >>
>> >>> >> Also in NCL V6.2.0, which was released between 6.1.2 and 6.3.0,
>> >>> >> there
>> >>> >> were
>> >>> >> some updates to the GRIB reader. Maybe this update is affecting
>> >>> >> you?
>> >>> >> You can
>> >>> >> read more about these updates at:
>> >>> >>
>> >>> >> http://www.ncl.ucar.edu/prev_releases.shtml#FileIOImprovements6.2.0
>> >>> >>
>> >>> >> Also, you might want to read the "file I/O improvements" section of
>> >>> >> our
>> >>> >> 6.3.0 release notes:
>> >>> >>
>> >>> >> http://www.ncl.ucar.edu/current_release.shtml#Improvements6.3.0
>> >>> >>
>> >>> >> --Mary
>> >>> >>
>> >>> >>
>> >>> >> On Thu, Feb 2, 2017 at 12:17 PM, Daniel Barandiaran - NOAA
>> >>> >> Affiliate
>> >>> >> <daniel.barandiaran at noaa.gov> wrote:
>> >>> >>>
>> >>> >>> It seems a little strange to me as well, but the bottom line is
>> >>> >>> that
>> >>> >>> I
>> >>> >>> could open the GRIB file no problem using the 6.1.2 installed on
>> >>> >>> our
>> >>> >>> server,
>> >>> >>> but could not using my test-install of 6.3.0. Paths in our machine
>> >>> >>> are
>> >>> >>> preset, and I'm not even sure exactly where they all point to. In
>> >>> >>> order to
>> >>> >>> ensure I was running the test install I put an alias in my
>> >>> >>> .bashrc,
>> >>> >>> and I
>> >>> >>> made sure to set $NCAR_ROOT to point to the test install as well.
>> >>> >>>
>> >>> >>> On Thu, Feb 2, 2017 at 2:03 PM, Guido Cioni <guidocioni at gmail.com>
>> >>> >>> wrote:
>> >>> >>>>
>> >>> >>>> Hey Daniel,
>> >>> >>>> I’m sure that someone will be more useful than me but…aren’t GRIB
>> >>> >>>> library supposed to be independent from NCL?
>> >>> >>>> Maybe there is something wrong on GRIB API and not on the
>> >>> >>>> installation
>> >>> >>>> of NCL, or maybe you didn’t set the GRIB_API path properly? Just
>> >>> >>>> saying,
>> >>> >>>> don’t know whether it is relevant or not :)
>> >>> >>>>
>> >>> >>>> Cheers
>> >>> >>>>
>> >>> >>>> Il giorno 02 feb 2017, alle ore 19:13, Daniel Barandiaran - NOAA
>> >>> >>>> Affiliate <daniel.barandiaran at noaa.gov> ha scritto:
>> >>> >>>>
>> >>> >>>> Our work machine here at CPC is running ncl-6.1.2. I requested an
>> >>> >>>> update
>> >>> >>>> and got turned down by the IT folks; their reason had something
>> >>> >>>> to
>> >>> >>>> do with
>> >>> >>>> old netCDF packages and an eventual upgrade of our Linux
>> >>> >>>> environment, at
>> >>> >>>> which time it would be easier to upgrade. In the meantime I tried
>> >>> >>>> to
>> >>> >>>> unpack
>> >>> >>>> the latest binaries into my personal work directory and set paths
>> >>> >>>> appropriately, but sure enough there were some serious problems,
>> >>> >>>> like not
>> >>> >>>> being able to read variables out of a GRIB file. I have therefore
>> >>> >>>> mostly
>> >>> >>>> resigned myself to working with 6.1.2, but this morning I thought
>> >>> >>>> that a lot
>> >>> >>>> of the functions I am pining for reside in contributed.ncl, so I
>> >>> >>>> was
>> >>> >>>> wondering if I could point to the contributed.ncl that came with
>> >>> >>>> 6.3.0 while
>> >>> >>>> still running 6.1.2? And what about the ESMF regridding, could I
>> >>> >>>> point to
>> >>> >>>> the ESMF_regridding.ncl script and use those functions as well?
>> >>> >>>>
>> >>> >>>> --
>> >>> >>>> Danny Barandiaran, Ph.D.
>> >>> >>>> Contract Scientist-INNOVIM
>> >>> >>>> Climate Prediction Center/NCEP/NWS, W/NP5
>> >>> >>>> National Center for Weather and Climate Prediction
>> >>> >>>>
>> >>> >>>> _______________________________________________
>> >>> >>>> ncl-talk mailing list
>> >>> >>>> ncl-talk at ucar.edu
>> >>> >>>> List instructions, subscriber options, unsubscribe:
>> >>> >>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >>> >>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>> --
>> >>> >>> Danny Barandiaran, Ph.D.
>> >>> >>> Contract Scientist-INNOVIM
>> >>> >>> Climate Prediction Center/NCEP/NWS, W/NP5
>> >>> >>> National Center for Weather and Climate Prediction
>> >>> >>>
>> >>> >>>
>> >>> >>> _______________________________________________
>> >>> >>> ncl-talk mailing list
>> >>> >>> ncl-talk at ucar.edu
>> >>> >>> List instructions, subscriber options, unsubscribe:
>> >>> >>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >>> >>>
>> >>> >>
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Danny Barandiaran, Ph.D.
>> >>> > Contract Scientist-INNOVIM
>> >>> > Climate Prediction Center/NCEP/NWS, W/NP5
>> >>> > National Center for Weather and Climate Prediction
>> >>> >
>> >>> >
>> >>> > _______________________________________________
>> >>> > ncl-talk mailing list
>> >>> > ncl-talk at ucar.edu
>> >>> > List instructions, subscriber options, unsubscribe:
>> >>> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >>> >
>> >>> _______________________________________________
>> >>> ncl-talk mailing list
>> >>> ncl-talk at ucar.edu
>> >>> List instructions, subscriber options, unsubscribe:
>> >>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > Danny Barandiaran, Ph.D.
>> > Contract Scientist-INNOVIM
>> > Climate Prediction Center/NCEP/NWS, W/NP5
>> > National Center for Weather and Climate Prediction
>> >
>
>
>
>
> --
> Danny Barandiaran, Ph.D.
> Contract Scientist-INNOVIM
> Climate Prediction Center/NCEP/NWS, W/NP5
> National Center for Weather and Climate Prediction
>
More information about the ncl-talk
mailing list