[ncl-talk] Missing data in ERA5?

[Dennis Shea]

Hi Anne,
THX for the update and posting to ncl-talk.  It is good information to have.
Stay Healthy
D

On Tue, Feb 9, 2021 at 4:03 AM Anne Seidenglanz <annework985 at gmail.com>
wrote:

> The data has been closely examined. The problem was related to downloading
> ERA5 data, for some reason a dimension "expver" (presumably experiment
> version?) appeared in the variable. This dimension did not appear after a
> fresh download of the data (minor changes in the script but the exact
> reason remains obscure to me).
> In any case, if someone happens to have problems plotting ERA5 data, pls
> check if the dimension "expver" appears, and if necessary re-download the
> data to end up with e.g. z(time, lev, lat, lon).
>
> On Mon, Feb 8, 2021 at 8:38 PM Dennis Shea <shea at ucar.edu> wrote:
>
>> The 'golden rule' of data processing is to look at the data.
>>
>> netcdf geop_500_975_89_20 {
>> dimensions:
>>         longitude = 1440 ;
>>         latitude = 721 ;                * ; has pole points*
>>         level = 2 ;
>>         expver = 2 ;                      *; not sure what this is BUT,
>> possibly, two different physics packages*
>>                                                  *; OR, possibly, an
>> ensemble of size two  OR, ???*
>>         time = 160 ;
>> variables:
>>         float longitude(longitude) ;
>>                 longitude:units = "degrees_east" ;
>>                 longitude:long_name = "longitude" ;
>>         float latitude(latitude) ;
>>                 latitude:units = "degrees_north" ;
>>                 latitude:long_name = "latitude" ;
>>         int level(level) ;
>>                 level:units = "millibars" ;
>>                 level:long_name = "pressure_level" ;
>>         int expver(expver) ;
>>                 expver:long_name = "expver" ;
>>         int time(time) ;
>>                 time:units = "hours since 1900-01-01 00:00:00.0" ;
>>                 time:long_name = "time" ;
>>                 time:calendar = "gregorian" ;
>>         *short z(time, expver, level, latitude, longitude)* ;
>> <====== rank 5
>>
>> *                z:scale_factor = 0.898208868468462          ; netCDF CF
>> [and COARDS] conventions*                *z:add_offset =
>> 28656.6907881439 *;             *; scale_factor and add_offet must be
>> present*
>>                 *z:_FillValue = -32767s ;
>> ; CF                    *
>>                 z:missing_value = -32767s ;                          ;
>> COARDS
>>                 z:units = "m**2 s**-2"
>> ;                                   ; COARDS and CF conventions
>>                 z:long_name = "Geopotential" ;                     ;
>> COARDS convention
>>                 z:standard_name = "geopotential" ;              ; CF
>> convention
>>
>> ======================================================
>>
>> dir="/Users/nuovoaccount/Documents/PYTHON/"
>> fil ="geop_500_975_89_20.nc"
>> f  = addfile(dir+fil,"r")
>> z = *short2flt*
>> <http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>
>> (f->z)
>> printVarSummary(z)    ;  *(time, expver, level, latitude, longitude)*
>> printMinMax(z,0)
>>
>> ---
>> or,
>>     [a] noting that NCL* eliminates degenerate dimensions* [size 1]
>>     [b] no information is lost, since NCL will add variable attributes
>> for the degenerate dimensions: *expver, level*
>>
>> z = *short2flt*
>> <http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>(
>> f->z(:,1,1,:,:) )    ; second ensemble member and second level
>> printVarSummary(z)    ; *(time, latitude, longitude) *with attributes *level
>> *and* expver*
>> printMinMax(z,0)
>>
>> HTH
>>
>>
>> On Mon, Feb 8, 2021 at 8:46 AM Anne Seidenglanz via ncl-talk <
>> ncl-talk at mailman.ucar.edu> wrote:
>>
>>> Hi,
>>>
>>> I am not new to NCL but encounter a rather new problem when dealing with
>>> ERA5 data (in .nc format, see script attached). I am trying to produce a
>>> simple test plot for geopotential height, and somehow all my data seems to
>>> be missing? The error message appears:
>>> warning:ContourPlotInitialize: no valid values in scalar field;
>>> ContourPlot not possible:[errno=1101]
>>> When I printed a subset of the dataset, all the values seemed to be
>>> missing values.
>>>
>>> I have:
>>> - tested that not all data is missing in Z with "print("Is Z all missing
>>> ? " + all(ismissing(Z)))". It gave me False.
>>> - I have tried both, reading in the data using "short2flt" to account
>>> for scale factor and offset value, and without short2flt. In both cases
>>> appears the missing value problem and the same error message.
>>>
>>> Where have I gone wrong?
>>>
>>> Thx for any help
>>> Anne
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at mailman.ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20210209/9f8f782f/attachment.html>