[ncl-talk] Missing data in ERA5?
Anne Seidenglanz
annework985 at gmail.com
Wed Feb 10 01:39:32 MST 2021
Thanks Kyle for this information. Indeed, yesterday I realized the problem
is related to the transition into using final analysis in ERA5, so this
problem occurs if one wants to incorporate the latest analysis data of the
present year with re-analysis data of previous years.
Thanks for pointing this out.
Anne
On Wed, Feb 10, 2021 at 4:53 AM Kyle Griffin <windrunnerxc at gmail.com> wrote:
> Anne and all,
>
> For more information about ERA5 'expver', please read over the ERA5
> documentation. The page below is directly relevant and discusses how the
> "near-real-time" analysis transitions to the final analysis after a period
> of time. If you re-downloaded your data and did not see this variable, you
> likely had your time period of interest age out of the "tentative" analysis
> and into the final.
>
> https://confluence.ecmwf.int/pages/viewpage.action?pageId=173385064
>
>
> Kyle
>
> On Tue, Feb 9, 2021 at 12:06 PM Dennis Shea via ncl-talk <
> ncl-talk at mailman.ucar.edu> wrote:
>
>> Hi Anne,
>> THX for the update and posting to ncl-talk. It is good information to
>> have.
>> Stay Healthy
>> D
>>
>> On Tue, Feb 9, 2021 at 4:03 AM Anne Seidenglanz <annework985 at gmail.com>
>> wrote:
>>
>>> The data has been closely examined. The problem was related to
>>> downloading ERA5 data, for some reason a dimension "expver" (presumably
>>> experiment version?) appeared in the variable. This dimension did not
>>> appear after a fresh download of the data (minor changes in the script but
>>> the exact reason remains obscure to me).
>>> In any case, if someone happens to have problems plotting ERA5 data, pls
>>> check if the dimension "expver" appears, and if necessary re-download the
>>> data to end up with e.g. z(time, lev, lat, lon).
>>>
>>> On Mon, Feb 8, 2021 at 8:38 PM Dennis Shea <shea at ucar.edu> wrote:
>>>
>>>> The 'golden rule' of data processing is to look at the data.
>>>>
>>>> netcdf geop_500_975_89_20 {
>>>> dimensions:
>>>> longitude = 1440 ;
>>>> latitude = 721 ; * ; has pole points*
>>>> level = 2 ;
>>>> expver = 2 ; *; not sure what this is
>>>> BUT, possibly, two different physics packages*
>>>> *; OR, possibly, an
>>>> ensemble of size two OR, ???*
>>>> time = 160 ;
>>>> variables:
>>>> float longitude(longitude) ;
>>>> longitude:units = "degrees_east" ;
>>>> longitude:long_name = "longitude" ;
>>>> float latitude(latitude) ;
>>>> latitude:units = "degrees_north" ;
>>>> latitude:long_name = "latitude" ;
>>>> int level(level) ;
>>>> level:units = "millibars" ;
>>>> level:long_name = "pressure_level" ;
>>>> int expver(expver) ;
>>>> expver:long_name = "expver" ;
>>>> int time(time) ;
>>>> time:units = "hours since 1900-01-01 00:00:00.0" ;
>>>> time:long_name = "time" ;
>>>> time:calendar = "gregorian" ;
>>>> *short z(time, expver, level, latitude, longitude)* ;
>>>> <====== rank 5
>>>>
>>>> * z:scale_factor = 0.898208868468462 ; netCDF
>>>> CF [and COARDS] conventions* *z:add_offset =
>>>> 28656.6907881439 *; *; scale_factor and add_offet must be
>>>> present*
>>>> *z:_FillValue = -32767s ;
>>>> ; CF *
>>>> z:missing_value = -32767s ; ;
>>>> COARDS
>>>> z:units = "m**2 s**-2"
>>>> ; ; COARDS and CF conventions
>>>> z:long_name = "Geopotential" ; ;
>>>> COARDS convention
>>>> z:standard_name = "geopotential" ; ; CF
>>>> convention
>>>>
>>>> ======================================================
>>>>
>>>> dir="/Users/nuovoaccount/Documents/PYTHON/"
>>>> fil ="geop_500_975_89_20.nc"
>>>> f = addfile(dir+fil,"r")
>>>> z = *short2flt*
>>>> <http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>
>>>> (f->z)
>>>> printVarSummary(z) ; *(time, expver, level, latitude, longitude)*
>>>> printMinMax(z,0)
>>>>
>>>> ---
>>>> or,
>>>> [a] noting that NCL* eliminates degenerate dimensions* [size 1]
>>>> [b] no information is lost, since NCL will add variable attributes
>>>> for the degenerate dimensions: *expver, level*
>>>>
>>>> z = *short2flt*
>>>> <http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>(
>>>> f->z(:,1,1,:,:) ) ; second ensemble member and second level
>>>> printVarSummary(z) ; *(time, latitude, longitude) *with attributes *level
>>>> *and* expver*
>>>> printMinMax(z,0)
>>>>
>>>> HTH
>>>>
>>>>
>>>> On Mon, Feb 8, 2021 at 8:46 AM Anne Seidenglanz via ncl-talk <
>>>> ncl-talk at mailman.ucar.edu> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am not new to NCL but encounter a rather new problem when dealing
>>>>> with ERA5 data (in .nc format, see script attached). I am trying to produce
>>>>> a simple test plot for geopotential height, and somehow all my data seems
>>>>> to be missing? The error message appears:
>>>>> warning:ContourPlotInitialize: no valid values in scalar field;
>>>>> ContourPlot not possible:[errno=1101]
>>>>> When I printed a subset of the dataset, all the values seemed to be
>>>>> missing values.
>>>>>
>>>>> I have:
>>>>> - tested that not all data is missing in Z with "print("Is Z all
>>>>> missing ? " + all(ismissing(Z)))". It gave me False.
>>>>> - I have tried both, reading in the data using "short2flt" to account
>>>>> for scale factor and offset value, and without short2flt. In both cases
>>>>> appears the missing value problem and the same error message.
>>>>>
>>>>> Where have I gone wrong?
>>>>>
>>>>> Thx for any help
>>>>> Anne
>>>>> _______________________________________________
>>>>> ncl-talk mailing list
>>>>> ncl-talk at mailman.ucar.edu
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at mailman.ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20210210/42a99c8e/attachment.html>
More information about the ncl-talk
mailing list