[Met_help] [rt.rap.ucar.edu #95186] History for Fwd: real time verification?
John Halley Gotway via RT
met_help at ucar.edu
Thu May 7 10:25:51 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
Below is a forward of a forward. Essentially the question is in regards to
how MET handles the wind rotation. There are no assumptions made if the
winds are earth-relative or grid-relative? Is MET using the metadata to
determine that and handle that information as needed?
---------- Forwarded message ---------
From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
Date: Thu, May 7, 2020 at 10:23 AM
Subject: Fwd: real time verification?
To: Perry Shafran <perry.shafran at noaa.gov>, Logan Dawson - NOAA Affiliate <
logan.dawson at noaa.gov>, Mallory Row - NOAA Affiliate <mallory.row at noaa.gov>
Hi Perry, Logan, and Mallory:
FYI - GSL figured out the source of their poor wind verification stats with
their SAR runs. It turns out their verification code was assuming the winds
from the model were grid relative and was treating them improperly (that
used to be the convention in our regional models). However with the SAR
all winds are output as earth-relative.
I think out MET-based stats are okay, but I wanted to pass this along to
check and ensure that MET handles all wind vector rotation appropriately,
i.e. without assumption and uses the metadata in the grib2 records to act
accordingly.
Thanks!
Jacob
---------- Forwarded message ---------
From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
Date: Thu, May 7, 2020 at 10:19 AM
Subject: Re: real time verification?
To: Jeff Beck - NOAA Affiliate <jeff.beck at noaa.gov>
Cc: Benjamin Blake - NOAA Affiliate <benjamin.blake at noaa.gov>, Logan Dawson
- NOAA Federal <logan.dawson at noaa.gov>, Curtis Alexander - NOAA Federal <
curtis.alexander at noaa.gov>, Gerard Ketefian - NOAA Affiliate <
gerard.ketefian at noaa.gov>, Stephen Weygandt - NOAA Federal <
stephen.weygandt at noaa.gov>, Trevor Alcott - NOAA Federal <
trevor.alcott at noaa.gov>, Jim Purser - NOAA Affiliate <jim.purser at noaa.gov>,
Jeffrey Hamilton - NOAA Affiliate <jeffrey.a.hamilton at noaa.gov>, Ming Hu -
NOAA Affiliate <ming.hu at noaa.gov>, Jili Dong - NOAA Affiliate <
jili.dong at noaa.gov>
Hi Jeff,
This is excellent news! This change in how the winds are treated has bitten
us several times (in the post, now verification, and we recently
discovered an issue in the GSI - as Ming already knows). Hopefully this is
the last of it.
Also great that the ESG grid has been exonerated :-)
I think the MET-based verification is okay since your stats are now
comparable to ours, which does use MET. I'll pass this along to a few folks
in EMC to be on the safe side.
Thanks,
Jacob
On Wed, May 6, 2020 at 6:18 PM Jili Dong - NOAA Affiliate <
jili.dong at noaa.gov> wrote:
> Thanks for the update Jeff. It is really great news! This gives us more
> confidence in running ESG grid for hurricanes.
>
> Jili
>
> On Wed, May 6, 2020 at 4:45 PM Jeff Beck - NOAA Affiliate <
> jeff.beck at noaa.gov> wrote:
>
>> Hi Everyone,
>>
>> After a considerable amount of sleuthing by a number of people at GSL,
>> we've identified that the wind rotation problem was occurring in our
>> verification system, and not in the model data. Since the regional models
>> we've dealt with in the past have output grib2 files with grid-relative
>> winds, MATS was rotating the RAOB data to the LCC grid prior to
>> verification, resulting in large RMSE/bias. Jeff Hamilton will be
>> modifying the code to read the grib2 metadata flag that defines either grid
>> or earth-relative winds (fixed for SAR output by Jili Dong in January) to
>> tell the verification code whether rotation of the RAOB data is required or
>> not. Please see the attached plot that Jeff put together, showing the
>> difference when the RAOB data aren't rotated (windtest). Jeff, please feel
>> free to correct or add to anything I've said.
>>
>> Ultimately (and fortunately!), this likely means that the FV3 wind
>> initialization code does not need to be modified for the JP/ESG grid, and
>> it's possible for use to switch back over to this much more uniform
>> option. We have a number of parallel FV3-SAR runs going at GSL on Hera and
>> Jet which will enable us to compare the GFDL and JP/ESG grids directly to
>> make sure. Jim, we may even be able to leverage some of these direct
>> comparisons for the manuscript you're working on for the ESG grid.
>>
>> Finally, I wanted to ask Jacob and Ben whether the EMC verification knows
>> that the SAR grib2 output is earth-relative, and whether it is correctly
>> verifying earth-relative RAOB to earth-relative grib2 output. We weren't
>> able to plot a subset of dates for the wind profile RMSE/bias plots on the
>> EMC verification website, so I wasn't sure if there was contamination from
>> solely the earlier grib2 metadata "N/S" vs "grid" bug, or if something was
>> still present that was throwing off your verification for recent runs.
>>
>> - Jeff
>>
>> On Thu, Apr 30, 2020 at 5:57 AM Jacob Carley - NOAA Federal <
>> jacob.carley at noaa.gov> wrote:
>>
>>> Hi Ben,
>>>
>>> Yup - and Jeff and I confirmed that his files do as well. So I don't
>>> think there's a UPP issue.
>>>
>>> -Jacob
>>>
>>> On Thu, Apr 30, 2020 at 7:54 AM Benjamin Blake - NOAA Affiliate <
>>> benjamin.blake at noaa.gov> wrote:
>>>
>>>> Hi Gerard,
>>>>
>>>> On April 1st we switched from using global_chgres to chgres_cube when
>>>> we moved our real-time runs to the WCOSS Cray.
>>>>
>>>> Jacob, here is what wgrib2 -V shows for the 10-m U wind component in
>>>> our UPP output. Does winds (N/S) indicate the winds are Earth relative?
>>>>
>>>> 18:17311820:vt=2020050206:1 hybrid level:54 hour fcst:UGRD U-Component
>>>> of Wind [m/s]:
>>>> ndata=1905141:undef=0:mean=1.44973:min=-14.13:max=23.3
>>>> grid_template=30:winds(N/S):
>>>> Lambert Conformal: (1799 x 1059) input WE:SN output WE:SN res 0
>>>> Lat1 21.138123 Lon1 237.280472 LoV 262.500000
>>>> LatD 38.500000 Latin1 38.500000 Latin2 38.500000
>>>> LatSP 0.000000 LonSP 0.000000
>>>> North Pole (1799 x 1059) Dx 3000.000000 m Dy 3000.000000 m mode
>>>> 0
>>>>
>>>> Thanks,
>>>> Ben
>>>>
>>>> On Wed, Apr 29, 2020 at 6:37 PM Jacob Carley - NOAA Federal <
>>>> jacob.carley at noaa.gov> wrote:
>>>>
>>>>> Do a simple 'wgrib2 -V' on your files, it should tell you if the winds
>>>>> are encoded as grid or earth relative.
>>>>>
>>>>> On Wed, Apr 29, 2020 at 6:30 PM Jeff Beck - NOAA Affiliate <
>>>>> jeff.beck at noaa.gov> wrote:
>>>>>
>>>>>> My understanding was that the fix was implemented into UPP itself, is
>>>>>> that not correct? We're using a Lambert conformal write component grid and
>>>>>> the top-of-the-trunk EMC_post develop branch code.
>>>>>>
>>>>>> - Jeff
>>>>>>
>>>>>> On Wed, Apr 29, 2020 at 4:28 PM Logan Dawson - NOAA Federal <
>>>>>> logan.dawson at noaa.gov> wrote:
>>>>>>
>>>>>>> Hi Jeff,
>>>>>>>
>>>>>>> I still need to update the “Current Experiment Stats” to account for
>>>>>>> the latest SAR/-X configuration changes.
>>>>>>>
>>>>>>> Those plots going back to last November are definitely affected by
>>>>>>> the grid rotation issue, which wasn’t addressed in UPP until sometime in
>>>>>>> January.
>>>>>>>
>>>>>>> I’m juggling several things with the GEFS evaluation wrapping up and
>>>>>>> HREF about to start, but I’ll try to get those updated very soon since I
>>>>>>> know there’s increased interest with HWT kicking off this week.
>>>>>>>
>>>>>>> -Logan
>>>>>>>
>>>>>>> On Wed, Apr 29, 2020 at 6:23 PM Jeff Beck - NOAA Affiliate <
>>>>>>> jeff.beck at noaa.gov> wrote:
>>>>>>>
>>>>>>>> Helps if I attach the plots.
>>>>>>>>
>>>>>>>> - Jeff
>>>>>>>>
>>>>>>>> On Wed, Apr 29, 2020 at 4:22 PM Jeff Beck - NOAA Affiliate <
>>>>>>>> jeff.beck at noaa.gov> wrote:
>>>>>>>>
>>>>>>>>> Jacob et al.,
>>>>>>>>>
>>>>>>>>> Just to clarify, here are two plots from the SARX verification
>>>>>>>>> webpage showing the large bias for wind vector verification (that we're
>>>>>>>>> also seeing) when compared to the HRRR.
>>>>>>>>>
>>>>>>>>> - Jeff
>>>>>>>>>
>>>>>>>>> On Wed, Apr 29, 2020 at 4:03 PM Gerard Ketefian - NOAA Affiliate <
>>>>>>>>> gerard.ketefian at noaa.gov> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Jacob,
>>>>>>>>>>
>>>>>>>>>> My mistake: We're also using UPP output for verification, not
>>>>>>>>>> the history files (I think by history files you mean the dynf and phyf,
>>>>>>>>>> right?).
>>>>>>>>>>
>>>>>>>>>> Jeff says we have a very recent UPP version, so that bug is
>>>>>>>>>> probably not it.
>>>>>>>>>>
>>>>>>>>>> He also noticed from the page you sent that your results show the
>>>>>>>>>> same problem that we're seeing -- under upper air, vector wind, the RMSVE
>>>>>>>>>> from the FV3-SAR and (-X) are way to the right (or above, if you look at it
>>>>>>>>>> by forecast lead) of the others.
>>>>>>>>>>
>>>>>>>>>> Gerard
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 29, 2020 at 3:53 PM Jacob Carley - NOAA Federal <
>>>>>>>>>> jacob.carley at noaa.gov> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Gerard,
>>>>>>>>>>>
>>>>>>>>>>> 1) Which output file (and/or code) was the metadata bug in?
>>>>>>>>>>>> We're currently doing verification based on the write-component files
>>>>>>>>>>>> (dynf*.nc and phyf*.nc).
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The bug was in the UPP.
>>>>>>>>>>>
>>>>>>>>>>> However, the winds in the history files are re-gridded to cell
>>>>>>>>>>> centers and then rotated to be the zonal and meridional winds (earth
>>>>>>>>>>> relative). So they don't require any extra rotation. Could they be getting
>>>>>>>>>>> an extra rotation in the verification code? If you plot a 500 hPa height
>>>>>>>>>>> and wind plot and look near the domain corners, do you see the barbs align
>>>>>>>>>>> with the heights as you would expect for the flow regime? That helped us
>>>>>>>>>>> figure out the UPP issue.
>>>>>>>>>>>
>>>>>>>>>>> Make sure that you are using the 'grid_lont' and 'grid_latt'
>>>>>>>>>>> arrays for lats and lons, as those are defined for the cell centroids (the
>>>>>>>>>>> 'T-cells').
>>>>>>>>>>>
>>>>>>>>>>> 2) Which chgres was used for the runs shown on the page you
>>>>>>>>>>>> sent? Was it the old chgres or the new chgres_cube?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> That's a Ben question!
>>>>>>>>>>>
>>>>>>>>>>> Hope that helps,
>>>>>>>>>>> Jacob
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 29, 2020 at 5:44 PM Gerard Ketefian - NOAA Affiliate
>>>>>>>>>>> <gerard.ketefian at noaa.gov> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thank you Jacob. Couple of questions:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) Which output file (and/or code) was the metadata bug in?
>>>>>>>>>>>> We're currently doing verification based on the write-component files
>>>>>>>>>>>> (dynf*.nc and phyf*.nc).
>>>>>>>>>>>> 2) Which chgres was used for the runs shown on the page you
>>>>>>>>>>>> sent? Was it the old chgres or the new chgres_cube?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Gerard
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:34 PM Jacob Carley - NOAA Federal <
>>>>>>>>>>>> jacob.carley at noaa.gov> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Gerard,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'll step in real quick and point you to this page, which was
>>>>>>>>>>>>> developed by Logan Dawson (cc'd):
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://www.emc.ncep.noaa.gov/users/Logan.Dawson/fv3cam_verif/
>>>>>>>>>>>>>
>>>>>>>>>>>>> We had a wind issue in the SAR with the HREF, but that was due
>>>>>>>>>>>>> to a double wind rotation being applied. The winds coming out of the model
>>>>>>>>>>>>> are already earth relative and they were getting marked as grid relative in
>>>>>>>>>>>>> the post (and thus all downstream codes were rotating the winds again).
>>>>>>>>>>>>> Once we changed the metadata to the appropriate flag, the issue was fixed.
>>>>>>>>>>>>> Though I think you have this bug fix? It might be worth checking to see if
>>>>>>>>>>>>> that's happening.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Jacob
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 29, 2020 at 5:24 PM Gerard Ketefian - NOAA
>>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Looping in Trevor.
>>>>>>>>>>>>>> Gerard
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:22 PM Gerard Ketefian - NOAA
>>>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Ben,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Do you or have you run real-time runs with verification with
>>>>>>>>>>>>>>> the FV3-SAR (on a GFDLgrid)? You may have told us previously, but I forgot.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> GSL is having issues with verification with the FV3-SAR.
>>>>>>>>>>>>>>> THe problem is that the winds are pretty bad, even at hour 0. We first
>>>>>>>>>>>>>>> thought that it might be a JPgrid issue so switched to a GFDLgrid, but
>>>>>>>>>>>>>>> that's showing a similar problem. Just wanted to see if you have any plots
>>>>>>>>>>>>>>> we can look at.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Now I'm thinking it might be a chgres_cube problem. I know
>>>>>>>>>>>>>>> you were using the old chgres in the past. Are you still using that?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Gerard
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Gerard Ketefian
>>>>>>>>>>>>>>> Research Scientist
>>>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>>>>>>>>>>>>>>> 325 Broadway
>>>>>>>>>>>>>>> <https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g>
>>>>>>>>>>>>>>> Boulder, CO 80305
>>>>>>>>>>>>>>> <https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g>
>>>>>>>>>>>>>>> phone: 303-497-6209
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Gerard Ketefian
>>>>>>>>>>>>>> Research Scientist
>>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>>>>>>>>>>>>>> 325 Broadway
>>>>>>>>>>>>>> <https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g>
>>>>>>>>>>>>>> Boulder, CO 80305
>>>>>>>>>>>>>> <https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g>
>>>>>>>>>>>>>> phone: 303-497-6209
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Gerard Ketefian
>>>>>>>>>>>> Research Scientist
>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>>>>>>>>>>>> 325 Broadway
>>>>>>>>>>>> <https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g>
>>>>>>>>>>>> Boulder, CO 80305
>>>>>>>>>>>> <https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g>
>>>>>>>>>>>> phone: 303-497-6209
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Gerard Ketefian
>>>>>>>>>> Research Scientist
>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>>>>>>>>>> 325 Broadway
>>>>>>>>>> <https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g>
>>>>>>>>>> Boulder, CO 80305
>>>>>>>>>> <https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g>
>>>>>>>>>> phone: 303-497-6209
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>> Logan C. Dawson, Ph.D.
>>>>>>> Physical Scientist, VPPPG/MEG
>>>>>>> NOAA/NWS/NCEP/EMC
>>>>>>> (301) 683-3944
>>>>>>>
>>>>>>
>
> --
> Jili Dong
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct., # 2002
>
> College Park, MD 20740
>
> Jili.Dong at noaa.gov
>
> 301-683-3745
>
--
Mallory Row
IMSG at NOAA/NWS/NCEP/EMC
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Fwd: real time verification?
From: Minna Win
Time: Thu May 07 09:14:03 2020
Hi Mallory,
Thanks for summarizing the MET question about wind rotation. I'll
assign
this issue to John Halley Gotway.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, May 7, 2020 at 8:58 AM Mallory Row - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> Thu May 07 08:58:02 2020: Request 95186 was acted upon.
> Transaction: Ticket created by mallory.row at noaa.gov
> Queue: met_help
> Subject: Fwd: real time verification?
> Owner: Nobody
> Requestors: mallory.row at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95186 >
>
>
> Hi,
>
> Below is a forward of a forward. Essentially the question is in
regards to
> how MET handles the wind rotation. There are no assumptions made if
the
> winds are earth-relative or grid-relative? Is MET using the metadata
to
> determine that and handle that information as needed?
>
> ---------- Forwarded message ---------
> From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
> Date: Thu, May 7, 2020 at 10:23 AM
> Subject: Fwd: real time verification?
> To: Perry Shafran <perry.shafran at noaa.gov>, Logan Dawson - NOAA
Affiliate
> <
> logan.dawson at noaa.gov>, Mallory Row - NOAA Affiliate
<mallory.row at noaa.gov
> >
>
>
> Hi Perry, Logan, and Mallory:
>
> FYI - GSL figured out the source of their poor wind verification
stats with
> their SAR runs. It turns out their verification code was assuming
the winds
> from the model were grid relative and was treating them improperly
(that
> used to be the convention in our regional models). However with the
SAR
> all winds are output as earth-relative.
>
> I think out MET-based stats are okay, but I wanted to pass this
along to
> check and ensure that MET handles all wind vector rotation
appropriately,
> i.e. without assumption and uses the metadata in the grib2 records
to act
> accordingly.
>
> Thanks!
> Jacob
>
> ---------- Forwarded message ---------
> From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
> Date: Thu, May 7, 2020 at 10:19 AM
> Subject: Re: real time verification?
> To: Jeff Beck - NOAA Affiliate <jeff.beck at noaa.gov>
> Cc: Benjamin Blake - NOAA Affiliate <benjamin.blake at noaa.gov>, Logan
> Dawson
> - NOAA Federal <logan.dawson at noaa.gov>, Curtis Alexander - NOAA
Federal <
> curtis.alexander at noaa.gov>, Gerard Ketefian - NOAA Affiliate <
> gerard.ketefian at noaa.gov>, Stephen Weygandt - NOAA Federal <
> stephen.weygandt at noaa.gov>, Trevor Alcott - NOAA Federal <
> trevor.alcott at noaa.gov>, Jim Purser - NOAA Affiliate
<jim.purser at noaa.gov
> >,
> Jeffrey Hamilton - NOAA Affiliate <jeffrey.a.hamilton at noaa.gov>,
Ming Hu -
> NOAA Affiliate <ming.hu at noaa.gov>, Jili Dong - NOAA Affiliate <
> jili.dong at noaa.gov>
>
>
> Hi Jeff,
>
> This is excellent news! This change in how the winds are treated has
bitten
> us several times (in the post, now verification, and we recently
> discovered an issue in the GSI - as Ming already knows). Hopefully
this is
> the last of it.
>
> Also great that the ESG grid has been exonerated :-)
>
> I think the MET-based verification is okay since your stats are now
> comparable to ours, which does use MET. I'll pass this along to a
few folks
> in EMC to be on the safe side.
>
> Thanks,
> Jacob
>
> On Wed, May 6, 2020 at 6:18 PM Jili Dong - NOAA Affiliate <
> jili.dong at noaa.gov> wrote:
>
> > Thanks for the update Jeff. It is really great news! This gives us
more
> > confidence in running ESG grid for hurricanes.
> >
> > Jili
> >
> > On Wed, May 6, 2020 at 4:45 PM Jeff Beck - NOAA Affiliate <
> > jeff.beck at noaa.gov> wrote:
> >
> >> Hi Everyone,
> >>
> >> After a considerable amount of sleuthing by a number of people at
GSL,
> >> we've identified that the wind rotation problem was occurring in
our
> >> verification system, and not in the model data. Since the
regional
> models
> >> we've dealt with in the past have output grib2 files with grid-
relative
> >> winds, MATS was rotating the RAOB data to the LCC grid prior to
> >> verification, resulting in large RMSE/bias. Jeff Hamilton will
be
> >> modifying the code to read the grib2 metadata flag that defines
either
> grid
> >> or earth-relative winds (fixed for SAR output by Jili Dong in
January)
> to
> >> tell the verification code whether rotation of the RAOB data is
> required or
> >> not. Please see the attached plot that Jeff put together,
showing the
> >> difference when the RAOB data aren't rotated (windtest). Jeff,
please
> feel
> >> free to correct or add to anything I've said.
> >>
> >> Ultimately (and fortunately!), this likely means that the FV3
wind
> >> initialization code does not need to be modified for the JP/ESG
grid,
> and
> >> it's possible for use to switch back over to this much more
uniform
> >> option. We have a number of parallel FV3-SAR runs going at GSL
on Hera
> and
> >> Jet which will enable us to compare the GFDL and JP/ESG grids
directly
> to
> >> make sure. Jim, we may even be able to leverage some of these
direct
> >> comparisons for the manuscript you're working on for the ESG
grid.
> >>
> >> Finally, I wanted to ask Jacob and Ben whether the EMC
verification
> knows
> >> that the SAR grib2 output is earth-relative, and whether it is
correctly
> >> verifying earth-relative RAOB to earth-relative grib2 output. We
> weren't
> >> able to plot a subset of dates for the wind profile RMSE/bias
plots on
> the
> >> EMC verification website, so I wasn't sure if there was
contamination
> from
> >> solely the earlier grib2 metadata "N/S" vs "grid" bug, or if
something
> was
> >> still present that was throwing off your verification for recent
runs.
> >>
> >> - Jeff
> >>
> >> On Thu, Apr 30, 2020 at 5:57 AM Jacob Carley - NOAA Federal <
> >> jacob.carley at noaa.gov> wrote:
> >>
> >>> Hi Ben,
> >>>
> >>> Yup - and Jeff and I confirmed that his files do as well. So I
don't
> >>> think there's a UPP issue.
> >>>
> >>> -Jacob
> >>>
> >>> On Thu, Apr 30, 2020 at 7:54 AM Benjamin Blake - NOAA Affiliate
<
> >>> benjamin.blake at noaa.gov> wrote:
> >>>
> >>>> Hi Gerard,
> >>>>
> >>>> On April 1st we switched from using global_chgres to
chgres_cube when
> >>>> we moved our real-time runs to the WCOSS Cray.
> >>>>
> >>>> Jacob, here is what wgrib2 -V shows for the 10-m U wind
component in
> >>>> our UPP output. Does winds (N/S) indicate the winds are Earth
> relative?
> >>>>
> >>>> 18:17311820:vt=2020050206:1 hybrid level:54 hour fcst:UGRD U-
Component
> >>>> of Wind [m/s]:
> >>>> ndata=1905141:undef=0:mean=1.44973:min=-14.13:max=23.3
> >>>> grid_template=30:winds(N/S):
> >>>> Lambert Conformal: (1799 x 1059) input WE:SN output
WE:SN res
> 0
> >>>> Lat1 21.138123 Lon1 237.280472 LoV 262.500000
> >>>> LatD 38.500000 Latin1 38.500000 Latin2 38.500000
> >>>> LatSP 0.000000 LonSP 0.000000
> >>>> North Pole (1799 x 1059) Dx 3000.000000 m Dy
3000.000000 m
> mode
> >>>> 0
> >>>>
> >>>> Thanks,
> >>>> Ben
> >>>>
> >>>> On Wed, Apr 29, 2020 at 6:37 PM Jacob Carley - NOAA Federal <
> >>>> jacob.carley at noaa.gov> wrote:
> >>>>
> >>>>> Do a simple 'wgrib2 -V' on your files, it should tell you if
the
> winds
> >>>>> are encoded as grid or earth relative.
> >>>>>
> >>>>> On Wed, Apr 29, 2020 at 6:30 PM Jeff Beck - NOAA Affiliate <
> >>>>> jeff.beck at noaa.gov> wrote:
> >>>>>
> >>>>>> My understanding was that the fix was implemented into UPP
itself,
> is
> >>>>>> that not correct? We're using a Lambert conformal write
component
> grid and
> >>>>>> the top-of-the-trunk EMC_post develop branch code.
> >>>>>>
> >>>>>> - Jeff
> >>>>>>
> >>>>>> On Wed, Apr 29, 2020 at 4:28 PM Logan Dawson - NOAA Federal <
> >>>>>> logan.dawson at noaa.gov> wrote:
> >>>>>>
> >>>>>>> Hi Jeff,
> >>>>>>>
> >>>>>>> I still need to update the “Current Experiment Stats” to
account
> for
> >>>>>>> the latest SAR/-X configuration changes.
> >>>>>>>
> >>>>>>> Those plots going back to last November are definitely
affected by
> >>>>>>> the grid rotation issue, which wasn’t addressed in UPP until
> sometime in
> >>>>>>> January.
> >>>>>>>
> >>>>>>> I’m juggling several things with the GEFS evaluation
wrapping up
> and
> >>>>>>> HREF about to start, but I’ll try to get those updated very
soon
> since I
> >>>>>>> know there’s increased interest with HWT kicking off this
week.
> >>>>>>>
> >>>>>>> -Logan
> >>>>>>>
> >>>>>>> On Wed, Apr 29, 2020 at 6:23 PM Jeff Beck - NOAA Affiliate <
> >>>>>>> jeff.beck at noaa.gov> wrote:
> >>>>>>>
> >>>>>>>> Helps if I attach the plots.
> >>>>>>>>
> >>>>>>>> - Jeff
> >>>>>>>>
> >>>>>>>> On Wed, Apr 29, 2020 at 4:22 PM Jeff Beck - NOAA Affiliate
<
> >>>>>>>> jeff.beck at noaa.gov> wrote:
> >>>>>>>>
> >>>>>>>>> Jacob et al.,
> >>>>>>>>>
> >>>>>>>>> Just to clarify, here are two plots from the SARX
verification
> >>>>>>>>> webpage showing the large bias for wind vector
verification
> (that we're
> >>>>>>>>> also seeing) when compared to the HRRR.
> >>>>>>>>>
> >>>>>>>>> - Jeff
> >>>>>>>>>
> >>>>>>>>> On Wed, Apr 29, 2020 at 4:03 PM Gerard Ketefian - NOAA
Affiliate
> <
> >>>>>>>>> gerard.ketefian at noaa.gov> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Jacob,
> >>>>>>>>>>
> >>>>>>>>>> My mistake: We're also using UPP output for
verification, not
> >>>>>>>>>> the history files (I think by history files you mean the
dynf
> and phyf,
> >>>>>>>>>> right?).
> >>>>>>>>>>
> >>>>>>>>>> Jeff says we have a very recent UPP version, so that bug
is
> >>>>>>>>>> probably not it.
> >>>>>>>>>>
> >>>>>>>>>> He also noticed from the page you sent that your results
show
> the
> >>>>>>>>>> same problem that we're seeing -- under upper air, vector
wind,
> the RMSVE
> >>>>>>>>>> from the FV3-SAR and (-X) are way to the right (or above,
if
> you look at it
> >>>>>>>>>> by forecast lead) of the others.
> >>>>>>>>>>
> >>>>>>>>>> Gerard
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Apr 29, 2020 at 3:53 PM Jacob Carley - NOAA
Federal <
> >>>>>>>>>> jacob.carley at noaa.gov> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Gerard,
> >>>>>>>>>>>
> >>>>>>>>>>> 1) Which output file (and/or code) was the metadata bug
in?
> >>>>>>>>>>>> We're currently doing verification based on the
> write-component files
> >>>>>>>>>>>> (dynf*.nc and phyf*.nc).
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> The bug was in the UPP.
> >>>>>>>>>>>
> >>>>>>>>>>> However, the winds in the history files are re-gridded
to cell
> >>>>>>>>>>> centers and then rotated to be the zonal and meridional
winds
> (earth
> >>>>>>>>>>> relative). So they don't require any extra rotation.
Could
> they be getting
> >>>>>>>>>>> an extra rotation in the verification code? If you plot
a 500
> hPa height
> >>>>>>>>>>> and wind plot and look near the domain corners, do you
see the
> barbs align
> >>>>>>>>>>> with the heights as you would expect for the flow
regime?
> That helped us
> >>>>>>>>>>> figure out the UPP issue.
> >>>>>>>>>>>
> >>>>>>>>>>> Make sure that you are using the 'grid_lont' and
'grid_latt'
> >>>>>>>>>>> arrays for lats and lons, as those are defined for the
cell
> centroids (the
> >>>>>>>>>>> 'T-cells').
> >>>>>>>>>>>
> >>>>>>>>>>> 2) Which chgres was used for the runs shown on the page
you
> >>>>>>>>>>>> sent? Was it the old chgres or the new chgres_cube?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> That's a Ben question!
> >>>>>>>>>>>
> >>>>>>>>>>> Hope that helps,
> >>>>>>>>>>> Jacob
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Apr 29, 2020 at 5:44 PM Gerard Ketefian - NOAA
> Affiliate
> >>>>>>>>>>> <gerard.ketefian at noaa.gov> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Thank you Jacob. Couple of questions:
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1) Which output file (and/or code) was the metadata bug
in?
> >>>>>>>>>>>> We're currently doing verification based on the
> write-component files
> >>>>>>>>>>>> (dynf*.nc and phyf*.nc).
> >>>>>>>>>>>> 2) Which chgres was used for the runs shown on the page
you
> >>>>>>>>>>>> sent? Was it the old chgres or the new chgres_cube?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Gerard
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:34 PM Jacob Carley - NOAA
Federal <
> >>>>>>>>>>>> jacob.carley at noaa.gov> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Gerard,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I'll step in real quick and point you to this page,
which was
> >>>>>>>>>>>>> developed by Logan Dawson (cc'd):
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> https://www.emc.ncep.noaa.gov/users/Logan.Dawson/fv3cam_verif/
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We had a wind issue in the SAR with the HREF, but that
was
> due
> >>>>>>>>>>>>> to a double wind rotation being applied. The winds
coming
> out of the model
> >>>>>>>>>>>>> are already earth relative and they were getting
marked as
> grid relative in
> >>>>>>>>>>>>> the post (and thus all downstream codes were rotating
the
> winds again).
> >>>>>>>>>>>>> Once we changed the metadata to the appropriate flag,
the
> issue was fixed.
> >>>>>>>>>>>>> Though I think you have this bug fix? It might be
worth
> checking to see if
> >>>>>>>>>>>>> that's happening.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Jacob
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Apr 29, 2020 at 5:24 PM Gerard Ketefian - NOAA
> >>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Looping in Trevor.
> >>>>>>>>>>>>>> Gerard
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:22 PM Gerard Ketefian -
NOAA
> >>>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Ben,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Do you or have you run real-time runs with
verification
> with
> >>>>>>>>>>>>>>> the FV3-SAR (on a GFDLgrid)? You may have told us
> previously, but I forgot.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> GSL is having issues with verification with the FV3-
SAR.
> >>>>>>>>>>>>>>> THe problem is that the winds are pretty bad, even
at hour
> 0. We first
> >>>>>>>>>>>>>>> thought that it might be a JPgrid issue so switched
to a
> GFDLgrid, but
> >>>>>>>>>>>>>>> that's showing a similar problem. Just wanted to
see if
> you have any plots
> >>>>>>>>>>>>>>> we can look at.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Now I'm thinking it might be a chgres_cube problem.
I know
> >>>>>>>>>>>>>>> you were using the old chgres in the past. Are you
still
> using that?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Gerard
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> Gerard Ketefian
> >>>>>>>>>>>>>>> Research Scientist
> >>>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> >>>>>>>>>>>>>>> 325 Broadway
> >>>>>>>>>>>>>>> <
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> >
> >>>>>>>>>>>>>>> Boulder, CO 80305
> >>>>>>>>>>>>>>> <
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> >
> >>>>>>>>>>>>>>> phone: 303-497-6209
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> Gerard Ketefian
> >>>>>>>>>>>>>> Research Scientist
> >>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> >>>>>>>>>>>>>> 325 Broadway
> >>>>>>>>>>>>>> <
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> >
> >>>>>>>>>>>>>> Boulder, CO 80305
> >>>>>>>>>>>>>> <
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> >
> >>>>>>>>>>>>>> phone: 303-497-6209
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Gerard Ketefian
> >>>>>>>>>>>> Research Scientist
> >>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> >>>>>>>>>>>> 325 Broadway
> >>>>>>>>>>>> <
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> >
> >>>>>>>>>>>> Boulder, CO 80305
> >>>>>>>>>>>> <
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> >
> >>>>>>>>>>>> phone: 303-497-6209
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Gerard Ketefian
> >>>>>>>>>> Research Scientist
> >>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> >>>>>>>>>> 325 Broadway
> >>>>>>>>>> <
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> >
> >>>>>>>>>> Boulder, CO 80305
> >>>>>>>>>> <
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> >
> >>>>>>>>>> phone: 303-497-6209
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>> Logan C. Dawson, Ph.D.
> >>>>>>> Physical Scientist, VPPPG/MEG
> >>>>>>> NOAA/NWS/NCEP/EMC
> >>>>>>> (301) 683-3944
> >>>>>>>
> >>>>>>
> >
> > --
> > Jili Dong
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct., # 2002
> >
> > College Park, MD 20740
> >
> > Jili.Dong at noaa.gov
> >
> > 301-683-3745
> >
>
>
> --
>
> Mallory Row
>
> IMSG at NOAA/NWS/NCEP/EMC
>
>
------------------------------------------------
Subject: Fwd: real time verification?
From: John Halley Gotway
Time: Thu May 07 10:06:23 2020
Mallory,
When MET reads wind variables from GRIB 1 and 2 files (e.g. UGRD,
VGRD, and
WDIR), it checks the metadata to see if the winds are defined as grid-
or
earth-relative. If they are grid-relative, MET automatically reads the
corresponding vector component and uses them to rotate the winds from
being
grid-relative to earth-relative. This is necessary prior to
verification
since point observations of winds should always be earth-relative.
When running at verbosity level 3 or higher, you should see log
messages
such as:
*"Rotating wind direction from grid-relative to earth-relative."*
or
*"Rotating U and V wind components from grid-relative to earth-
relative."*
In GRIB1, this flag is contained in the Grid Description Section, in
the
5th bit of 17-th octet:
https://www.nco.ncep.noaa.gov/pmb/docs/on388/table7.html
And here's the corresponding table for GRIB2:
https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table3-
3.shtml
MET checks these flags and handles them correctly, but this is ONLY
when
reading GRIB1/GRIB2 data. No such logic is applied for gridded NetCDF
data. It does seem like to should consider adding support to rotate
winds
as needed in future versions of MET.
Does that answer the question?
Thanks,
John
On Thu, May 7, 2020 at 9:26 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95186 >
>
> Hi Mallory,
>
> Thanks for summarizing the MET question about wind rotation. I'll
assign
> this issue to John Halley Gotway.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Thu, May 7, 2020 at 8:58 AM Mallory Row - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu May 07 08:58:02 2020: Request 95186 was acted upon.
> > Transaction: Ticket created by mallory.row at noaa.gov
> > Queue: met_help
> > Subject: Fwd: real time verification?
> > Owner: Nobody
> > Requestors: mallory.row at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95186 >
> >
> >
> > Hi,
> >
> > Below is a forward of a forward. Essentially the question is in
regards
> to
> > how MET handles the wind rotation. There are no assumptions made
if the
> > winds are earth-relative or grid-relative? Is MET using the
metadata to
> > determine that and handle that information as needed?
> >
> > ---------- Forwarded message ---------
> > From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
> > Date: Thu, May 7, 2020 at 10:23 AM
> > Subject: Fwd: real time verification?
> > To: Perry Shafran <perry.shafran at noaa.gov>, Logan Dawson - NOAA
> Affiliate
> > <
> > logan.dawson at noaa.gov>, Mallory Row - NOAA Affiliate <
> mallory.row at noaa.gov
> > >
> >
> >
> > Hi Perry, Logan, and Mallory:
> >
> > FYI - GSL figured out the source of their poor wind verification
stats
> with
> > their SAR runs. It turns out their verification code was assuming
the
> winds
> > from the model were grid relative and was treating them improperly
(that
> > used to be the convention in our regional models). However with
the SAR
> > all winds are output as earth-relative.
> >
> > I think out MET-based stats are okay, but I wanted to pass this
along to
> > check and ensure that MET handles all wind vector rotation
appropriately,
> > i.e. without assumption and uses the metadata in the grib2 records
to act
> > accordingly.
> >
> > Thanks!
> > Jacob
> >
> > ---------- Forwarded message ---------
> > From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
> > Date: Thu, May 7, 2020 at 10:19 AM
> > Subject: Re: real time verification?
> > To: Jeff Beck - NOAA Affiliate <jeff.beck at noaa.gov>
> > Cc: Benjamin Blake - NOAA Affiliate <benjamin.blake at noaa.gov>,
Logan
> > Dawson
> > - NOAA Federal <logan.dawson at noaa.gov>, Curtis Alexander - NOAA
Federal
> <
> > curtis.alexander at noaa.gov>, Gerard Ketefian - NOAA Affiliate <
> > gerard.ketefian at noaa.gov>, Stephen Weygandt - NOAA Federal <
> > stephen.weygandt at noaa.gov>, Trevor Alcott - NOAA Federal <
> > trevor.alcott at noaa.gov>, Jim Purser - NOAA Affiliate <
> jim.purser at noaa.gov
> > >,
> > Jeffrey Hamilton - NOAA Affiliate <jeffrey.a.hamilton at noaa.gov>,
Ming
> Hu -
> > NOAA Affiliate <ming.hu at noaa.gov>, Jili Dong - NOAA Affiliate <
> > jili.dong at noaa.gov>
> >
> >
> > Hi Jeff,
> >
> > This is excellent news! This change in how the winds are treated
has
> bitten
> > us several times (in the post, now verification, and we recently
> > discovered an issue in the GSI - as Ming already knows).
Hopefully this
> is
> > the last of it.
> >
> > Also great that the ESG grid has been exonerated :-)
> >
> > I think the MET-based verification is okay since your stats are
now
> > comparable to ours, which does use MET. I'll pass this along to a
few
> folks
> > in EMC to be on the safe side.
> >
> > Thanks,
> > Jacob
> >
> > On Wed, May 6, 2020 at 6:18 PM Jili Dong - NOAA Affiliate <
> > jili.dong at noaa.gov> wrote:
> >
> > > Thanks for the update Jeff. It is really great news! This gives
us more
> > > confidence in running ESG grid for hurricanes.
> > >
> > > Jili
> > >
> > > On Wed, May 6, 2020 at 4:45 PM Jeff Beck - NOAA Affiliate <
> > > jeff.beck at noaa.gov> wrote:
> > >
> > >> Hi Everyone,
> > >>
> > >> After a considerable amount of sleuthing by a number of people
at GSL,
> > >> we've identified that the wind rotation problem was occurring
in our
> > >> verification system, and not in the model data. Since the
regional
> > models
> > >> we've dealt with in the past have output grib2 files with
> grid-relative
> > >> winds, MATS was rotating the RAOB data to the LCC grid prior to
> > >> verification, resulting in large RMSE/bias. Jeff Hamilton will
be
> > >> modifying the code to read the grib2 metadata flag that defines
either
> > grid
> > >> or earth-relative winds (fixed for SAR output by Jili Dong in
January)
> > to
> > >> tell the verification code whether rotation of the RAOB data is
> > required or
> > >> not. Please see the attached plot that Jeff put together,
showing the
> > >> difference when the RAOB data aren't rotated (windtest). Jeff,
please
> > feel
> > >> free to correct or add to anything I've said.
> > >>
> > >> Ultimately (and fortunately!), this likely means that the FV3
wind
> > >> initialization code does not need to be modified for the JP/ESG
grid,
> > and
> > >> it's possible for use to switch back over to this much more
uniform
> > >> option. We have a number of parallel FV3-SAR runs going at GSL
on
> Hera
> > and
> > >> Jet which will enable us to compare the GFDL and JP/ESG grids
directly
> > to
> > >> make sure. Jim, we may even be able to leverage some of these
direct
> > >> comparisons for the manuscript you're working on for the ESG
grid.
> > >>
> > >> Finally, I wanted to ask Jacob and Ben whether the EMC
verification
> > knows
> > >> that the SAR grib2 output is earth-relative, and whether it is
> correctly
> > >> verifying earth-relative RAOB to earth-relative grib2 output.
We
> > weren't
> > >> able to plot a subset of dates for the wind profile RMSE/bias
plots on
> > the
> > >> EMC verification website, so I wasn't sure if there was
contamination
> > from
> > >> solely the earlier grib2 metadata "N/S" vs "grid" bug, or if
something
> > was
> > >> still present that was throwing off your verification for
recent runs.
> > >>
> > >> - Jeff
> > >>
> > >> On Thu, Apr 30, 2020 at 5:57 AM Jacob Carley - NOAA Federal <
> > >> jacob.carley at noaa.gov> wrote:
> > >>
> > >>> Hi Ben,
> > >>>
> > >>> Yup - and Jeff and I confirmed that his files do as well. So
I don't
> > >>> think there's a UPP issue.
> > >>>
> > >>> -Jacob
> > >>>
> > >>> On Thu, Apr 30, 2020 at 7:54 AM Benjamin Blake - NOAA
Affiliate <
> > >>> benjamin.blake at noaa.gov> wrote:
> > >>>
> > >>>> Hi Gerard,
> > >>>>
> > >>>> On April 1st we switched from using global_chgres to
chgres_cube
> when
> > >>>> we moved our real-time runs to the WCOSS Cray.
> > >>>>
> > >>>> Jacob, here is what wgrib2 -V shows for the 10-m U wind
component in
> > >>>> our UPP output. Does winds (N/S) indicate the winds are
Earth
> > relative?
> > >>>>
> > >>>> 18:17311820:vt=2020050206:1 hybrid level:54 hour fcst:UGRD
> U-Component
> > >>>> of Wind [m/s]:
> > >>>> ndata=1905141:undef=0:mean=1.44973:min=-14.13:max=23.3
> > >>>> grid_template=30:winds(N/S):
> > >>>> Lambert Conformal: (1799 x 1059) input WE:SN output
WE:SN
> res
> > 0
> > >>>> Lat1 21.138123 Lon1 237.280472 LoV 262.500000
> > >>>> LatD 38.500000 Latin1 38.500000 Latin2 38.500000
> > >>>> LatSP 0.000000 LonSP 0.000000
> > >>>> North Pole (1799 x 1059) Dx 3000.000000 m Dy
3000.000000 m
> > mode
> > >>>> 0
> > >>>>
> > >>>> Thanks,
> > >>>> Ben
> > >>>>
> > >>>> On Wed, Apr 29, 2020 at 6:37 PM Jacob Carley - NOAA Federal <
> > >>>> jacob.carley at noaa.gov> wrote:
> > >>>>
> > >>>>> Do a simple 'wgrib2 -V' on your files, it should tell you if
the
> > winds
> > >>>>> are encoded as grid or earth relative.
> > >>>>>
> > >>>>> On Wed, Apr 29, 2020 at 6:30 PM Jeff Beck - NOAA Affiliate <
> > >>>>> jeff.beck at noaa.gov> wrote:
> > >>>>>
> > >>>>>> My understanding was that the fix was implemented into UPP
itself,
> > is
> > >>>>>> that not correct? We're using a Lambert conformal write
component
> > grid and
> > >>>>>> the top-of-the-trunk EMC_post develop branch code.
> > >>>>>>
> > >>>>>> - Jeff
> > >>>>>>
> > >>>>>> On Wed, Apr 29, 2020 at 4:28 PM Logan Dawson - NOAA Federal
<
> > >>>>>> logan.dawson at noaa.gov> wrote:
> > >>>>>>
> > >>>>>>> Hi Jeff,
> > >>>>>>>
> > >>>>>>> I still need to update the “Current Experiment Stats” to
account
> > for
> > >>>>>>> the latest SAR/-X configuration changes.
> > >>>>>>>
> > >>>>>>> Those plots going back to last November are definitely
affected
> by
> > >>>>>>> the grid rotation issue, which wasn’t addressed in UPP
until
> > sometime in
> > >>>>>>> January.
> > >>>>>>>
> > >>>>>>> I’m juggling several things with the GEFS evaluation
wrapping up
> > and
> > >>>>>>> HREF about to start, but I’ll try to get those updated
very soon
> > since I
> > >>>>>>> know there’s increased interest with HWT kicking off this
week.
> > >>>>>>>
> > >>>>>>> -Logan
> > >>>>>>>
> > >>>>>>> On Wed, Apr 29, 2020 at 6:23 PM Jeff Beck - NOAA Affiliate
<
> > >>>>>>> jeff.beck at noaa.gov> wrote:
> > >>>>>>>
> > >>>>>>>> Helps if I attach the plots.
> > >>>>>>>>
> > >>>>>>>> - Jeff
> > >>>>>>>>
> > >>>>>>>> On Wed, Apr 29, 2020 at 4:22 PM Jeff Beck - NOAA
Affiliate <
> > >>>>>>>> jeff.beck at noaa.gov> wrote:
> > >>>>>>>>
> > >>>>>>>>> Jacob et al.,
> > >>>>>>>>>
> > >>>>>>>>> Just to clarify, here are two plots from the SARX
verification
> > >>>>>>>>> webpage showing the large bias for wind vector
verification
> > (that we're
> > >>>>>>>>> also seeing) when compared to the HRRR.
> > >>>>>>>>>
> > >>>>>>>>> - Jeff
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Apr 29, 2020 at 4:03 PM Gerard Ketefian - NOAA
> Affiliate
> > <
> > >>>>>>>>> gerard.ketefian at noaa.gov> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi Jacob,
> > >>>>>>>>>>
> > >>>>>>>>>> My mistake: We're also using UPP output for
verification, not
> > >>>>>>>>>> the history files (I think by history files you mean
the dynf
> > and phyf,
> > >>>>>>>>>> right?).
> > >>>>>>>>>>
> > >>>>>>>>>> Jeff says we have a very recent UPP version, so that
bug is
> > >>>>>>>>>> probably not it.
> > >>>>>>>>>>
> > >>>>>>>>>> He also noticed from the page you sent that your
results show
> > the
> > >>>>>>>>>> same problem that we're seeing -- under upper air,
vector
> wind,
> > the RMSVE
> > >>>>>>>>>> from the FV3-SAR and (-X) are way to the right (or
above, if
> > you look at it
> > >>>>>>>>>> by forecast lead) of the others.
> > >>>>>>>>>>
> > >>>>>>>>>> Gerard
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Wed, Apr 29, 2020 at 3:53 PM Jacob Carley - NOAA
Federal <
> > >>>>>>>>>> jacob.carley at noaa.gov> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Hi Gerard,
> > >>>>>>>>>>>
> > >>>>>>>>>>> 1) Which output file (and/or code) was the metadata
bug in?
> > >>>>>>>>>>>> We're currently doing verification based on the
> > write-component files
> > >>>>>>>>>>>> (dynf*.nc and phyf*.nc).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> The bug was in the UPP.
> > >>>>>>>>>>>
> > >>>>>>>>>>> However, the winds in the history files are re-gridded
to
> cell
> > >>>>>>>>>>> centers and then rotated to be the zonal and
meridional winds
> > (earth
> > >>>>>>>>>>> relative). So they don't require any extra rotation.
Could
> > they be getting
> > >>>>>>>>>>> an extra rotation in the verification code? If you
plot a
> 500
> > hPa height
> > >>>>>>>>>>> and wind plot and look near the domain corners, do you
see
> the
> > barbs align
> > >>>>>>>>>>> with the heights as you would expect for the flow
regime?
> > That helped us
> > >>>>>>>>>>> figure out the UPP issue.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Make sure that you are using the 'grid_lont' and
'grid_latt'
> > >>>>>>>>>>> arrays for lats and lons, as those are defined for the
cell
> > centroids (the
> > >>>>>>>>>>> 'T-cells').
> > >>>>>>>>>>>
> > >>>>>>>>>>> 2) Which chgres was used for the runs shown on the
page you
> > >>>>>>>>>>>> sent? Was it the old chgres or the new chgres_cube?
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> That's a Ben question!
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hope that helps,
> > >>>>>>>>>>> Jacob
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Apr 29, 2020 at 5:44 PM Gerard Ketefian - NOAA
> > Affiliate
> > >>>>>>>>>>> <gerard.ketefian at noaa.gov> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Thank you Jacob. Couple of questions:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 1) Which output file (and/or code) was the metadata
bug in?
> > >>>>>>>>>>>> We're currently doing verification based on the
> > write-component files
> > >>>>>>>>>>>> (dynf*.nc and phyf*.nc).
> > >>>>>>>>>>>> 2) Which chgres was used for the runs shown on the
page you
> > >>>>>>>>>>>> sent? Was it the old chgres or the new chgres_cube?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>> Gerard
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:34 PM Jacob Carley - NOAA
Federal
> <
> > >>>>>>>>>>>> jacob.carley at noaa.gov> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi Gerard,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I'll step in real quick and point you to this page,
which
> was
> > >>>>>>>>>>>>> developed by Logan Dawson (cc'd):
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > https://www.emc.ncep.noaa.gov/users/Logan.Dawson/fv3cam_verif/
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We had a wind issue in the SAR with the HREF, but
that was
> > due
> > >>>>>>>>>>>>> to a double wind rotation being applied. The winds
coming
> > out of the model
> > >>>>>>>>>>>>> are already earth relative and they were getting
marked as
> > grid relative in
> > >>>>>>>>>>>>> the post (and thus all downstream codes were
rotating the
> > winds again).
> > >>>>>>>>>>>>> Once we changed the metadata to the appropriate
flag, the
> > issue was fixed.
> > >>>>>>>>>>>>> Though I think you have this bug fix? It might be
worth
> > checking to see if
> > >>>>>>>>>>>>> that's happening.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> -Jacob
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Wed, Apr 29, 2020 at 5:24 PM Gerard Ketefian -
NOAA
> > >>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Looping in Trevor.
> > >>>>>>>>>>>>>> Gerard
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:22 PM Gerard Ketefian -
NOAA
> > >>>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi Ben,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Do you or have you run real-time runs with
verification
> > with
> > >>>>>>>>>>>>>>> the FV3-SAR (on a GFDLgrid)? You may have told us
> > previously, but I forgot.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> GSL is having issues with verification with the
FV3-SAR.
> > >>>>>>>>>>>>>>> THe problem is that the winds are pretty bad, even
at
> hour
> > 0. We first
> > >>>>>>>>>>>>>>> thought that it might be a JPgrid issue so
switched to a
> > GFDLgrid, but
> > >>>>>>>>>>>>>>> that's showing a similar problem. Just wanted to
see if
> > you have any plots
> > >>>>>>>>>>>>>>> we can look at.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Now I'm thinking it might be a chgres_cube
problem. I
> know
> > >>>>>>>>>>>>>>> you were using the old chgres in the past. Are
you still
> > using that?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>> Gerard
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>> Gerard Ketefian
> > >>>>>>>>>>>>>>> Research Scientist
> > >>>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> > >>>>>>>>>>>>>>> 325 Broadway
> > >>>>>>>>>>>>>>> <
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > >
> > >>>>>>>>>>>>>>> Boulder, CO 80305
> > >>>>>>>>>>>>>>> <
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > >
> > >>>>>>>>>>>>>>> phone: 303-497-6209
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> Gerard Ketefian
> > >>>>>>>>>>>>>> Research Scientist
> > >>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> > >>>>>>>>>>>>>> 325 Broadway
> > >>>>>>>>>>>>>> <
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > >
> > >>>>>>>>>>>>>> Boulder, CO 80305
> > >>>>>>>>>>>>>> <
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > >
> > >>>>>>>>>>>>>> phone: 303-497-6209
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> --
> > >>>>>>>>>>>> Gerard Ketefian
> > >>>>>>>>>>>> Research Scientist
> > >>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> > >>>>>>>>>>>> 325 Broadway
> > >>>>>>>>>>>> <
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > >
> > >>>>>>>>>>>> Boulder, CO 80305
> > >>>>>>>>>>>> <
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > >
> > >>>>>>>>>>>> phone: 303-497-6209
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> Gerard Ketefian
> > >>>>>>>>>> Research Scientist
> > >>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> > >>>>>>>>>> 325 Broadway
> > >>>>>>>>>> <
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > >
> > >>>>>>>>>> Boulder, CO 80305
> > >>>>>>>>>> <
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > >
> > >>>>>>>>>> phone: 303-497-6209
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>> Logan C. Dawson, Ph.D.
> > >>>>>>> Physical Scientist, VPPPG/MEG
> > >>>>>>> NOAA/NWS/NCEP/EMC
> > >>>>>>> (301) 683-3944
> > >>>>>>>
> > >>>>>>
> > >
> > > --
> > > Jili Dong
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct., # 2002
> > >
> > > College Park, MD 20740
> > >
> > > Jili.Dong at noaa.gov
> > >
> > > 301-683-3745
> > >
> >
> >
> > --
> >
> > Mallory Row
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> >
>
>
------------------------------------------------
Subject: Fwd: real time verification?
From: John Halley Gotway
Time: Thu May 07 10:10:47 2020
Mallory,
FYI, I added a note to this GitHub issue to reference this RT ticket:
https://github.com/NCAR/MET/issues/1020
When we work on the "set_hdr" config file functionality, I think we
should
address wind rotation support at the same time.
John
On Thu, May 7, 2020 at 10:06 AM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Mallory,
>
> When MET reads wind variables from GRIB 1 and 2 files (e.g. UGRD,
VGRD,
> and WDIR), it checks the metadata to see if the winds are defined as
grid-
> or earth-relative. If they are grid-relative, MET automatically
reads the
> corresponding vector component and uses them to rotate the winds
from being
> grid-relative to earth-relative. This is necessary prior to
verification
> since point observations of winds should always be earth-relative.
>
> When running at verbosity level 3 or higher, you should see log
messages
> such as:
> *"Rotating wind direction from grid-relative to earth-relative."*
> or
> *"Rotating U and V wind components from grid-relative to earth-
relative."*
>
> In GRIB1, this flag is contained in the Grid Description Section, in
the
> 5th bit of 17-th octet:
> https://www.nco.ncep.noaa.gov/pmb/docs/on388/table7.html
>
> And here's the corresponding table for GRIB2:
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table3-
3.shtml
>
> MET checks these flags and handles them correctly, but this is ONLY
when
> reading GRIB1/GRIB2 data. No such logic is applied for gridded
NetCDF
> data. It does seem like to should consider adding support to rotate
winds
> as needed in future versions of MET.
>
> Does that answer the question?
>
> Thanks,
> John
>
> On Thu, May 7, 2020 at 9:26 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95186 >
>>
>> Hi Mallory,
>>
>> Thanks for summarizing the MET question about wind rotation. I'll
assign
>> this issue to John Halley Gotway.
>>
>> Regards,
>> Minna
>> ---------------
>> Minna Win
>> National Center for Atmospheric Research
>> Developmental Testbed Center
>> Phone: 303-497-8423
>> Fax: 303-497-8401
>>
>>
>>
>> On Thu, May 7, 2020 at 8:58 AM Mallory Row - NOAA Affiliate via RT
<
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Thu May 07 08:58:02 2020: Request 95186 was acted upon.
>> > Transaction: Ticket created by mallory.row at noaa.gov
>> > Queue: met_help
>> > Subject: Fwd: real time verification?
>> > Owner: Nobody
>> > Requestors: mallory.row at noaa.gov
>> > Status: new
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95186 >
>> >
>> >
>> > Hi,
>> >
>> > Below is a forward of a forward. Essentially the question is in
regards
>> to
>> > how MET handles the wind rotation. There are no assumptions made
if the
>> > winds are earth-relative or grid-relative? Is MET using the
metadata to
>> > determine that and handle that information as needed?
>> >
>> > ---------- Forwarded message ---------
>> > From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
>> > Date: Thu, May 7, 2020 at 10:23 AM
>> > Subject: Fwd: real time verification?
>> > To: Perry Shafran <perry.shafran at noaa.gov>, Logan Dawson - NOAA
>> Affiliate
>> > <
>> > logan.dawson at noaa.gov>, Mallory Row - NOAA Affiliate <
>> mallory.row at noaa.gov
>> > >
>> >
>> >
>> > Hi Perry, Logan, and Mallory:
>> >
>> > FYI - GSL figured out the source of their poor wind verification
stats
>> with
>> > their SAR runs. It turns out their verification code was assuming
the
>> winds
>> > from the model were grid relative and was treating them
improperly (that
>> > used to be the convention in our regional models). However with
the SAR
>> > all winds are output as earth-relative.
>> >
>> > I think out MET-based stats are okay, but I wanted to pass this
along to
>> > check and ensure that MET handles all wind vector rotation
>> appropriately,
>> > i.e. without assumption and uses the metadata in the grib2
records to
>> act
>> > accordingly.
>> >
>> > Thanks!
>> > Jacob
>> >
>> > ---------- Forwarded message ---------
>> > From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
>> > Date: Thu, May 7, 2020 at 10:19 AM
>> > Subject: Re: real time verification?
>> > To: Jeff Beck - NOAA Affiliate <jeff.beck at noaa.gov>
>> > Cc: Benjamin Blake - NOAA Affiliate <benjamin.blake at noaa.gov>,
Logan
>> > Dawson
>> > - NOAA Federal <logan.dawson at noaa.gov>, Curtis Alexander - NOAA
>> Federal <
>> > curtis.alexander at noaa.gov>, Gerard Ketefian - NOAA Affiliate <
>> > gerard.ketefian at noaa.gov>, Stephen Weygandt - NOAA Federal <
>> > stephen.weygandt at noaa.gov>, Trevor Alcott - NOAA Federal <
>> > trevor.alcott at noaa.gov>, Jim Purser - NOAA Affiliate <
>> jim.purser at noaa.gov
>> > >,
>> > Jeffrey Hamilton - NOAA Affiliate <jeffrey.a.hamilton at noaa.gov>,
Ming
>> Hu -
>> > NOAA Affiliate <ming.hu at noaa.gov>, Jili Dong - NOAA Affiliate <
>> > jili.dong at noaa.gov>
>> >
>> >
>> > Hi Jeff,
>> >
>> > This is excellent news! This change in how the winds are treated
has
>> bitten
>> > us several times (in the post, now verification, and we recently
>> > discovered an issue in the GSI - as Ming already knows).
Hopefully
>> this is
>> > the last of it.
>> >
>> > Also great that the ESG grid has been exonerated :-)
>> >
>> > I think the MET-based verification is okay since your stats are
now
>> > comparable to ours, which does use MET. I'll pass this along to a
few
>> folks
>> > in EMC to be on the safe side.
>> >
>> > Thanks,
>> > Jacob
>> >
>> > On Wed, May 6, 2020 at 6:18 PM Jili Dong - NOAA Affiliate <
>> > jili.dong at noaa.gov> wrote:
>> >
>> > > Thanks for the update Jeff. It is really great news! This gives
us
>> more
>> > > confidence in running ESG grid for hurricanes.
>> > >
>> > > Jili
>> > >
>> > > On Wed, May 6, 2020 at 4:45 PM Jeff Beck - NOAA Affiliate <
>> > > jeff.beck at noaa.gov> wrote:
>> > >
>> > >> Hi Everyone,
>> > >>
>> > >> After a considerable amount of sleuthing by a number of people
at
>> GSL,
>> > >> we've identified that the wind rotation problem was occurring
in our
>> > >> verification system, and not in the model data. Since the
regional
>> > models
>> > >> we've dealt with in the past have output grib2 files with
>> grid-relative
>> > >> winds, MATS was rotating the RAOB data to the LCC grid prior
to
>> > >> verification, resulting in large RMSE/bias. Jeff Hamilton
will be
>> > >> modifying the code to read the grib2 metadata flag that
defines
>> either
>> > grid
>> > >> or earth-relative winds (fixed for SAR output by Jili Dong in
>> January)
>> > to
>> > >> tell the verification code whether rotation of the RAOB data
is
>> > required or
>> > >> not. Please see the attached plot that Jeff put together,
showing
>> the
>> > >> difference when the RAOB data aren't rotated (windtest).
Jeff,
>> please
>> > feel
>> > >> free to correct or add to anything I've said.
>> > >>
>> > >> Ultimately (and fortunately!), this likely means that the FV3
wind
>> > >> initialization code does not need to be modified for the
JP/ESG grid,
>> > and
>> > >> it's possible for use to switch back over to this much more
uniform
>> > >> option. We have a number of parallel FV3-SAR runs going at
GSL on
>> Hera
>> > and
>> > >> Jet which will enable us to compare the GFDL and JP/ESG grids
>> directly
>> > to
>> > >> make sure. Jim, we may even be able to leverage some of these
direct
>> > >> comparisons for the manuscript you're working on for the ESG
grid.
>> > >>
>> > >> Finally, I wanted to ask Jacob and Ben whether the EMC
verification
>> > knows
>> > >> that the SAR grib2 output is earth-relative, and whether it is
>> correctly
>> > >> verifying earth-relative RAOB to earth-relative grib2 output.
We
>> > weren't
>> > >> able to plot a subset of dates for the wind profile RMSE/bias
plots
>> on
>> > the
>> > >> EMC verification website, so I wasn't sure if there was
contamination
>> > from
>> > >> solely the earlier grib2 metadata "N/S" vs "grid" bug, or if
>> something
>> > was
>> > >> still present that was throwing off your verification for
recent
>> runs.
>> > >>
>> > >> - Jeff
>> > >>
>> > >> On Thu, Apr 30, 2020 at 5:57 AM Jacob Carley - NOAA Federal <
>> > >> jacob.carley at noaa.gov> wrote:
>> > >>
>> > >>> Hi Ben,
>> > >>>
>> > >>> Yup - and Jeff and I confirmed that his files do as well. So
I
>> don't
>> > >>> think there's a UPP issue.
>> > >>>
>> > >>> -Jacob
>> > >>>
>> > >>> On Thu, Apr 30, 2020 at 7:54 AM Benjamin Blake - NOAA
Affiliate <
>> > >>> benjamin.blake at noaa.gov> wrote:
>> > >>>
>> > >>>> Hi Gerard,
>> > >>>>
>> > >>>> On April 1st we switched from using global_chgres to
chgres_cube
>> when
>> > >>>> we moved our real-time runs to the WCOSS Cray.
>> > >>>>
>> > >>>> Jacob, here is what wgrib2 -V shows for the 10-m U wind
component
>> in
>> > >>>> our UPP output. Does winds (N/S) indicate the winds are
Earth
>> > relative?
>> > >>>>
>> > >>>> 18:17311820:vt=2020050206:1 hybrid level:54 hour fcst:UGRD
>> U-Component
>> > >>>> of Wind [m/s]:
>> > >>>> ndata=1905141:undef=0:mean=1.44973:min=-14.13:max=23.3
>> > >>>> grid_template=30:winds(N/S):
>> > >>>> Lambert Conformal: (1799 x 1059) input WE:SN output
WE:SN
>> res
>> > 0
>> > >>>> Lat1 21.138123 Lon1 237.280472 LoV 262.500000
>> > >>>> LatD 38.500000 Latin1 38.500000 Latin2 38.500000
>> > >>>> LatSP 0.000000 LonSP 0.000000
>> > >>>> North Pole (1799 x 1059) Dx 3000.000000 m Dy
3000.000000 m
>> > mode
>> > >>>> 0
>> > >>>>
>> > >>>> Thanks,
>> > >>>> Ben
>> > >>>>
>> > >>>> On Wed, Apr 29, 2020 at 6:37 PM Jacob Carley - NOAA Federal
<
>> > >>>> jacob.carley at noaa.gov> wrote:
>> > >>>>
>> > >>>>> Do a simple 'wgrib2 -V' on your files, it should tell you
if the
>> > winds
>> > >>>>> are encoded as grid or earth relative.
>> > >>>>>
>> > >>>>> On Wed, Apr 29, 2020 at 6:30 PM Jeff Beck - NOAA Affiliate
<
>> > >>>>> jeff.beck at noaa.gov> wrote:
>> > >>>>>
>> > >>>>>> My understanding was that the fix was implemented into UPP
>> itself,
>> > is
>> > >>>>>> that not correct? We're using a Lambert conformal write
>> component
>> > grid and
>> > >>>>>> the top-of-the-trunk EMC_post develop branch code.
>> > >>>>>>
>> > >>>>>> - Jeff
>> > >>>>>>
>> > >>>>>> On Wed, Apr 29, 2020 at 4:28 PM Logan Dawson - NOAA
Federal <
>> > >>>>>> logan.dawson at noaa.gov> wrote:
>> > >>>>>>
>> > >>>>>>> Hi Jeff,
>> > >>>>>>>
>> > >>>>>>> I still need to update the “Current Experiment Stats” to
account
>> > for
>> > >>>>>>> the latest SAR/-X configuration changes.
>> > >>>>>>>
>> > >>>>>>> Those plots going back to last November are definitely
affected
>> by
>> > >>>>>>> the grid rotation issue, which wasn’t addressed in UPP
until
>> > sometime in
>> > >>>>>>> January.
>> > >>>>>>>
>> > >>>>>>> I’m juggling several things with the GEFS evaluation
wrapping up
>> > and
>> > >>>>>>> HREF about to start, but I’ll try to get those updated
very soon
>> > since I
>> > >>>>>>> know there’s increased interest with HWT kicking off this
week.
>> > >>>>>>>
>> > >>>>>>> -Logan
>> > >>>>>>>
>> > >>>>>>> On Wed, Apr 29, 2020 at 6:23 PM Jeff Beck - NOAA
Affiliate <
>> > >>>>>>> jeff.beck at noaa.gov> wrote:
>> > >>>>>>>
>> > >>>>>>>> Helps if I attach the plots.
>> > >>>>>>>>
>> > >>>>>>>> - Jeff
>> > >>>>>>>>
>> > >>>>>>>> On Wed, Apr 29, 2020 at 4:22 PM Jeff Beck - NOAA
Affiliate <
>> > >>>>>>>> jeff.beck at noaa.gov> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Jacob et al.,
>> > >>>>>>>>>
>> > >>>>>>>>> Just to clarify, here are two plots from the SARX
verification
>> > >>>>>>>>> webpage showing the large bias for wind vector
verification
>> > (that we're
>> > >>>>>>>>> also seeing) when compared to the HRRR.
>> > >>>>>>>>>
>> > >>>>>>>>> - Jeff
>> > >>>>>>>>>
>> > >>>>>>>>> On Wed, Apr 29, 2020 at 4:03 PM Gerard Ketefian - NOAA
>> Affiliate
>> > <
>> > >>>>>>>>> gerard.ketefian at noaa.gov> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> Hi Jacob,
>> > >>>>>>>>>>
>> > >>>>>>>>>> My mistake: We're also using UPP output for
verification,
>> not
>> > >>>>>>>>>> the history files (I think by history files you mean
the dynf
>> > and phyf,
>> > >>>>>>>>>> right?).
>> > >>>>>>>>>>
>> > >>>>>>>>>> Jeff says we have a very recent UPP version, so that
bug is
>> > >>>>>>>>>> probably not it.
>> > >>>>>>>>>>
>> > >>>>>>>>>> He also noticed from the page you sent that your
results show
>> > the
>> > >>>>>>>>>> same problem that we're seeing -- under upper air,
vector
>> wind,
>> > the RMSVE
>> > >>>>>>>>>> from the FV3-SAR and (-X) are way to the right (or
above, if
>> > you look at it
>> > >>>>>>>>>> by forecast lead) of the others.
>> > >>>>>>>>>>
>> > >>>>>>>>>> Gerard
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Wed, Apr 29, 2020 at 3:53 PM Jacob Carley - NOAA
Federal <
>> > >>>>>>>>>> jacob.carley at noaa.gov> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>>> Hi Gerard,
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> 1) Which output file (and/or code) was the metadata
bug in?
>> > >>>>>>>>>>>> We're currently doing verification based on the
>> > write-component files
>> > >>>>>>>>>>>> (dynf*.nc and phyf*.nc).
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> The bug was in the UPP.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> However, the winds in the history files are re-
gridded to
>> cell
>> > >>>>>>>>>>> centers and then rotated to be the zonal and
meridional
>> winds
>> > (earth
>> > >>>>>>>>>>> relative). So they don't require any extra rotation.
Could
>> > they be getting
>> > >>>>>>>>>>> an extra rotation in the verification code? If you
plot a
>> 500
>> > hPa height
>> > >>>>>>>>>>> and wind plot and look near the domain corners, do
you see
>> the
>> > barbs align
>> > >>>>>>>>>>> with the heights as you would expect for the flow
regime?
>> > That helped us
>> > >>>>>>>>>>> figure out the UPP issue.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Make sure that you are using the 'grid_lont' and
'grid_latt'
>> > >>>>>>>>>>> arrays for lats and lons, as those are defined for
the cell
>> > centroids (the
>> > >>>>>>>>>>> 'T-cells').
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> 2) Which chgres was used for the runs shown on the
page you
>> > >>>>>>>>>>>> sent? Was it the old chgres or the new chgres_cube?
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> That's a Ben question!
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Hope that helps,
>> > >>>>>>>>>>> Jacob
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Wed, Apr 29, 2020 at 5:44 PM Gerard Ketefian -
NOAA
>> > Affiliate
>> > >>>>>>>>>>> <gerard.ketefian at noaa.gov> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>> Thank you Jacob. Couple of questions:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> 1) Which output file (and/or code) was the metadata
bug in?
>> > >>>>>>>>>>>> We're currently doing verification based on the
>> > write-component files
>> > >>>>>>>>>>>> (dynf*.nc and phyf*.nc).
>> > >>>>>>>>>>>> 2) Which chgres was used for the runs shown on the
page you
>> > >>>>>>>>>>>> sent? Was it the old chgres or the new chgres_cube?
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Thanks,
>> > >>>>>>>>>>>> Gerard
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:34 PM Jacob Carley - NOAA
>> Federal <
>> > >>>>>>>>>>>> jacob.carley at noaa.gov> wrote:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> Hi Gerard,
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> I'll step in real quick and point you to this page,
which
>> was
>> > >>>>>>>>>>>>> developed by Logan Dawson (cc'd):
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > https://www.emc.ncep.noaa.gov/users/Logan.Dawson/fv3cam_verif/
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> We had a wind issue in the SAR with the HREF, but
that was
>> > due
>> > >>>>>>>>>>>>> to a double wind rotation being applied. The winds
coming
>> > out of the model
>> > >>>>>>>>>>>>> are already earth relative and they were getting
marked as
>> > grid relative in
>> > >>>>>>>>>>>>> the post (and thus all downstream codes were
rotating the
>> > winds again).
>> > >>>>>>>>>>>>> Once we changed the metadata to the appropriate
flag, the
>> > issue was fixed.
>> > >>>>>>>>>>>>> Though I think you have this bug fix? It might be
worth
>> > checking to see if
>> > >>>>>>>>>>>>> that's happening.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> -Jacob
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Wed, Apr 29, 2020 at 5:24 PM Gerard Ketefian -
NOAA
>> > >>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Looping in Trevor.
>> > >>>>>>>>>>>>>> Gerard
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:22 PM Gerard Ketefian -
NOAA
>> > >>>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Hi Ben,
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Do you or have you run real-time runs with
verification
>> > with
>> > >>>>>>>>>>>>>>> the FV3-SAR (on a GFDLgrid)? You may have told
us
>> > previously, but I forgot.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> GSL is having issues with verification with the
FV3-SAR.
>> > >>>>>>>>>>>>>>> THe problem is that the winds are pretty bad,
even at
>> hour
>> > 0. We first
>> > >>>>>>>>>>>>>>> thought that it might be a JPgrid issue so
switched to a
>> > GFDLgrid, but
>> > >>>>>>>>>>>>>>> that's showing a similar problem. Just wanted to
see if
>> > you have any plots
>> > >>>>>>>>>>>>>>> we can look at.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Now I'm thinking it might be a chgres_cube
problem. I
>> know
>> > >>>>>>>>>>>>>>> you were using the old chgres in the past. Are
you
>> still
>> > using that?
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Thanks,
>> > >>>>>>>>>>>>>>> Gerard
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>> Gerard Ketefian
>> > >>>>>>>>>>>>>>> Research Scientist
>> > >>>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>> > >>>>>>>>>>>>>>> 325 Broadway
>> > >>>>>>>>>>>>>>> <
>> >
>>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
>> > >
>> > >>>>>>>>>>>>>>> Boulder, CO 80305
>> > >>>>>>>>>>>>>>> <
>> >
>>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
>> > >
>> > >>>>>>>>>>>>>>> phone: 303-497-6209
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>> Gerard Ketefian
>> > >>>>>>>>>>>>>> Research Scientist
>> > >>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>> > >>>>>>>>>>>>>> 325 Broadway
>> > >>>>>>>>>>>>>> <
>> >
>>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
>> > >
>> > >>>>>>>>>>>>>> Boulder, CO 80305
>> > >>>>>>>>>>>>>> <
>> >
>>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
>> > >
>> > >>>>>>>>>>>>>> phone: 303-497-6209
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> --
>> > >>>>>>>>>>>> Gerard Ketefian
>> > >>>>>>>>>>>> Research Scientist
>> > >>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>> > >>>>>>>>>>>> 325 Broadway
>> > >>>>>>>>>>>> <
>> >
>>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
>> > >
>> > >>>>>>>>>>>> Boulder, CO 80305
>> > >>>>>>>>>>>> <
>> >
>>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
>> > >
>> > >>>>>>>>>>>> phone: 303-497-6209
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> --
>> > >>>>>>>>>> Gerard Ketefian
>> > >>>>>>>>>> Research Scientist
>> > >>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
>> > >>>>>>>>>> 325 Broadway
>> > >>>>>>>>>> <
>> >
>>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
>> > >
>> > >>>>>>>>>> Boulder, CO 80305
>> > >>>>>>>>>> <
>> >
>>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
>> > >
>> > >>>>>>>>>> phone: 303-497-6209
>> > >>>>>>>>>>
>> > >>>>>>>>>> --
>> > >>>>>>> Logan C. Dawson, Ph.D.
>> > >>>>>>> Physical Scientist, VPPPG/MEG
>> > >>>>>>> NOAA/NWS/NCEP/EMC
>> > >>>>>>> (301) 683-3944
>> > >>>>>>>
>> > >>>>>>
>> > >
>> > > --
>> > > Jili Dong
>> > >
>> > > IMSG at NOAA/NWS/NCEP/EMC
>> > >
>> > > 5830 University Research Ct., # 2002
>> > >
>> > > College Park, MD 20740
>> > >
>> > > Jili.Dong at noaa.gov
>> > >
>> > > 301-683-3745
>> > >
>> >
>> >
>> > --
>> >
>> > Mallory Row
>> >
>> > IMSG at NOAA/NWS/NCEP/EMC
>> >
>> >
>>
>>
------------------------------------------------
Subject: Fwd: real time verification?
From: Mallory Row - NOAA Affiliate
Time: Thu May 07 10:11:35 2020
It does! Thank you!
On Thu, May 7, 2020 at 12:06 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Mallory,
>
> When MET reads wind variables from GRIB 1 and 2 files (e.g. UGRD,
VGRD, and
> WDIR), it checks the metadata to see if the winds are defined as
grid- or
> earth-relative. If they are grid-relative, MET automatically reads
the
> corresponding vector component and uses them to rotate the winds
from being
> grid-relative to earth-relative. This is necessary prior to
verification
> since point observations of winds should always be earth-relative.
>
> When running at verbosity level 3 or higher, you should see log
messages
> such as:
> *"Rotating wind direction from grid-relative to earth-relative."*
> or
> *"Rotating U and V wind components from grid-relative to earth-
relative."*
>
> In GRIB1, this flag is contained in the Grid Description Section, in
the
> 5th bit of 17-th octet:
> https://www.nco.ncep.noaa.gov/pmb/docs/on388/table7.html
>
> And here's the corresponding table for GRIB2:
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table3-
3.shtml
>
> MET checks these flags and handles them correctly, but this is ONLY
when
> reading GRIB1/GRIB2 data. No such logic is applied for gridded
NetCDF
> data. It does seem like to should consider adding support to rotate
winds
> as needed in future versions of MET.
>
> Does that answer the question?
>
> Thanks,
> John
>
> On Thu, May 7, 2020 at 9:26 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95186 >
> >
> > Hi Mallory,
> >
> > Thanks for summarizing the MET question about wind rotation. I'll
assign
> > this issue to John Halley Gotway.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> >
> >
> >
> > On Thu, May 7, 2020 at 8:58 AM Mallory Row - NOAA Affiliate via RT
<
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Thu May 07 08:58:02 2020: Request 95186 was acted upon.
> > > Transaction: Ticket created by mallory.row at noaa.gov
> > > Queue: met_help
> > > Subject: Fwd: real time verification?
> > > Owner: Nobody
> > > Requestors: mallory.row at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95186
> >
> > >
> > >
> > > Hi,
> > >
> > > Below is a forward of a forward. Essentially the question is in
regards
> > to
> > > how MET handles the wind rotation. There are no assumptions made
if the
> > > winds are earth-relative or grid-relative? Is MET using the
metadata to
> > > determine that and handle that information as needed?
> > >
> > > ---------- Forwarded message ---------
> > > From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
> > > Date: Thu, May 7, 2020 at 10:23 AM
> > > Subject: Fwd: real time verification?
> > > To: Perry Shafran <perry.shafran at noaa.gov>, Logan Dawson - NOAA
> > Affiliate
> > > <
> > > logan.dawson at noaa.gov>, Mallory Row - NOAA Affiliate <
> > mallory.row at noaa.gov
> > > >
> > >
> > >
> > > Hi Perry, Logan, and Mallory:
> > >
> > > FYI - GSL figured out the source of their poor wind verification
stats
> > with
> > > their SAR runs. It turns out their verification code was
assuming the
> > winds
> > > from the model were grid relative and was treating them
improperly
> (that
> > > used to be the convention in our regional models). However with
the
> SAR
> > > all winds are output as earth-relative.
> > >
> > > I think out MET-based stats are okay, but I wanted to pass this
along
> to
> > > check and ensure that MET handles all wind vector rotation
> appropriately,
> > > i.e. without assumption and uses the metadata in the grib2
records to
> act
> > > accordingly.
> > >
> > > Thanks!
> > > Jacob
> > >
> > > ---------- Forwarded message ---------
> > > From: Jacob Carley - NOAA Federal <jacob.carley at noaa.gov>
> > > Date: Thu, May 7, 2020 at 10:19 AM
> > > Subject: Re: real time verification?
> > > To: Jeff Beck - NOAA Affiliate <jeff.beck at noaa.gov>
> > > Cc: Benjamin Blake - NOAA Affiliate <benjamin.blake at noaa.gov>,
Logan
> > > Dawson
> > > - NOAA Federal <logan.dawson at noaa.gov>, Curtis Alexander - NOAA
> Federal
> > <
> > > curtis.alexander at noaa.gov>, Gerard Ketefian - NOAA Affiliate <
> > > gerard.ketefian at noaa.gov>, Stephen Weygandt - NOAA Federal <
> > > stephen.weygandt at noaa.gov>, Trevor Alcott - NOAA Federal <
> > > trevor.alcott at noaa.gov>, Jim Purser - NOAA Affiliate <
> > jim.purser at noaa.gov
> > > >,
> > > Jeffrey Hamilton - NOAA Affiliate <jeffrey.a.hamilton at noaa.gov>,
Ming
> > Hu -
> > > NOAA Affiliate <ming.hu at noaa.gov>, Jili Dong - NOAA Affiliate <
> > > jili.dong at noaa.gov>
> > >
> > >
> > > Hi Jeff,
> > >
> > > This is excellent news! This change in how the winds are treated
has
> > bitten
> > > us several times (in the post, now verification, and we recently
> > > discovered an issue in the GSI - as Ming already knows).
Hopefully
> this
> > is
> > > the last of it.
> > >
> > > Also great that the ESG grid has been exonerated :-)
> > >
> > > I think the MET-based verification is okay since your stats are
now
> > > comparable to ours, which does use MET. I'll pass this along to
a few
> > folks
> > > in EMC to be on the safe side.
> > >
> > > Thanks,
> > > Jacob
> > >
> > > On Wed, May 6, 2020 at 6:18 PM Jili Dong - NOAA Affiliate <
> > > jili.dong at noaa.gov> wrote:
> > >
> > > > Thanks for the update Jeff. It is really great news! This
gives us
> more
> > > > confidence in running ESG grid for hurricanes.
> > > >
> > > > Jili
> > > >
> > > > On Wed, May 6, 2020 at 4:45 PM Jeff Beck - NOAA Affiliate <
> > > > jeff.beck at noaa.gov> wrote:
> > > >
> > > >> Hi Everyone,
> > > >>
> > > >> After a considerable amount of sleuthing by a number of
people at
> GSL,
> > > >> we've identified that the wind rotation problem was occurring
in our
> > > >> verification system, and not in the model data. Since the
regional
> > > models
> > > >> we've dealt with in the past have output grib2 files with
> > grid-relative
> > > >> winds, MATS was rotating the RAOB data to the LCC grid prior
to
> > > >> verification, resulting in large RMSE/bias. Jeff Hamilton
will be
> > > >> modifying the code to read the grib2 metadata flag that
defines
> either
> > > grid
> > > >> or earth-relative winds (fixed for SAR output by Jili Dong in
> January)
> > > to
> > > >> tell the verification code whether rotation of the RAOB data
is
> > > required or
> > > >> not. Please see the attached plot that Jeff put together,
showing
> the
> > > >> difference when the RAOB data aren't rotated (windtest).
Jeff,
> please
> > > feel
> > > >> free to correct or add to anything I've said.
> > > >>
> > > >> Ultimately (and fortunately!), this likely means that the FV3
wind
> > > >> initialization code does not need to be modified for the
JP/ESG
> grid,
> > > and
> > > >> it's possible for use to switch back over to this much more
uniform
> > > >> option. We have a number of parallel FV3-SAR runs going at
GSL on
> > Hera
> > > and
> > > >> Jet which will enable us to compare the GFDL and JP/ESG grids
> directly
> > > to
> > > >> make sure. Jim, we may even be able to leverage some of
these
> direct
> > > >> comparisons for the manuscript you're working on for the ESG
grid.
> > > >>
> > > >> Finally, I wanted to ask Jacob and Ben whether the EMC
verification
> > > knows
> > > >> that the SAR grib2 output is earth-relative, and whether it
is
> > correctly
> > > >> verifying earth-relative RAOB to earth-relative grib2 output.
We
> > > weren't
> > > >> able to plot a subset of dates for the wind profile RMSE/bias
plots
> on
> > > the
> > > >> EMC verification website, so I wasn't sure if there was
> contamination
> > > from
> > > >> solely the earlier grib2 metadata "N/S" vs "grid" bug, or if
> something
> > > was
> > > >> still present that was throwing off your verification for
recent
> runs.
> > > >>
> > > >> - Jeff
> > > >>
> > > >> On Thu, Apr 30, 2020 at 5:57 AM Jacob Carley - NOAA Federal <
> > > >> jacob.carley at noaa.gov> wrote:
> > > >>
> > > >>> Hi Ben,
> > > >>>
> > > >>> Yup - and Jeff and I confirmed that his files do as well.
So I
> don't
> > > >>> think there's a UPP issue.
> > > >>>
> > > >>> -Jacob
> > > >>>
> > > >>> On Thu, Apr 30, 2020 at 7:54 AM Benjamin Blake - NOAA
Affiliate <
> > > >>> benjamin.blake at noaa.gov> wrote:
> > > >>>
> > > >>>> Hi Gerard,
> > > >>>>
> > > >>>> On April 1st we switched from using global_chgres to
chgres_cube
> > when
> > > >>>> we moved our real-time runs to the WCOSS Cray.
> > > >>>>
> > > >>>> Jacob, here is what wgrib2 -V shows for the 10-m U wind
component
> in
> > > >>>> our UPP output. Does winds (N/S) indicate the winds are
Earth
> > > relative?
> > > >>>>
> > > >>>> 18:17311820:vt=2020050206:1 hybrid level:54 hour fcst:UGRD
> > U-Component
> > > >>>> of Wind [m/s]:
> > > >>>> ndata=1905141:undef=0:mean=1.44973:min=-14.13:max=23.3
> > > >>>> grid_template=30:winds(N/S):
> > > >>>> Lambert Conformal: (1799 x 1059) input WE:SN output
WE:SN
> > res
> > > 0
> > > >>>> Lat1 21.138123 Lon1 237.280472 LoV 262.500000
> > > >>>> LatD 38.500000 Latin1 38.500000 Latin2 38.500000
> > > >>>> LatSP 0.000000 LonSP 0.000000
> > > >>>> North Pole (1799 x 1059) Dx 3000.000000 m Dy
3000.000000 m
> > > mode
> > > >>>> 0
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Ben
> > > >>>>
> > > >>>> On Wed, Apr 29, 2020 at 6:37 PM Jacob Carley - NOAA Federal
<
> > > >>>> jacob.carley at noaa.gov> wrote:
> > > >>>>
> > > >>>>> Do a simple 'wgrib2 -V' on your files, it should tell you
if the
> > > winds
> > > >>>>> are encoded as grid or earth relative.
> > > >>>>>
> > > >>>>> On Wed, Apr 29, 2020 at 6:30 PM Jeff Beck - NOAA Affiliate
<
> > > >>>>> jeff.beck at noaa.gov> wrote:
> > > >>>>>
> > > >>>>>> My understanding was that the fix was implemented into
UPP
> itself,
> > > is
> > > >>>>>> that not correct? We're using a Lambert conformal write
> component
> > > grid and
> > > >>>>>> the top-of-the-trunk EMC_post develop branch code.
> > > >>>>>>
> > > >>>>>> - Jeff
> > > >>>>>>
> > > >>>>>> On Wed, Apr 29, 2020 at 4:28 PM Logan Dawson - NOAA
Federal <
> > > >>>>>> logan.dawson at noaa.gov> wrote:
> > > >>>>>>
> > > >>>>>>> Hi Jeff,
> > > >>>>>>>
> > > >>>>>>> I still need to update the “Current Experiment Stats” to
> account
> > > for
> > > >>>>>>> the latest SAR/-X configuration changes.
> > > >>>>>>>
> > > >>>>>>> Those plots going back to last November are definitely
affected
> > by
> > > >>>>>>> the grid rotation issue, which wasn’t addressed in UPP
until
> > > sometime in
> > > >>>>>>> January.
> > > >>>>>>>
> > > >>>>>>> I’m juggling several things with the GEFS evaluation
wrapping
> up
> > > and
> > > >>>>>>> HREF about to start, but I’ll try to get those updated
very
> soon
> > > since I
> > > >>>>>>> know there’s increased interest with HWT kicking off
this week.
> > > >>>>>>>
> > > >>>>>>> -Logan
> > > >>>>>>>
> > > >>>>>>> On Wed, Apr 29, 2020 at 6:23 PM Jeff Beck - NOAA
Affiliate <
> > > >>>>>>> jeff.beck at noaa.gov> wrote:
> > > >>>>>>>
> > > >>>>>>>> Helps if I attach the plots.
> > > >>>>>>>>
> > > >>>>>>>> - Jeff
> > > >>>>>>>>
> > > >>>>>>>> On Wed, Apr 29, 2020 at 4:22 PM Jeff Beck - NOAA
Affiliate <
> > > >>>>>>>> jeff.beck at noaa.gov> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Jacob et al.,
> > > >>>>>>>>>
> > > >>>>>>>>> Just to clarify, here are two plots from the SARX
> verification
> > > >>>>>>>>> webpage showing the large bias for wind vector
verification
> > > (that we're
> > > >>>>>>>>> also seeing) when compared to the HRRR.
> > > >>>>>>>>>
> > > >>>>>>>>> - Jeff
> > > >>>>>>>>>
> > > >>>>>>>>> On Wed, Apr 29, 2020 at 4:03 PM Gerard Ketefian - NOAA
> > Affiliate
> > > <
> > > >>>>>>>>> gerard.ketefian at noaa.gov> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hi Jacob,
> > > >>>>>>>>>>
> > > >>>>>>>>>> My mistake: We're also using UPP output for
verification,
> not
> > > >>>>>>>>>> the history files (I think by history files you mean
the
> dynf
> > > and phyf,
> > > >>>>>>>>>> right?).
> > > >>>>>>>>>>
> > > >>>>>>>>>> Jeff says we have a very recent UPP version, so that
bug is
> > > >>>>>>>>>> probably not it.
> > > >>>>>>>>>>
> > > >>>>>>>>>> He also noticed from the page you sent that your
results
> show
> > > the
> > > >>>>>>>>>> same problem that we're seeing -- under upper air,
vector
> > wind,
> > > the RMSVE
> > > >>>>>>>>>> from the FV3-SAR and (-X) are way to the right (or
above, if
> > > you look at it
> > > >>>>>>>>>> by forecast lead) of the others.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Gerard
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Wed, Apr 29, 2020 at 3:53 PM Jacob Carley - NOAA
Federal
> <
> > > >>>>>>>>>> jacob.carley at noaa.gov> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Hi Gerard,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> 1) Which output file (and/or code) was the metadata
bug in?
> > > >>>>>>>>>>>> We're currently doing verification based on the
> > > write-component files
> > > >>>>>>>>>>>> (dynf*.nc and phyf*.nc).
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> The bug was in the UPP.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> However, the winds in the history files are re-
gridded to
> > cell
> > > >>>>>>>>>>> centers and then rotated to be the zonal and
meridional
> winds
> > > (earth
> > > >>>>>>>>>>> relative). So they don't require any extra rotation.
Could
> > > they be getting
> > > >>>>>>>>>>> an extra rotation in the verification code? If you
plot a
> > 500
> > > hPa height
> > > >>>>>>>>>>> and wind plot and look near the domain corners, do
you see
> > the
> > > barbs align
> > > >>>>>>>>>>> with the heights as you would expect for the flow
regime?
> > > That helped us
> > > >>>>>>>>>>> figure out the UPP issue.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Make sure that you are using the 'grid_lont' and
> 'grid_latt'
> > > >>>>>>>>>>> arrays for lats and lons, as those are defined for
the cell
> > > centroids (the
> > > >>>>>>>>>>> 'T-cells').
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> 2) Which chgres was used for the runs shown on the
page you
> > > >>>>>>>>>>>> sent? Was it the old chgres or the new
chgres_cube?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> That's a Ben question!
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Hope that helps,
> > > >>>>>>>>>>> Jacob
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Wed, Apr 29, 2020 at 5:44 PM Gerard Ketefian -
NOAA
> > > Affiliate
> > > >>>>>>>>>>> <gerard.ketefian at noaa.gov> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> Thank you Jacob. Couple of questions:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> 1) Which output file (and/or code) was the metadata
bug
> in?
> > > >>>>>>>>>>>> We're currently doing verification based on the
> > > write-component files
> > > >>>>>>>>>>>> (dynf*.nc and phyf*.nc).
> > > >>>>>>>>>>>> 2) Which chgres was used for the runs shown on the
page
> you
> > > >>>>>>>>>>>> sent? Was it the old chgres or the new
chgres_cube?
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>> Gerard
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:34 PM Jacob Carley - NOAA
> Federal
> > <
> > > >>>>>>>>>>>> jacob.carley at noaa.gov> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> Hi Gerard,
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I'll step in real quick and point you to this
page, which
> > was
> > > >>>>>>>>>>>>> developed by Logan Dawson (cc'd):
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > https://www.emc.ncep.noaa.gov/users/Logan.Dawson/fv3cam_verif/
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> We had a wind issue in the SAR with the HREF, but
that
> was
> > > due
> > > >>>>>>>>>>>>> to a double wind rotation being applied. The
winds
> coming
> > > out of the model
> > > >>>>>>>>>>>>> are already earth relative and they were getting
marked
> as
> > > grid relative in
> > > >>>>>>>>>>>>> the post (and thus all downstream codes were
rotating the
> > > winds again).
> > > >>>>>>>>>>>>> Once we changed the metadata to the appropriate
flag, the
> > > issue was fixed.
> > > >>>>>>>>>>>>> Though I think you have this bug fix? It might be
worth
> > > checking to see if
> > > >>>>>>>>>>>>> that's happening.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> -Jacob
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Wed, Apr 29, 2020 at 5:24 PM Gerard Ketefian -
NOAA
> > > >>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Looping in Trevor.
> > > >>>>>>>>>>>>>> Gerard
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:22 PM Gerard Ketefian -
NOAA
> > > >>>>>>>>>>>>>> Affiliate <gerard.ketefian at noaa.gov> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Hi Ben,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Do you or have you run real-time runs with
verification
> > > with
> > > >>>>>>>>>>>>>>> the FV3-SAR (on a GFDLgrid)? You may have told
us
> > > previously, but I forgot.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> GSL is having issues with verification with the
> FV3-SAR.
> > > >>>>>>>>>>>>>>> THe problem is that the winds are pretty bad,
even at
> > hour
> > > 0. We first
> > > >>>>>>>>>>>>>>> thought that it might be a JPgrid issue so
switched to
> a
> > > GFDLgrid, but
> > > >>>>>>>>>>>>>>> that's showing a similar problem. Just wanted
to see
> if
> > > you have any plots
> > > >>>>>>>>>>>>>>> we can look at.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Now I'm thinking it might be a chgres_cube
problem. I
> > know
> > > >>>>>>>>>>>>>>> you were using the old chgres in the past. Are
you
> still
> > > using that?
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>> Gerard
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>> Gerard Ketefian
> > > >>>>>>>>>>>>>>> Research Scientist
> > > >>>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> > > >>>>>>>>>>>>>>> 325 Broadway
> > > >>>>>>>>>>>>>>> <
> > >
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > > >
> > > >>>>>>>>>>>>>>> Boulder, CO 80305
> > > >>>>>>>>>>>>>>> <
> > >
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > > >
> > > >>>>>>>>>>>>>>> phone: 303-497-6209
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>> Gerard Ketefian
> > > >>>>>>>>>>>>>> Research Scientist
> > > >>>>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> > > >>>>>>>>>>>>>> 325 Broadway
> > > >>>>>>>>>>>>>> <
> > >
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > > >
> > > >>>>>>>>>>>>>> Boulder, CO 80305
> > > >>>>>>>>>>>>>> <
> > >
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > > >
> > > >>>>>>>>>>>>>> phone: 303-497-6209
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> --
> > > >>>>>>>>>>>> Gerard Ketefian
> > > >>>>>>>>>>>> Research Scientist
> > > >>>>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> > > >>>>>>>>>>>> 325 Broadway
> > > >>>>>>>>>>>> <
> > >
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > > >
> > > >>>>>>>>>>>> Boulder, CO 80305
> > > >>>>>>>>>>>> <
> > >
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > > >
> > > >>>>>>>>>>>> phone: 303-497-6209
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> --
> > > >>>>>>>>>> Gerard Ketefian
> > > >>>>>>>>>> Research Scientist
> > > >>>>>>>>>> NOAA/OAR/ESRL/GSD/EMB, R/GSD1
> > > >>>>>>>>>> 325 Broadway
> > > >>>>>>>>>> <
> > >
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > > >
> > > >>>>>>>>>> Boulder, CO 80305
> > > >>>>>>>>>> <
> > >
> >
>
https://www.google.com/maps/search/325+Broadway+Boulder,+CO+80305?entry=gmail&source=g
> > > >
> > > >>>>>>>>>> phone: 303-497-6209
> > > >>>>>>>>>>
> > > >>>>>>>>>> --
> > > >>>>>>> Logan C. Dawson, Ph.D.
> > > >>>>>>> Physical Scientist, VPPPG/MEG
> > > >>>>>>> NOAA/NWS/NCEP/EMC
> > > >>>>>>> (301) 683-3944
> > > >>>>>>>
> > > >>>>>>
> > > >
> > > > --
> > > > Jili Dong
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct., # 2002
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Jili.Dong at noaa.gov
> > > >
> > > > 301-683-3745
> > > >
> > >
> > >
> > > --
> > >
> > > Mallory Row
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > >
> >
> >
>
>
--
Mallory Row
IMSG at NOAA/NWS/NCEP/EMC
------------------------------------------------
More information about the Met_help
mailing list