[Met_help] [rt.rap.ucar.edu #97919] History for madis2nc and point_stat
Minna Win via RT
met_help at ucar.edu
Thu Mar 18 10:03:04 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm trying to verify a WRF
simulation against some MADIS metar observations. I have two questions
about the madis2nc function and the point_stat function, if you could help
please?
*madis2nc*
When I run the command “madis2nc madis_20171001.nc out.nc -type metar -v 2” it
executes with the following message:
DEBUG 1: Reading MADIS File: madis_20171001.nc
DEBUG 2: Processing METAR recs = -9999
DEBUG 2: Rejected recs based on masking grid = 0
DEBUG 2: Rejected recs based on masking poly = 0
DEBUG 2: Rejected recs based on masking station ID's = 0
DEBUG 2: Rejected obs based on fill value = 0
DEBUG 2: Rejected obs based on quality control = 0
DEBUG 2: Total observations retained or derived = 0
DEBUG 1: Writing MET File: out.nc
However, the output file out.nc is empty (except for some metadata). I
tried running the same command on a different MADIS file that I took from
Cheyenne (
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
and
got the below message:
DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
DEBUG 2: Processing METAR recs = 9044
DEBUG 2: Rejected recs based on masking grid = 0
DEBUG 2: Rejected recs based on masking poly = 0
DEBUG 2: Rejected recs based on masking station ID's = 0
DEBUG 2: Rejected obs based on fill value = 79932
DEBUG 2: Rejected obs based on quality control = 0
DEBUG 2: Total observations retained or derived = 64060
DEBUG 1: Writing MET File: out.nc
I noticed the difference in the "Processing METAR recs” line: can you
advise what that means? I've attached the output from running ncdump -h for
each input file, if it helps.
*point_stat*
When I get to running point_stat with my WRF output file, do I need to do
any reformatting first? (e.g. does it matter if I use GRIB vs NetCDF, will
I need to regrid from my Lambert projection to a lat-lon grid?).
Thanks very much for any help!
Best Regards,
Callum Thompson
--
Dr Callum Thompson
Earth Research Institute
Email: callum-thompson at ucsb.edu
[image: UC Santa Barbara]
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: madis2nc and point_stat
From: Minna Win
Time: Thu Dec 17 08:59:11 2020
Hello Callum,
It looks like you are having issues with your input MADIS data. The
debug
message of "Processing METAR recs = -9999" indicates that there is
some
missing data (-9999 is used in MET to indicate missing and fill
values).
This debug message reports the difference between the first and second
record number to indicate how many records were processed. The only
difference I see between the two txt files you provided is the
presence of
'var_xyz:missing_value' and 'var_xyz:fill_value' in the madis input.
I'll
need to ask someone who is more familiar with MADIS data to see if
this is
the cause of your issues.
As far as your point-stat data the only requirement is that your point
data
is in NetCDF format.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
Pronouns: she/her
On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT
<met_help at ucar.edu>
wrote:
>
> Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
> Transaction: Ticket created by callum-thompson at ucsb.edu
> Queue: met_help
> Subject: madis2nc and point_stat
> Owner: Nobody
> Requestors: callum-thompson at ucsb.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>
>
> Hello,
>
> I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm trying to verify
a WRF
> simulation against some MADIS metar observations. I have two
questions
> about the madis2nc function and the point_stat function, if you
could help
> please?
>
>
> *madis2nc*
>
> When I run the command “madis2nc madis_20171001.nc out.nc -type
metar -v
> 2” it
> executes with the following message:
>
>
> DEBUG 1: Reading MADIS File: madis_20171001.nc
>
> DEBUG 2: Processing METAR recs = -9999
>
> DEBUG 2: Rejected recs based on masking grid = 0
>
> DEBUG 2: Rejected recs based on masking poly = 0
>
> DEBUG 2: Rejected recs based on masking station ID's = 0
>
> DEBUG 2: Rejected obs based on fill value = 0
>
> DEBUG 2: Rejected obs based on quality control = 0
>
> DEBUG 2: Total observations retained or derived = 0
>
> DEBUG 1: Writing MET File: out.nc
>
>
> However, the output file out.nc is empty (except for some metadata).
I
> tried running the same command on a different MADIS file that I took
from
> Cheyenne (
>
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> and
> got the below message:
>
>
> DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
>
> DEBUG 2: Processing METAR recs = 9044
>
> DEBUG 2: Rejected recs based on masking grid = 0
>
> DEBUG 2: Rejected recs based on masking poly = 0
>
> DEBUG 2: Rejected recs based on masking station ID's = 0
>
> DEBUG 2: Rejected obs based on fill value = 79932
>
> DEBUG 2: Rejected obs based on quality control = 0
>
> DEBUG 2: Total observations retained or derived = 64060
>
> DEBUG 1: Writing MET File: out.nc
>
>
> I noticed the difference in the "Processing METAR recs” line: can
you
> advise what that means? I've attached the output from running ncdump
-h for
> each input file, if it helps.
>
>
> *point_stat*
>
> When I get to running point_stat with my WRF output file, do I need
to do
> any reformatting first? (e.g. does it matter if I use GRIB vs
NetCDF, will
> I need to regrid from my Lambert projection to a lat-lon grid?).
>
>
> Thanks very much for any help!
>
>
> Best Regards,
>
> Callum Thompson
>
>
>
> --
> Dr Callum Thompson
> Earth Research Institute
> Email: callum-thompson at ucsb.edu
> [image: UC Santa Barbara]
>
>
------------------------------------------------
Subject: madis2nc and point_stat
From: John Halley Gotway
Time: Thu Dec 17 09:25:48 2020
Hi Callum,
Wanted to jump in with some more information about preparing your WRF
output for Point-Stat.
The Point-Stat tool takes 3 required inputs:
(1) Gridded model output data in any gridded data file format
supported by
MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output of the WRF-
Input
utility).
(2) Point observations in NetCDF format... these must be the output of
one
of MET's point pre-processing tools (ascii2nc, madis2nc, pb2nc, and so
on).
(3) An ascii configuration file.
It sounds like you're asking about (1), your gridded model output. You
have
2 options:
(1) Run the wrf-interp post-processing utility on your WRF output.
(2) Post-process your output using UPP (the Unified Post Processor)
which
writes GRIB1 or GRIB2 output files.
http://www.dtcenter.org/community-code/unified-post-processor-upp
WRF-Interp does interpolate from native model levels to pressure
levels,
which is good, but it does not de-stagger the winds. And MET won't
process
the winds on the staggered dimensions.
So we recommend using UPP which destaggers the winds, interpolates to
pressure levels, and optionally regrids the data.
That being said, if you don't care about the winds and are only
interested
in surface fields, MET can actually read the WRF outptu files
directly. We
just need to use a config option to tell it to interpret the input
data as
being output from the wrf-interp utility.
How would you like to proceed?
Thanks,
John Halley Gotway
On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>
> Hello Callum,
>
> It looks like you are having issues with your input MADIS data. The
debug
> message of "Processing METAR recs = -9999" indicates that there is
some
> missing data (-9999 is used in MET to indicate missing and fill
values).
> This debug message reports the difference between the first and
second
> record number to indicate how many records were processed. The only
> difference I see between the two txt files you provided is the
presence of
> 'var_xyz:missing_value' and 'var_xyz:fill_value' in the madis input.
I'll
> need to ask someone who is more familiar with MADIS data to see if
this is
> the cause of your issues.
>
> As far as your point-stat data the only requirement is that your
point data
> is in NetCDF format.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
> ---------------
> Pronouns: she/her
>
>
> On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
> > Transaction: Ticket created by callum-thompson at ucsb.edu
> > Queue: met_help
> > Subject: madis2nc and point_stat
> > Owner: Nobody
> > Requestors: callum-thompson at ucsb.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> >
> >
> > Hello,
> >
> > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm trying to
verify a
> WRF
> > simulation against some MADIS metar observations. I have two
questions
> > about the madis2nc function and the point_stat function, if you
could
> help
> > please?
> >
> >
> > *madis2nc*
> >
> > When I run the command “madis2nc madis_20171001.nc out.nc -type
metar -v
> > 2” it
> > executes with the following message:
> >
> >
> > DEBUG 1: Reading MADIS File: madis_20171001.nc
> >
> > DEBUG 2: Processing METAR recs = -9999
> >
> > DEBUG 2: Rejected recs based on masking grid = 0
> >
> > DEBUG 2: Rejected recs based on masking poly = 0
> >
> > DEBUG 2: Rejected recs based on masking station ID's = 0
> >
> > DEBUG 2: Rejected obs based on fill value = 0
> >
> > DEBUG 2: Rejected obs based on quality control = 0
> >
> > DEBUG 2: Total observations retained or derived = 0
> >
> > DEBUG 1: Writing MET File: out.nc
> >
> >
> > However, the output file out.nc is empty (except for some
metadata). I
> > tried running the same command on a different MADIS file that I
took from
> > Cheyenne (
> >
> >
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> > and
> > got the below message:
> >
> >
> > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> >
> > DEBUG 2: Processing METAR recs = 9044
> >
> > DEBUG 2: Rejected recs based on masking grid = 0
> >
> > DEBUG 2: Rejected recs based on masking poly = 0
> >
> > DEBUG 2: Rejected recs based on masking station ID's = 0
> >
> > DEBUG 2: Rejected obs based on fill value = 79932
> >
> > DEBUG 2: Rejected obs based on quality control = 0
> >
> > DEBUG 2: Total observations retained or derived = 64060
> >
> > DEBUG 1: Writing MET File: out.nc
> >
> >
> > I noticed the difference in the "Processing METAR recs” line: can
you
> > advise what that means? I've attached the output from running
ncdump -h
> for
> > each input file, if it helps.
> >
> >
> > *point_stat*
> >
> > When I get to running point_stat with my WRF output file, do I
need to do
> > any reformatting first? (e.g. does it matter if I use GRIB vs
NetCDF,
> will
> > I need to regrid from my Lambert projection to a lat-lon grid?).
> >
> >
> > Thanks very much for any help!
> >
> >
> > Best Regards,
> >
> > Callum Thompson
> >
> >
> >
> > --
> > Dr Callum Thompson
> > Earth Research Institute
> > Email: callum-thompson at ucsb.edu
> > [image: UC Santa Barbara]
> >
> >
>
>
------------------------------------------------
Subject: madis2nc and point_stat
From: Callum Thompson
Time: Thu Dec 17 09:45:51 2020
Hi John,
This is really helpful and quick feedback, thanks!
For the MADIS data, it sounds like there's a possible
'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll wait for
Minna
to get back to me for that,
For the WRF files, it is the winds (particularly surface winds) that
I'm
looking to verify, so it sounds like UPP is the better option.
However,
I've never
used it before. Is it already installed on Cheyenne/Casper, or would I
have
to use it on my local machine?
Thanks,
Callum
On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Callum,
>
> Wanted to jump in with some more information about preparing your
WRF
> output for Point-Stat.
>
> The Point-Stat tool takes 3 required inputs:
> (1) Gridded model output data in any gridded data file format
supported by
> MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output of the
WRF-Input
> utility).
> (2) Point observations in NetCDF format... these must be the output
of one
> of MET's point pre-processing tools (ascii2nc, madis2nc, pb2nc, and
so on).
> (3) An ascii configuration file.
>
> It sounds like you're asking about (1), your gridded model output.
You have
> 2 options:
> (1) Run the wrf-interp post-processing utility on your WRF output.
> (2) Post-process your output using UPP (the Unified Post Processor)
which
> writes GRIB1 or GRIB2 output files.
> http://www.dtcenter.org/community-code/unified-post-processor-upp
>
> WRF-Interp does interpolate from native model levels to pressure
levels,
> which is good, but it does not de-stagger the winds. And MET won't
process
> the winds on the staggered dimensions.
>
> So we recommend using UPP which destaggers the winds, interpolates
to
> pressure levels, and optionally regrids the data.
>
> That being said, if you don't care about the winds and are only
interested
> in surface fields, MET can actually read the WRF outptu files
directly. We
> just need to use a config option to tell it to interpret the input
data as
> being output from the wrf-interp utility.
>
> How would you like to proceed?
>
> Thanks,
> John Halley Gotway
>
> On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT <met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> >
> > Hello Callum,
> >
> > It looks like you are having issues with your input MADIS data.
The
> debug
> > message of "Processing METAR recs = -9999" indicates that there is
some
> > missing data (-9999 is used in MET to indicate missing and fill
values).
> > This debug message reports the difference between the first and
second
> > record number to indicate how many records were processed. The
only
> > difference I see between the two txt files you provided is the
presence
> of
> > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the madis
input.
> I'll
> > need to ask someone who is more familiar with MADIS data to see if
this
> is
> > the cause of your issues.
> >
> > As far as your point-stat data the only requirement is that your
point
> data
> > is in NetCDF format.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> > ---------------
> > Pronouns: she/her
> >
> >
> > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
> > > Transaction: Ticket created by callum-thompson at ucsb.edu
> > > Queue: met_help
> > > Subject: madis2nc and point_stat
> > > Owner: Nobody
> > > Requestors: callum-thompson at ucsb.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> >
> > >
> > >
> > > Hello,
> > >
> > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm trying to
verify a
> > WRF
> > > simulation against some MADIS metar observations. I have two
questions
> > > about the madis2nc function and the point_stat function, if you
could
> > help
> > > please?
> > >
> > >
> > > *madis2nc*
> > >
> > > When I run the command “madis2nc madis_20171001.nc out.nc -type
metar
> -v
> > > 2” it
> > > executes with the following message:
> > >
> > >
> > > DEBUG 1: Reading MADIS File: madis_20171001.nc
> > >
> > > DEBUG 2: Processing METAR recs = -9999
> > >
> > > DEBUG 2: Rejected recs based on masking grid = 0
> > >
> > > DEBUG 2: Rejected recs based on masking poly = 0
> > >
> > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > >
> > > DEBUG 2: Rejected obs based on fill value = 0
> > >
> > > DEBUG 2: Rejected obs based on quality control = 0
> > >
> > > DEBUG 2: Total observations retained or derived = 0
> > >
> > > DEBUG 1: Writing MET File: out.nc
> > >
> > >
> > > However, the output file out.nc is empty (except for some
metadata). I
> > > tried running the same command on a different MADIS file that I
took
> from
> > > Cheyenne (
> > >
> > >
> >
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> > > and
> > > got the below message:
> > >
> > >
> > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> > >
> > > DEBUG 2: Processing METAR recs = 9044
> > >
> > > DEBUG 2: Rejected recs based on masking grid = 0
> > >
> > > DEBUG 2: Rejected recs based on masking poly = 0
> > >
> > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > >
> > > DEBUG 2: Rejected obs based on fill value = 79932
> > >
> > > DEBUG 2: Rejected obs based on quality control = 0
> > >
> > > DEBUG 2: Total observations retained or derived = 64060
> > >
> > > DEBUG 1: Writing MET File: out.nc
> > >
> > >
> > > I noticed the difference in the "Processing METAR recs” line:
can you
> > > advise what that means? I've attached the output from running
ncdump -h
> > for
> > > each input file, if it helps.
> > >
> > >
> > > *point_stat*
> > >
> > > When I get to running point_stat with my WRF output file, do I
need to
> do
> > > any reformatting first? (e.g. does it matter if I use GRIB vs
NetCDF,
> > will
> > > I need to regrid from my Lambert projection to a lat-lon grid?).
> > >
> > >
> > > Thanks very much for any help!
> > >
> > >
> > > Best Regards,
> > >
> > > Callum Thompson
> > >
> > >
> > >
> > > --
> > > Dr Callum Thompson
> > > Earth Research Institute
> > > Email: callum-thompson at ucsb.edu
> > > [image: UC Santa Barbara]
> > >
> > >
> >
> >
>
>
--
Dr Callum Thompson
Earth Research Institute
Email: callum-thompson at ucsb.edu
[image: UC Santa Barbara]
------------------------------------------------
Subject: madis2nc and point_stat
From: Minna Win
Time: Thu Dec 17 11:15:38 2020
Hi Callum,
Can you please send me your NetCDF input files you used in madis2nc?
Please follow the instructions at the bottom of this page:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
I'd like to play around with some commands to trouble-shoot.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
Pronouns: she/her
On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>
> Hi John,
>
> This is really helpful and quick feedback, thanks!
>
> For the MADIS data, it sounds like there's a possible
> 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll wait for
Minna
> to get back to me for that,
>
> For the WRF files, it is the winds (particularly surface winds) that
I'm
> looking to verify, so it sounds like UPP is the better option.
However,
> I've never
> used it before. Is it already installed on Cheyenne/Casper, or would
I have
> to use it on my local machine?
>
> Thanks,
>
> Callum
>
>
>
> On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Callum,
> >
> > Wanted to jump in with some more information about preparing your
WRF
> > output for Point-Stat.
> >
> > The Point-Stat tool takes 3 required inputs:
> > (1) Gridded model output data in any gridded data file format
supported
> by
> > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output of the
> WRF-Input
> > utility).
> > (2) Point observations in NetCDF format... these must be the
output of
> one
> > of MET's point pre-processing tools (ascii2nc, madis2nc, pb2nc,
and so
> on).
> > (3) An ascii configuration file.
> >
> > It sounds like you're asking about (1), your gridded model output.
You
> have
> > 2 options:
> > (1) Run the wrf-interp post-processing utility on your WRF output.
> > (2) Post-process your output using UPP (the Unified Post
Processor) which
> > writes GRIB1 or GRIB2 output files.
> > http://www.dtcenter.org/community-code/unified-post-processor-upp
> >
> > WRF-Interp does interpolate from native model levels to pressure
levels,
> > which is good, but it does not de-stagger the winds. And MET won't
> process
> > the winds on the staggered dimensions.
> >
> > So we recommend using UPP which destaggers the winds, interpolates
to
> > pressure levels, and optionally regrids the data.
> >
> > That being said, if you don't care about the winds and are only
> interested
> > in surface fields, MET can actually read the WRF outptu files
directly.
> We
> > just need to use a config option to tell it to interpret the input
data
> as
> > being output from the wrf-interp utility.
> >
> > How would you like to proceed?
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> > >
> > > Hello Callum,
> > >
> > > It looks like you are having issues with your input MADIS data.
The
> > debug
> > > message of "Processing METAR recs = -9999" indicates that there
is some
> > > missing data (-9999 is used in MET to indicate missing and fill
> values).
> > > This debug message reports the difference between the first and
second
> > > record number to indicate how many records were processed. The
only
> > > difference I see between the two txt files you provided is the
presence
> > of
> > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the madis
input.
> > I'll
> > > need to ask someone who is more familiar with MADIS data to see
if this
> > is
> > > the cause of your issues.
> > >
> > > As far as your point-stat data the only requirement is that your
point
> > data
> > > is in NetCDF format.
> > >
> > > Regards,
> > > Minna
> > > ---------------
> > > Minna Win
> > > National Center for Atmospheric Research
> > > Developmental Testbed Center
> > > Phone: 303-497-8423
> > > Fax: 303-497-8401
> > > ---------------
> > > Pronouns: she/her
> > >
> > >
> > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
> > > > Transaction: Ticket created by callum-thompson at ucsb.edu
> > > > Queue: met_help
> > > > Subject: madis2nc and point_stat
> > > > Owner: Nobody
> > > > Requestors: callum-thompson at ucsb.edu
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> > >
> > > >
> > > >
> > > > Hello,
> > > >
> > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm trying to
verify
> a
> > > WRF
> > > > simulation against some MADIS metar observations. I have two
> questions
> > > > about the madis2nc function and the point_stat function, if
you could
> > > help
> > > > please?
> > > >
> > > >
> > > > *madis2nc*
> > > >
> > > > When I run the command “madis2nc madis_20171001.nc out.nc
-type
> metar
> > -v
> > > > 2” it
> > > > executes with the following message:
> > > >
> > > >
> > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
> > > >
> > > > DEBUG 2: Processing METAR recs = -9999
> > > >
> > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > >
> > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > >
> > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > >
> > > > DEBUG 2: Rejected obs based on fill value = 0
> > > >
> > > > DEBUG 2: Rejected obs based on quality control = 0
> > > >
> > > > DEBUG 2: Total observations retained or derived = 0
> > > >
> > > > DEBUG 1: Writing MET File: out.nc
> > > >
> > > >
> > > > However, the output file out.nc is empty (except for some
> metadata). I
> > > > tried running the same command on a different MADIS file that
I took
> > from
> > > > Cheyenne (
> > > >
> > > >
> > >
> >
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> > > > and
> > > > got the below message:
> > > >
> > > >
> > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> > > >
> > > > DEBUG 2: Processing METAR recs = 9044
> > > >
> > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > >
> > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > >
> > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > >
> > > > DEBUG 2: Rejected obs based on fill value = 79932
> > > >
> > > > DEBUG 2: Rejected obs based on quality control = 0
> > > >
> > > > DEBUG 2: Total observations retained or derived = 64060
> > > >
> > > > DEBUG 1: Writing MET File: out.nc
> > > >
> > > >
> > > > I noticed the difference in the "Processing METAR recs” line:
can
> you
> > > > advise what that means? I've attached the output from running
ncdump
> -h
> > > for
> > > > each input file, if it helps.
> > > >
> > > >
> > > > *point_stat*
> > > >
> > > > When I get to running point_stat with my WRF output file, do I
need
> to
> > do
> > > > any reformatting first? (e.g. does it matter if I use GRIB vs
NetCDF,
> > > will
> > > > I need to regrid from my Lambert projection to a lat-lon
grid?).
> > > >
> > > >
> > > > Thanks very much for any help!
> > > >
> > > >
> > > > Best Regards,
> > > >
> > > > Callum Thompson
> > > >
> > > >
> > > >
> > > > --
> > > > Dr Callum Thompson
> > > > Earth Research Institute
> > > > Email: callum-thompson at ucsb.edu
> > > > [image: UC Santa Barbara]
> > > >
> > > >
> > >
> > >
> >
> >
>
> --
> Dr Callum Thompson
> Earth Research Institute
> Email: callum-thompson at ucsb.edu
> [image: UC Santa Barbara]
>
>
------------------------------------------------
Subject: madis2nc and point_stat
From: Callum Thompson
Time: Thu Dec 17 11:27:21 2020
Hi Minna,
I just did it now. For reference, madis_20171001.nc is my data,
metar_2012040912_F000.nc is the test data I took from Cheyenne.
Thanks,
Callum
On Thu, Dec 17, 2020 at 6:15 PM Minna Win via RT <met_help at ucar.edu>
wrote:
> Hi Callum,
>
> Can you please send me your NetCDF input files you used in madis2nc?
> Please follow the instructions at the bottom of this page:
>
>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
>
> I'd like to play around with some commands to trouble-shoot.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
> ---------------
> Pronouns: she/her
>
>
> On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> >
> > Hi John,
> >
> > This is really helpful and quick feedback, thanks!
> >
> > For the MADIS data, it sounds like there's a possible
> > 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll wait
for
> Minna
> > to get back to me for that,
> >
> > For the WRF files, it is the winds (particularly surface winds)
that I'm
> > looking to verify, so it sounds like UPP is the better option.
However,
> > I've never
> > used it before. Is it already installed on Cheyenne/Casper, or
would I
> have
> > to use it on my local machine?
> >
> > Thanks,
> >
> > Callum
> >
> >
> >
> > On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Callum,
> > >
> > > Wanted to jump in with some more information about preparing
your WRF
> > > output for Point-Stat.
> > >
> > > The Point-Stat tool takes 3 required inputs:
> > > (1) Gridded model output data in any gridded data file format
supported
> > by
> > > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output of the
> > WRF-Input
> > > utility).
> > > (2) Point observations in NetCDF format... these must be the
output of
> > one
> > > of MET's point pre-processing tools (ascii2nc, madis2nc, pb2nc,
and so
> > on).
> > > (3) An ascii configuration file.
> > >
> > > It sounds like you're asking about (1), your gridded model
output. You
> > have
> > > 2 options:
> > > (1) Run the wrf-interp post-processing utility on your WRF
output.
> > > (2) Post-process your output using UPP (the Unified Post
Processor)
> which
> > > writes GRIB1 or GRIB2 output files.
> > > http://www.dtcenter.org/community-code/unified-post-processor-
upp
> > >
> > > WRF-Interp does interpolate from native model levels to pressure
> levels,
> > > which is good, but it does not de-stagger the winds. And MET
won't
> > process
> > > the winds on the staggered dimensions.
> > >
> > > So we recommend using UPP which destaggers the winds,
interpolates to
> > > pressure levels, and optionally regrids the data.
> > >
> > > That being said, if you don't care about the winds and are only
> > interested
> > > in surface fields, MET can actually read the WRF outptu files
directly.
> > We
> > > just need to use a config option to tell it to interpret the
input data
> > as
> > > being output from the wrf-interp utility.
> > >
> > > How would you like to proceed?
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT
<met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
>
> > > >
> > > > Hello Callum,
> > > >
> > > > It looks like you are having issues with your input MADIS
data. The
> > > debug
> > > > message of "Processing METAR recs = -9999" indicates that
there is
> some
> > > > missing data (-9999 is used in MET to indicate missing and
fill
> > values).
> > > > This debug message reports the difference between the first
and
> second
> > > > record number to indicate how many records were processed.
The only
> > > > difference I see between the two txt files you provided is the
> presence
> > > of
> > > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the madis
input.
> > > I'll
> > > > need to ask someone who is more familiar with MADIS data to
see if
> this
> > > is
> > > > the cause of your issues.
> > > >
> > > > As far as your point-stat data the only requirement is that
your
> point
> > > data
> > > > is in NetCDF format.
> > > >
> > > > Regards,
> > > > Minna
> > > > ---------------
> > > > Minna Win
> > > > National Center for Atmospheric Research
> > > > Developmental Testbed Center
> > > > Phone: 303-497-8423
> > > > Fax: 303-497-8401
> > > > ---------------
> > > > Pronouns: she/her
> > > >
> > > >
> > > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
> > > > > Transaction: Ticket created by callum-thompson at ucsb.edu
> > > > > Queue: met_help
> > > > > Subject: madis2nc and point_stat
> > > > > Owner: Nobody
> > > > > Requestors: callum-thompson at ucsb.edu
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> > > >
> > > > >
> > > > >
> > > > > Hello,
> > > > >
> > > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm trying
to
> verify
> > a
> > > > WRF
> > > > > simulation against some MADIS metar observations. I have two
> > questions
> > > > > about the madis2nc function and the point_stat function, if
you
> could
> > > > help
> > > > > please?
> > > > >
> > > > >
> > > > > *madis2nc*
> > > > >
> > > > > When I run the command “madis2nc madis_20171001.nc out.nc
-type
> > metar
> > > -v
> > > > > 2” it
> > > > > executes with the following message:
> > > > >
> > > > >
> > > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
> > > > >
> > > > > DEBUG 2: Processing METAR recs = -9999
> > > > >
> > > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > > >
> > > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > > >
> > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > > >
> > > > > DEBUG 2: Rejected obs based on fill value = 0
> > > > >
> > > > > DEBUG 2: Rejected obs based on quality control = 0
> > > > >
> > > > > DEBUG 2: Total observations retained or derived = 0
> > > > >
> > > > > DEBUG 1: Writing MET File: out.nc
> > > > >
> > > > >
> > > > > However, the output file out.nc is empty (except for some
> > metadata). I
> > > > > tried running the same command on a different MADIS file
that I
> took
> > > from
> > > > > Cheyenne (
> > > > >
> > > > >
> > > >
> > >
> >
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> > > > > and
> > > > > got the below message:
> > > > >
> > > > >
> > > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> > > > >
> > > > > DEBUG 2: Processing METAR recs = 9044
> > > > >
> > > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > > >
> > > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > > >
> > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > > >
> > > > > DEBUG 2: Rejected obs based on fill value = 79932
> > > > >
> > > > > DEBUG 2: Rejected obs based on quality control = 0
> > > > >
> > > > > DEBUG 2: Total observations retained or derived = 64060
> > > > >
> > > > > DEBUG 1: Writing MET File: out.nc
> > > > >
> > > > >
> > > > > I noticed the difference in the "Processing METAR recs”
line: can
> > you
> > > > > advise what that means? I've attached the output from
running
> ncdump
> > -h
> > > > for
> > > > > each input file, if it helps.
> > > > >
> > > > >
> > > > > *point_stat*
> > > > >
> > > > > When I get to running point_stat with my WRF output file, do
I need
> > to
> > > do
> > > > > any reformatting first? (e.g. does it matter if I use GRIB
vs
> NetCDF,
> > > > will
> > > > > I need to regrid from my Lambert projection to a lat-lon
grid?).
> > > > >
> > > > >
> > > > > Thanks very much for any help!
> > > > >
> > > > >
> > > > > Best Regards,
> > > > >
> > > > > Callum Thompson
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Dr Callum Thompson
> > > > > Earth Research Institute
> > > > > Email: callum-thompson at ucsb.edu
> > > > > [image: UC Santa Barbara]
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> > --
> > Dr Callum Thompson
> > Earth Research Institute
> > Email: callum-thompson at ucsb.edu
> > [image: UC Santa Barbara]
> >
> >
>
>
--
Dr Callum Thompson
Earth Research Institute
Email: callum-thompson at ucsb.edu
[image: UC Santa Barbara]
------------------------------------------------
Subject: madis2nc and point_stat
From: John Halley Gotway
Time: Thu Dec 17 11:54:59 2020
Callum,
I did check on the UPP website and found this page about existing
builds of
UPP:
http://www.dtcenter.org/community-code/unified-post-processor-
upp/uppv9-0-existing-builds
However, details about builds on Cheyenne are conspicuously absent! So
I
emailed a couple of DTC folks asking for details but haven't heard
back
from anyone yet.
FYI, over the last few months, support for UPP has moved from direct
email
to a community forum page:
https://forums.ufscommunity.org/forum/post-processing
I'd encourage you to post your question about Cheyenne there. The
folks who
used to answer direct email questions are now monitoring that forum.
Hope that helps.
Thanks,
John
On Thu, Dec 17, 2020 at 11:35 AM Callum Thompson via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>
> Hi Minna,
>
> I just did it now. For reference, madis_20171001.nc is my data,
> metar_2012040912_F000.nc is the test data I took from Cheyenne.
>
> Thanks,
> Callum
>
>
>
> On Thu, Dec 17, 2020 at 6:15 PM Minna Win via RT <met_help at ucar.edu>
> wrote:
>
> > Hi Callum,
> >
> > Can you please send me your NetCDF input files you used in
madis2nc?
> > Please follow the instructions at the bottom of this page:
> >
> >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> >
> > I'd like to play around with some commands to trouble-shoot.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> > ---------------
> > Pronouns: she/her
> >
> >
> > On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> > >
> > > Hi John,
> > >
> > > This is really helpful and quick feedback, thanks!
> > >
> > > For the MADIS data, it sounds like there's a possible
> > > 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll wait
for
> > Minna
> > > to get back to me for that,
> > >
> > > For the WRF files, it is the winds (particularly surface winds)
that
> I'm
> > > looking to verify, so it sounds like UPP is the better option.
However,
> > > I've never
> > > used it before. Is it already installed on Cheyenne/Casper, or
would I
> > have
> > > to use it on my local machine?
> > >
> > > Thanks,
> > >
> > > Callum
> > >
> > >
> > >
> > > On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Callum,
> > > >
> > > > Wanted to jump in with some more information about preparing
your WRF
> > > > output for Point-Stat.
> > > >
> > > > The Point-Stat tool takes 3 required inputs:
> > > > (1) Gridded model output data in any gridded data file format
> supported
> > > by
> > > > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output of
the
> > > WRF-Input
> > > > utility).
> > > > (2) Point observations in NetCDF format... these must be the
output
> of
> > > one
> > > > of MET's point pre-processing tools (ascii2nc, madis2nc,
pb2nc, and
> so
> > > on).
> > > > (3) An ascii configuration file.
> > > >
> > > > It sounds like you're asking about (1), your gridded model
output.
> You
> > > have
> > > > 2 options:
> > > > (1) Run the wrf-interp post-processing utility on your WRF
output.
> > > > (2) Post-process your output using UPP (the Unified Post
Processor)
> > which
> > > > writes GRIB1 or GRIB2 output files.
> > > > http://www.dtcenter.org/community-code/unified-post-processor-
upp
> > > >
> > > > WRF-Interp does interpolate from native model levels to
pressure
> > levels,
> > > > which is good, but it does not de-stagger the winds. And MET
won't
> > > process
> > > > the winds on the staggered dimensions.
> > > >
> > > > So we recommend using UPP which destaggers the winds,
interpolates to
> > > > pressure levels, and optionally regrids the data.
> > > >
> > > > That being said, if you don't care about the winds and are
only
> > > interested
> > > > in surface fields, MET can actually read the WRF outptu files
> directly.
> > > We
> > > > just need to use a config option to tell it to interpret the
input
> data
> > > as
> > > > being output from the wrf-interp utility.
> > > >
> > > > How would you like to proceed?
> > > >
> > > > Thanks,
> > > > John Halley Gotway
> > > >
> > > > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT
<met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> > > > >
> > > > > Hello Callum,
> > > > >
> > > > > It looks like you are having issues with your input MADIS
data.
> The
> > > > debug
> > > > > message of "Processing METAR recs = -9999" indicates that
there is
> > some
> > > > > missing data (-9999 is used in MET to indicate missing and
fill
> > > values).
> > > > > This debug message reports the difference between the first
and
> > second
> > > > > record number to indicate how many records were processed.
The
> only
> > > > > difference I see between the two txt files you provided is
the
> > presence
> > > > of
> > > > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the
madis
> input.
> > > > I'll
> > > > > need to ask someone who is more familiar with MADIS data to
see if
> > this
> > > > is
> > > > > the cause of your issues.
> > > > >
> > > > > As far as your point-stat data the only requirement is that
your
> > point
> > > > data
> > > > > is in NetCDF format.
> > > > >
> > > > > Regards,
> > > > > Minna
> > > > > ---------------
> > > > > Minna Win
> > > > > National Center for Atmospheric Research
> > > > > Developmental Testbed Center
> > > > > Phone: 303-497-8423
> > > > > Fax: 303-497-8401
> > > > > ---------------
> > > > > Pronouns: she/her
> > > > >
> > > > >
> > > > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
> > > > > > Transaction: Ticket created by callum-thompson at ucsb.edu
> > > > > > Queue: met_help
> > > > > > Subject: madis2nc and point_stat
> > > > > > Owner: Nobody
> > > > > > Requestors: callum-thompson at ucsb.edu
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> > > > >
> > > > > >
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm trying
to
> > verify
> > > a
> > > > > WRF
> > > > > > simulation against some MADIS metar observations. I have
two
> > > questions
> > > > > > about the madis2nc function and the point_stat function,
if you
> > could
> > > > > help
> > > > > > please?
> > > > > >
> > > > > >
> > > > > > *madis2nc*
> > > > > >
> > > > > > When I run the command “madis2nc madis_20171001.nc out.nc
-type
> > > metar
> > > > -v
> > > > > > 2” it
> > > > > > executes with the following message:
> > > > > >
> > > > > >
> > > > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
> > > > > >
> > > > > > DEBUG 2: Processing METAR recs = -9999
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > > > >
> > > > > > DEBUG 2: Rejected obs based on fill value = 0
> > > > > >
> > > > > > DEBUG 2: Rejected obs based on quality control = 0
> > > > > >
> > > > > > DEBUG 2: Total observations retained or derived = 0
> > > > > >
> > > > > > DEBUG 1: Writing MET File: out.nc
> > > > > >
> > > > > >
> > > > > > However, the output file out.nc is empty (except for some
> > > metadata). I
> > > > > > tried running the same command on a different MADIS file
that I
> > took
> > > > from
> > > > > > Cheyenne (
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> > > > > > and
> > > > > > got the below message:
> > > > > >
> > > > > >
> > > > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> > > > > >
> > > > > > DEBUG 2: Processing METAR recs = 9044
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > > > >
> > > > > > DEBUG 2: Rejected obs based on fill value = 79932
> > > > > >
> > > > > > DEBUG 2: Rejected obs based on quality control = 0
> > > > > >
> > > > > > DEBUG 2: Total observations retained or derived = 64060
> > > > > >
> > > > > > DEBUG 1: Writing MET File: out.nc
> > > > > >
> > > > > >
> > > > > > I noticed the difference in the "Processing METAR recs”
line:
> can
> > > you
> > > > > > advise what that means? I've attached the output from
running
> > ncdump
> > > -h
> > > > > for
> > > > > > each input file, if it helps.
> > > > > >
> > > > > >
> > > > > > *point_stat*
> > > > > >
> > > > > > When I get to running point_stat with my WRF output file,
do I
> need
> > > to
> > > > do
> > > > > > any reformatting first? (e.g. does it matter if I use GRIB
vs
> > NetCDF,
> > > > > will
> > > > > > I need to regrid from my Lambert projection to a lat-lon
grid?).
> > > > > >
> > > > > >
> > > > > > Thanks very much for any help!
> > > > > >
> > > > > >
> > > > > > Best Regards,
> > > > > >
> > > > > > Callum Thompson
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Dr Callum Thompson
> > > > > > Earth Research Institute
> > > > > > Email: callum-thompson at ucsb.edu
> > > > > > [image: UC Santa Barbara]
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > Dr Callum Thompson
> > > Earth Research Institute
> > > Email: callum-thompson at ucsb.edu
> > > [image: UC Santa Barbara]
> > >
> > >
> >
> >
>
> --
> Dr Callum Thompson
> Earth Research Institute
> Email: callum-thompson at ucsb.edu
> [image: UC Santa Barbara]
>
>
------------------------------------------------
Subject: madis2nc and point_stat
From: John Halley Gotway
Time: Thu Dec 17 13:29:15 2020
Callum,
Below is the response I received back from UPP staff...
================================
Yes - our current website is poised for the upcoming release (v9.0.0)
that
recently got pushed to February with the UFS SRW App release. Since we
plan
to release at the same time as the UFS SRW app, our main website is
stuck
in transition.
You may redirect the user to
https://dtcenter.org/community-code/unified-post-processor-upp-wrf,
which
is our frozen UPP for WRF website (since we are no longer supporting
WRF
next AOP). We did include a couple banners on the website stating this
(welcome/downloads page), so to try and minimize the confusion. That
page
will have all info for the latest release, v4.1, including existing
builds
on Cheyenne. Let me know if you have further questions.
================================
So please take a look at this page:
https://dtcenter.org/community-code/unified-post-processor-upp/uppv4-
1-existing-builds
Thanks,
John
On Thu, Dec 17, 2020 at 11:54 AM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Callum,
>
> I did check on the UPP website and found this page about existing
builds
> of UPP:
>
> http://www.dtcenter.org/community-code/unified-post-processor-
upp/uppv9-0-existing-builds
>
> However, details about builds on Cheyenne are conspicuously absent!
So I
> emailed a couple of DTC folks asking for details but haven't heard
back
> from anyone yet.
>
> FYI, over the last few months, support for UPP has moved from direct
email
> to a community forum page:
> https://forums.ufscommunity.org/forum/post-processing
>
> I'd encourage you to post your question about Cheyenne there. The
folks
> who used to answer direct email questions are now monitoring that
forum.
>
> Hope that helps.
>
> Thanks,
> John
>
> On Thu, Dec 17, 2020 at 11:35 AM Callum Thompson via RT
<met_help at ucar.edu>
> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>>
>> Hi Minna,
>>
>> I just did it now. For reference, madis_20171001.nc is my data,
>> metar_2012040912_F000.nc is the test data I took from Cheyenne.
>>
>> Thanks,
>> Callum
>>
>>
>>
>> On Thu, Dec 17, 2020 at 6:15 PM Minna Win via RT
<met_help at ucar.edu>
>> wrote:
>>
>> > Hi Callum,
>> >
>> > Can you please send me your NetCDF input files you used in
madis2nc?
>> > Please follow the instructions at the bottom of this page:
>> >
>> >
>> >
>> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
>> >
>> > I'd like to play around with some commands to trouble-shoot.
>> >
>> > Regards,
>> > Minna
>> > ---------------
>> > Minna Win
>> > National Center for Atmospheric Research
>> > Developmental Testbed Center
>> > Phone: 303-497-8423
>> > Fax: 303-497-8401
>> > ---------------
>> > Pronouns: she/her
>> >
>> >
>> > On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT <
>> met_help at ucar.edu>
>> > wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>> > >
>> > > Hi John,
>> > >
>> > > This is really helpful and quick feedback, thanks!
>> > >
>> > > For the MADIS data, it sounds like there's a possible
>> > > 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll
wait for
>> > Minna
>> > > to get back to me for that,
>> > >
>> > > For the WRF files, it is the winds (particularly surface winds)
that
>> I'm
>> > > looking to verify, so it sounds like UPP is the better option.
>> However,
>> > > I've never
>> > > used it before. Is it already installed on Cheyenne/Casper, or
would I
>> > have
>> > > to use it on my local machine?
>> > >
>> > > Thanks,
>> > >
>> > > Callum
>> > >
>> > >
>> > >
>> > > On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
>> > > met_help at ucar.edu>
>> > > wrote:
>> > >
>> > > > Hi Callum,
>> > > >
>> > > > Wanted to jump in with some more information about preparing
your
>> WRF
>> > > > output for Point-Stat.
>> > > >
>> > > > The Point-Stat tool takes 3 required inputs:
>> > > > (1) Gridded model output data in any gridded data file format
>> supported
>> > > by
>> > > > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output of
the
>> > > WRF-Input
>> > > > utility).
>> > > > (2) Point observations in NetCDF format... these must be the
output
>> of
>> > > one
>> > > > of MET's point pre-processing tools (ascii2nc, madis2nc,
pb2nc, and
>> so
>> > > on).
>> > > > (3) An ascii configuration file.
>> > > >
>> > > > It sounds like you're asking about (1), your gridded model
output.
>> You
>> > > have
>> > > > 2 options:
>> > > > (1) Run the wrf-interp post-processing utility on your WRF
output.
>> > > > (2) Post-process your output using UPP (the Unified Post
Processor)
>> > which
>> > > > writes GRIB1 or GRIB2 output files.
>> > > > http://www.dtcenter.org/community-code/unified-post-
processor-upp
>> > > >
>> > > > WRF-Interp does interpolate from native model levels to
pressure
>> > levels,
>> > > > which is good, but it does not de-stagger the winds. And MET
won't
>> > > process
>> > > > the winds on the staggered dimensions.
>> > > >
>> > > > So we recommend using UPP which destaggers the winds,
interpolates
>> to
>> > > > pressure levels, and optionally regrids the data.
>> > > >
>> > > > That being said, if you don't care about the winds and are
only
>> > > interested
>> > > > in surface fields, MET can actually read the WRF outptu files
>> directly.
>> > > We
>> > > > just need to use a config option to tell it to interpret the
input
>> data
>> > > as
>> > > > being output from the wrf-interp utility.
>> > > >
>> > > > How would you like to proceed?
>> > > >
>> > > > Thanks,
>> > > > John Halley Gotway
>> > > >
>> > > > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT
<met_help at ucar.edu
>> >
>> > > > wrote:
>> > > >
>> > > > >
>> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>> > > > >
>> > > > > Hello Callum,
>> > > > >
>> > > > > It looks like you are having issues with your input MADIS
data.
>> The
>> > > > debug
>> > > > > message of "Processing METAR recs = -9999" indicates that
there is
>> > some
>> > > > > missing data (-9999 is used in MET to indicate missing and
fill
>> > > values).
>> > > > > This debug message reports the difference between the first
and
>> > second
>> > > > > record number to indicate how many records were processed.
The
>> only
>> > > > > difference I see between the two txt files you provided is
the
>> > presence
>> > > > of
>> > > > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the
madis
>> input.
>> > > > I'll
>> > > > > need to ask someone who is more familiar with MADIS data to
see if
>> > this
>> > > > is
>> > > > > the cause of your issues.
>> > > > >
>> > > > > As far as your point-stat data the only requirement is that
your
>> > point
>> > > > data
>> > > > > is in NetCDF format.
>> > > > >
>> > > > > Regards,
>> > > > > Minna
>> > > > > ---------------
>> > > > > Minna Win
>> > > > > National Center for Atmospheric Research
>> > > > > Developmental Testbed Center
>> > > > > Phone: 303-497-8423
>> > > > > Fax: 303-497-8401
>> > > > > ---------------
>> > > > > Pronouns: she/her
>> > > > >
>> > > > >
>> > > > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT <
>> > > > met_help at ucar.edu>
>> > > > > wrote:
>> > > > >
>> > > > > >
>> > > > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
>> > > > > > Transaction: Ticket created by callum-thompson at ucsb.edu
>> > > > > > Queue: met_help
>> > > > > > Subject: madis2nc and point_stat
>> > > > > > Owner: Nobody
>> > > > > > Requestors: callum-thompson at ucsb.edu
>> > > > > > Status: new
>> > > > > > Ticket <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
>> > > > >
>> > > > > >
>> > > > > >
>> > > > > > Hello,
>> > > > > >
>> > > > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm
trying to
>> > verify
>> > > a
>> > > > > WRF
>> > > > > > simulation against some MADIS metar observations. I have
two
>> > > questions
>> > > > > > about the madis2nc function and the point_stat function,
if you
>> > could
>> > > > > help
>> > > > > > please?
>> > > > > >
>> > > > > >
>> > > > > > *madis2nc*
>> > > > > >
>> > > > > > When I run the command “madis2nc madis_20171001.nc out.nc
-type
>> > > metar
>> > > > -v
>> > > > > > 2” it
>> > > > > > executes with the following message:
>> > > > > >
>> > > > > >
>> > > > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
>> > > > > >
>> > > > > > DEBUG 2: Processing METAR recs = -9999
>> > > > > >
>> > > > > > DEBUG 2: Rejected recs based on masking grid = 0
>> > > > > >
>> > > > > > DEBUG 2: Rejected recs based on masking poly = 0
>> > > > > >
>> > > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
>> > > > > >
>> > > > > > DEBUG 2: Rejected obs based on fill value = 0
>> > > > > >
>> > > > > > DEBUG 2: Rejected obs based on quality control = 0
>> > > > > >
>> > > > > > DEBUG 2: Total observations retained or derived = 0
>> > > > > >
>> > > > > > DEBUG 1: Writing MET File: out.nc
>> > > > > >
>> > > > > >
>> > > > > > However, the output file out.nc is empty (except for some
>> > > metadata). I
>> > > > > > tried running the same command on a different MADIS file
that I
>> > took
>> > > > from
>> > > > > > Cheyenne (
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
>> > > > > > and
>> > > > > > got the below message:
>> > > > > >
>> > > > > >
>> > > > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
>> > > > > >
>> > > > > > DEBUG 2: Processing METAR recs = 9044
>> > > > > >
>> > > > > > DEBUG 2: Rejected recs based on masking grid = 0
>> > > > > >
>> > > > > > DEBUG 2: Rejected recs based on masking poly = 0
>> > > > > >
>> > > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
>> > > > > >
>> > > > > > DEBUG 2: Rejected obs based on fill value = 79932
>> > > > > >
>> > > > > > DEBUG 2: Rejected obs based on quality control = 0
>> > > > > >
>> > > > > > DEBUG 2: Total observations retained or derived = 64060
>> > > > > >
>> > > > > > DEBUG 1: Writing MET File: out.nc
>> > > > > >
>> > > > > >
>> > > > > > I noticed the difference in the "Processing METAR recs”
line:
>> can
>> > > you
>> > > > > > advise what that means? I've attached the output from
running
>> > ncdump
>> > > -h
>> > > > > for
>> > > > > > each input file, if it helps.
>> > > > > >
>> > > > > >
>> > > > > > *point_stat*
>> > > > > >
>> > > > > > When I get to running point_stat with my WRF output file,
do I
>> need
>> > > to
>> > > > do
>> > > > > > any reformatting first? (e.g. does it matter if I use
GRIB vs
>> > NetCDF,
>> > > > > will
>> > > > > > I need to regrid from my Lambert projection to a lat-lon
grid?).
>> > > > > >
>> > > > > >
>> > > > > > Thanks very much for any help!
>> > > > > >
>> > > > > >
>> > > > > > Best Regards,
>> > > > > >
>> > > > > > Callum Thompson
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Dr Callum Thompson
>> > > > > > Earth Research Institute
>> > > > > > Email: callum-thompson at ucsb.edu
>> > > > > > [image: UC Santa Barbara]
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>> > > --
>> > > Dr Callum Thompson
>> > > Earth Research Institute
>> > > Email: callum-thompson at ucsb.edu
>> > > [image: UC Santa Barbara]
>> > >
>> > >
>> >
>> >
>>
>> --
>> Dr Callum Thompson
>> Earth Research Institute
>> Email: callum-thompson at ucsb.edu
>> [image: UC Santa Barbara]
>>
>>
------------------------------------------------
Subject: madis2nc and point_stat
From: Minna Win
Time: Thu Dec 17 16:05:22 2020
Hi Callum,
It looks like your input is missing the dimension variable, 'recNum',
which
is what the madis2nc tool is looking for. This explains why there was
an
error when attempting to calculate the number of records. Do you know
who
created your madis_20171001.nc file?
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
Pronouns: she/her
On Thu, Dec 17, 2020 at 11:35 AM Callum Thompson via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>
> Hi Minna,
>
> I just did it now. For reference, madis_20171001.nc is my data,
> metar_2012040912_F000.nc is the test data I took from Cheyenne.
>
> Thanks,
> Callum
>
>
>
> On Thu, Dec 17, 2020 at 6:15 PM Minna Win via RT <met_help at ucar.edu>
> wrote:
>
> > Hi Callum,
> >
> > Can you please send me your NetCDF input files you used in
madis2nc?
> > Please follow the instructions at the bottom of this page:
> >
> >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> >
> > I'd like to play around with some commands to trouble-shoot.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> > ---------------
> > Pronouns: she/her
> >
> >
> > On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> > >
> > > Hi John,
> > >
> > > This is really helpful and quick feedback, thanks!
> > >
> > > For the MADIS data, it sounds like there's a possible
> > > 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll wait
for
> > Minna
> > > to get back to me for that,
> > >
> > > For the WRF files, it is the winds (particularly surface winds)
that
> I'm
> > > looking to verify, so it sounds like UPP is the better option.
However,
> > > I've never
> > > used it before. Is it already installed on Cheyenne/Casper, or
would I
> > have
> > > to use it on my local machine?
> > >
> > > Thanks,
> > >
> > > Callum
> > >
> > >
> > >
> > > On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Callum,
> > > >
> > > > Wanted to jump in with some more information about preparing
your WRF
> > > > output for Point-Stat.
> > > >
> > > > The Point-Stat tool takes 3 required inputs:
> > > > (1) Gridded model output data in any gridded data file format
> supported
> > > by
> > > > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output of
the
> > > WRF-Input
> > > > utility).
> > > > (2) Point observations in NetCDF format... these must be the
output
> of
> > > one
> > > > of MET's point pre-processing tools (ascii2nc, madis2nc,
pb2nc, and
> so
> > > on).
> > > > (3) An ascii configuration file.
> > > >
> > > > It sounds like you're asking about (1), your gridded model
output.
> You
> > > have
> > > > 2 options:
> > > > (1) Run the wrf-interp post-processing utility on your WRF
output.
> > > > (2) Post-process your output using UPP (the Unified Post
Processor)
> > which
> > > > writes GRIB1 or GRIB2 output files.
> > > > http://www.dtcenter.org/community-code/unified-post-processor-
upp
> > > >
> > > > WRF-Interp does interpolate from native model levels to
pressure
> > levels,
> > > > which is good, but it does not de-stagger the winds. And MET
won't
> > > process
> > > > the winds on the staggered dimensions.
> > > >
> > > > So we recommend using UPP which destaggers the winds,
interpolates to
> > > > pressure levels, and optionally regrids the data.
> > > >
> > > > That being said, if you don't care about the winds and are
only
> > > interested
> > > > in surface fields, MET can actually read the WRF outptu files
> directly.
> > > We
> > > > just need to use a config option to tell it to interpret the
input
> data
> > > as
> > > > being output from the wrf-interp utility.
> > > >
> > > > How would you like to proceed?
> > > >
> > > > Thanks,
> > > > John Halley Gotway
> > > >
> > > > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT
<met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> > > > >
> > > > > Hello Callum,
> > > > >
> > > > > It looks like you are having issues with your input MADIS
data.
> The
> > > > debug
> > > > > message of "Processing METAR recs = -9999" indicates that
there is
> > some
> > > > > missing data (-9999 is used in MET to indicate missing and
fill
> > > values).
> > > > > This debug message reports the difference between the first
and
> > second
> > > > > record number to indicate how many records were processed.
The
> only
> > > > > difference I see between the two txt files you provided is
the
> > presence
> > > > of
> > > > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the
madis
> input.
> > > > I'll
> > > > > need to ask someone who is more familiar with MADIS data to
see if
> > this
> > > > is
> > > > > the cause of your issues.
> > > > >
> > > > > As far as your point-stat data the only requirement is that
your
> > point
> > > > data
> > > > > is in NetCDF format.
> > > > >
> > > > > Regards,
> > > > > Minna
> > > > > ---------------
> > > > > Minna Win
> > > > > National Center for Atmospheric Research
> > > > > Developmental Testbed Center
> > > > > Phone: 303-497-8423
> > > > > Fax: 303-497-8401
> > > > > ---------------
> > > > > Pronouns: she/her
> > > > >
> > > > >
> > > > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
> > > > > > Transaction: Ticket created by callum-thompson at ucsb.edu
> > > > > > Queue: met_help
> > > > > > Subject: madis2nc and point_stat
> > > > > > Owner: Nobody
> > > > > > Requestors: callum-thompson at ucsb.edu
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> > > > >
> > > > > >
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm trying
to
> > verify
> > > a
> > > > > WRF
> > > > > > simulation against some MADIS metar observations. I have
two
> > > questions
> > > > > > about the madis2nc function and the point_stat function,
if you
> > could
> > > > > help
> > > > > > please?
> > > > > >
> > > > > >
> > > > > > *madis2nc*
> > > > > >
> > > > > > When I run the command “madis2nc madis_20171001.nc out.nc
-type
> > > metar
> > > > -v
> > > > > > 2” it
> > > > > > executes with the following message:
> > > > > >
> > > > > >
> > > > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
> > > > > >
> > > > > > DEBUG 2: Processing METAR recs = -9999
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > > > >
> > > > > > DEBUG 2: Rejected obs based on fill value = 0
> > > > > >
> > > > > > DEBUG 2: Rejected obs based on quality control = 0
> > > > > >
> > > > > > DEBUG 2: Total observations retained or derived = 0
> > > > > >
> > > > > > DEBUG 1: Writing MET File: out.nc
> > > > > >
> > > > > >
> > > > > > However, the output file out.nc is empty (except for some
> > > metadata). I
> > > > > > tried running the same command on a different MADIS file
that I
> > took
> > > > from
> > > > > > Cheyenne (
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> > > > > > and
> > > > > > got the below message:
> > > > > >
> > > > > >
> > > > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> > > > > >
> > > > > > DEBUG 2: Processing METAR recs = 9044
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > > > >
> > > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > > > >
> > > > > > DEBUG 2: Rejected obs based on fill value = 79932
> > > > > >
> > > > > > DEBUG 2: Rejected obs based on quality control = 0
> > > > > >
> > > > > > DEBUG 2: Total observations retained or derived = 64060
> > > > > >
> > > > > > DEBUG 1: Writing MET File: out.nc
> > > > > >
> > > > > >
> > > > > > I noticed the difference in the "Processing METAR recs”
line:
> can
> > > you
> > > > > > advise what that means? I've attached the output from
running
> > ncdump
> > > -h
> > > > > for
> > > > > > each input file, if it helps.
> > > > > >
> > > > > >
> > > > > > *point_stat*
> > > > > >
> > > > > > When I get to running point_stat with my WRF output file,
do I
> need
> > > to
> > > > do
> > > > > > any reformatting first? (e.g. does it matter if I use GRIB
vs
> > NetCDF,
> > > > > will
> > > > > > I need to regrid from my Lambert projection to a lat-lon
grid?).
> > > > > >
> > > > > >
> > > > > > Thanks very much for any help!
> > > > > >
> > > > > >
> > > > > > Best Regards,
> > > > > >
> > > > > > Callum Thompson
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Dr Callum Thompson
> > > > > > Earth Research Institute
> > > > > > Email: callum-thompson at ucsb.edu
> > > > > > [image: UC Santa Barbara]
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > Dr Callum Thompson
> > > Earth Research Institute
> > > Email: callum-thompson at ucsb.edu
> > > [image: UC Santa Barbara]
> > >
> > >
> >
> >
>
> --
> Dr Callum Thompson
> Earth Research Institute
> Email: callum-thompson at ucsb.edu
> [image: UC Santa Barbara]
>
>
------------------------------------------------
Subject: madis2nc and point_stat
From: Callum Thompson
Time: Fri Dec 18 07:56:52 2020
Hello,
Thanks for the info on UPP, John. I'll have a look into using it on
Cheyenne once I have the madis2nc step sorted.
It was a colleague who gave me the MADIS file, so I'll ask him how he
obtained it and get back to you.
Best,
Callum
On Thu, Dec 17, 2020 at 11:05 PM Minna Win via RT <met_help at ucar.edu>
wrote:
> Hi Callum,
>
> It looks like your input is missing the dimension variable,
'recNum', which
> is what the madis2nc tool is looking for. This explains why there
was an
> error when attempting to calculate the number of records. Do you
know who
> created your madis_20171001.nc file?
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
> ---------------
> Pronouns: she/her
>
>
> On Thu, Dec 17, 2020 at 11:35 AM Callum Thompson via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> >
> > Hi Minna,
> >
> > I just did it now. For reference, madis_20171001.nc is my data,
> > metar_2012040912_F000.nc is the test data I took from Cheyenne.
> >
> > Thanks,
> > Callum
> >
> >
> >
> > On Thu, Dec 17, 2020 at 6:15 PM Minna Win via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Callum,
> > >
> > > Can you please send me your NetCDF input files you used in
madis2nc?
> > > Please follow the instructions at the bottom of this page:
> > >
> > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> > >
> > > I'd like to play around with some commands to trouble-shoot.
> > >
> > > Regards,
> > > Minna
> > > ---------------
> > > Minna Win
> > > National Center for Atmospheric Research
> > > Developmental Testbed Center
> > > Phone: 303-497-8423
> > > Fax: 303-497-8401
> > > ---------------
> > > Pronouns: she/her
> > >
> > >
> > > On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
>
> > > >
> > > > Hi John,
> > > >
> > > > This is really helpful and quick feedback, thanks!
> > > >
> > > > For the MADIS data, it sounds like there's a possible
> > > > 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll
wait for
> > > Minna
> > > > to get back to me for that,
> > > >
> > > > For the WRF files, it is the winds (particularly surface
winds) that
> > I'm
> > > > looking to verify, so it sounds like UPP is the better option.
> However,
> > > > I've never
> > > > used it before. Is it already installed on Cheyenne/Casper, or
would
> I
> > > have
> > > > to use it on my local machine?
> > > >
> > > > Thanks,
> > > >
> > > > Callum
> > > >
> > > >
> > > >
> > > > On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Callum,
> > > > >
> > > > > Wanted to jump in with some more information about preparing
your
> WRF
> > > > > output for Point-Stat.
> > > > >
> > > > > The Point-Stat tool takes 3 required inputs:
> > > > > (1) Gridded model output data in any gridded data file
format
> > supported
> > > > by
> > > > > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output of
the
> > > > WRF-Input
> > > > > utility).
> > > > > (2) Point observations in NetCDF format... these must be the
output
> > of
> > > > one
> > > > > of MET's point pre-processing tools (ascii2nc, madis2nc,
pb2nc, and
> > so
> > > > on).
> > > > > (3) An ascii configuration file.
> > > > >
> > > > > It sounds like you're asking about (1), your gridded model
output.
> > You
> > > > have
> > > > > 2 options:
> > > > > (1) Run the wrf-interp post-processing utility on your WRF
output.
> > > > > (2) Post-process your output using UPP (the Unified Post
Processor)
> > > which
> > > > > writes GRIB1 or GRIB2 output files.
> > > > > http://www.dtcenter.org/community-code/unified-post-
processor-upp
> > > > >
> > > > > WRF-Interp does interpolate from native model levels to
pressure
> > > levels,
> > > > > which is good, but it does not de-stagger the winds. And MET
won't
> > > > process
> > > > > the winds on the staggered dimensions.
> > > > >
> > > > > So we recommend using UPP which destaggers the winds,
interpolates
> to
> > > > > pressure levels, and optionally regrids the data.
> > > > >
> > > > > That being said, if you don't care about the winds and are
only
> > > > interested
> > > > > in surface fields, MET can actually read the WRF outptu
files
> > directly.
> > > > We
> > > > > just need to use a config option to tell it to interpret the
input
> > data
> > > > as
> > > > > being output from the wrf-interp utility.
> > > > >
> > > > > How would you like to proceed?
> > > > >
> > > > > Thanks,
> > > > > John Halley Gotway
> > > > >
> > > > > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT <
> met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> > > > > >
> > > > > > Hello Callum,
> > > > > >
> > > > > > It looks like you are having issues with your input MADIS
data.
> > The
> > > > > debug
> > > > > > message of "Processing METAR recs = -9999" indicates that
there
> is
> > > some
> > > > > > missing data (-9999 is used in MET to indicate missing and
fill
> > > > values).
> > > > > > This debug message reports the difference between the
first and
> > > second
> > > > > > record number to indicate how many records were processed.
The
> > only
> > > > > > difference I see between the two txt files you provided is
the
> > > presence
> > > > > of
> > > > > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the
madis
> > input.
> > > > > I'll
> > > > > > need to ask someone who is more familiar with MADIS data
to see
> if
> > > this
> > > > > is
> > > > > > the cause of your issues.
> > > > > >
> > > > > > As far as your point-stat data the only requirement is
that your
> > > point
> > > > > data
> > > > > > is in NetCDF format.
> > > > > >
> > > > > > Regards,
> > > > > > Minna
> > > > > > ---------------
> > > > > > Minna Win
> > > > > > National Center for Atmospheric Research
> > > > > > Developmental Testbed Center
> > > > > > Phone: 303-497-8423
> > > > > > Fax: 303-497-8401
> > > > > > ---------------
> > > > > > Pronouns: she/her
> > > > > >
> > > > > >
> > > > > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
> > > > > > > Transaction: Ticket created by callum-thompson at ucsb.edu
> > > > > > > Queue: met_help
> > > > > > > Subject: madis2nc and point_stat
> > > > > > > Owner: Nobody
> > > > > > > Requestors: callum-thompson at ucsb.edu
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm
trying to
> > > verify
> > > > a
> > > > > > WRF
> > > > > > > simulation against some MADIS metar observations. I have
two
> > > > questions
> > > > > > > about the madis2nc function and the point_stat function,
if you
> > > could
> > > > > > help
> > > > > > > please?
> > > > > > >
> > > > > > >
> > > > > > > *madis2nc*
> > > > > > >
> > > > > > > When I run the command “madis2nc madis_20171001.nc
out.nc
> -type
> > > > metar
> > > > > -v
> > > > > > > 2” it
> > > > > > > executes with the following message:
> > > > > > >
> > > > > > >
> > > > > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
> > > > > > >
> > > > > > > DEBUG 2: Processing METAR recs = -9999
> > > > > > >
> > > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > > > > >
> > > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > > > > >
> > > > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > > > > >
> > > > > > > DEBUG 2: Rejected obs based on fill value = 0
> > > > > > >
> > > > > > > DEBUG 2: Rejected obs based on quality control = 0
> > > > > > >
> > > > > > > DEBUG 2: Total observations retained or derived = 0
> > > > > > >
> > > > > > > DEBUG 1: Writing MET File: out.nc
> > > > > > >
> > > > > > >
> > > > > > > However, the output file out.nc is empty (except for
some
> > > > metadata). I
> > > > > > > tried running the same command on a different MADIS file
that I
> > > took
> > > > > from
> > > > > > > Cheyenne (
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> > > > > > > and
> > > > > > > got the below message:
> > > > > > >
> > > > > > >
> > > > > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> > > > > > >
> > > > > > > DEBUG 2: Processing METAR recs = 9044
> > > > > > >
> > > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> > > > > > >
> > > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> > > > > > >
> > > > > > > DEBUG 2: Rejected recs based on masking station ID's = 0
> > > > > > >
> > > > > > > DEBUG 2: Rejected obs based on fill value = 79932
> > > > > > >
> > > > > > > DEBUG 2: Rejected obs based on quality control = 0
> > > > > > >
> > > > > > > DEBUG 2: Total observations retained or derived = 64060
> > > > > > >
> > > > > > > DEBUG 1: Writing MET File: out.nc
> > > > > > >
> > > > > > >
> > > > > > > I noticed the difference in the "Processing METAR recs”
line:
> > can
> > > > you
> > > > > > > advise what that means? I've attached the output from
running
> > > ncdump
> > > > -h
> > > > > > for
> > > > > > > each input file, if it helps.
> > > > > > >
> > > > > > >
> > > > > > > *point_stat*
> > > > > > >
> > > > > > > When I get to running point_stat with my WRF output
file, do I
> > need
> > > > to
> > > > > do
> > > > > > > any reformatting first? (e.g. does it matter if I use
GRIB vs
> > > NetCDF,
> > > > > > will
> > > > > > > I need to regrid from my Lambert projection to a lat-lon
> grid?).
> > > > > > >
> > > > > > >
> > > > > > > Thanks very much for any help!
> > > > > > >
> > > > > > >
> > > > > > > Best Regards,
> > > > > > >
> > > > > > > Callum Thompson
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Dr Callum Thompson
> > > > > > > Earth Research Institute
> > > > > > > Email: callum-thompson at ucsb.edu
> > > > > > > [image: UC Santa Barbara]
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > Dr Callum Thompson
> > > > Earth Research Institute
> > > > Email: callum-thompson at ucsb.edu
> > > > [image: UC Santa Barbara]
> > > >
> > > >
> > >
> > >
> >
> > --
> > Dr Callum Thompson
> > Earth Research Institute
> > Email: callum-thompson at ucsb.edu
> > [image: UC Santa Barbara]
> >
> >
>
>
--
Dr Callum Thompson
Earth Research Institute
Email: callum-thompson at ucsb.edu
[image: UC Santa Barbara]
------------------------------------------------
Subject: madis2nc and point_stat
From: Callum Thompson
Time: Fri Dec 18 11:26:21 2020
Hi Minna,
I checked with my colleague and I think I know where the problem is
now. He
gave me individual files for each hour and I used the 'cdo mergetime'
command to generate a single file.
However, the recNum variable isn't retained that way. I tried madis2nc
for
one of the individual file times and it worked fine :)
That said, is there a way to handle input data over multiple times? If
not,
I might still be able to use the cdo mergetime output file and try to
manually set the recNum variable.
Best,
Callum
On Fri, Dec 18, 2020 at 2:56 PM Callum Thompson <callum-
thompson at ucsb.edu>
wrote:
> Hello,
>
> Thanks for the info on UPP, John. I'll have a look into using it on
> Cheyenne once I have the madis2nc step sorted.
>
> It was a colleague who gave me the MADIS file, so I'll ask him how
he
> obtained it and get back to you.
>
> Best,
> Callum
>
>
>
>
> On Thu, Dec 17, 2020 at 11:05 PM Minna Win via RT
<met_help at ucar.edu>
> wrote:
>
>> Hi Callum,
>>
>> It looks like your input is missing the dimension variable,
'recNum',
>> which
>> is what the madis2nc tool is looking for. This explains why there
was an
>> error when attempting to calculate the number of records. Do you
know who
>> created your madis_20171001.nc file?
>>
>> Regards,
>> Minna
>> ---------------
>> Minna Win
>> National Center for Atmospheric Research
>> Developmental Testbed Center
>> Phone: 303-497-8423
>> Fax: 303-497-8401
>> ---------------
>> Pronouns: she/her
>>
>>
>> On Thu, Dec 17, 2020 at 11:35 AM Callum Thompson via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>> >
>> > Hi Minna,
>> >
>> > I just did it now. For reference, madis_20171001.nc is my data,
>> > metar_2012040912_F000.nc is the test data I took from Cheyenne.
>> >
>> > Thanks,
>> > Callum
>> >
>> >
>> >
>> > On Thu, Dec 17, 2020 at 6:15 PM Minna Win via RT
<met_help at ucar.edu>
>> > wrote:
>> >
>> > > Hi Callum,
>> > >
>> > > Can you please send me your NetCDF input files you used in
madis2nc?
>> > > Please follow the instructions at the bottom of this page:
>> > >
>> > >
>> > >
>> >
>> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
>> > >
>> > > I'd like to play around with some commands to trouble-shoot.
>> > >
>> > > Regards,
>> > > Minna
>> > > ---------------
>> > > Minna Win
>> > > National Center for Atmospheric Research
>> > > Developmental Testbed Center
>> > > Phone: 303-497-8423
>> > > Fax: 303-497-8401
>> > > ---------------
>> > > Pronouns: she/her
>> > >
>> > >
>> > > On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT <
>> > met_help at ucar.edu>
>> > > wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
>
>> > > >
>> > > > Hi John,
>> > > >
>> > > > This is really helpful and quick feedback, thanks!
>> > > >
>> > > > For the MADIS data, it sounds like there's a possible
>> > > > 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll
wait for
>> > > Minna
>> > > > to get back to me for that,
>> > > >
>> > > > For the WRF files, it is the winds (particularly surface
winds) that
>> > I'm
>> > > > looking to verify, so it sounds like UPP is the better
option.
>> However,
>> > > > I've never
>> > > > used it before. Is it already installed on Cheyenne/Casper,
or
>> would I
>> > > have
>> > > > to use it on my local machine?
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Callum
>> > > >
>> > > >
>> > > >
>> > > > On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
>> > > > met_help at ucar.edu>
>> > > > wrote:
>> > > >
>> > > > > Hi Callum,
>> > > > >
>> > > > > Wanted to jump in with some more information about
preparing your
>> WRF
>> > > > > output for Point-Stat.
>> > > > >
>> > > > > The Point-Stat tool takes 3 required inputs:
>> > > > > (1) Gridded model output data in any gridded data file
format
>> > supported
>> > > > by
>> > > > > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output
of the
>> > > > WRF-Input
>> > > > > utility).
>> > > > > (2) Point observations in NetCDF format... these must be
the
>> output
>> > of
>> > > > one
>> > > > > of MET's point pre-processing tools (ascii2nc, madis2nc,
pb2nc,
>> and
>> > so
>> > > > on).
>> > > > > (3) An ascii configuration file.
>> > > > >
>> > > > > It sounds like you're asking about (1), your gridded model
output.
>> > You
>> > > > have
>> > > > > 2 options:
>> > > > > (1) Run the wrf-interp post-processing utility on your WRF
output.
>> > > > > (2) Post-process your output using UPP (the Unified Post
>> Processor)
>> > > which
>> > > > > writes GRIB1 or GRIB2 output files.
>> > > > > http://www.dtcenter.org/community-code/unified-post-
processor-upp
>> > > > >
>> > > > > WRF-Interp does interpolate from native model levels to
pressure
>> > > levels,
>> > > > > which is good, but it does not de-stagger the winds. And
MET won't
>> > > > process
>> > > > > the winds on the staggered dimensions.
>> > > > >
>> > > > > So we recommend using UPP which destaggers the winds,
>> interpolates to
>> > > > > pressure levels, and optionally regrids the data.
>> > > > >
>> > > > > That being said, if you don't care about the winds and are
only
>> > > > interested
>> > > > > in surface fields, MET can actually read the WRF outptu
files
>> > directly.
>> > > > We
>> > > > > just need to use a config option to tell it to interpret
the input
>> > data
>> > > > as
>> > > > > being output from the wrf-interp utility.
>> > > > >
>> > > > > How would you like to proceed?
>> > > > >
>> > > > > Thanks,
>> > > > > John Halley Gotway
>> > > > >
>> > > > > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT <
>> met_help at ucar.edu>
>> > > > > wrote:
>> > > > >
>> > > > > >
>> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>> > > > > >
>> > > > > > Hello Callum,
>> > > > > >
>> > > > > > It looks like you are having issues with your input MADIS
data.
>> > The
>> > > > > debug
>> > > > > > message of "Processing METAR recs = -9999" indicates that
there
>> is
>> > > some
>> > > > > > missing data (-9999 is used in MET to indicate missing
and fill
>> > > > values).
>> > > > > > This debug message reports the difference between the
first and
>> > > second
>> > > > > > record number to indicate how many records were
processed. The
>> > only
>> > > > > > difference I see between the two txt files you provided
is the
>> > > presence
>> > > > > of
>> > > > > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the
madis
>> > input.
>> > > > > I'll
>> > > > > > need to ask someone who is more familiar with MADIS data
to see
>> if
>> > > this
>> > > > > is
>> > > > > > the cause of your issues.
>> > > > > >
>> > > > > > As far as your point-stat data the only requirement is
that your
>> > > point
>> > > > > data
>> > > > > > is in NetCDF format.
>> > > > > >
>> > > > > > Regards,
>> > > > > > Minna
>> > > > > > ---------------
>> > > > > > Minna Win
>> > > > > > National Center for Atmospheric Research
>> > > > > > Developmental Testbed Center
>> > > > > > Phone: 303-497-8423
>> > > > > > Fax: 303-497-8401
>> > > > > > ---------------
>> > > > > > Pronouns: she/her
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT <
>> > > > > met_help at ucar.edu>
>> > > > > > wrote:
>> > > > > >
>> > > > > > >
>> > > > > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted upon.
>> > > > > > > Transaction: Ticket created by callum-thompson at ucsb.edu
>> > > > > > > Queue: met_help
>> > > > > > > Subject: madis2nc and point_stat
>> > > > > > > Owner: Nobody
>> > > > > > > Requestors: callum-thompson at ucsb.edu
>> > > > > > > Status: new
>> > > > > > > Ticket <URL:
>> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
>> > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > Hello,
>> > > > > > >
>> > > > > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm
trying to
>> > > verify
>> > > > a
>> > > > > > WRF
>> > > > > > > simulation against some MADIS metar observations. I
have two
>> > > > questions
>> > > > > > > about the madis2nc function and the point_stat
function, if
>> you
>> > > could
>> > > > > > help
>> > > > > > > please?
>> > > > > > >
>> > > > > > >
>> > > > > > > *madis2nc*
>> > > > > > >
>> > > > > > > When I run the command “madis2nc madis_20171001.nc
out.nc
>> -type
>> > > > metar
>> > > > > -v
>> > > > > > > 2” it
>> > > > > > > executes with the following message:
>> > > > > > >
>> > > > > > >
>> > > > > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
>> > > > > > >
>> > > > > > > DEBUG 2: Processing METAR recs = -9999
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected recs based on masking grid = 0
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected recs based on masking poly = 0
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected recs based on masking station ID's =
0
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected obs based on fill value = 0
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected obs based on quality control = 0
>> > > > > > >
>> > > > > > > DEBUG 2: Total observations retained or derived = 0
>> > > > > > >
>> > > > > > > DEBUG 1: Writing MET File: out.nc
>> > > > > > >
>> > > > > > >
>> > > > > > > However, the output file out.nc is empty (except for
some
>> > > > metadata). I
>> > > > > > > tried running the same command on a different MADIS
file that
>> I
>> > > took
>> > > > > from
>> > > > > > > Cheyenne (
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
>> > > > > > > and
>> > > > > > > got the below message:
>> > > > > > >
>> > > > > > >
>> > > > > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
>> > > > > > >
>> > > > > > > DEBUG 2: Processing METAR recs = 9044
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected recs based on masking grid = 0
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected recs based on masking poly = 0
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected recs based on masking station ID's =
0
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected obs based on fill value = 79932
>> > > > > > >
>> > > > > > > DEBUG 2: Rejected obs based on quality control = 0
>> > > > > > >
>> > > > > > > DEBUG 2: Total observations retained or derived = 64060
>> > > > > > >
>> > > > > > > DEBUG 1: Writing MET File: out.nc
>> > > > > > >
>> > > > > > >
>> > > > > > > I noticed the difference in the "Processing METAR
recs” line:
>> > can
>> > > > you
>> > > > > > > advise what that means? I've attached the output from
running
>> > > ncdump
>> > > > -h
>> > > > > > for
>> > > > > > > each input file, if it helps.
>> > > > > > >
>> > > > > > >
>> > > > > > > *point_stat*
>> > > > > > >
>> > > > > > > When I get to running point_stat with my WRF output
file, do I
>> > need
>> > > > to
>> > > > > do
>> > > > > > > any reformatting first? (e.g. does it matter if I use
GRIB vs
>> > > NetCDF,
>> > > > > > will
>> > > > > > > I need to regrid from my Lambert projection to a lat-
lon
>> grid?).
>> > > > > > >
>> > > > > > >
>> > > > > > > Thanks very much for any help!
>> > > > > > >
>> > > > > > >
>> > > > > > > Best Regards,
>> > > > > > >
>> > > > > > > Callum Thompson
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Dr Callum Thompson
>> > > > > > > Earth Research Institute
>> > > > > > > Email: callum-thompson at ucsb.edu
>> > > > > > > [image: UC Santa Barbara]
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > > --
>> > > > Dr Callum Thompson
>> > > > Earth Research Institute
>> > > > Email: callum-thompson at ucsb.edu
>> > > > [image: UC Santa Barbara]
>> > > >
>> > > >
>> > >
>> > >
>> >
>> > --
>> > Dr Callum Thompson
>> > Earth Research Institute
>> > Email: callum-thompson at ucsb.edu
>> > [image: UC Santa Barbara]
>> >
>> >
>>
>>
>
> --
> Dr Callum Thompson
> Earth Research Institute
> Email: callum-thompson at ucsb.edu
> [image: UC Santa Barbara]
>
--
Dr Callum Thompson
Earth Research Institute
Email: callum-thompson at ucsb.edu
[image: UC Santa Barbara]
------------------------------------------------
Subject: madis2nc and point_stat
From: John Halley Gotway
Time: Fri Dec 18 12:04:54 2020
Callum,
Listed below is the usage statement for madis2nc. Note that you can
tell it
to process multiple madis input files in a single call. Please try
processing them that way instead of using cdo to preprocess them
first.
Thanks,
John
Usage: madis2nc
madis_file [madis_file2 ... madis_filen]
out_file
-type str
[-config file]
[-qc_dd list]
[-lvl_dim list]
[-rec_beg n]
[-rec_end n]
[-mask_grid string]
[-mask_poly file]
[-mask_sid file|list]
[-log file]
[-v level]
[-compress level]
On Fri, Dec 18, 2020 at 11:26 AM Callum Thompson via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>
> Hi Minna,
>
> I checked with my colleague and I think I know where the problem is
now. He
> gave me individual files for each hour and I used the 'cdo
mergetime'
> command to generate a single file.
> However, the recNum variable isn't retained that way. I tried
madis2nc for
> one of the individual file times and it worked fine :)
>
> That said, is there a way to handle input data over multiple times?
If not,
> I might still be able to use the cdo mergetime output file and try
to
> manually set the recNum variable.
>
> Best,
>
> Callum
>
>
> On Fri, Dec 18, 2020 at 2:56 PM Callum Thompson <callum-
thompson at ucsb.edu>
> wrote:
>
> > Hello,
> >
> > Thanks for the info on UPP, John. I'll have a look into using it
on
> > Cheyenne once I have the madis2nc step sorted.
> >
> > It was a colleague who gave me the MADIS file, so I'll ask him how
he
> > obtained it and get back to you.
> >
> > Best,
> > Callum
> >
> >
> >
> >
> > On Thu, Dec 17, 2020 at 11:05 PM Minna Win via RT
<met_help at ucar.edu>
> > wrote:
> >
> >> Hi Callum,
> >>
> >> It looks like your input is missing the dimension variable,
'recNum',
> >> which
> >> is what the madis2nc tool is looking for. This explains why
there was
> an
> >> error when attempting to calculate the number of records. Do you
know
> who
> >> created your madis_20171001.nc file?
> >>
> >> Regards,
> >> Minna
> >> ---------------
> >> Minna Win
> >> National Center for Atmospheric Research
> >> Developmental Testbed Center
> >> Phone: 303-497-8423
> >> Fax: 303-497-8401
> >> ---------------
> >> Pronouns: she/her
> >>
> >>
> >> On Thu, Dec 17, 2020 at 11:35 AM Callum Thompson via RT <
> >> met_help at ucar.edu>
> >> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> >> >
> >> > Hi Minna,
> >> >
> >> > I just did it now. For reference, madis_20171001.nc is my data,
> >> > metar_2012040912_F000.nc is the test data I took from Cheyenne.
> >> >
> >> > Thanks,
> >> > Callum
> >> >
> >> >
> >> >
> >> > On Thu, Dec 17, 2020 at 6:15 PM Minna Win via RT
<met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Hi Callum,
> >> > >
> >> > > Can you please send me your NetCDF input files you used in
madis2nc?
> >> > > Please follow the instructions at the bottom of this page:
> >> > >
> >> > >
> >> > >
> >> >
> >>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> >> > >
> >> > > I'd like to play around with some commands to trouble-shoot.
> >> > >
> >> > > Regards,
> >> > > Minna
> >> > > ---------------
> >> > > Minna Win
> >> > > National Center for Atmospheric Research
> >> > > Developmental Testbed Center
> >> > > Phone: 303-497-8423
> >> > > Fax: 303-497-8401
> >> > > ---------------
> >> > > Pronouns: she/her
> >> > >
> >> > >
> >> > > On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT <
> >> > met_help at ucar.edu>
> >> > > wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> >> > > >
> >> > > > Hi John,
> >> > > >
> >> > > > This is really helpful and quick feedback, thanks!
> >> > > >
> >> > > > For the MADIS data, it sounds like there's a possible
> >> > > > 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll
wait
> for
> >> > > Minna
> >> > > > to get back to me for that,
> >> > > >
> >> > > > For the WRF files, it is the winds (particularly surface
winds)
> that
> >> > I'm
> >> > > > looking to verify, so it sounds like UPP is the better
option.
> >> However,
> >> > > > I've never
> >> > > > used it before. Is it already installed on Cheyenne/Casper,
or
> >> would I
> >> > > have
> >> > > > to use it on my local machine?
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Callum
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
> >> > > > met_help at ucar.edu>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Callum,
> >> > > > >
> >> > > > > Wanted to jump in with some more information about
preparing
> your
> >> WRF
> >> > > > > output for Point-Stat.
> >> > > > >
> >> > > > > The Point-Stat tool takes 3 required inputs:
> >> > > > > (1) Gridded model output data in any gridded data file
format
> >> > supported
> >> > > > by
> >> > > > > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output
of the
> >> > > > WRF-Input
> >> > > > > utility).
> >> > > > > (2) Point observations in NetCDF format... these must be
the
> >> output
> >> > of
> >> > > > one
> >> > > > > of MET's point pre-processing tools (ascii2nc, madis2nc,
pb2nc,
> >> and
> >> > so
> >> > > > on).
> >> > > > > (3) An ascii configuration file.
> >> > > > >
> >> > > > > It sounds like you're asking about (1), your gridded
model
> output.
> >> > You
> >> > > > have
> >> > > > > 2 options:
> >> > > > > (1) Run the wrf-interp post-processing utility on your
WRF
> output.
> >> > > > > (2) Post-process your output using UPP (the Unified Post
> >> Processor)
> >> > > which
> >> > > > > writes GRIB1 or GRIB2 output files.
> >> > > > >
> http://www.dtcenter.org/community-code/unified-post-processor-upp
> >> > > > >
> >> > > > > WRF-Interp does interpolate from native model levels to
pressure
> >> > > levels,
> >> > > > > which is good, but it does not de-stagger the winds. And
MET
> won't
> >> > > > process
> >> > > > > the winds on the staggered dimensions.
> >> > > > >
> >> > > > > So we recommend using UPP which destaggers the winds,
> >> interpolates to
> >> > > > > pressure levels, and optionally regrids the data.
> >> > > > >
> >> > > > > That being said, if you don't care about the winds and
are only
> >> > > > interested
> >> > > > > in surface fields, MET can actually read the WRF outptu
files
> >> > directly.
> >> > > > We
> >> > > > > just need to use a config option to tell it to interpret
the
> input
> >> > data
> >> > > > as
> >> > > > > being output from the wrf-interp utility.
> >> > > > >
> >> > > > > How would you like to proceed?
> >> > > > >
> >> > > > > Thanks,
> >> > > > > John Halley Gotway
> >> > > > >
> >> > > > > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT <
> >> met_help at ucar.edu>
> >> > > > > wrote:
> >> > > > >
> >> > > > > >
> >> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> >
> >> > > > > >
> >> > > > > > Hello Callum,
> >> > > > > >
> >> > > > > > It looks like you are having issues with your input
MADIS
> data.
> >> > The
> >> > > > > debug
> >> > > > > > message of "Processing METAR recs = -9999" indicates
that
> there
> >> is
> >> > > some
> >> > > > > > missing data (-9999 is used in MET to indicate missing
and
> fill
> >> > > > values).
> >> > > > > > This debug message reports the difference between the
first
> and
> >> > > second
> >> > > > > > record number to indicate how many records were
processed.
> The
> >> > only
> >> > > > > > difference I see between the two txt files you provided
is the
> >> > > presence
> >> > > > > of
> >> > > > > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the
madis
> >> > input.
> >> > > > > I'll
> >> > > > > > need to ask someone who is more familiar with MADIS
data to
> see
> >> if
> >> > > this
> >> > > > > is
> >> > > > > > the cause of your issues.
> >> > > > > >
> >> > > > > > As far as your point-stat data the only requirement is
that
> your
> >> > > point
> >> > > > > data
> >> > > > > > is in NetCDF format.
> >> > > > > >
> >> > > > > > Regards,
> >> > > > > > Minna
> >> > > > > > ---------------
> >> > > > > > Minna Win
> >> > > > > > National Center for Atmospheric Research
> >> > > > > > Developmental Testbed Center
> >> > > > > > Phone: 303-497-8423
> >> > > > > > Fax: 303-497-8401
> >> > > > > > ---------------
> >> > > > > > Pronouns: she/her
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT
<
> >> > > > > met_help at ucar.edu>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > >
> >> > > > > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted
upon.
> >> > > > > > > Transaction: Ticket created by callum-
thompson at ucsb.edu
> >> > > > > > > Queue: met_help
> >> > > > > > > Subject: madis2nc and point_stat
> >> > > > > > > Owner: Nobody
> >> > > > > > > Requestors: callum-thompson at ucsb.edu
> >> > > > > > > Status: new
> >> > > > > > > Ticket <URL:
> >> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> >> > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Hello,
> >> > > > > > >
> >> > > > > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm
trying
> to
> >> > > verify
> >> > > > a
> >> > > > > > WRF
> >> > > > > > > simulation against some MADIS metar observations. I
have two
> >> > > > questions
> >> > > > > > > about the madis2nc function and the point_stat
function, if
> >> you
> >> > > could
> >> > > > > > help
> >> > > > > > > please?
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > *madis2nc*
> >> > > > > > >
> >> > > > > > > When I run the command “madis2nc madis_20171001.nc
out.nc
> >> -type
> >> > > > metar
> >> > > > > -v
> >> > > > > > > 2” it
> >> > > > > > > executes with the following message:
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
> >> > > > > > >
> >> > > > > > > DEBUG 2: Processing METAR recs = -9999
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking station ID's
= 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected obs based on fill value = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected obs based on quality control = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Total observations retained or derived = 0
> >> > > > > > >
> >> > > > > > > DEBUG 1: Writing MET File: out.nc
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > However, the output file out.nc is empty (except for
some
> >> > > > metadata). I
> >> > > > > > > tried running the same command on a different MADIS
file
> that
> >> I
> >> > > took
> >> > > > > from
> >> > > > > > > Cheyenne (
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> >> > > > > > > and
> >> > > > > > > got the below message:
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> >> > > > > > >
> >> > > > > > > DEBUG 2: Processing METAR recs = 9044
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking station ID's
= 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected obs based on fill value = 79932
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected obs based on quality control = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Total observations retained or derived =
64060
> >> > > > > > >
> >> > > > > > > DEBUG 1: Writing MET File: out.nc
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > I noticed the difference in the "Processing METAR
recs”
> line:
> >> > can
> >> > > > you
> >> > > > > > > advise what that means? I've attached the output from
> running
> >> > > ncdump
> >> > > > -h
> >> > > > > > for
> >> > > > > > > each input file, if it helps.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > *point_stat*
> >> > > > > > >
> >> > > > > > > When I get to running point_stat with my WRF output
file,
> do I
> >> > need
> >> > > > to
> >> > > > > do
> >> > > > > > > any reformatting first? (e.g. does it matter if I use
GRIB
> vs
> >> > > NetCDF,
> >> > > > > > will
> >> > > > > > > I need to regrid from my Lambert projection to a lat-
lon
> >> grid?).
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Thanks very much for any help!
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Best Regards,
> >> > > > > > >
> >> > > > > > > Callum Thompson
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > Dr Callum Thompson
> >> > > > > > > Earth Research Institute
> >> > > > > > > Email: callum-thompson at ucsb.edu
> >> > > > > > > [image: UC Santa Barbara]
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > > > --
> >> > > > Dr Callum Thompson
> >> > > > Earth Research Institute
> >> > > > Email: callum-thompson at ucsb.edu
> >> > > > [image: UC Santa Barbara]
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> > --
> >> > Dr Callum Thompson
> >> > Earth Research Institute
> >> > Email: callum-thompson at ucsb.edu
> >> > [image: UC Santa Barbara]
> >> >
> >> >
> >>
> >>
> >
> > --
> > Dr Callum Thompson
> > Earth Research Institute
> > Email: callum-thompson at ucsb.edu
> > [image: UC Santa Barbara]
> >
>
>
> --
> Dr Callum Thompson
> Earth Research Institute
> Email: callum-thompson at ucsb.edu
> [image: UC Santa Barbara]
>
>
------------------------------------------------
Subject: madis2nc and point_stat
From: Minna Win
Time: Fri Dec 18 13:27:06 2020
Hi Callum,
If we had a wrapper to the madis2nc MET tool like we have for ascii2nc
MET
tool:
https://dtcenter.github.io/METplus/latest/generated/met_tool_wrapper/ASCII2NC/ASCII2NC.html#sphx-
glr-generated-met-tool-wrapper-ascii2nc-ascii2nc-py
I would suggest you use that. Unfortunately, we don't have a wrapper
for
the madis2nc tool yet. I can create a Github issue to create one, but
it
might not be ready for your time frame.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
Pronouns: she/her
On Fri, Dec 18, 2020 at 11:26 AM Callum Thompson via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
>
> Hi Minna,
>
> I checked with my colleague and I think I know where the problem is
now. He
> gave me individual files for each hour and I used the 'cdo
mergetime'
> command to generate a single file.
> However, the recNum variable isn't retained that way. I tried
madis2nc for
> one of the individual file times and it worked fine :)
>
> That said, is there a way to handle input data over multiple times?
If not,
> I might still be able to use the cdo mergetime output file and try
to
> manually set the recNum variable.
>
> Best,
>
> Callum
>
>
> On Fri, Dec 18, 2020 at 2:56 PM Callum Thompson <callum-
thompson at ucsb.edu>
> wrote:
>
> > Hello,
> >
> > Thanks for the info on UPP, John. I'll have a look into using it
on
> > Cheyenne once I have the madis2nc step sorted.
> >
> > It was a colleague who gave me the MADIS file, so I'll ask him how
he
> > obtained it and get back to you.
> >
> > Best,
> > Callum
> >
> >
> >
> >
> > On Thu, Dec 17, 2020 at 11:05 PM Minna Win via RT
<met_help at ucar.edu>
> > wrote:
> >
> >> Hi Callum,
> >>
> >> It looks like your input is missing the dimension variable,
'recNum',
> >> which
> >> is what the madis2nc tool is looking for. This explains why
there was
> an
> >> error when attempting to calculate the number of records. Do you
know
> who
> >> created your madis_20171001.nc file?
> >>
> >> Regards,
> >> Minna
> >> ---------------
> >> Minna Win
> >> National Center for Atmospheric Research
> >> Developmental Testbed Center
> >> Phone: 303-497-8423
> >> Fax: 303-497-8401
> >> ---------------
> >> Pronouns: she/her
> >>
> >>
> >> On Thu, Dec 17, 2020 at 11:35 AM Callum Thompson via RT <
> >> met_help at ucar.edu>
> >> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> >> >
> >> > Hi Minna,
> >> >
> >> > I just did it now. For reference, madis_20171001.nc is my data,
> >> > metar_2012040912_F000.nc is the test data I took from Cheyenne.
> >> >
> >> > Thanks,
> >> > Callum
> >> >
> >> >
> >> >
> >> > On Thu, Dec 17, 2020 at 6:15 PM Minna Win via RT
<met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Hi Callum,
> >> > >
> >> > > Can you please send me your NetCDF input files you used in
madis2nc?
> >> > > Please follow the instructions at the bottom of this page:
> >> > >
> >> > >
> >> > >
> >> >
> >>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> >> > >
> >> > > I'd like to play around with some commands to trouble-shoot.
> >> > >
> >> > > Regards,
> >> > > Minna
> >> > > ---------------
> >> > > Minna Win
> >> > > National Center for Atmospheric Research
> >> > > Developmental Testbed Center
> >> > > Phone: 303-497-8423
> >> > > Fax: 303-497-8401
> >> > > ---------------
> >> > > Pronouns: she/her
> >> > >
> >> > >
> >> > > On Thu, Dec 17, 2020 at 9:46 AM Callum Thompson via RT <
> >> > met_help at ucar.edu>
> >> > > wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919 >
> >> > > >
> >> > > > Hi John,
> >> > > >
> >> > > > This is really helpful and quick feedback, thanks!
> >> > > >
> >> > > > For the MADIS data, it sounds like there's a possible
> >> > > > 'var_xyz:missing_value' or 'var_xyz:fill_value' issue. I'll
wait
> for
> >> > > Minna
> >> > > > to get back to me for that,
> >> > > >
> >> > > > For the WRF files, it is the winds (particularly surface
winds)
> that
> >> > I'm
> >> > > > looking to verify, so it sounds like UPP is the better
option.
> >> However,
> >> > > > I've never
> >> > > > used it before. Is it already installed on Cheyenne/Casper,
or
> >> would I
> >> > > have
> >> > > > to use it on my local machine?
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Callum
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Thu, Dec 17, 2020 at 4:25 PM John Halley Gotway via RT <
> >> > > > met_help at ucar.edu>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Callum,
> >> > > > >
> >> > > > > Wanted to jump in with some more information about
preparing
> your
> >> WRF
> >> > > > > output for Point-Stat.
> >> > > > >
> >> > > > > The Point-Stat tool takes 3 required inputs:
> >> > > > > (1) Gridded model output data in any gridded data file
format
> >> > supported
> >> > > > by
> >> > > > > MET (GRIB1, GRIB2, CF-compliant NetCDF, the NetCDF output
of the
> >> > > > WRF-Input
> >> > > > > utility).
> >> > > > > (2) Point observations in NetCDF format... these must be
the
> >> output
> >> > of
> >> > > > one
> >> > > > > of MET's point pre-processing tools (ascii2nc, madis2nc,
pb2nc,
> >> and
> >> > so
> >> > > > on).
> >> > > > > (3) An ascii configuration file.
> >> > > > >
> >> > > > > It sounds like you're asking about (1), your gridded
model
> output.
> >> > You
> >> > > > have
> >> > > > > 2 options:
> >> > > > > (1) Run the wrf-interp post-processing utility on your
WRF
> output.
> >> > > > > (2) Post-process your output using UPP (the Unified Post
> >> Processor)
> >> > > which
> >> > > > > writes GRIB1 or GRIB2 output files.
> >> > > > >
> http://www.dtcenter.org/community-code/unified-post-processor-upp
> >> > > > >
> >> > > > > WRF-Interp does interpolate from native model levels to
pressure
> >> > > levels,
> >> > > > > which is good, but it does not de-stagger the winds. And
MET
> won't
> >> > > > process
> >> > > > > the winds on the staggered dimensions.
> >> > > > >
> >> > > > > So we recommend using UPP which destaggers the winds,
> >> interpolates to
> >> > > > > pressure levels, and optionally regrids the data.
> >> > > > >
> >> > > > > That being said, if you don't care about the winds and
are only
> >> > > > interested
> >> > > > > in surface fields, MET can actually read the WRF outptu
files
> >> > directly.
> >> > > > We
> >> > > > > just need to use a config option to tell it to interpret
the
> input
> >> > data
> >> > > > as
> >> > > > > being output from the wrf-interp utility.
> >> > > > >
> >> > > > > How would you like to proceed?
> >> > > > >
> >> > > > > Thanks,
> >> > > > > John Halley Gotway
> >> > > > >
> >> > > > > On Thu, Dec 17, 2020 at 8:59 AM Minna Win via RT <
> >> met_help at ucar.edu>
> >> > > > > wrote:
> >> > > > >
> >> > > > > >
> >> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> >
> >> > > > > >
> >> > > > > > Hello Callum,
> >> > > > > >
> >> > > > > > It looks like you are having issues with your input
MADIS
> data.
> >> > The
> >> > > > > debug
> >> > > > > > message of "Processing METAR recs = -9999" indicates
that
> there
> >> is
> >> > > some
> >> > > > > > missing data (-9999 is used in MET to indicate missing
and
> fill
> >> > > > values).
> >> > > > > > This debug message reports the difference between the
first
> and
> >> > > second
> >> > > > > > record number to indicate how many records were
processed.
> The
> >> > only
> >> > > > > > difference I see between the two txt files you provided
is the
> >> > > presence
> >> > > > > of
> >> > > > > > 'var_xyz:missing_value' and 'var_xyz:fill_value' in the
madis
> >> > input.
> >> > > > > I'll
> >> > > > > > need to ask someone who is more familiar with MADIS
data to
> see
> >> if
> >> > > this
> >> > > > > is
> >> > > > > > the cause of your issues.
> >> > > > > >
> >> > > > > > As far as your point-stat data the only requirement is
that
> your
> >> > > point
> >> > > > > data
> >> > > > > > is in NetCDF format.
> >> > > > > >
> >> > > > > > Regards,
> >> > > > > > Minna
> >> > > > > > ---------------
> >> > > > > > Minna Win
> >> > > > > > National Center for Atmospheric Research
> >> > > > > > Developmental Testbed Center
> >> > > > > > Phone: 303-497-8423
> >> > > > > > Fax: 303-497-8401
> >> > > > > > ---------------
> >> > > > > > Pronouns: she/her
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu, Dec 17, 2020 at 5:51 AM Callum Thompson via RT
<
> >> > > > > met_help at ucar.edu>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > >
> >> > > > > > > Thu Dec 17 05:51:16 2020: Request 97919 was acted
upon.
> >> > > > > > > Transaction: Ticket created by callum-
thompson at ucsb.edu
> >> > > > > > > Queue: met_help
> >> > > > > > > Subject: madis2nc and point_stat
> >> > > > > > > Owner: Nobody
> >> > > > > > > Requestors: callum-thompson at ucsb.edu
> >> > > > > > > Status: new
> >> > > > > > > Ticket <URL:
> >> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97919
> >> > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Hello,
> >> > > > > > >
> >> > > > > > > I'm a new user to MET (v.9.0.3 on Cheyenne) and I'm
trying
> to
> >> > > verify
> >> > > > a
> >> > > > > > WRF
> >> > > > > > > simulation against some MADIS metar observations. I
have two
> >> > > > questions
> >> > > > > > > about the madis2nc function and the point_stat
function, if
> >> you
> >> > > could
> >> > > > > > help
> >> > > > > > > please?
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > *madis2nc*
> >> > > > > > >
> >> > > > > > > When I run the command “madis2nc madis_20171001.nc
out.nc
> >> -type
> >> > > > metar
> >> > > > > -v
> >> > > > > > > 2” it
> >> > > > > > > executes with the following message:
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > DEBUG 1: Reading MADIS File: madis_20171001.nc
> >> > > > > > >
> >> > > > > > > DEBUG 2: Processing METAR recs = -9999
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking station ID's
= 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected obs based on fill value = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected obs based on quality control = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Total observations retained or derived = 0
> >> > > > > > >
> >> > > > > > > DEBUG 1: Writing MET File: out.nc
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > However, the output file out.nc is empty (except for
some
> >> > > > metadata). I
> >> > > > > > > tried running the same command on a different MADIS
file
> that
> >> I
> >> > > took
> >> > > > > from
> >> > > > > > > Cheyenne (
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/glade/p/ral/jntp/MET/MET_Tutorial/8.0/MET_Tutorial/input/sample_obs/madis/metar/metar_2012040912_F000.nc)
> >> > > > > > > and
> >> > > > > > > got the below message:
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > DEBUG 1: Reading MADIS File: metar_2012040912_F000.nc
> >> > > > > > >
> >> > > > > > > DEBUG 2: Processing METAR recs = 9044
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking grid = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking poly = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected recs based on masking station ID's
= 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected obs based on fill value = 79932
> >> > > > > > >
> >> > > > > > > DEBUG 2: Rejected obs based on quality control = 0
> >> > > > > > >
> >> > > > > > > DEBUG 2: Total observations retained or derived =
64060
> >> > > > > > >
> >> > > > > > > DEBUG 1: Writing MET File: out.nc
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > I noticed the difference in the "Processing METAR
recs”
> line:
> >> > can
> >> > > > you
> >> > > > > > > advise what that means? I've attached the output from
> running
> >> > > ncdump
> >> > > > -h
> >> > > > > > for
> >> > > > > > > each input file, if it helps.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > *point_stat*
> >> > > > > > >
> >> > > > > > > When I get to running point_stat with my WRF output
file,
> do I
> >> > need
> >> > > > to
> >> > > > > do
> >> > > > > > > any reformatting first? (e.g. does it matter if I use
GRIB
> vs
> >> > > NetCDF,
> >> > > > > > will
> >> > > > > > > I need to regrid from my Lambert projection to a lat-
lon
> >> grid?).
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Thanks very much for any help!
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Best Regards,
> >> > > > > > >
> >> > > > > > > Callum Thompson
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > Dr Callum Thompson
> >> > > > > > > Earth Research Institute
> >> > > > > > > Email: callum-thompson at ucsb.edu
> >> > > > > > > [image: UC Santa Barbara]
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > > > --
> >> > > > Dr Callum Thompson
> >> > > > Earth Research Institute
> >> > > > Email: callum-thompson at ucsb.edu
> >> > > > [image: UC Santa Barbara]
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> > --
> >> > Dr Callum Thompson
> >> > Earth Research Institute
> >> > Email: callum-thompson at ucsb.edu
> >> > [image: UC Santa Barbara]
> >> >
> >> >
> >>
> >>
> >
> > --
> > Dr Callum Thompson
> > Earth Research Institute
> > Email: callum-thompson at ucsb.edu
> > [image: UC Santa Barbara]
> >
>
>
> --
> Dr Callum Thompson
> Earth Research Institute
> Email: callum-thompson at ucsb.edu
> [image: UC Santa Barbara]
>
>
------------------------------------------------
More information about the Met_help
mailing list