[Met_help] [rt.rap.ucar.edu #49485] History for Ensemble Stat
RAL HelpDesk {for John Halley Gotway}
met_help at ucar.edu
Tue Sep 13 14:42:43 MDT 2011
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Good afternoon,
Brian E., over at NOAA. I have run into two odd errors with ensemble_stat,
that have to do with reading in the ensemble data.
Error #1 comes when I try to read in raw SREF ensemble data. Well, not
exactly raw, but GRIB2 files that I converted into GRIB1 using "cnvgrib".
The error I get is this:
ERROR: WrfData::two_to_one() -> range check error: (nx, ny) = (185, 129),
(x, y) = (0, 129)
Which seems to indicate that something is getting lost in translation.
Error #2 comes when I try to use my bias-corrected and/or downscaled
ensembles. The error I get is this:
Memory fault(coredump)
Which I attribute to me using copygb to put the ensemble on the 5km RTMA
grid, and thus, my ensembles have 1079x768, which I am thinking is too darn
big.
Any insights onto these errors would be appreciated.
As a side note: I am presently trying this using "grid_obs", the 5km RTMA
files. I hope to use surface obs too, and think that files of the sort
"gdas1.t12z.adpsfc.tm00.bufr_d.nr", downloaded from
http://nomads.ncdc.noaa.gov/data/gdas/201106/20110609/ will do the trick.
Regards,
Brian Etherton
Forecast Applications Branch
NOAA/OAR/ESRL/GSD
Phone: 303-497-6735, Fax: 303-497-7262
325 Broadway, R/GSD7, Boulder, CO 80305
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #49485] Ensemble Stat
From: John Halley Gotway
Time: Fri Sep 09 14:22:52 2011
Hey Brian,
Can you either send me some data that demonstrates the two errors
you're seeing (see
http://www.dtcenter.org/met/users/support/met_help.php#ftp for
instructions for posting to our anonymous ftp site)
- or point me to the data on jet?
Also, it'd be helpful to see the exact command line and config file
you're using. I'll need to try to replicate the problem(s) to figure
out what's going on.
Thanks,
John
On 09/09/2011 02:14 PM, RAL HelpDesk {for Brian Etherton} wrote:
>
> Fri Sep 09 14:14:13 2011: Request 49485 was acted upon.
> Transaction: Ticket created by Brian.Etherton at noaa.gov
> Queue: met_help
> Subject: Ensemble Stat
> Owner: Nobody
> Requestors: Brian.Etherton at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=49485 >
>
>
> Good afternoon,
>
> Brian E., over at NOAA. I have run into two odd errors with
ensemble_stat,
> that have to do with reading in the ensemble data.
>
> Error #1 comes when I try to read in raw SREF ensemble data. Well,
not
> exactly raw, but GRIB2 files that I converted into GRIB1 using
"cnvgrib".
> The error I get is this:
> ERROR: WrfData::two_to_one() -> range check error: (nx, ny) = (185,
129),
> (x, y) = (0, 129)
> Which seems to indicate that something is getting lost in
translation.
>
> Error #2 comes when I try to use my bias-corrected and/or downscaled
> ensembles. The error I get is this:
> Memory fault(coredump)
> Which I attribute to me using copygb to put the ensemble on the 5km
RTMA
> grid, and thus, my ensembles have 1079x768, which I am thinking is
too darn
> big.
>
> Any insights onto these errors would be appreciated.
>
> As a side note: I am presently trying this using "grid_obs", the 5km
RTMA
> files. I hope to use surface obs too, and think that files of the
sort
> "gdas1.t12z.adpsfc.tm00.bufr_d.nr", downloaded from
> http://nomads.ncdc.noaa.gov/data/gdas/201106/20110609/ will do the
trick.
>
> Regards,
> Brian Etherton
> Forecast Applications Branch
> NOAA/OAR/ESRL/GSD
> Phone: 303-497-6735, Fax: 303-497-7262
> 325 Broadway, R/GSD7, Boulder, CO 80305
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #49485] Ensemble Stat
From: Brian Etherton
Time: Fri Sep 09 15:12:54 2011
Good afternoon John,
Side note: nice fall day, don't you think?
On the the issues:
I am running these things in this directory, on jet:
/lfs0/projects/det/sref-emc/biasdown
I am using the script "evaluate.sh" to make these calls to
ensemble_stat.
The file "met-config-ens.txt" as my config file.
To just do the simple (raw) ensemble, my command is:
./bin/ensemble_stat 10 sref/sref.20110609/09/pgrb/sref*[en]m*15
met-config-ens.txt -grid_obs rtma.X -outdir ./ -v 3 &> testing_raw.log
To do the bias-corrected ensemble, my command is:
./bin/ensemble_stat 10 sref/sref.20110609/09/pgrb/sref_bc*[en]m*15*5km
met-config-ens.txt -grid_obs rtma.X -outdir ./ -v 3 &> testing_bc.log
Interestingly... It looks like the bc stuff works fine, when not in my
script. Hmm...
Also, from the command line, I get a different error in the 'raw'
case:
config() -> syntax error in file
"sref/sref.20110609/09/pgrb/sref_bc_t09z.pgrb212_em.ctl.f15"
More digging needed!
Thanks for the help,
Brian E.
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #49485] Ensemble Stat
From: John Halley Gotway
Time: Fri Sep 09 15:54:48 2011
Brian,
Before I begin, I think your "evaluate.sh" script has issues. On line
36, where you tell ensemble_stat that there are 10 members, there
really are 30:
[johnhg at fe3 etherton_data_20110909]$ ls
sref/sref.20110609/09/pgrb/sref*[en]m*15 | wc -w
30
So you should look more closely how you're selecting the 10 members
you want to use.
The first issue is caused by the fact that the forecast and
observations are not on the same grid:
wgrib -d 1 -V /lfs0/projects/det/sref-
emc/vertical/sref/sref.20110609/09/pgrb/sref_bc_t09z.pgrb212.em.n2.f15
Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov -95.000000
Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP 0.000000
North Pole (185 x 129) Dx 40.635000 Dy 40.635000 scan 64 mode 8
wgrib -d 1 -V /lfs0/projects/det/sref-emc/biasdown/rtma.X
Lambert Conf: Lat1 20.191000 Lon1 238.445000 Lov 265.000000
Latin1 25.000000 Latin2 25.000000 LatSP -8364.400000 LonSP
0.000000
North Pole (1073 x 689) Dx 5.079000 Dy 5.079000 scan 64 mode 8
I tried to run copygb to put the rtma.X file onto the same grid:
/whome/det/hwt/2011/tools/WPPV3/exec/copygb.exe \
-xg"255 3 185 129 12190 -133459 8 -95000 40635 40635 0 64 25000
25000" \
rtma.X rtma.X_regrid
FYI, to do this I followed these instructions:
http://www.dtcenter.org/met/users/support/online_tutorial/METv3.0/copygb/run3.php
However that yielded a seg fault from copygb. I suspect that this
setting "LatSP -8364.400000" in the grid definition is causing
problems. I've only seen that set to zero.
So I'd say that your "rtma.X" gridded observation file has problems.
Of course, I don't like the behavior from Ensemble-Stat. Instead of
giving you a weird error, it should tell you that the fcst
and obs aren't on the same grid. I'll look into that - as well as the
second issue.
Thanks,
John
On 09/09/2011 03:12 PM, RAL HelpDesk {for Brian Etherton} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=49485 >
>
> Good afternoon John,
>
> Side note: nice fall day, don't you think?
>
> On the the issues:
>
> I am running these things in this directory, on jet:
> /lfs0/projects/det/sref-emc/biasdown
>
> I am using the script "evaluate.sh" to make these calls to
ensemble_stat.
> The file "met-config-ens.txt" as my config file.
>
> To just do the simple (raw) ensemble, my command is:
> ./bin/ensemble_stat 10 sref/sref.20110609/09/pgrb/sref*[en]m*15
> met-config-ens.txt -grid_obs rtma.X -outdir ./ -v 3 &>
testing_raw.log
>
> To do the bias-corrected ensemble, my command is:
> ./bin/ensemble_stat 10
sref/sref.20110609/09/pgrb/sref_bc*[en]m*15*5km
> met-config-ens.txt -grid_obs rtma.X -outdir ./ -v 3 &>
testing_bc.log
>
> Interestingly... It looks like the bc stuff works fine, when not in
my
> script. Hmm...
>
> Also, from the command line, I get a different error in the 'raw'
case:
> config() -> syntax error in file
> "sref/sref.20110609/09/pgrb/sref_bc_t09z.pgrb212_em.ctl.f15"
>
> More digging needed!
>
> Thanks for the help,
> Brian E.
>
>
>
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #49485] Ensemble Stat
From: Brian Etherton
Time: Fri Sep 09 16:04:29 2011
Hi there,
Indeed - put all the data on the same grid, cleaned up the script a
bit,
and we are in business.
Regards,
Brian Etherton
Forecast Applications Branch
NOAA/OAR/ESRL/GSD
Phone: 303-497-6735, Fax: 303-497-7262
325 Broadway, R/GSD7, Boulder, CO 80305
On 9/9/11 3:54 PM, "RAL HelpDesk {for John Halley Gotway}"
<met_help at ucar.edu> wrote:
>Brian,
>
>Before I begin, I think your "evaluate.sh" script has issues. On
line
>36, where you tell ensemble_stat that there are 10 members, there
really
>are 30:
> [johnhg at fe3 etherton_data_20110909]$ ls
>sref/sref.20110609/09/pgrb/sref*[en]m*15 | wc -w
> 30
>So you should look more closely how you're selecting the 10 members
you
>want to use.
>
>The first issue is caused by the fact that the forecast and
observations
>are not on the same grid:
>
>wgrib -d 1 -V
>/lfs0/projects/det/sref-
emc/vertical/sref/sref.20110609/09/pgrb/sref_bc_t0
>9z.pgrb212.em.n2.f15
>
> Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov -95.000000
> Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP 0.000000
> North Pole (185 x 129) Dx 40.635000 Dy 40.635000 scan 64 mode 8
>
>wgrib -d 1 -V /lfs0/projects/det/sref-emc/biasdown/rtma.X
>
> Lambert Conf: Lat1 20.191000 Lon1 238.445000 Lov 265.000000
> Latin1 25.000000 Latin2 25.000000 LatSP -8364.400000 LonSP
0.000000
> North Pole (1073 x 689) Dx 5.079000 Dy 5.079000 scan 64 mode 8
>
>I tried to run copygb to put the rtma.X file onto the same grid:
> /whome/det/hwt/2011/tools/WPPV3/exec/copygb.exe \
> -xg"255 3 185 129 12190 -133459 8 -95000 40635 40635 0 64 25000
>25000" \
> rtma.X rtma.X_regrid
>
>FYI, to do this I followed these instructions:
>
>http://www.dtcenter.org/met/users/support/online_tutorial/METv3.0/copygb/r
>un3.php
>
>However that yielded a seg fault from copygb. I suspect that this
>setting "LatSP -8364.400000" in the grid definition is causing
problems.
>I've only seen that set to zero.
>
>So I'd say that your "rtma.X" gridded observation file has problems.
Of
>course, I don't like the behavior from Ensemble-Stat. Instead of
giving
>you a weird error, it should tell you that the fcst
>and obs aren't on the same grid. I'll look into that - as well as
the
>second issue.
>
>Thanks,
>John
>
>
>On 09/09/2011 03:12 PM, RAL HelpDesk {for Brian Etherton} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=49485 >
>>
>> Good afternoon John,
>>
>> Side note: nice fall day, don't you think?
>>
>> On the the issues:
>>
>> I am running these things in this directory, on jet:
>> /lfs0/projects/det/sref-emc/biasdown
>>
>> I am using the script "evaluate.sh" to make these calls to
>>ensemble_stat.
>> The file "met-config-ens.txt" as my config file.
>>
>> To just do the simple (raw) ensemble, my command is:
>> ./bin/ensemble_stat 10 sref/sref.20110609/09/pgrb/sref*[en]m*15
>> met-config-ens.txt -grid_obs rtma.X -outdir ./ -v 3 &>
testing_raw.log
>>
>> To do the bias-corrected ensemble, my command is:
>> ./bin/ensemble_stat 10
sref/sref.20110609/09/pgrb/sref_bc*[en]m*15*5km
>> met-config-ens.txt -grid_obs rtma.X -outdir ./ -v 3 &>
testing_bc.log
>>
>> Interestingly... It looks like the bc stuff works fine, when not in
my
>> script. Hmm...
>>
>> Also, from the command line, I get a different error in the 'raw'
case:
>> config() -> syntax error in file
>> "sref/sref.20110609/09/pgrb/sref_bc_t09z.pgrb212_em.ctl.f15"
>>
>> More digging needed!
>>
>> Thanks for the help,
>> Brian E.
>>
>>
>>
>>>
>>
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #49485] Ensemble Stat
From: John Halley Gotway
Time: Fri Sep 09 16:18:27 2011
Brian,
I looked at the development version of the code and saw that we're
missing a call to "set_grid()" after reading the gridded observations.
That checks to ensure everything is on the same grid. So
I've committed that change to the development trunk, but won't plan to
post it as a bugfix - since that incurs some overhead. As long as
your data is all on a common grid it will run fine.
Are you still experiencing the second error, or has that gone away?
Thanks,
John
On 09/09/2011 04:04 PM, RAL HelpDesk {for Brian Etherton} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=49485 >
>
> Hi there,
>
> Indeed - put all the data on the same grid, cleaned up the script a
bit,
> and we are in business.
>
> Regards,
> Brian Etherton
> Forecast Applications Branch
> NOAA/OAR/ESRL/GSD
> Phone: 303-497-6735, Fax: 303-497-7262
> 325 Broadway, R/GSD7, Boulder, CO 80305
>
>
>
>
> On 9/9/11 3:54 PM, "RAL HelpDesk {for John Halley Gotway}"
> <met_help at ucar.edu> wrote:
>
>> Brian,
>>
>> Before I begin, I think your "evaluate.sh" script has issues. On
line
>> 36, where you tell ensemble_stat that there are 10 members, there
really
>> are 30:
>> [johnhg at fe3 etherton_data_20110909]$ ls
>> sref/sref.20110609/09/pgrb/sref*[en]m*15 | wc -w
>> 30
>> So you should look more closely how you're selecting the 10 members
you
>> want to use.
>>
>> The first issue is caused by the fact that the forecast and
observations
>> are not on the same grid:
>>
>> wgrib -d 1 -V
>> /lfs0/projects/det/sref-
emc/vertical/sref/sref.20110609/09/pgrb/sref_bc_t0
>> 9z.pgrb212.em.n2.f15
>>
>> Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov -95.000000
>> Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP
0.000000
>> North Pole (185 x 129) Dx 40.635000 Dy 40.635000 scan 64 mode
8
>>
>> wgrib -d 1 -V /lfs0/projects/det/sref-emc/biasdown/rtma.X
>>
>> Lambert Conf: Lat1 20.191000 Lon1 238.445000 Lov 265.000000
>> Latin1 25.000000 Latin2 25.000000 LatSP -8364.400000 LonSP
0.000000
>> North Pole (1073 x 689) Dx 5.079000 Dy 5.079000 scan 64 mode 8
>>
>> I tried to run copygb to put the rtma.X file onto the same grid:
>> /whome/det/hwt/2011/tools/WPPV3/exec/copygb.exe \
>> -xg"255 3 185 129 12190 -133459 8 -95000 40635 40635 0 64 25000
>> 25000" \
>> rtma.X rtma.X_regrid
>>
>> FYI, to do this I followed these instructions:
>>
>>
http://www.dtcenter.org/met/users/support/online_tutorial/METv3.0/copygb/r
>> un3.php
>>
>> However that yielded a seg fault from copygb. I suspect that this
>> setting "LatSP -8364.400000" in the grid definition is causing
problems.
>> I've only seen that set to zero.
>>
>> So I'd say that your "rtma.X" gridded observation file has
problems. Of
>> course, I don't like the behavior from Ensemble-Stat. Instead of
giving
>> you a weird error, it should tell you that the fcst
>> and obs aren't on the same grid. I'll look into that - as well as
the
>> second issue.
>>
>> Thanks,
>> John
>>
>>
>> On 09/09/2011 03:12 PM, RAL HelpDesk {for Brian Etherton} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=49485 >
>>>
>>> Good afternoon John,
>>>
>>> Side note: nice fall day, don't you think?
>>>
>>> On the the issues:
>>>
>>> I am running these things in this directory, on jet:
>>> /lfs0/projects/det/sref-emc/biasdown
>>>
>>> I am using the script "evaluate.sh" to make these calls to
>>> ensemble_stat.
>>> The file "met-config-ens.txt" as my config file.
>>>
>>> To just do the simple (raw) ensemble, my command is:
>>> ./bin/ensemble_stat 10 sref/sref.20110609/09/pgrb/sref*[en]m*15
>>> met-config-ens.txt -grid_obs rtma.X -outdir ./ -v 3 &>
testing_raw.log
>>>
>>> To do the bias-corrected ensemble, my command is:
>>> ./bin/ensemble_stat 10
sref/sref.20110609/09/pgrb/sref_bc*[en]m*15*5km
>>> met-config-ens.txt -grid_obs rtma.X -outdir ./ -v 3 &>
testing_bc.log
>>>
>>> Interestingly... It looks like the bc stuff works fine, when not
in my
>>> script. Hmm...
>>>
>>> Also, from the command line, I get a different error in the 'raw'
case:
>>> config() -> syntax error in file
>>> "sref/sref.20110609/09/pgrb/sref_bc_t09z.pgrb212_em.ctl.f15"
>>>
>>> More digging needed!
>>>
>>> Thanks for the help,
>>> Brian E.
>>>
>>>
>>>
>>>>
>>>
>>>
>>
>
>
------------------------------------------------
More information about the Met_help
mailing list