[Met_help] [rt.rap.ucar.edu #80202] History for question about Grid-Stat (UNCLASSIFIED)
John Halley Gotway via RT
met_help at ucar.edu
Mon May 1 13:52:45 MDT 2017
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
CLASSIFICATION: UNCLASSIFIED
I'm running MET V5.2 Grid-Stat on a high performance computer and
encountered an "out of memory" error. See attached log file generated using
verbosity level 4.
I tried adding the block_size into the config file and setting it to
3,000,000 to accommodate the gridded observations grid I'm using which has
dimensions of 2145X1377 (larger than the fcst grid), but I still have the
same error.
Is there a setting I can change to resolve this error? Any other
suggestions?
Thanks.
R/
John
Mr John W. Raby, Meteorologist
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
(575) 678-2004 DSN 258-2004
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
CLASSIFICATION: UNCLASSIFIED
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: question about Grid-Stat (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Apr 18 22:43:04 2017
John,
I see you're having a memory issue with grid_stat in version 5.2.
Unfortunately, the block_size config option doesn't apply to
grid_stat.
Instead it's used by series_analysis to define how many grid points to
process concurrently. Setting it lowers in series_analysis uses less
memory, but it would have no effect in the grid_stat confit file.
If possible, it'd be worth trying this same run using version 6.0 of
grid_stat. We found and fixed some very inefficient memory usage in
grid_stat, as listed in the release notes:
http://www.dtcenter.org/met/users/support/release_notes/METv6.0_release_notes.php
These changes won't necessarily fix your memory issue but will
certainly
make it run faster.
Often when you submit a job to a batch queuing system, there is an
option
to request more memory. I'd suggest looking into that and requesting
more
memory, if possible.
You might also test to see if using the "regrid" option to evaluate on
a
more coarse grid enables it to actually run.
Hope that helps you get started.
Thanks,
John
On Tue, Apr 18, 2017 at 3:55 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Tue Apr 18 15:55:16 2017: Request 80202 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: question about Grid-Stat (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80202 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
> I'm running MET V5.2 Grid-Stat on a high performance computer and
> encountered an "out of memory" error. See attached log file
generated using
> verbosity level 4.
>
> I tried adding the block_size into the config file and setting it to
> 3,000,000 to accommodate the gridded observations grid I'm using
which has
> dimensions of 2145X1377 (larger than the fcst grid), but I still
have the
> same error.
>
> Is there a setting I can change to resolve this error? Any other
> suggestions?
>
> Thanks.
>
> R/
>
> John
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
>
>
>
>
>
>
>
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
>
------------------------------------------------
Subject: question about Grid-Stat (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed Apr 19 08:57:03 2017
CLASSIFICATION: UNCLASSIFIED
John -
Thanks for your suggestions. I'll start looking into trying them. Will
installing V6.0 be easier because I have V5.2 installed or is it an
independent set of libraries and code which requires the same process
I went
through for V5.2?
Is the large size of the observation grid (2145X1377) alone causing
the
problem or is it the combination of the fcst grid (278X278) and the
observation grids together that result in the exceedance of the
available
memory size?
Is there a way to relate available memory size to the grid sizes I
need to
process?
I ran a command called "free -g" yesterday to see if this would show
the
available memory and I received the following:
Free - 13GB
Swap - 61GB
The HPC User's says that you shouldn't use more than 8 GB at a time,
so even
though 13 is "free", they may be restricting me to 8. Is this as a
potential
limitation?
In my config file (attached) I did use the "regrid" option to set the
verification grid to "FCST" so that the verification would occur on
that grid
(9 km spacing). How does the grid matching occur? Will the OBS grid
(2.5 km
spacing) be interpolated to the FCST grid (9 km spacing)?
Maybe I have used other config file settings improperly somewhere
which might
cause a problem. I have n_rep set to 1000 which was cited in the past
as a
process which slows things down.
If I don't want to use smoothing prior to verification, can I use some
null
settings to prevent it from running while leaving that section in the
config
file for future reference?
I had not envisioned having to run MET using a batch queuing system,
and I'm
not very familiar with it, but I know that my colleagues here use it
for
running the WRF model, so I should be able to tap that experience to
explore
the possibility that I can allocate more memory for running Grid-Stat.
The
compute nodes can only be used through batch job submission. Good
suggestion.
I thought of the possibility of using the options in "wgrib2" to
window out a
subset of the URMA gridded observations to pare down the grid size
some. I've
never done this. Have you tried this and would this cut down on the
memory
resources needed to process this data?
Can I anticipate this same problem if I use the same input fields into
MODE
and Series-Analysis?
Thanks.
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, April 18, 2017 10:43 PM
To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #80202] question about
Grid-Stat (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the
identity of the sender, and confirm the authenticity of all links
contained
within the message prior to copying and pasting the address to a Web
browser.
----
John,
I see you're having a memory issue with grid_stat in version 5.2.
Unfortunately, the block_size config option doesn't apply to
grid_stat.
Instead it's used by series_analysis to define how many grid points to
process
concurrently. Setting it lowers in series_analysis uses less memory,
but it
would have no effect in the grid_stat confit file.
If possible, it'd be worth trying this same run using version 6.0 of
grid_stat. We found and fixed some very inefficient memory usage in
grid_stat, as listed in the release notes:
Caution-
http://www.dtcenter.org/met/users/support/release_notes/METv6.0_release_notes.php
These changes won't necessarily fix your memory issue but will
certainly make
it run faster.
Often when you submit a job to a batch queuing system, there is an
option to
request more memory. I'd suggest looking into that and requesting
more
memory, if possible.
You might also test to see if using the "regrid" option to evaluate on
a more
coarse grid enables it to actually run.
Hope that helps you get started.
Thanks,
John
On Tue, Apr 18, 2017 at 3:55 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Tue Apr 18 15:55:16 2017: Request 80202 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: question about Grid-Stat (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80202 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
> I'm running MET V5.2 Grid-Stat on a high performance computer and
> encountered an "out of memory" error. See attached log file
generated
> using verbosity level 4.
>
> I tried adding the block_size into the config file and setting it to
> 3,000,000 to accommodate the gridded observations grid I'm using
which
> has dimensions of 2145X1377 (larger than the fcst grid), but I still
> have the same error.
>
> Is there a setting I can change to resolve this error? Any other
> suggestions?
>
> Thanks.
>
> R/
>
> John
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
>
>
>
>
>
>
>
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: question about Grid-Stat (UNCLASSIFIED)
From: John Halley Gotway
Time: Thu Apr 20 16:15:30 2017
John,
Just curious if you've made any progress resolving these memory
issues?
Thanks,
John
On Wed, Apr 19, 2017 at 8:57 AM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80202 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks for your suggestions. I'll start looking into trying them.
Will
> installing V6.0 be easier because I have V5.2 installed or is it an
> independent set of libraries and code which requires the same
process I
> went
> through for V5.2?
>
> Is the large size of the observation grid (2145X1377) alone causing
the
> problem or is it the combination of the fcst grid (278X278) and the
> observation grids together that result in the exceedance of the
available
> memory size?
>
> Is there a way to relate available memory size to the grid sizes I
need to
> process?
>
> I ran a command called "free -g" yesterday to see if this would show
the
> available memory and I received the following:
>
> Free - 13GB
> Swap - 61GB
>
> The HPC User's says that you shouldn't use more than 8 GB at a time,
so
> even
> though 13 is "free", they may be restricting me to 8. Is this as a
> potential
> limitation?
>
> In my config file (attached) I did use the "regrid" option to set
the
> verification grid to "FCST" so that the verification would occur on
that
> grid
> (9 km spacing). How does the grid matching occur? Will the OBS grid
(2.5 km
> spacing) be interpolated to the FCST grid (9 km spacing)?
>
> Maybe I have used other config file settings improperly somewhere
which
> might
> cause a problem. I have n_rep set to 1000 which was cited in the
past as a
> process which slows things down.
>
> If I don't want to use smoothing prior to verification, can I use
some null
> settings to prevent it from running while leaving that section in
the
> config
> file for future reference?
>
> I had not envisioned having to run MET using a batch queuing system,
and
> I'm
> not very familiar with it, but I know that my colleagues here use it
for
> running the WRF model, so I should be able to tap that experience to
> explore
> the possibility that I can allocate more memory for running Grid-
Stat. The
> compute nodes can only be used through batch job submission. Good
> suggestion.
>
> I thought of the possibility of using the options in "wgrib2" to
window
> out a
> subset of the URMA gridded observations to pare down the grid size
some.
> I've
> never done this. Have you tried this and would this cut down on the
memory
> resources needed to process this data?
>
> Can I anticipate this same problem if I use the same input fields
into MODE
> and Series-Analysis?
>
> Thanks.
>
> R/
> John
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, April 18, 2017 10:43 PM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #80202] question
about
> Grid-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> I see you're having a memory issue with grid_stat in version 5.2.
> Unfortunately, the block_size config option doesn't apply to
grid_stat.
> Instead it's used by series_analysis to define how many grid points
to
> process
> concurrently. Setting it lowers in series_analysis uses less
memory, but
> it
> would have no effect in the grid_stat confit file.
>
> If possible, it'd be worth trying this same run using version 6.0 of
> grid_stat. We found and fixed some very inefficient memory usage in
> grid_stat, as listed in the release notes:
>
> Caution-http://www.dtcenter.org/met/users/support/release_
> notes/METv6.0_release_notes.php
>
> These changes won't necessarily fix your memory issue but will
certainly
> make
> it run faster.
>
> Often when you submit a job to a batch queuing system, there is an
option
> to
> request more memory. I'd suggest looking into that and requesting
more
> memory, if possible.
>
> You might also test to see if using the "regrid" option to evaluate
on a
> more
> coarse grid enables it to actually run.
>
> Hope that helps you get started.
>
> Thanks,
> John
>
>
> On Tue, Apr 18, 2017 at 3:55 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Apr 18 15:55:16 2017: Request 80202 was acted upon.
> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > Queue: met_help
> > Subject: question about Grid-Stat (UNCLASSIFIED)
> > Owner: Nobody
> > Requestors: john.w.raby2.civ at mail.mil
> > Status: new
> > Ticket <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80202 >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
> > I'm running MET V5.2 Grid-Stat on a high performance computer and
> > encountered an "out of memory" error. See attached log file
generated
> > using verbosity level 4.
> >
> > I tried adding the block_size into the config file and setting it
to
> > 3,000,000 to accommodate the gridded observations grid I'm using
which
> > has dimensions of 2145X1377 (larger than the fcst grid), but I
still
> > have the same error.
> >
> > Is there a setting I can change to resolve this error? Any other
> > suggestions?
> >
> > Thanks.
> >
> > R/
> >
> > John
> >
> > Mr John W. Raby, Meteorologist
> >
> > U.S. Army Research Laboratory
> >
> > White Sands Missile Range, NM 88002
> >
> > (575) 678-2004 DSN 258-2004
> >
> > FAX (575) 678-1230 DSN 258-1230
> >
> > Email: john.w.raby2.civ at mail.mil
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
> >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: question about Grid-Stat (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Thu Apr 20 16:32:17 2017
CLASSIFICATION: UNCLASSIFIED
John -
I haven't been able to get back to that. I have begun looking into the
Portable Batch system (PBS) which is used on the HPC and I'm told that
you can
access a much larger memory on the compute nodes than where I was
running
Grid-Stat before. The login node has 2GB available while the compute
node has
128 GB. So, I'm going to try that approach which you suggested.
Would running regrid as a utility to pare down the size of the larger
grid
prior to running Grid-Stat be problematic in the same way? (i.e.
likely to run
out of memory)
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, April 20, 2017 4:16 PM
To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #80202] question
about
Grid-Stat (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the
identity of the sender, and confirm the authenticity of all links
contained
within the message prior to copying and pasting the address to a Web
browser.
----
John,
Just curious if you've made any progress resolving these memory
issues?
Thanks,
John
On Wed, Apr 19, 2017 at 8:57 AM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80202 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks for your suggestions. I'll start looking into trying them.
Will
> installing V6.0 be easier because I have V5.2 installed or is it an
> independent set of libraries and code which requires the same
process
> I went through for V5.2?
>
> Is the large size of the observation grid (2145X1377) alone causing
> the problem or is it the combination of the fcst grid (278X278) and
> the observation grids together that result in the exceedance of the
> available memory size?
>
> Is there a way to relate available memory size to the grid sizes I
> need to process?
>
> I ran a command called "free -g" yesterday to see if this would show
> the available memory and I received the following:
>
> Free - 13GB
> Swap - 61GB
>
> The HPC User's says that you shouldn't use more than 8 GB at a time,
> so even though 13 is "free", they may be restricting me to 8. Is
this
> as a potential limitation?
>
> In my config file (attached) I did use the "regrid" option to set
the
> verification grid to "FCST" so that the verification would occur on
> that grid
> (9 km spacing). How does the grid matching occur? Will the OBS grid
> (2.5 km
> spacing) be interpolated to the FCST grid (9 km spacing)?
>
> Maybe I have used other config file settings improperly somewhere
> which might cause a problem. I have n_rep set to 1000 which was
cited
> in the past as a process which slows things down.
>
> If I don't want to use smoothing prior to verification, can I use
some
> null settings to prevent it from running while leaving that section
in
> the config file for future reference?
>
> I had not envisioned having to run MET using a batch queuing system,
> and I'm not very familiar with it, but I know that my colleagues
here
> use it for running the WRF model, so I should be able to tap that
> experience to explore the possibility that I can allocate more
memory
> for running Grid-Stat. The compute nodes can only be used through
> batch job submission. Good suggestion.
>
> I thought of the possibility of using the options in "wgrib2" to
> window out a subset of the URMA gridded observations to pare down
the
> grid size some.
> I've
> never done this. Have you tried this and would this cut down on the
> memory resources needed to process this data?
>
> Can I anticipate this same problem if I use the same input fields
into
> MODE and Series-Analysis?
>
> Thanks.
>
> R/
> John
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [Caution-mailto:met_help at ucar.edu]
> Sent: Tuesday, April 18, 2017 10:43 PM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
> <john.w.raby2.civ at mail.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #80202] question
about
> Grid-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> I see you're having a memory issue with grid_stat in version 5.2.
> Unfortunately, the block_size config option doesn't apply to
grid_stat.
> Instead it's used by series_analysis to define how many grid points
to
> process concurrently. Setting it lowers in series_analysis uses
less
> memory, but it would have no effect in the grid_stat confit file.
>
> If possible, it'd be worth trying this same run using version 6.0 of
> grid_stat. We found and fixed some very inefficient memory usage in
> grid_stat, as listed in the release notes:
>
> Caution-Caution-http://www.dtcenter.org/met/users/support/release_
> notes/METv6.0_release_notes.php
>
> These changes won't necessarily fix your memory issue but will
> certainly make it run faster.
>
> Often when you submit a job to a batch queuing system, there is an
> option to request more memory. I'd suggest looking into that and
> requesting more memory, if possible.
>
> You might also test to see if using the "regrid" option to evaluate
on
> a more coarse grid enables it to actually run.
>
> Hope that helps you get started.
>
> Thanks,
> John
>
>
> On Tue, Apr 18, 2017 at 3:55 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Apr 18 15:55:16 2017: Request 80202 was acted upon.
> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > Queue: met_help
> > Subject: question about Grid-Stat (UNCLASSIFIED)
> > Owner: Nobody
> > Requestors: john.w.raby2.civ at mail.mil
> > Status: new
> > Ticket <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80
> > 202 >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
> > I'm running MET V5.2 Grid-Stat on a high performance computer and
> > encountered an "out of memory" error. See attached log file
> > generated using verbosity level 4.
> >
> > I tried adding the block_size into the config file and setting it
to
> > 3,000,000 to accommodate the gridded observations grid I'm using
> > which has dimensions of 2145X1377 (larger than the fcst grid), but
I
> > still have the same error.
> >
> > Is there a setting I can change to resolve this error? Any other
> > suggestions?
> >
> > Thanks.
> >
> > R/
> >
> > John
> >
> > Mr John W. Raby, Meteorologist
> >
> > U.S. Army Research Laboratory
> >
> > White Sands Missile Range, NM 88002
> >
> > (575) 678-2004 DSN 258-2004
> >
> > FAX (575) 678-1230 DSN 258-1230
> >
> > Email: john.w.raby2.civ at mail.mil
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
> >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: question about Grid-Stat (UNCLASSIFIED)
From: John Halley Gotway
Time: Fri Apr 21 09:58:19 2017
John,
Unfortunately, I'm not sure if you'll run into more memory issues with
regrid_data_plane. I'd just suggest testing it out.
John
On Thu, Apr 20, 2017 at 4:32 PM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80202 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> I haven't been able to get back to that. I have begun looking into
the
> Portable Batch system (PBS) which is used on the HPC and I'm told
that you
> can
> access a much larger memory on the compute nodes than where I was
running
> Grid-Stat before. The login node has 2GB available while the compute
node
> has
> 128 GB. So, I'm going to try that approach which you suggested.
>
> Would running regrid as a utility to pare down the size of the
larger grid
> prior to running Grid-Stat be problematic in the same way? (i.e.
likely to
> run
> out of memory)
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, April 20, 2017 4:16 PM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #80202] question
about
> Grid-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Just curious if you've made any progress resolving these memory
issues?
>
> Thanks,
> John
>
> On Wed, Apr 19, 2017 at 8:57 AM, Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80202 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > Thanks for your suggestions. I'll start looking into trying them.
Will
> > installing V6.0 be easier because I have V5.2 installed or is it
an
> > independent set of libraries and code which requires the same
process
> > I went through for V5.2?
> >
> > Is the large size of the observation grid (2145X1377) alone
causing
> > the problem or is it the combination of the fcst grid (278X278)
and
> > the observation grids together that result in the exceedance of
the
> > available memory size?
> >
> > Is there a way to relate available memory size to the grid sizes I
> > need to process?
> >
> > I ran a command called "free -g" yesterday to see if this would
show
> > the available memory and I received the following:
> >
> > Free - 13GB
> > Swap - 61GB
> >
> > The HPC User's says that you shouldn't use more than 8 GB at a
time,
> > so even though 13 is "free", they may be restricting me to 8. Is
this
> > as a potential limitation?
> >
> > In my config file (attached) I did use the "regrid" option to set
the
> > verification grid to "FCST" so that the verification would occur
on
> > that grid
> > (9 km spacing). How does the grid matching occur? Will the OBS
grid
> > (2.5 km
> > spacing) be interpolated to the FCST grid (9 km spacing)?
> >
> > Maybe I have used other config file settings improperly somewhere
> > which might cause a problem. I have n_rep set to 1000 which was
cited
> > in the past as a process which slows things down.
> >
> > If I don't want to use smoothing prior to verification, can I use
some
> > null settings to prevent it from running while leaving that
section in
> > the config file for future reference?
> >
> > I had not envisioned having to run MET using a batch queuing
system,
> > and I'm not very familiar with it, but I know that my colleagues
here
> > use it for running the WRF model, so I should be able to tap that
> > experience to explore the possibility that I can allocate more
memory
> > for running Grid-Stat. The compute nodes can only be used through
> > batch job submission. Good suggestion.
> >
> > I thought of the possibility of using the options in "wgrib2" to
> > window out a subset of the URMA gridded observations to pare down
the
> > grid size some.
> > I've
> > never done this. Have you tried this and would this cut down on
the
> > memory resources needed to process this data?
> >
> > Can I anticipate this same problem if I use the same input fields
into
> > MODE and Series-Analysis?
> >
> > Thanks.
> >
> > R/
> > John
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [Caution-mailto:met_help at ucar.edu]
> > Sent: Tuesday, April 18, 2017 10:43 PM
> > To: Raby, John W CIV USARMY RDECOM ARL (US)
> > <john.w.raby2.civ at mail.mil>
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #80202] question
about
> > Grid-Stat (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > I see you're having a memory issue with grid_stat in version 5.2.
> > Unfortunately, the block_size config option doesn't apply to
grid_stat.
> > Instead it's used by series_analysis to define how many grid
points to
> > process concurrently. Setting it lowers in series_analysis uses
less
> > memory, but it would have no effect in the grid_stat confit file.
> >
> > If possible, it'd be worth trying this same run using version 6.0
of
> > grid_stat. We found and fixed some very inefficient memory usage
in
> > grid_stat, as listed in the release notes:
> >
> > Caution-Caution-http://www.dtcenter.org/met/users/support/release_
> > notes/METv6.0_release_notes.php
> >
> > These changes won't necessarily fix your memory issue but will
> > certainly make it run faster.
> >
> > Often when you submit a job to a batch queuing system, there is an
> > option to request more memory. I'd suggest looking into that and
> > requesting more memory, if possible.
> >
> > You might also test to see if using the "regrid" option to
evaluate on
> > a more coarse grid enables it to actually run.
> >
> > Hope that helps you get started.
> >
> > Thanks,
> > John
> >
> >
> > On Tue, Apr 18, 2017 at 3:55 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Apr 18 15:55:16 2017: Request 80202 was acted upon.
> > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > Queue: met_help
> > > Subject: question about Grid-Stat (UNCLASSIFIED)
> > > Owner: Nobody
> > > Requestors: john.w.raby2.civ at mail.mil
> > > Status: new
> > > Ticket <Caution-Caution-url:
> > > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80
> > > 202 >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> > > I'm running MET V5.2 Grid-Stat on a high performance computer
and
> > > encountered an "out of memory" error. See attached log file
> > > generated using verbosity level 4.
> > >
> > > I tried adding the block_size into the config file and setting
it to
> > > 3,000,000 to accommodate the gridded observations grid I'm using
> > > which has dimensions of 2145X1377 (larger than the fcst grid),
but I
> > > still have the same error.
> > >
> > > Is there a setting I can change to resolve this error? Any other
> > > suggestions?
> > >
> > > Thanks.
> > >
> > > R/
> > >
> > > John
> > >
> > > Mr John W. Raby, Meteorologist
> > >
> > > U.S. Army Research Laboratory
> > >
> > > White Sands Missile Range, NM 88002
> > >
> > > (575) 678-2004 DSN 258-2004
> > >
> > > FAX (575) 678-1230 DSN 258-1230
> > >
> > > Email: john.w.raby2.civ at mail.mil
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> > >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
More information about the Met_help
mailing list