[Met_help] [rt.rap.ucar.edu #86164] History for longtime waiting for generating Gen_Vx_mask
John Halley Gotway via RT
met_help at ucar.edu
Mon Jul 16 14:08:50 MDT 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Zhang, Mandy has shared OneDrive for Business files with you. To view them, click the links below.
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ>
2018041800acc01_003600(1).nc<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ>
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w>
MASK.poly<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w>
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ>
test_gen_vx_mask.sh<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ>
Dear All,
I try to generate mask region, the grid data (model output) covers CONUS domain with 1623*1123 grids, and the mask input file has 691966 points. when I run sh test_gen_vx_mask.sh, screen shows it is running without any errors, but it looks like keeping computing and will not be completed.
*** Running Gen-Vx-Mask to generate a polyline mask file for the Continental United States ***
DEBUG 1: Input File: /cnproj1/mzhang/met_test/ensm/model_data/20180418-acc/m01/2018041800acc01_003600.nc
DEBUG 1: Mask File: MASK.poly
DEBUG 2: Parsed Data Grid: Lambert Conformal (1623 x 1123)
DEBUG 2: Parsed Lat/Lon Mask File: MASK containing 691966 points
*********************************************************
please find my attached files for details.
Does anyone know what's the issue?
Thank you!
Mandy
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: longtime waiting for generating Gen_Vx_mask
From: John Halley Gotway
Time: Mon Jul 16 09:57:31 2018
Mandy,
The runtime is so slow because that is such a large number of lat/lon
points to include in a masking region. The Gen-Vx-Mask tool is
checking
whether the lat/lon corresponding to each grid point is inside that
polyline region. For a polyline consisting of nearly 700,000 points,
that
takes a long time! I'm running this case on my machine now, and I see
that
it's taking a long time as well. But it's not consuming very much
memory.
Presumably, it would eventually finish but I really don't know how
long it
would take.
I can tell you that applying the CONUS.poly file to this grid takes
about
20 seconds on my machine. And the CONUS polyline only has 243 points
in it.
How did you end up defining a polyline with 691966 points in it?
Thanks,
John
On Thu, Jul 12, 2018 at 9:10 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> Thu Jul 12 09:10:01 2018: Request 86164 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
> Queue: met_help
> Subject: longtime waiting for generating Gen_Vx_mask
> Owner: Nobody
> Requestors: Man.Zhang-1 at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> >
>
> 2018041800acc01_003600(1).nc<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> >
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> >
>
> MASK.poly<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> >
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> >
>
> test_gen_vx_mask.sh<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> >
>
>
>
>
> Dear All,
>
> I try to generate mask region, the grid data (model output)
> covers CONUS domain with 1623*1123 grids, and the mask input file
has
> 691966 points. when I run sh test_gen_vx_mask.sh, screen shows it is
> running without any errors, but it looks like keeping computing and
will
> not be completed.
>
> *** Running Gen-Vx-Mask to generate a polyline mask file for the
> Continental United States ***
> DEBUG 1: Input File:
> /cnproj1/mzhang/met_test/ensm/model_data/20180418-acc/m01/
> 2018041800acc01_003600.nc
> DEBUG 1: Mask File: MASK.poly
> DEBUG 2: Parsed Data Grid: Lambert Conformal (1623 x 1123)
> DEBUG 2: Parsed Lat/Lon Mask File: MASK containing 691966 points
> *********************************************************
> please find my attached files for details.
> Does anyone know what's the issue?
> Thank you!
> Mandy
>
>
------------------------------------------------
Subject: longtime waiting for generating Gen_Vx_mask
From: Zhang, Mandy
Time: Mon Jul 16 10:12:37 2018
Hi John,
I define a polyline with longitude from 103w to 75w and latitude
from 30n~47 N at 3km grid space. And it spend 4 days to finish.
Do you have any suggestion to do it faster?
Thank you very much!
Mandy
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, July 16, 2018 10:57:31 AM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for generating
Gen_Vx_mask
Mandy,
The runtime is so slow because that is such a large number of lat/lon
points to include in a masking region. The Gen-Vx-Mask tool is
checking
whether the lat/lon corresponding to each grid point is inside that
polyline region. For a polyline consisting of nearly 700,000 points,
that
takes a long time! I'm running this case on my machine now, and I see
that
it's taking a long time as well. But it's not consuming very much
memory.
Presumably, it would eventually finish but I really don't know how
long it
would take.
I can tell you that applying the CONUS.poly file to this grid takes
about
20 seconds on my machine. And the CONUS polyline only has 243 points
in it.
How did you end up defining a polyline with 691966 points in it?
Thanks,
John
On Thu, Jul 12, 2018 at 9:10 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> Thu Jul 12 09:10:01 2018: Request 86164 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
> Queue: met_help
> Subject: longtime waiting for generating Gen_Vx_mask
> Owner: Nobody
> Requestors: Man.Zhang-1 at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> >
>
> 2018041800acc01_003600(1).nc<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> >
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> >
>
> MASK.poly<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> >
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> >
>
> test_gen_vx_mask.sh<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> >
>
>
>
>
> Dear All,
>
> I try to generate mask region, the grid data (model output)
> covers CONUS domain with 1623*1123 grids, and the mask input file
has
> 691966 points. when I run sh test_gen_vx_mask.sh, screen shows it is
> running without any errors, but it looks like keeping computing and
will
> not be completed.
>
> *** Running Gen-Vx-Mask to generate a polyline mask file for the
> Continental United States ***
> DEBUG 1: Input File:
> /cnproj1/mzhang/met_test/ensm/model_data/20180418-acc/m01/
> 2018041800acc01_003600.nc
> DEBUG 1: Mask File: MASK.poly
> DEBUG 2: Parsed Data Grid: Lambert Conformal (1623 x 1123)
> DEBUG 2: Parsed Lat/Lon Mask File: MASK containing 691966 points
> *********************************************************
> please find my attached files for details.
> Does anyone know what's the issue?
> Thank you!
> Mandy
>
>
------------------------------------------------
Subject: longtime waiting for generating Gen_Vx_mask
From: John Halley Gotway
Time: Mon Jul 16 11:25:49 2018
What is the region you're trying to include? Is it the continental
US?
John
On Mon, Jul 16, 2018 at 10:13 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
>
> Hi John,
>
> I define a polyline with longitude from 103w to 75w and
latitude from
> 30n~47 N at 3km grid space. And it spend 4 days to finish.
>
> Do you have any suggestion to do it faster?
>
> Thank you very much!
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, July 16, 2018 10:57:31 AM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for
generating
> Gen_Vx_mask
>
> Mandy,
>
> The runtime is so slow because that is such a large number of
lat/lon
> points to include in a masking region. The Gen-Vx-Mask tool is
checking
> whether the lat/lon corresponding to each grid point is inside that
> polyline region. For a polyline consisting of nearly 700,000
points, that
> takes a long time! I'm running this case on my machine now, and I
see that
> it's taking a long time as well. But it's not consuming very much
memory.
> Presumably, it would eventually finish but I really don't know how
long it
> would take.
>
> I can tell you that applying the CONUS.poly file to this grid takes
about
> 20 seconds on my machine. And the CONUS polyline only has 243
points in
> it.
>
> How did you end up defining a polyline with 691966 points in it?
>
> Thanks,
> John
>
>
>
>
>
> On Thu, Jul 12, 2018 at 9:10 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Jul 12 09:10:01 2018: Request 86164 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > Queue: met_help
> > Subject: longtime waiting for generating Gen_Vx_mask
> > Owner: Nobody
> > Requestors: Man.Zhang-1 at ou.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > >
> >
> > 2018041800acc01_003600(1).nc<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > >
> >
> > MASK.poly<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > >
> >
> > test_gen_vx_mask.sh<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > >
> >
> >
> >
> >
> > Dear All,
> >
> > I try to generate mask region, the grid data (model
output)
> > covers CONUS domain with 1623*1123 grids, and the mask input file
has
> > 691966 points. when I run sh test_gen_vx_mask.sh, screen shows it
is
> > running without any errors, but it looks like keeping computing
and will
> > not be completed.
> >
> > *** Running Gen-Vx-Mask to generate a polyline mask file for the
> > Continental United States ***
> > DEBUG 1: Input File:
> > /cnproj1/mzhang/met_test/ensm/model_data/20180418-acc/m01/
> > 2018041800acc01_003600.nc
> > DEBUG 1: Mask File: MASK.poly
> > DEBUG 2: Parsed Data Grid: Lambert Conformal (1623 x 1123)
> > DEBUG 2: Parsed Lat/Lon Mask File: MASK containing 691966 points
> > *********************************************************
> > please find my attached files for details.
> > Does anyone know what's the issue?
> > Thank you!
> > Mandy
> >
> >
>
>
>
------------------------------------------------
Subject: longtime waiting for generating Gen_Vx_mask
From: Zhang, Mandy
Time: Mon Jul 16 13:04:17 2018
Hi John
yes , it is continental US and excludes the west of Rocky mountain.
Thank you!
Mandy
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, July 16, 2018 12:25:49 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for generating
Gen_Vx_mask
What is the region you're trying to include? Is it the continental
US?
John
On Mon, Jul 16, 2018 at 10:13 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
>
> Hi John,
>
> I define a polyline with longitude from 103w to 75w and
latitude from
> 30n~47 N at 3km grid space. And it spend 4 days to finish.
>
> Do you have any suggestion to do it faster?
>
> Thank you very much!
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, July 16, 2018 10:57:31 AM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for
generating
> Gen_Vx_mask
>
> Mandy,
>
> The runtime is so slow because that is such a large number of
lat/lon
> points to include in a masking region. The Gen-Vx-Mask tool is
checking
> whether the lat/lon corresponding to each grid point is inside that
> polyline region. For a polyline consisting of nearly 700,000
points, that
> takes a long time! I'm running this case on my machine now, and I
see that
> it's taking a long time as well. But it's not consuming very much
memory.
> Presumably, it would eventually finish but I really don't know how
long it
> would take.
>
> I can tell you that applying the CONUS.poly file to this grid takes
about
> 20 seconds on my machine. And the CONUS polyline only has 243
points in
> it.
>
> How did you end up defining a polyline with 691966 points in it?
>
> Thanks,
> John
>
>
>
>
>
> On Thu, Jul 12, 2018 at 9:10 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Jul 12 09:10:01 2018: Request 86164 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > Queue: met_help
> > Subject: longtime waiting for generating Gen_Vx_mask
> > Owner: Nobody
> > Requestors: Man.Zhang-1 at ou.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > >
> >
> > 2018041800acc01_003600(1).nc<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > >
> >
> > MASK.poly<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > >
> >
> > test_gen_vx_mask.sh<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > >
> >
> >
> >
> >
> > Dear All,
> >
> > I try to generate mask region, the grid data (model
output)
> > covers CONUS domain with 1623*1123 grids, and the mask input file
has
> > 691966 points. when I run sh test_gen_vx_mask.sh, screen shows it
is
> > running without any errors, but it looks like keeping computing
and will
> > not be completed.
> >
> > *** Running Gen-Vx-Mask to generate a polyline mask file for the
> > Continental United States ***
> > DEBUG 1: Input File:
> > /cnproj1/mzhang/met_test/ensm/model_data/20180418-acc/m01/
> > 2018041800acc01_003600.nc
> > DEBUG 1: Mask File: MASK.poly
> > DEBUG 2: Parsed Data Grid: Lambert Conformal (1623 x 1123)
> > DEBUG 2: Parsed Lat/Lon Mask File: MASK containing 691966 points
> > *********************************************************
> > please find my attached files for details.
> > Does anyone know what's the issue?
> > Thank you!
> > Mandy
> >
> >
>
>
>
------------------------------------------------
Subject: longtime waiting for generating Gen_Vx_mask
From: John Halley Gotway
Time: Mon Jul 16 13:25:21 2018
Mandy,
We just need to figure out a way of defining that mask using fewer
points.
Here's an idea.
(1) Run gen_vx_mask to apply the CONUS mask from the MET release to
your
grid:
time /usr/local/met-7.0/bin/gen_vx_mask \
2018041800acc01_003600\(1\).nc \
/usr/local/met-7.0/share/met/poly/CONUS.poly \
conus_mask.nc -v 3
Takes about 20 seconds.
(2) Run gen_vx_mask a second time keeping only points east of -104
degrees
longitude:
/usr/local/met-7.0/bin/gen_vx_mask \
conus_mask.nc conus_mask.nc \
conus_east_of_rockies.nc \
-type lon -thresh ge-104 -intersection \
-name CONUS_EAST
(3) Plot the result (see attached):
/usr/local/met-7.0/bin/plot_data_plane \
conus_east_of_rockies.nc \
conus_east.ps \
'name="CONUS_EAST"; level="(*,*)";'
Is that the type of mask you're looking for?
Another alternative would be using a GIS shapefile to define the CONUS
region, but that isn't available until the next release of MET. But
we do
have beta versions out there for testing:
https://dtcenter.org/met/users/downloads/existing_met_builds/METv7.1_beta6_existing_met_builds.php
Thanks,
John
On Mon, Jul 16, 2018 at 1:04 PM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
>
> Hi John
>
> yes , it is continental US and excludes the west of Rocky mountain.
>
> Thank you!
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, July 16, 2018 12:25:49 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for
generating
> Gen_Vx_mask
>
> What is the region you're trying to include? Is it the continental
US?
>
> John
>
> On Mon, Jul 16, 2018 at 10:13 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
> >
> > Hi John,
> >
> > I define a polyline with longitude from 103w to 75w and
latitude
> from
> > 30n~47 N at 3km grid space. And it spend 4 days to finish.
> >
> > Do you have any suggestion to do it faster?
> >
> > Thank you very much!
> >
> > Mandy
> >
> > ________________________________
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, July 16, 2018 10:57:31 AM
> > To: Zhang, Mandy
> > Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for
generating
> > Gen_Vx_mask
> >
> > Mandy,
> >
> > The runtime is so slow because that is such a large number of
lat/lon
> > points to include in a masking region. The Gen-Vx-Mask tool is
checking
> > whether the lat/lon corresponding to each grid point is inside
that
> > polyline region. For a polyline consisting of nearly 700,000
points,
> that
> > takes a long time! I'm running this case on my machine now, and I
see
> that
> > it's taking a long time as well. But it's not consuming very much
> memory.
> > Presumably, it would eventually finish but I really don't know how
long
> it
> > would take.
> >
> > I can tell you that applying the CONUS.poly file to this grid
takes about
> > 20 seconds on my machine. And the CONUS polyline only has 243
points in
> > it.
> >
> > How did you end up defining a polyline with 691966 points in it?
> >
> > Thanks,
> > John
> >
> >
> >
> >
> >
> > On Thu, Jul 12, 2018 at 9:10 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Thu Jul 12 09:10:01 2018: Request 86164 was acted upon.
> > > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > > Queue: met_help
> > > Subject: longtime waiting for generating Gen_Vx_mask
> > > Owner: Nobody
> > > Requestors: Man.Zhang-1 at ou.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164
> >
> > >
> > >
> > > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > > them, click the links below.
> > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > > >
> > >
> > > 2018041800acc01_003600(1).nc<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > > >
> > >
> > > MASK.poly<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > > >
> > >
> > > test_gen_vx_mask.sh<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > > >
> > >
> > >
> > >
> > >
> > > Dear All,
> > >
> > > I try to generate mask region, the grid data (model
output)
> > > covers CONUS domain with 1623*1123 grids, and the mask input
file has
> > > 691966 points. when I run sh test_gen_vx_mask.sh, screen shows
it is
> > > running without any errors, but it looks like keeping computing
and
> will
> > > not be completed.
> > >
> > > *** Running Gen-Vx-Mask to generate a polyline mask file for the
> > > Continental United States ***
> > > DEBUG 1: Input File:
> > > /cnproj1/mzhang/met_test/ensm/model_data/20180418-acc/m01/
> > > 2018041800acc01_003600.nc
> > > DEBUG 1: Mask File: MASK.poly
> > > DEBUG 2: Parsed Data Grid: Lambert Conformal (1623 x 1123)
> > > DEBUG 2: Parsed Lat/Lon Mask File: MASK containing 691966 points
> > > *********************************************************
> > > please find my attached files for details.
> > > Does anyone know what's the issue?
> > > Thank you!
> > > Mandy
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: longtime waiting for generating Gen_Vx_mask
From: Zhang, Mandy
Time: Mon Jul 16 13:37:31 2018
Hi John
That's really a better way, thank you very much!
Mandy
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, July 16, 2018 2:25:21 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for generating
Gen_Vx_mask
Mandy,
We just need to figure out a way of defining that mask using fewer
points.
Here's an idea.
(1) Run gen_vx_mask to apply the CONUS mask from the MET release to
your
grid:
time /usr/local/met-7.0/bin/gen_vx_mask \
2018041800acc01_003600\(1\).nc \
/usr/local/met-7.0/share/met/poly/CONUS.poly \
conus_mask.nc -v 3
Takes about 20 seconds.
(2) Run gen_vx_mask a second time keeping only points east of -104
degrees
longitude:
/usr/local/met-7.0/bin/gen_vx_mask \
conus_mask.nc conus_mask.nc \
conus_east_of_rockies.nc \
-type lon -thresh ge-104 -intersection \
-name CONUS_EAST
(3) Plot the result (see attached):
/usr/local/met-7.0/bin/plot_data_plane \
conus_east_of_rockies.nc \
conus_east.ps \
'name="CONUS_EAST"; level="(*,*)";'
Is that the type of mask you're looking for?
Another alternative would be using a GIS shapefile to define the CONUS
region, but that isn't available until the next release of MET. But
we do
have beta versions out there for testing:
https://dtcenter.org/met/users/downloads/existing_met_builds/METv7.1_beta6_existing_met_builds.php
Thanks,
John
On Mon, Jul 16, 2018 at 1:04 PM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
>
> Hi John
>
> yes , it is continental US and excludes the west of Rocky mountain.
>
> Thank you!
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, July 16, 2018 12:25:49 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for
generating
> Gen_Vx_mask
>
> What is the region you're trying to include? Is it the continental
US?
>
> John
>
> On Mon, Jul 16, 2018 at 10:13 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164 >
> >
> > Hi John,
> >
> > I define a polyline with longitude from 103w to 75w and
latitude
> from
> > 30n~47 N at 3km grid space. And it spend 4 days to finish.
> >
> > Do you have any suggestion to do it faster?
> >
> > Thank you very much!
> >
> > Mandy
> >
> > ________________________________
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, July 16, 2018 10:57:31 AM
> > To: Zhang, Mandy
> > Subject: Re: [rt.rap.ucar.edu #86164] longtime waiting for
generating
> > Gen_Vx_mask
> >
> > Mandy,
> >
> > The runtime is so slow because that is such a large number of
lat/lon
> > points to include in a masking region. The Gen-Vx-Mask tool is
checking
> > whether the lat/lon corresponding to each grid point is inside
that
> > polyline region. For a polyline consisting of nearly 700,000
points,
> that
> > takes a long time! I'm running this case on my machine now, and I
see
> that
> > it's taking a long time as well. But it's not consuming very much
> memory.
> > Presumably, it would eventually finish but I really don't know how
long
> it
> > would take.
> >
> > I can tell you that applying the CONUS.poly file to this grid
takes about
> > 20 seconds on my machine. And the CONUS polyline only has 243
points in
> > it.
> >
> > How did you end up defining a polyline with 691966 points in it?
> >
> > Thanks,
> > John
> >
> >
> >
> >
> >
> > On Thu, Jul 12, 2018 at 9:10 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Thu Jul 12 09:10:01 2018: Request 86164 was acted upon.
> > > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > > Queue: met_help
> > > Subject: longtime waiting for generating Gen_Vx_mask
> > > Owner: Nobody
> > > Requestors: Man.Zhang-1 at ou.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86164
> >
> > >
> > >
> > > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > > them, click the links below.
> > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > > >
> > >
> > > 2018041800acc01_003600(1).nc<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EeftP16GEVdOlUXCX7sBBckB7peiQmJ9mK6JkIJs1PuyoQ
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > > >
> > >
> > > MASK.poly<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EbXVc47CBbpFiW0P2QU4I5wBGhDqPQNOrgyrAhqG1Xnz1w
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > > >
> > >
> > > test_gen_vx_mask.sh<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EdNMvXnu6ElLuePXFG10TKMBpYgMzmK2xg1b2JqUJETrcQ
> > > >
> > >
> > >
> > >
> > >
> > > Dear All,
> > >
> > > I try to generate mask region, the grid data (model
output)
> > > covers CONUS domain with 1623*1123 grids, and the mask input
file has
> > > 691966 points. when I run sh test_gen_vx_mask.sh, screen shows
it is
> > > running without any errors, but it looks like keeping computing
and
> will
> > > not be completed.
> > >
> > > *** Running Gen-Vx-Mask to generate a polyline mask file for the
> > > Continental United States ***
> > > DEBUG 1: Input File:
> > > /cnproj1/mzhang/met_test/ensm/model_data/20180418-acc/m01/
> > > 2018041800acc01_003600.nc
> > > DEBUG 1: Mask File: MASK.poly
> > > DEBUG 2: Parsed Data Grid: Lambert Conformal (1623 x 1123)
> > > DEBUG 2: Parsed Lat/Lon Mask File: MASK containing 691966 points
> > > *********************************************************
> > > please find my attached files for details.
> > > Does anyone know what's the issue?
> > > Thank you!
> > > Mandy
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list