[Met_help] [rt.rap.ucar.edu #89469] History for MET V7.0, help with Ensemble_stat
David Fillmore via RT
met_help at ucar.edu
Thu Apr 18 15:12:26 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am trying to test out the ensemble_stat routine on some COAMPS flatfile 10
m wind output I have from a 20 member ensemble run. I wasn't quite sure how
to get the flatfile into a netcdf format that could be read in by MET so I
just used ascii2nc to on an ascii file containing a "flattened" list of
uwind forecasts at each latitude and longitude within my COAMPS regional
grid. Once I had my files named ens00x_nest1.nc, I ran ensemble_stat on an
asci file containing a list of the locations of these netcdf files, changed
my config file and copied the 20th ensemble member (ens020_nest1.nc) to be
my obs netcdf file (obs.nc). For a 10 member stat run, I get the following:
ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc -outdir ./ -v 2
DEBUG 1: Default Config File:
/software/depot/met-7.0/share/met/config/EnsembleStatConfig_default
DEBUG 1: User Config File: ./config.rc
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=322607452
DEBUG 1: Ensemble Files[10]:
DEBUG 1: ./nest1/ens001_nest1.nc
DEBUG 1: ./nest1/ens002_nest1.nc
DEBUG 1: ./nest1/ens003_nest1.nc
DEBUG 1: ./nest1/ens004_nest1.nc
DEBUG 1: ./nest1/ens005_nest1.nc
DEBUG 1: ./nest1/ens006_nest1.nc
DEBUG 1: ./nest1/ens007_nest1.nc
DEBUG 1: ./nest1/ens008_nest1.nc
DEBUG 1: ./nest1/ens009_nest1.nc
DEBUG 1: ./nest1/ens010_nest1.nc
DEBUG 1: Gridded Observation Files[1]:
DEBUG 1: obs/obs.nc
ERROR :
ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &) const ->
star found in bad slot
ERROR :
I know I definitely made some errors in the formatting of the netcdf file
seeing as ascii2nc is actually meant for ob conversion to netcdf, but this
was the best way I could find seeing as there is currently no way to read in
a IEEE-BE data file into MET.
Let me know if you need more information from me.
Thanks,
Justin
Justin Tsu
Marine Meteorology Division
Data Assimilation/Mesoscale Modeling
Building 704 Room 212
Naval Research Laboratory, Code 7531
7 Grace Hopper Avenue
Monterey, CA 93943-5502
Ph. (831) 656-4111
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: David Fillmore
Time: Tue Mar 26 12:54:41 2019
Hi Justin -
ascii2nc reformats observations as input for the Point-Stat tool,
so I don't expect Ensemble Stat to work on ascii2nc output used for
forecast files.
The best way may be to convert the COAMPS flat files to CF compliant
netcdf.
Much of our support staff is out this week,
so let us get back to you early next week with other possible
solutions.
Are you able to share sample COAMPS data files with us here?
If so there are instructions for uploading files to our ftp site at
this link:
https://dtcenter.org/met/users/support/met_help.php
thanks,
David
On Mon Mar 25 14:35:19 2019, justin.tsu at nrlmry.navy.mil wrote:
> Hello,
>
>
>
> I am trying to test out the ensemble_stat routine on some COAMPS
flatfile 10
> m wind output I have from a 20 member ensemble run. I wasn't quite
sure how
> to get the flatfile into a netcdf format that could be read in by
MET so I
> just used ascii2nc to on an ascii file containing a "flattened" list
of
> uwind forecasts at each latitude and longitude within my COAMPS
regional
> grid. Once I had my files named ens00x_nest1.nc, I ran
ensemble_stat on an
> asci file containing a list of the locations of these netcdf files,
changed
> my config file and copied the 20th ensemble member (ens020_nest1.nc)
to be
> my obs netcdf file (obs.nc). For a 10 member stat run, I get the
following:
>
>
>
> ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc -outdir
./ -v 2
>
> DEBUG 1: Default Config File:
> /software/depot/met-7.0/share/met/config/EnsembleStatConfig_default
>
> DEBUG 1: User Config File: ./config.rc
>
> GSL_RNG_TYPE=mt19937
>
> GSL_RNG_SEED=322607452
>
> DEBUG 1: Ensemble Files[10]:
>
> DEBUG 1: ./nest1/ens001_nest1.nc
>
> DEBUG 1: ./nest1/ens002_nest1.nc
>
> DEBUG 1: ./nest1/ens003_nest1.nc
>
> DEBUG 1: ./nest1/ens004_nest1.nc
>
> DEBUG 1: ./nest1/ens005_nest1.nc
>
> DEBUG 1: ./nest1/ens006_nest1.nc
>
> DEBUG 1: ./nest1/ens007_nest1.nc
>
> DEBUG 1: ./nest1/ens008_nest1.nc
>
> DEBUG 1: ./nest1/ens009_nest1.nc
>
> DEBUG 1: ./nest1/ens010_nest1.nc
>
> DEBUG 1: Gridded Observation Files[1]:
>
> DEBUG 1: obs/obs.nc
>
> ERROR :
>
> ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
const ->
> star found in bad slot
>
> ERROR :
>
>
>
>
>
> I know I definitely made some errors in the formatting of the netcdf
file
> seeing as ascii2nc is actually meant for ob conversion to netcdf,
but this
> was the best way I could find seeing as there is currently no way to
read in
> a IEEE-BE data file into MET.
>
>
>
>
>
> Let me know if you need more information from me.
>
>
>
> Thanks,
>
> Justin
>
>
>
> Justin Tsu
>
> Marine Meteorology Division
>
> Data Assimilation/Mesoscale Modeling
>
> Building 704 Room 212
>
> Naval Research Laboratory, Code 7531
>
> 7 Grace Hopper Avenue
>
> Monterey, CA 93943-5502
>
>
>
> Ph. (831) 656-4111
>
>
>
------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: Tsu, Mr. Justin
Time: Wed Mar 27 13:30:18 2019
I think I am able to share some code. I've uploaded COAMPS data in
COAMPS.tar
under tsu/ in met_help/. The flatfiles are for 3 atmosphereic nests
(denoted
by 1a, 2a, 3a) and I've uploaded latitude, longitude, and 10 meter U-
WIND at
the analysis time at 2018051700. The data for each of the 20 ensemble
members
is in this tar file.
Thanks,
Justin
-----Original Message-----
From: David Fillmore via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, March 26, 2019 11:55 AM
To: Tsu, Mr. Justin
Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
Hi Justin -
ascii2nc reformats observations as input for the Point-Stat tool,
so I don't expect Ensemble Stat to work on ascii2nc output used for
forecast
files.
The best way may be to convert the COAMPS flat files to CF compliant
netcdf.
Much of our support staff is out this week,
so let us get back to you early next week with other possible
solutions.
Are you able to share sample COAMPS data files with us here?
If so there are instructions for uploading files to our ftp site at
this link:
https://dtcenter.org/met/users/support/met_help.php
thanks,
David
On Mon Mar 25 14:35:19 2019, justin.tsu at nrlmry.navy.mil wrote:
> Hello,
>
>
>
> I am trying to test out the ensemble_stat routine on some COAMPS
flatfile 10
> m wind output I have from a 20 member ensemble run. I wasn't quite
sure how
> to get the flatfile into a netcdf format that could be read in by
MET so I
> just used ascii2nc to on an ascii file containing a "flattened" list
of
> uwind forecasts at each latitude and longitude within my COAMPS
regional
> grid. Once I had my files named ens00x_nest1.nc, I ran
ensemble_stat on an
> asci file containing a list of the locations of these netcdf files,
changed
> my config file and copied the 20th ensemble member (ens020_nest1.nc)
to be
> my obs netcdf file (obs.nc). For a 10 member stat run, I get the
following:
>
>
>
> ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc -outdir
./ -v 2
>
> DEBUG 1: Default Config File:
> /software/depot/met-7.0/share/met/config/EnsembleStatConfig_default
>
> DEBUG 1: User Config File: ./config.rc
>
> GSL_RNG_TYPE=mt19937
>
> GSL_RNG_SEED=322607452
>
> DEBUG 1: Ensemble Files[10]:
>
> DEBUG 1: ./nest1/ens001_nest1.nc
>
> DEBUG 1: ./nest1/ens002_nest1.nc
>
> DEBUG 1: ./nest1/ens003_nest1.nc
>
> DEBUG 1: ./nest1/ens004_nest1.nc
>
> DEBUG 1: ./nest1/ens005_nest1.nc
>
> DEBUG 1: ./nest1/ens006_nest1.nc
>
> DEBUG 1: ./nest1/ens007_nest1.nc
>
> DEBUG 1: ./nest1/ens008_nest1.nc
>
> DEBUG 1: ./nest1/ens009_nest1.nc
>
> DEBUG 1: ./nest1/ens010_nest1.nc
>
> DEBUG 1: Gridded Observation Files[1]:
>
> DEBUG 1: obs/obs.nc
>
> ERROR :
>
> ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
const ->
> star found in bad slot
>
> ERROR :
>
>
>
>
>
> I know I definitely made some errors in the formatting of the netcdf
file
> seeing as ascii2nc is actually meant for ob conversion to netcdf,
but this
> was the best way I could find seeing as there is currently no way to
read in
> a IEEE-BE data file into MET.
>
>
>
>
>
> Let me know if you need more information from me.
>
>
>
> Thanks,
>
> Justin
>
>
>
> Justin Tsu
>
> Marine Meteorology Division
>
> Data Assimilation/Mesoscale Modeling
>
> Building 704 Room 212
>
> Naval Research Laboratory, Code 7531
>
> 7 Grace Hopper Avenue
>
> Monterey, CA 93943-5502
>
>
>
> Ph. (831) 656-4111
>
>
>
------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: David Fillmore
Time: Thu Mar 28 16:56:55 2019
Thanks Justin,
I have retrieved your COAMPS data files.
David
On Wed Mar 27 13:30:18 2019, justin.tsu at nrlmry.navy.mil wrote:
> I think I am able to share some code. I've uploaded COAMPS data in
> COAMPS.tar
> under tsu/ in met_help/. The flatfiles are for 3 atmosphereic nests
> (denoted
> by 1a, 2a, 3a) and I've uploaded latitude, longitude, and 10 meter
U-
> WIND at
> the analysis time at 2018051700. The data for each of the 20
ensemble
> members
> is in this tar file.
>
> Thanks,
> Justin
>
> -----Original Message-----
> From: David Fillmore via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, March 26, 2019 11:55 AM
> To: Tsu, Mr. Justin
> Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
>
> Hi Justin -
>
> ascii2nc reformats observations as input for the Point-Stat tool,
> so I don't expect Ensemble Stat to work on ascii2nc output used for
> forecast
> files.
>
> The best way may be to convert the COAMPS flat files to CF compliant
> netcdf.
>
> Much of our support staff is out this week,
> so let us get back to you early next week with other possible
> solutions.
>
> Are you able to share sample COAMPS data files with us here?
>
> If so there are instructions for uploading files to our ftp site at
> this link:
> https://dtcenter.org/met/users/support/met_help.php
>
> thanks,
> David
>
> On Mon Mar 25 14:35:19 2019, justin.tsu at nrlmry.navy.mil wrote:
> > Hello,
> >
> >
> >
> > I am trying to test out the ensemble_stat routine on some COAMPS
> > flatfile 10
> > m wind output I have from a 20 member ensemble run. I wasn't
quite
> > sure how
> > to get the flatfile into a netcdf format that could be read in by
MET
> > so I
> > just used ascii2nc to on an ascii file containing a "flattened"
list
> > of
> > uwind forecasts at each latitude and longitude within my COAMPS
> > regional
> > grid. Once I had my files named ens00x_nest1.nc, I ran
ensemble_stat
> > on an
> > asci file containing a list of the locations of these netcdf
files,
> > changed
> > my config file and copied the 20th ensemble member
(ens020_nest1.nc)
> > to be
> > my obs netcdf file (obs.nc). For a 10 member stat run, I get the
> > following:
> >
> >
> >
> > ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc
-outdir
> > ./ -v 2
> >
> > DEBUG 1: Default Config File:
> > /software/depot/met-
7.0/share/met/config/EnsembleStatConfig_default
> >
> > DEBUG 1: User Config File: ./config.rc
> >
> > GSL_RNG_TYPE=mt19937
> >
> > GSL_RNG_SEED=322607452
> >
> > DEBUG 1: Ensemble Files[10]:
> >
> > DEBUG 1: ./nest1/ens001_nest1.nc
> >
> > DEBUG 1: ./nest1/ens002_nest1.nc
> >
> > DEBUG 1: ./nest1/ens003_nest1.nc
> >
> > DEBUG 1: ./nest1/ens004_nest1.nc
> >
> > DEBUG 1: ./nest1/ens005_nest1.nc
> >
> > DEBUG 1: ./nest1/ens006_nest1.nc
> >
> > DEBUG 1: ./nest1/ens007_nest1.nc
> >
> > DEBUG 1: ./nest1/ens008_nest1.nc
> >
> > DEBUG 1: ./nest1/ens009_nest1.nc
> >
> > DEBUG 1: ./nest1/ens010_nest1.nc
> >
> > DEBUG 1: Gridded Observation Files[1]:
> >
> > DEBUG 1: obs/obs.nc
> >
> > ERROR :
> >
> > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
> > const ->
> > star found in bad slot
> >
> > ERROR :
> >
> >
> >
> >
> >
> > I know I definitely made some errors in the formatting of the
netcdf
> > file
> > seeing as ascii2nc is actually meant for ob conversion to netcdf,
but
> > this
> > was the best way I could find seeing as there is currently no way
to
> > read in
> > a IEEE-BE data file into MET.
> >
> >
> >
> >
> >
> > Let me know if you need more information from me.
> >
> >
> >
> > Thanks,
> >
> > Justin
> >
> >
> >
> > Justin Tsu
> >
> > Marine Meteorology Division
> >
> > Data Assimilation/Mesoscale Modeling
> >
> > Building 704 Room 212
> >
> > Naval Research Laboratory, Code 7531
> >
> > 7 Grace Hopper Avenue
> >
> > Monterey, CA 93943-5502
> >
> >
> >
> > Ph. (831) 656-4111
> >
> >
> >
------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: Tsu, Mr. Justin
Time: Fri Mar 29 15:58:38 2019
Hi David,
Im having some trouble creating a CF compliant netcdf. I am a little
bit
confused because the data that I have is as follows
For a grid that has 75 x points and 73 y points:
*lat(75,73)
*lon(75,73)
*uwind(75,73)
So basically each of these 2-D arrays has 75*73 points on a grid. For
most of
these CF-compliant netcdf examples I've seen online, I see that they
have the
following dimensions for their lat, lon, and var arrays:
*lat(lat)
*lon(lon)
*uwind(lon,lat)
So what I end up doing is I flatten my 2-D arrays so that they are all
75*73
in length and put these into my netcdf file with the dimensions
*lat(fcst)
*lon(fcst)
*uwind(fcst)
Where fcst is 75*73. I do not see this method anywhere online which
confuses
me because I've been reading the examples laid out in
http://cfconventions.org/Data/cf-conventions/cf-conventions-
1.6/build/cf-conventions.html
particularly the section where they have coordinate projections in
their data.
In these examples, they dimension their variables with lat(lat),
lon(lon),
var(lon,lat). How does that work? I'm assuming the one dimensional
lat and
lon arrays are expanded to be 2-D (lon,lat) by using something like a
meshgrid
to duplicate the latitudes columnwise and the longitudes rowwise.
However,
you cant use a meshgrid to replicate latitudes and longitudes when
using
projections, can you?
Sorry if this is a confusing email, I am just very much confused
myself since
my data is so wacky.
Feel free to ask me to elaborate on anything I just wrote here.
Thanks,
Justin
-----Original Message-----
From: David Fillmore via RT [mailto:met_help at ucar.edu]
Sent: Thursday, March 28, 2019 3:57 PM
To: Tsu, Mr. Justin
Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
Thanks Justin,
I have retrieved your COAMPS data files.
David
On Wed Mar 27 13:30:18 2019, justin.tsu at nrlmry.navy.mil wrote:
> I think I am able to share some code. I've uploaded COAMPS data in
> COAMPS.tar
> under tsu/ in met_help/. The flatfiles are for 3 atmosphereic nests
> (denoted
> by 1a, 2a, 3a) and I've uploaded latitude, longitude, and 10 meter
U-
> WIND at
> the analysis time at 2018051700. The data for each of the 20
ensemble
> members
> is in this tar file.
>
> Thanks,
> Justin
>
> -----Original Message-----
> From: David Fillmore via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, March 26, 2019 11:55 AM
> To: Tsu, Mr. Justin
> Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
>
> Hi Justin -
>
> ascii2nc reformats observations as input for the Point-Stat tool,
> so I don't expect Ensemble Stat to work on ascii2nc output used for
> forecast
> files.
>
> The best way may be to convert the COAMPS flat files to CF compliant
> netcdf.
>
> Much of our support staff is out this week,
> so let us get back to you early next week with other possible
> solutions.
>
> Are you able to share sample COAMPS data files with us here?
>
> If so there are instructions for uploading files to our ftp site at
> this link:
> https://dtcenter.org/met/users/support/met_help.php
>
> thanks,
> David
>
> On Mon Mar 25 14:35:19 2019, justin.tsu at nrlmry.navy.mil wrote:
> > Hello,
> >
> >
> >
> > I am trying to test out the ensemble_stat routine on some COAMPS
> > flatfile 10
> > m wind output I have from a 20 member ensemble run. I wasn't
quite
> > sure how
> > to get the flatfile into a netcdf format that could be read in by
MET
> > so I
> > just used ascii2nc to on an ascii file containing a "flattened"
list
> > of
> > uwind forecasts at each latitude and longitude within my COAMPS
> > regional
> > grid. Once I had my files named ens00x_nest1.nc, I ran
ensemble_stat
> > on an
> > asci file containing a list of the locations of these netcdf
files,
> > changed
> > my config file and copied the 20th ensemble member
(ens020_nest1.nc)
> > to be
> > my obs netcdf file (obs.nc). For a 10 member stat run, I get the
> > following:
> >
> >
> >
> > ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc
-outdir
> > ./ -v 2
> >
> > DEBUG 1: Default Config File:
> > /software/depot/met-
7.0/share/met/config/EnsembleStatConfig_default
> >
> > DEBUG 1: User Config File: ./config.rc
> >
> > GSL_RNG_TYPE=mt19937
> >
> > GSL_RNG_SEED=322607452
> >
> > DEBUG 1: Ensemble Files[10]:
> >
> > DEBUG 1: ./nest1/ens001_nest1.nc
> >
> > DEBUG 1: ./nest1/ens002_nest1.nc
> >
> > DEBUG 1: ./nest1/ens003_nest1.nc
> >
> > DEBUG 1: ./nest1/ens004_nest1.nc
> >
> > DEBUG 1: ./nest1/ens005_nest1.nc
> >
> > DEBUG 1: ./nest1/ens006_nest1.nc
> >
> > DEBUG 1: ./nest1/ens007_nest1.nc
> >
> > DEBUG 1: ./nest1/ens008_nest1.nc
> >
> > DEBUG 1: ./nest1/ens009_nest1.nc
> >
> > DEBUG 1: ./nest1/ens010_nest1.nc
> >
> > DEBUG 1: Gridded Observation Files[1]:
> >
> > DEBUG 1: obs/obs.nc
> >
> > ERROR :
> >
> > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
> > const ->
> > star found in bad slot
> >
> > ERROR :
> >
> >
> >
> >
> >
> > I know I definitely made some errors in the formatting of the
netcdf
> > file
> > seeing as ascii2nc is actually meant for ob conversion to netcdf,
but
> > this
> > was the best way I could find seeing as there is currently no way
to
> > read in
> > a IEEE-BE data file into MET.
> >
> >
> >
> >
> >
> > Let me know if you need more information from me.
> >
> >
> >
> > Thanks,
> >
> > Justin
> >
> >
> >
> > Justin Tsu
> >
> > Marine Meteorology Division
> >
> > Data Assimilation/Mesoscale Modeling
> >
> > Building 704 Room 212
> >
> > Naval Research Laboratory, Code 7531
> >
> > 7 Grace Hopper Avenue
> >
> > Monterey, CA 93943-5502
> >
> >
> >
> > Ph. (831) 656-4111
> >
> >
> >
------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: David Fillmore
Time: Mon Apr 01 15:40:04 2019
Hi Justin -
For projected coordinate systems see the section 'Two-Dimensional
Latitude, Longitude, Coordinate Variables'
in the CF document:
http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-
conventions.pdf
I think in your case you just want to reshape your flattened lon(75 *
73) to lon(75, 73),
and similarly for lat. meshgrid may not be necessary.
Does that make sense?
thanks,
David
On Fri Mar 29 15:58:38 2019, justin.tsu at nrlmry.navy.mil wrote:
> Hi David,
>
> Im having some trouble creating a CF compliant netcdf. I am a
little
> bit
> confused because the data that I have is as follows
>
> For a grid that has 75 x points and 73 y points:
> *lat(75,73)
> *lon(75,73)
> *uwind(75,73)
>
> So basically each of these 2-D arrays has 75*73 points on a grid.
For
> most of
> these CF-compliant netcdf examples I've seen online, I see that they
> have the
> following dimensions for their lat, lon, and var arrays:
>
> *lat(lat)
> *lon(lon)
> *uwind(lon,lat)
>
> So what I end up doing is I flatten my 2-D arrays so that they are
all
> 75*73
> in length and put these into my netcdf file with the dimensions
> *lat(fcst)
> *lon(fcst)
> *uwind(fcst)
>
> Where fcst is 75*73. I do not see this method anywhere online which
> confuses
> me because I've been reading the examples laid out in
> http://cfconventions.org/Data/cf-conventions/cf-conventions-
> 1.6/build/cf-conventions.html
> particularly the section where they have coordinate projections in
> their data.
> In these examples, they dimension their variables with lat(lat),
> lon(lon),
> var(lon,lat). How does that work? I'm assuming the one dimensional
> lat and
> lon arrays are expanded to be 2-D (lon,lat) by using something like
a
> meshgrid
> to duplicate the latitudes columnwise and the longitudes rowwise.
> However,
> you cant use a meshgrid to replicate latitudes and longitudes when
> using
> projections, can you?
>
> Sorry if this is a confusing email, I am just very much confused
> myself since
> my data is so wacky.
>
> Feel free to ask me to elaborate on anything I just wrote here.
>
> Thanks,
> Justin
>
> -----Original Message-----
> From: David Fillmore via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, March 28, 2019 3:57 PM
> To: Tsu, Mr. Justin
> Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
>
> Thanks Justin,
> I have retrieved your COAMPS data files.
> David
>
> On Wed Mar 27 13:30:18 2019, justin.tsu at nrlmry.navy.mil wrote:
> > I think I am able to share some code. I've uploaded COAMPS data in
> > COAMPS.tar
> > under tsu/ in met_help/. The flatfiles are for 3 atmosphereic
nests
> > (denoted
> > by 1a, 2a, 3a) and I've uploaded latitude, longitude, and 10 meter
U-
> > WIND at
> > the analysis time at 2018051700. The data for each of the 20
> > ensemble
> > members
> > is in this tar file.
> >
> > Thanks,
> > Justin
> >
> > -----Original Message-----
> > From: David Fillmore via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, March 26, 2019 11:55 AM
> > To: Tsu, Mr. Justin
> > Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with
Ensemble_stat
> >
> > Hi Justin -
> >
> > ascii2nc reformats observations as input for the Point-Stat tool,
> > so I don't expect Ensemble Stat to work on ascii2nc output used
for
> > forecast
> > files.
> >
> > The best way may be to convert the COAMPS flat files to CF
compliant
> > netcdf.
> >
> > Much of our support staff is out this week,
> > so let us get back to you early next week with other possible
> > solutions.
> >
> > Are you able to share sample COAMPS data files with us here?
> >
> > If so there are instructions for uploading files to our ftp site
at
> > this link:
> > https://dtcenter.org/met/users/support/met_help.php
> >
> > thanks,
> > David
> >
> > On Mon Mar 25 14:35:19 2019, justin.tsu at nrlmry.navy.mil wrote:
> > > Hello,
> > >
> > >
> > >
> > > I am trying to test out the ensemble_stat routine on some COAMPS
> > > flatfile 10
> > > m wind output I have from a 20 member ensemble run. I wasn't
quite
> > > sure how
> > > to get the flatfile into a netcdf format that could be read in
by
> > > MET
> > > so I
> > > just used ascii2nc to on an ascii file containing a "flattened"
> > > list
> > > of
> > > uwind forecasts at each latitude and longitude within my COAMPS
> > > regional
> > > grid. Once I had my files named ens00x_nest1.nc, I ran
> > > ensemble_stat
> > > on an
> > > asci file containing a list of the locations of these netcdf
files,
> > > changed
> > > my config file and copied the 20th ensemble member
> > > (ens020_nest1.nc)
> > > to be
> > > my obs netcdf file (obs.nc). For a 10 member stat run, I get
the
> > > following:
> > >
> > >
> > >
> > > ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc
> > > -outdir
> > > ./ -v 2
> > >
> > > DEBUG 1: Default Config File:
> > > /software/depot/met-
7.0/share/met/config/EnsembleStatConfig_default
> > >
> > > DEBUG 1: User Config File: ./config.rc
> > >
> > > GSL_RNG_TYPE=mt19937
> > >
> > > GSL_RNG_SEED=322607452
> > >
> > > DEBUG 1: Ensemble Files[10]:
> > >
> > > DEBUG 1: ./nest1/ens001_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens002_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens003_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens004_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens005_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens006_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens007_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens008_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens009_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens010_nest1.nc
> > >
> > > DEBUG 1: Gridded Observation Files[1]:
> > >
> > > DEBUG 1: obs/obs.nc
> > >
> > > ERROR :
> > >
> > > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane
&)
> > > const ->
> > > star found in bad slot
> > >
> > > ERROR :
> > >
> > >
> > >
> > >
> > >
> > > I know I definitely made some errors in the formatting of the
> > > netcdf
> > > file
> > > seeing as ascii2nc is actually meant for ob conversion to
netcdf,
> > > but
> > > this
> > > was the best way I could find seeing as there is currently no
way
> > > to
> > > read in
> > > a IEEE-BE data file into MET.
> > >
> > >
> > >
> > >
> > >
> > > Let me know if you need more information from me.
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Justin
> > >
> > >
> > >
> > > Justin Tsu
> > >
> > > Marine Meteorology Division
> > >
> > > Data Assimilation/Mesoscale Modeling
> > >
> > > Building 704 Room 212
> > >
> > > Naval Research Laboratory, Code 7531
> > >
> > > 7 Grace Hopper Avenue
> > >
> > > Monterey, CA 93943-5502
> > >
> > >
> > >
> > > Ph. (831) 656-4111
> > >
> > >
> > >
------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: David Fillmore
Time: Mon Apr 01 16:06:33 2019
Hi Justin -
There are also some sample python analysis scripts for NRL available
here:
https://dtcenter.org/met/users/downloads/analysis_scripts.php
I think you should be able to use those as a starting point.
thanks,
David
------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: Tsu, Mr. Justin
Time: Thu Apr 04 11:38:51 2019
Hi David,
Thanks for the response and I apologize for not getting back to you
sooner. Checking this link, I am particularly interested in the
section which displays the code:
dimensions:
xc = 128 ;
yc = 64 ;
lev = 18 ;
variables:
float T(lev,yc,xc) ;
T:long_name = "temperature" ;
T:units = "K" ;
T:coordinates = "lon lat" ;
float xc(xc) ;
xc:axis = "X" ;
xc:long_name = "x-coordinate in Cartesian system" ;
xc:units = "m" ;
float yc(yc) ;
yc:axis = "Y" ;
yc:long_name = "y-coordinate in Cartesian system" ;
yc:units = "m" ;
float lev(lev) ;
lev:long_name = "pressure level" ;
lev:units = "hPa" ;
float lon(yc,xc) ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
float lat(yc,xc) ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
It has been difficult for me to resolve the relationships between the
"coordinates" and "lon/lat". So I guess I'll just try and break down
my thought process and you can correct me if I am incorrect in my
assumptions
Okay, so it looks like they use some indexing value called (xc, yc)
that supposedly points to a particular latitude, longitude (in
lon(yc,xc), lat(yx,xc)), and climate variable value (T(lev,yc,xc)). I
am just going to ignore lev here because that part is pretty self
explanatory. The part that confuses me is the xc(xc) and yc(yc) part.
* Why are these two variables one dimensional? The only way that this
makes sense to me is if these two variables are literally just the
numbers 1 to 128 for xc and 1 to 64 for yc. The part that trips me up
is the unit... meter? If these were truly measurements of meters then
they wouldn’t be range(1,128) and range(1,64). That means that they
must be some discrete measurement distance, and since they are one
dimensional, how do you use a meter measurement as an index for
latitude and longitude?
I think that’s my only question for now.
Justin
-----Original Message-----
From: David Fillmore via RT [mailto:met_help at ucar.edu]
Sent: Monday, April 1, 2019 2:40 PM
To: Tsu, Mr. Justin
Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
Hi Justin -
For projected coordinate systems see the section 'Two-Dimensional
Latitude, Longitude, Coordinate Variables'
in the CF document:
http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-
conventions.pdf
I think in your case you just want to reshape your flattened lon(75 *
73) to lon(75, 73),
and similarly for lat. meshgrid may not be necessary.
Does that make sense?
thanks,
David
On Fri Mar 29 15:58:38 2019, justin.tsu at nrlmry.navy.mil wrote:
> Hi David,
>
> Im having some trouble creating a CF compliant netcdf. I am a
little
> bit
> confused because the data that I have is as follows
>
> For a grid that has 75 x points and 73 y points:
> *lat(75,73)
> *lon(75,73)
> *uwind(75,73)
>
> So basically each of these 2-D arrays has 75*73 points on a grid.
For
> most of
> these CF-compliant netcdf examples I've seen online, I see that they
> have the
> following dimensions for their lat, lon, and var arrays:
>
> *lat(lat)
> *lon(lon)
> *uwind(lon,lat)
>
> So what I end up doing is I flatten my 2-D arrays so that they are
all
> 75*73
> in length and put these into my netcdf file with the dimensions
> *lat(fcst)
> *lon(fcst)
> *uwind(fcst)
>
> Where fcst is 75*73. I do not see this method anywhere online which
> confuses
> me because I've been reading the examples laid out in
> http://cfconventions.org/Data/cf-conventions/cf-conventions-
> 1.6/build/cf-conventions.html
> particularly the section where they have coordinate projections in
> their data.
> In these examples, they dimension their variables with lat(lat),
> lon(lon),
> var(lon,lat). How does that work? I'm assuming the one dimensional
> lat and
> lon arrays are expanded to be 2-D (lon,lat) by using something like
a
> meshgrid
> to duplicate the latitudes columnwise and the longitudes rowwise.
> However,
> you cant use a meshgrid to replicate latitudes and longitudes when
> using
> projections, can you?
>
> Sorry if this is a confusing email, I am just very much confused
> myself since
> my data is so wacky.
>
> Feel free to ask me to elaborate on anything I just wrote here.
>
> Thanks,
> Justin
>
> -----Original Message-----
> From: David Fillmore via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, March 28, 2019 3:57 PM
> To: Tsu, Mr. Justin
> Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
>
> Thanks Justin,
> I have retrieved your COAMPS data files.
> David
>
> On Wed Mar 27 13:30:18 2019, justin.tsu at nrlmry.navy.mil wrote:
> > I think I am able to share some code. I've uploaded COAMPS data in
> > COAMPS.tar
> > under tsu/ in met_help/. The flatfiles are for 3 atmosphereic
nests
> > (denoted
> > by 1a, 2a, 3a) and I've uploaded latitude, longitude, and 10 meter
U-
> > WIND at
> > the analysis time at 2018051700. The data for each of the 20
> > ensemble
> > members
> > is in this tar file.
> >
> > Thanks,
> > Justin
> >
> > -----Original Message-----
> > From: David Fillmore via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, March 26, 2019 11:55 AM
> > To: Tsu, Mr. Justin
> > Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with
Ensemble_stat
> >
> > Hi Justin -
> >
> > ascii2nc reformats observations as input for the Point-Stat tool,
> > so I don't expect Ensemble Stat to work on ascii2nc output used
for
> > forecast
> > files.
> >
> > The best way may be to convert the COAMPS flat files to CF
compliant
> > netcdf.
> >
> > Much of our support staff is out this week,
> > so let us get back to you early next week with other possible
> > solutions.
> >
> > Are you able to share sample COAMPS data files with us here?
> >
> > If so there are instructions for uploading files to our ftp site
at
> > this link:
> > https://dtcenter.org/met/users/support/met_help.php
> >
> > thanks,
> > David
> >
> > On Mon Mar 25 14:35:19 2019, justin.tsu at nrlmry.navy.mil wrote:
> > > Hello,
> > >
> > >
> > >
> > > I am trying to test out the ensemble_stat routine on some COAMPS
> > > flatfile 10
> > > m wind output I have from a 20 member ensemble run. I wasn't
quite
> > > sure how
> > > to get the flatfile into a netcdf format that could be read in
by
> > > MET
> > > so I
> > > just used ascii2nc to on an ascii file containing a "flattened"
> > > list
> > > of
> > > uwind forecasts at each latitude and longitude within my COAMPS
> > > regional
> > > grid. Once I had my files named ens00x_nest1.nc, I ran
> > > ensemble_stat
> > > on an
> > > asci file containing a list of the locations of these netcdf
files,
> > > changed
> > > my config file and copied the 20th ensemble member
> > > (ens020_nest1.nc)
> > > to be
> > > my obs netcdf file (obs.nc). For a 10 member stat run, I get
the
> > > following:
> > >
> > >
> > >
> > > ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc
> > > -outdir
> > > ./ -v 2
> > >
> > > DEBUG 1: Default Config File:
> > > /software/depot/met-
7.0/share/met/config/EnsembleStatConfig_default
> > >
> > > DEBUG 1: User Config File: ./config.rc
> > >
> > > GSL_RNG_TYPE=mt19937
> > >
> > > GSL_RNG_SEED=322607452
> > >
> > > DEBUG 1: Ensemble Files[10]:
> > >
> > > DEBUG 1: ./nest1/ens001_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens002_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens003_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens004_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens005_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens006_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens007_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens008_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens009_nest1.nc
> > >
> > > DEBUG 1: ./nest1/ens010_nest1.nc
> > >
> > > DEBUG 1: Gridded Observation Files[1]:
> > >
> > > DEBUG 1: obs/obs.nc
> > >
> > > ERROR :
> > >
> > > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane
&)
> > > const ->
> > > star found in bad slot
> > >
> > > ERROR :
> > >
> > >
> > >
> > >
> > >
> > > I know I definitely made some errors in the formatting of the
> > > netcdf
> > > file
> > > seeing as ascii2nc is actually meant for ob conversion to
netcdf,
> > > but
> > > this
> > > was the best way I could find seeing as there is currently no
way
> > > to
> > > read in
> > > a IEEE-BE data file into MET.
> > >
> > >
> > >
> > >
> > >
> > > Let me know if you need more information from me.
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Justin
> > >
> > >
> > >
> > > Justin Tsu
> > >
> > > Marine Meteorology Division
> > >
> > > Data Assimilation/Mesoscale Modeling
> > >
> > > Building 704 Room 212
> > >
> > > Naval Research Laboratory, Code 7531
> > >
> > > 7 Grace Hopper Avenue
> > >
> > > Monterey, CA 93943-5502
> > >
> > >
> > >
> > > Ph. (831) 656-4111
> > >
> > >
> > >
------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: David Fillmore
Time: Thu Apr 11 13:53:09 2019
Hi Justin -
Sorry for the delay with the reply.
In this case xc = 128, yc = 64 dimension ranges do not imply that the
xc(xc), yc(yc) variables have the same range.
They can in fact take on physical ranges over the spatial domain that
are
in meters.
Have you had any success in converting your NRL binary to MET readable
NetCDF?
I am curious because there are other MET users at NRL that will likely
need
to do something similar
We are going to try craft a general purpose python script for this
task.
thanks,
David
On Thu, Apr 4, 2019 at 11:38 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89469 >
>
> Hi David,
>
> Thanks for the response and I apologize for not getting back to you
> sooner. Checking this link, I am particularly interested in the
section
> which displays the code:
>
> dimensions:
> xc = 128 ;
> yc = 64 ;
> lev = 18 ;
> variables:
> float T(lev,yc,xc) ;
> T:long_name = "temperature" ;
> T:units = "K" ;
> T:coordinates = "lon lat" ;
> float xc(xc) ;
> xc:axis = "X" ;
> xc:long_name = "x-coordinate in Cartesian system" ;
> xc:units = "m" ;
> float yc(yc) ;
> yc:axis = "Y" ;
> yc:long_name = "y-coordinate in Cartesian system" ;
> yc:units = "m" ;
> float lev(lev) ;
> lev:long_name = "pressure level" ;
> lev:units = "hPa" ;
> float lon(yc,xc) ;
> lon:long_name = "longitude" ;
> lon:units = "degrees_east" ;
> float lat(yc,xc) ;
> lat:long_name = "latitude" ;
> lat:units = "degrees_north" ;
>
> It has been difficult for me to resolve the relationships between
the
> "coordinates" and "lon/lat". So I guess I'll just try and break
down my
> thought process and you can correct me if I am incorrect in my
assumptions
>
> Okay, so it looks like they use some indexing value called (xc, yc)
that
> supposedly points to a particular latitude, longitude (in
lon(yc,xc),
> lat(yx,xc)), and climate variable value (T(lev,yc,xc)). I am just
going to
> ignore lev here because that part is pretty self explanatory. The
part
> that confuses me is the xc(xc) and yc(yc) part.
>
> * Why are these two variables one dimensional? The only way that
this
> makes sense to me is if these two variables are literally just the
numbers
> 1 to 128 for xc and 1 to 64 for yc. The part that trips me up is
the
> unit... meter? If these were truly measurements of meters then they
> wouldn’t be range(1,128) and range(1,64). That means that they must
be
> some discrete measurement distance, and since they are one
dimensional, how
> do you use a meter measurement as an index for latitude and
longitude?
>
> I think that’s my only question for now.
>
> Justin
> -----Original Message-----
> From: David Fillmore via RT [mailto:met_help at ucar.edu]
> Sent: Monday, April 1, 2019 2:40 PM
> To: Tsu, Mr. Justin
> Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
>
> Hi Justin -
>
> For projected coordinate systems see the section 'Two-Dimensional
> Latitude, Longitude, Coordinate Variables'
> in the CF document:
>
> http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-
conventions.pdf
>
> I think in your case you just want to reshape your flattened lon(75
* 73)
> to lon(75, 73),
> and similarly for lat. meshgrid may not be necessary.
>
> Does that make sense?
> thanks,
> David
>
> On Fri Mar 29 15:58:38 2019, justin.tsu at nrlmry.navy.mil wrote:
> > Hi David,
> >
> > Im having some trouble creating a CF compliant netcdf. I am a
little
> > bit
> > confused because the data that I have is as follows
> >
> > For a grid that has 75 x points and 73 y points:
> > *lat(75,73)
> > *lon(75,73)
> > *uwind(75,73)
> >
> > So basically each of these 2-D arrays has 75*73 points on a grid.
For
> > most of
> > these CF-compliant netcdf examples I've seen online, I see that
they
> > have the
> > following dimensions for their lat, lon, and var arrays:
> >
> > *lat(lat)
> > *lon(lon)
> > *uwind(lon,lat)
> >
> > So what I end up doing is I flatten my 2-D arrays so that they are
all
> > 75*73
> > in length and put these into my netcdf file with the dimensions
> > *lat(fcst)
> > *lon(fcst)
> > *uwind(fcst)
> >
> > Where fcst is 75*73. I do not see this method anywhere online
which
> > confuses
> > me because I've been reading the examples laid out in
> > http://cfconventions.org/Data/cf-conventions/cf-conventions-
> > 1.6/build/cf-conventions.html
> > particularly the section where they have coordinate projections in
> > their data.
> > In these examples, they dimension their variables with lat(lat),
> > lon(lon),
> > var(lon,lat). How does that work? I'm assuming the one
dimensional
> > lat and
> > lon arrays are expanded to be 2-D (lon,lat) by using something
like a
> > meshgrid
> > to duplicate the latitudes columnwise and the longitudes rowwise.
> > However,
> > you cant use a meshgrid to replicate latitudes and longitudes when
> > using
> > projections, can you?
> >
> > Sorry if this is a confusing email, I am just very much confused
> > myself since
> > my data is so wacky.
> >
> > Feel free to ask me to elaborate on anything I just wrote here.
> >
> > Thanks,
> > Justin
> >
> > -----Original Message-----
> > From: David Fillmore via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, March 28, 2019 3:57 PM
> > To: Tsu, Mr. Justin
> > Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with
Ensemble_stat
> >
> > Thanks Justin,
> > I have retrieved your COAMPS data files.
> > David
> >
> > On Wed Mar 27 13:30:18 2019, justin.tsu at nrlmry.navy.mil wrote:
> > > I think I am able to share some code. I've uploaded COAMPS data
in
> > > COAMPS.tar
> > > under tsu/ in met_help/. The flatfiles are for 3 atmosphereic
nests
> > > (denoted
> > > by 1a, 2a, 3a) and I've uploaded latitude, longitude, and 10
meter U-
> > > WIND at
> > > the analysis time at 2018051700. The data for each of the 20
> > > ensemble
> > > members
> > > is in this tar file.
> > >
> > > Thanks,
> > > Justin
> > >
> > > -----Original Message-----
> > > From: David Fillmore via RT [mailto:met_help at ucar.edu]
> > > Sent: Tuesday, March 26, 2019 11:55 AM
> > > To: Tsu, Mr. Justin
> > > Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with
Ensemble_stat
> > >
> > > Hi Justin -
> > >
> > > ascii2nc reformats observations as input for the Point-Stat
tool,
> > > so I don't expect Ensemble Stat to work on ascii2nc output used
for
> > > forecast
> > > files.
> > >
> > > The best way may be to convert the COAMPS flat files to CF
compliant
> > > netcdf.
> > >
> > > Much of our support staff is out this week,
> > > so let us get back to you early next week with other possible
> > > solutions.
> > >
> > > Are you able to share sample COAMPS data files with us here?
> > >
> > > If so there are instructions for uploading files to our ftp site
at
> > > this link:
> > > https://dtcenter.org/met/users/support/met_help.php
> > >
> > > thanks,
> > > David
> > >
> > > On Mon Mar 25 14:35:19 2019, justin.tsu at nrlmry.navy.mil wrote:
> > > > Hello,
> > > >
> > > >
> > > >
> > > > I am trying to test out the ensemble_stat routine on some
COAMPS
> > > > flatfile 10
> > > > m wind output I have from a 20 member ensemble run. I wasn't
quite
> > > > sure how
> > > > to get the flatfile into a netcdf format that could be read in
by
> > > > MET
> > > > so I
> > > > just used ascii2nc to on an ascii file containing a
"flattened"
> > > > list
> > > > of
> > > > uwind forecasts at each latitude and longitude within my
COAMPS
> > > > regional
> > > > grid. Once I had my files named ens00x_nest1.nc, I ran
> > > > ensemble_stat
> > > > on an
> > > > asci file containing a list of the locations of these netcdf
files,
> > > > changed
> > > > my config file and copied the 20th ensemble member
> > > > (ens020_nest1.nc)
> > > > to be
> > > > my obs netcdf file (obs.nc). For a 10 member stat run, I get
the
> > > > following:
> > > >
> > > >
> > > >
> > > > ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc
> > > > -outdir
> > > > ./ -v 2
> > > >
> > > > DEBUG 1: Default Config File:
> > > > /software/depot/met-
7.0/share/met/config/EnsembleStatConfig_default
> > > >
> > > > DEBUG 1: User Config File: ./config.rc
> > > >
> > > > GSL_RNG_TYPE=mt19937
> > > >
> > > > GSL_RNG_SEED=322607452
> > > >
> > > > DEBUG 1: Ensemble Files[10]:
> > > >
> > > > DEBUG 1: ./nest1/ens001_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens002_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens003_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens004_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens005_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens006_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens007_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens008_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens009_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens010_nest1.nc
> > > >
> > > > DEBUG 1: Gridded Observation Files[1]:
> > > >
> > > > DEBUG 1: obs/obs.nc
> > > >
> > > > ERROR :
> > > >
> > > > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane
&)
> > > > const ->
> > > > star found in bad slot
> > > >
> > > > ERROR :
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I know I definitely made some errors in the formatting of the
> > > > netcdf
> > > > file
> > > > seeing as ascii2nc is actually meant for ob conversion to
netcdf,
> > > > but
> > > > this
> > > > was the best way I could find seeing as there is currently no
way
> > > > to
> > > > read in
> > > > a IEEE-BE data file into MET.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Let me know if you need more information from me.
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Justin
> > > >
> > > >
> > > >
> > > > Justin Tsu
> > > >
> > > > Marine Meteorology Division
> > > >
> > > > Data Assimilation/Mesoscale Modeling
> > > >
> > > > Building 704 Room 212
> > > >
> > > > Naval Research Laboratory, Code 7531
> > > >
> > > > 7 Grace Hopper Avenue
> > > >
> > > > Monterey, CA 93943-5502
> > > >
> > > >
> > > >
> > > > Ph. (831) 656-4111
> > > >
> > > >
> > > >
>
>
>
>
>
------------------------------------------------
Subject: MET V7.0, help with Ensemble_stat
From: Tsu, Mr. Justin
Time: Wed Apr 17 15:23:10 2019
Hi David,
I ended up scrapping the whole CF compliance attempt at creating a
NETCDF file from scratch. I use the PYTHON_NUMPY python embedding
functionality and regrid_data_plane to convert a flatfile (with grid
info) into a netCDF. This gave me a netCDF that seems o have taken
care of everything for me:
[tsu at maury2 ens_work]$ ncdump -h ens003.nc
netcdf ens003 {
dimensions:
lat = 73 ;
lon = 75 ;
variables:
float lat(lat, lon) ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:standard_name = "latitude" ;
float lon(lat, lon) ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:standard_name = "longitude" ;
float uuwind_zht_000010(lat, lon) ;
uuwind_zht_000010:name = "uuwind_zht_000010" ;
uuwind_zht_000010:long_name = "UNKNOWN" ;
uuwind_zht_000010:level = "zht_000010" ;
uuwind_zht_000010:units = "UNKNOWN" ;
uuwind_zht_000010:_FillValue = -9999.f ;
uuwind_zht_000010:init_time = "20180517_000000" ;
uuwind_zht_000010:init_time_ut = "1526515200" ;
uuwind_zht_000010:valid_time = "20180517_000000" ;
uuwind_zht_000010:valid_time_ut = "1526515200" ;
// global attributes:
:FileOrigins = "File ens003.nc generated
20190417_185730 UTC on host maury2.local by the MET regrid_data_plane
tool" ;
:MET_version = "V8.0" ;
:MET_tool = "regrid_data_plane" ;
:RunCommand = "Regrid from Projection: Lambert
Conformal Nx: 75 Ny: 73 Lat_LL: 30.190 Lon_LL: -105.630 Lon_orient:
122.000 Alpha: 422.437 Cone: 0.716 Bx: 283.3939 By: -23.3975 to
Projection: Lambert Conformal Nx: 75 Ny: 73 Lat_LL: 30.190 Lon_LL:
105.630 Lon_orient: -122.000 Alpha: 422.437 Cone: 0.716 Bx: 83.6862
By: -271.7642" ;
:Projection = "Lambert Conformal" ;
:hemisphere = "N" ;
:scale_lat_1 = "60.000000" ;
:scale_lat_2 = "30.000000" ;
:lat_pin = "30.190000" ;
:lon_pin = "-105.630000" ;
:x_pin = "0.000000" ;
:y_pin = "0.000000" ;
:lon_orient = "122.000000" ;
:d_km = "27.000000" ;
:r_km = "6361.200000" ;
:nx = "75" ;
:ny = "73 grid_points" ;
However, now when I run ensemble_stat, it gets stuck in the
lookup_init() function looking for method type "AW_MEAN".... even
though my specified method type for interpoloation is "NEAREST"
[tsu at maury2 ens_work]$ ensemble_stat ens_file_list config.rc
DEBUG 1: Default Config File: /software/depot/met-
7.0/share/met/config/EnsembleStatConfig_default
DEBUG 1: User Config File: config.rc
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744072578119516
ERROR :
ERROR : Dictionary::lookup_int() -> lookup failed for name "AW_MEAN"
ERROR :
Justin
-----Original Message-----
From: David Fillmore via RT [mailto:met_help at ucar.edu]
Sent: Thursday, April 11, 2019 12:53 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #89469] MET V7.0, help with
Ensemble_stat
Hi Justin -
Sorry for the delay with the reply.
In this case xc = 128, yc = 64 dimension ranges do not imply that the
xc(xc), yc(yc) variables have the same range.
They can in fact take on physical ranges over the spatial domain that
are
in meters.
Have you had any success in converting your NRL binary to MET readable
NetCDF?
I am curious because there are other MET users at NRL that will likely
need
to do something similar
We are going to try craft a general purpose python script for this
task.
thanks,
David
On Thu, Apr 4, 2019 at 11:38 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89469 >
>
> Hi David,
>
> Thanks for the response and I apologize for not getting back to you
> sooner. Checking this link, I am particularly interested in the
section
> which displays the code:
>
> dimensions:
> xc = 128 ;
> yc = 64 ;
> lev = 18 ;
> variables:
> float T(lev,yc,xc) ;
> T:long_name = "temperature" ;
> T:units = "K" ;
> T:coordinates = "lon lat" ;
> float xc(xc) ;
> xc:axis = "X" ;
> xc:long_name = "x-coordinate in Cartesian system" ;
> xc:units = "m" ;
> float yc(yc) ;
> yc:axis = "Y" ;
> yc:long_name = "y-coordinate in Cartesian system" ;
> yc:units = "m" ;
> float lev(lev) ;
> lev:long_name = "pressure level" ;
> lev:units = "hPa" ;
> float lon(yc,xc) ;
> lon:long_name = "longitude" ;
> lon:units = "degrees_east" ;
> float lat(yc,xc) ;
> lat:long_name = "latitude" ;
> lat:units = "degrees_north" ;
>
> It has been difficult for me to resolve the relationships between
the
> "coordinates" and "lon/lat". So I guess I'll just try and break
down my
> thought process and you can correct me if I am incorrect in my
assumptions
>
> Okay, so it looks like they use some indexing value called (xc, yc)
that
> supposedly points to a particular latitude, longitude (in
lon(yc,xc),
> lat(yx,xc)), and climate variable value (T(lev,yc,xc)). I am just
going to
> ignore lev here because that part is pretty self explanatory. The
part
> that confuses me is the xc(xc) and yc(yc) part.
>
> * Why are these two variables one dimensional? The only way that
this
> makes sense to me is if these two variables are literally just the
numbers
> 1 to 128 for xc and 1 to 64 for yc. The part that trips me up is
the
> unit... meter? If these were truly measurements of meters then they
> wouldn’t be range(1,128) and range(1,64). That means that they must
be
> some discrete measurement distance, and since they are one
dimensional, how
> do you use a meter measurement as an index for latitude and
longitude?
>
> I think that’s my only question for now.
>
> Justin
> -----Original Message-----
> From: David Fillmore via RT [mailto:met_help at ucar.edu]
> Sent: Monday, April 1, 2019 2:40 PM
> To: Tsu, Mr. Justin
> Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with Ensemble_stat
>
> Hi Justin -
>
> For projected coordinate systems see the section 'Two-Dimensional
> Latitude, Longitude, Coordinate Variables'
> in the CF document:
>
> http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-
conventions.pdf
>
> I think in your case you just want to reshape your flattened lon(75
* 73)
> to lon(75, 73),
> and similarly for lat. meshgrid may not be necessary.
>
> Does that make sense?
> thanks,
> David
>
> On Fri Mar 29 15:58:38 2019, justin.tsu at nrlmry.navy.mil wrote:
> > Hi David,
> >
> > Im having some trouble creating a CF compliant netcdf. I am a
little
> > bit
> > confused because the data that I have is as follows
> >
> > For a grid that has 75 x points and 73 y points:
> > *lat(75,73)
> > *lon(75,73)
> > *uwind(75,73)
> >
> > So basically each of these 2-D arrays has 75*73 points on a grid.
For
> > most of
> > these CF-compliant netcdf examples I've seen online, I see that
they
> > have the
> > following dimensions for their lat, lon, and var arrays:
> >
> > *lat(lat)
> > *lon(lon)
> > *uwind(lon,lat)
> >
> > So what I end up doing is I flatten my 2-D arrays so that they are
all
> > 75*73
> > in length and put these into my netcdf file with the dimensions
> > *lat(fcst)
> > *lon(fcst)
> > *uwind(fcst)
> >
> > Where fcst is 75*73. I do not see this method anywhere online
which
> > confuses
> > me because I've been reading the examples laid out in
> > http://cfconventions.org/Data/cf-conventions/cf-conventions-
> > 1.6/build/cf-conventions.html
> > particularly the section where they have coordinate projections in
> > their data.
> > In these examples, they dimension their variables with lat(lat),
> > lon(lon),
> > var(lon,lat). How does that work? I'm assuming the one
dimensional
> > lat and
> > lon arrays are expanded to be 2-D (lon,lat) by using something
like a
> > meshgrid
> > to duplicate the latitudes columnwise and the longitudes rowwise.
> > However,
> > you cant use a meshgrid to replicate latitudes and longitudes when
> > using
> > projections, can you?
> >
> > Sorry if this is a confusing email, I am just very much confused
> > myself since
> > my data is so wacky.
> >
> > Feel free to ask me to elaborate on anything I just wrote here.
> >
> > Thanks,
> > Justin
> >
> > -----Original Message-----
> > From: David Fillmore via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, March 28, 2019 3:57 PM
> > To: Tsu, Mr. Justin
> > Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with
Ensemble_stat
> >
> > Thanks Justin,
> > I have retrieved your COAMPS data files.
> > David
> >
> > On Wed Mar 27 13:30:18 2019, justin.tsu at nrlmry.navy.mil wrote:
> > > I think I am able to share some code. I've uploaded COAMPS data
in
> > > COAMPS.tar
> > > under tsu/ in met_help/. The flatfiles are for 3 atmosphereic
nests
> > > (denoted
> > > by 1a, 2a, 3a) and I've uploaded latitude, longitude, and 10
meter U-
> > > WIND at
> > > the analysis time at 2018051700. The data for each of the 20
> > > ensemble
> > > members
> > > is in this tar file.
> > >
> > > Thanks,
> > > Justin
> > >
> > > -----Original Message-----
> > > From: David Fillmore via RT [mailto:met_help at ucar.edu]
> > > Sent: Tuesday, March 26, 2019 11:55 AM
> > > To: Tsu, Mr. Justin
> > > Subject: [rt.rap.ucar.edu #89469] MET V7.0, help with
Ensemble_stat
> > >
> > > Hi Justin -
> > >
> > > ascii2nc reformats observations as input for the Point-Stat
tool,
> > > so I don't expect Ensemble Stat to work on ascii2nc output used
for
> > > forecast
> > > files.
> > >
> > > The best way may be to convert the COAMPS flat files to CF
compliant
> > > netcdf.
> > >
> > > Much of our support staff is out this week,
> > > so let us get back to you early next week with other possible
> > > solutions.
> > >
> > > Are you able to share sample COAMPS data files with us here?
> > >
> > > If so there are instructions for uploading files to our ftp site
at
> > > this link:
> > > https://dtcenter.org/met/users/support/met_help.php
> > >
> > > thanks,
> > > David
> > >
> > > On Mon Mar 25 14:35:19 2019, justin.tsu at nrlmry.navy.mil wrote:
> > > > Hello,
> > > >
> > > >
> > > >
> > > > I am trying to test out the ensemble_stat routine on some
COAMPS
> > > > flatfile 10
> > > > m wind output I have from a 20 member ensemble run. I wasn't
quite
> > > > sure how
> > > > to get the flatfile into a netcdf format that could be read in
by
> > > > MET
> > > > so I
> > > > just used ascii2nc to on an ascii file containing a
"flattened"
> > > > list
> > > > of
> > > > uwind forecasts at each latitude and longitude within my
COAMPS
> > > > regional
> > > > grid. Once I had my files named ens00x_nest1.nc, I ran
> > > > ensemble_stat
> > > > on an
> > > > asci file containing a list of the locations of these netcdf
files,
> > > > changed
> > > > my config file and copied the 20th ensemble member
> > > > (ens020_nest1.nc)
> > > > to be
> > > > my obs netcdf file (obs.nc). For a 10 member stat run, I get
the
> > > > following:
> > > >
> > > >
> > > >
> > > > ensemble_stat ens_file_list ./config.rc -grid_obs obs/obs.nc
> > > > -outdir
> > > > ./ -v 2
> > > >
> > > > DEBUG 1: Default Config File:
> > > > /software/depot/met-
7.0/share/met/config/EnsembleStatConfig_default
> > > >
> > > > DEBUG 1: User Config File: ./config.rc
> > > >
> > > > GSL_RNG_TYPE=mt19937
> > > >
> > > > GSL_RNG_SEED=322607452
> > > >
> > > > DEBUG 1: Ensemble Files[10]:
> > > >
> > > > DEBUG 1: ./nest1/ens001_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens002_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens003_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens004_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens005_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens006_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens007_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens008_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens009_nest1.nc
> > > >
> > > > DEBUG 1: ./nest1/ens010_nest1.nc
> > > >
> > > > DEBUG 1: Gridded Observation Files[1]:
> > > >
> > > > DEBUG 1: obs/obs.nc
> > > >
> > > > ERROR :
> > > >
> > > > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane
&)
> > > > const ->
> > > > star found in bad slot
> > > >
> > > > ERROR :
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I know I definitely made some errors in the formatting of the
> > > > netcdf
> > > > file
> > > > seeing as ascii2nc is actually meant for ob conversion to
netcdf,
> > > > but
> > > > this
> > > > was the best way I could find seeing as there is currently no
way
> > > > to
> > > > read in
> > > > a IEEE-BE data file into MET.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Let me know if you need more information from me.
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Justin
> > > >
> > > >
> > > >
> > > > Justin Tsu
> > > >
> > > > Marine Meteorology Division
> > > >
> > > > Data Assimilation/Mesoscale Modeling
> > > >
> > > > Building 704 Room 212
> > > >
> > > > Naval Research Laboratory, Code 7531
> > > >
> > > > 7 Grace Hopper Avenue
> > > >
> > > > Monterey, CA 93943-5502
> > > >
> > > >
> > > >
> > > > Ph. (831) 656-4111
> > > >
> > > >
> > > >
>
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list