[Met_help] [rt.rap.ucar.edu #99169] History for Verification on MPAS forecast using MET
John Halley Gotway via RT
met_help at ucar.edu
Wed Apr 21 11:59:35 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
Could you help me with this?
Thanks a lot!!
Zhuming
I would like to verificate MPAS forecasts against obs. using MET
"Point-Stat" tool.
It looks METplus has successfully finished running, but when I checked the
results, point_stat*.txt are almost empty only with a head there.
I already convert accii point obs data from little-r format to NetCDF
format using MET "ASCII2NC" tool, and
convert MPAS NetCDF forecasts to MET compliant Climate and Forecast (CF)
NetCDF format.
MET results: /glade/scratch/zhuming/metplus/point_stat3
obs files : /glade/scratch/zhuming/metplus/ascii2nc
mpas forecast: /glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
mpas_wrf_cf_2015071500.nc
conf files: /gpfs/fs1/work/zhuming/METplus/PointStat3.conf
cheyenne4:/gpfs/fs1/work/zhuming/METplus>master_metplus.py -c
./PointStat3.conf -c ./metplus_system.conf.cheyenne
03/22 10:08:58.723 METplus (met_util.py:97) INFO: Starting METplus v3.1
03/22 10:08:58.723 metplus (config_metplus.py:76) INFO: Starting METplus
configuration setup.
03/22 10:08:58.725 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_system.conf:
Parsed this file
03/22 10:08:58.726 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_data.conf:
Parsed this file
03/22 10:08:58.726 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_runtime.conf:
Parsed this file
03/22 10:08:58.727 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_logging.conf:
Parsed this file
03/22 10:08:58.728 metplus (config_launcher.py:204) INFO:
./PointStat3.conf: Parsed this file
03/22 10:08:58.728 metplus (config_launcher.py:204) INFO:
./metplus_system.conf.cheyenne: Parsed this file
03/22 10:08:58.729 metplus (config_launcher.py:394) INFO: Adding:
config.LOG_TIMESTAMP='20210322100858'
03/22 10:08:58.729 metplus (config_launcher.py:400) INFO: Replace [config]
LOG_METPLUS with
'/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210322100858'
03/22 10:08:58.730 metplus (config_launcher.py:272) INFO: Setting [dir]
METPLUS_BASE to /glade/p/ral/jntp/MET/METplus/METplus-3.1
03/22 10:08:58.730 metplus (config_launcher.py:272) INFO: Setting [dir]
PARM_BASE to /gpfs/fs1/work/zhuming/METplus/parm
03/22 10:08:58.730 metplus (config_launcher.py:279) INFO: METPLUS_CONF:
/glade/scratch/zhuming/metplus/metplus_final.conf written here.
03/22 10:08:58.730 metplus (config_metplus.py:147) INFO: Completed METplus
configuration setup.
03/22 10:08:58.731 metplus (met_util.py:107) INFO: Running METplus v3.1
called with command:
/glade/p/ral/jntp/MET/METplus/METplus-3.1/ush/master_metplus.py -c
./PointStat3.conf -c ./metplus_system.conf.cheyenne
03/22 10:08:58.731 metplus (met_util.py:109) INFO: Log file:
/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210322100858
03/22 10:09:00.193 metplus (met_util.py:920) INFO:
****************************************
03/22 10:09:00.193 metplus (met_util.py:921) INFO: * Running METplus
03/22 10:09:00.193 metplus (met_util.py:925) INFO: * at valid time:
201507150000
03/22 10:09:00.193 metplus (met_util.py:926) INFO:
****************************************
03/22 10:09:00.194 metplus.PointStat (compare_gridded_wrapper.py:176) INFO:
Processing forecast lead 0 hours
03/22 10:09:00.200 metplus.PointStat (command_runner.py:89) INFO: COMMAND:
/glade/p/ral/jntp/MET/MET_releases/9.1/bin/point_stat -v 3
/glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
mpas_wrf_cf_2015071500.nc /glade/scratch/zhuming/metplus/ascii2nc/
obs.2015071500.nc
/gpfs/fs1/work/zhuming/METplus/parm/met_config/PointStatConfig_wrapped
-outdir /glade/scratch/zhuming/metplus/point_stat3
03/22 10:09:25.848 metplus (met_util.py:812) INFO:
/glade/scratch/zhuming/metplus/metplus_final.conf: write metplus.conf here
03/22 10:09:25.851 metplus (met_util.py:239) INFO: METplus has successfully
finished running.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Verification on MPAS forecast using MET
From: George McCabe
Time: Tue Mar 23 17:06:47 2021
Hi Zhuming,
I took a look at your log file and it looks like you have fields
specified in your config file that don't exist in the NetCDF forecast
file:
WARNING:
WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find NetCDF variable "t" in file
"/glade/scratch/zhuming/test_mpas/r60-
3km/cf/2015071500/mpas_wrf_cf_2015071500.nc".
WARNING:
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for tP925.
WARNING:
WARNING: process_fcst_climo_files() -> no fields matching tP925 found
in file: /glade/scratch/zhuming/test_mpas/r60-
3km/cf/2015071500/mpas_wrf_cf_2015071500.nc
WARNING:
WARNING:
WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find NetCDF variable "t" in file
"/glade/scratch/zhuming/test_mpas/r60-
3km/cf/2015071500/mpas_wrf_cf_2015071500.nc".
WARNING:
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for tP850.
WARNING:
WARNING: process_fcst_climo_files() -> no fields matching tP850 found
in file: /glade/scratch/zhuming/test_mpas/r60-
3km/cf/2015071500/mpas_wrf_cf_2015071500.nc
WARNING:
WARNING:
WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find NetCDF variable "t" in file
"/glade/scratch/zhuming/test_mpas/r60-
3km/cf/2015071500/mpas_wrf_cf_2015071500.nc".
I copied your config file to my directory and commented out the
variables that reference "t" since it does not exist. When I ran it
again, it found the t_2m and v_10m fields, but it looks like it
rejected all of the point observations:
DEBUG 2: Processing t_2m(0,*,*) versus TMP/Z2, for observation type
ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
matched pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 1090016
DEBUG 3: Rejected: station id = 0
DEBUG 3: Rejected: obs var name = 898087
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 0
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 191929
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: bad climo mean = 0
DEBUG 3: Rejected: bad climo stdev = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing v_10m(0,*,*) versus VGRD/Z10, for observation type
ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
matched pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 1090016
DEBUG 3: Rejected: station id = 0
DEBUG 3: Rejected: obs var name = 1090012
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 0
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 4
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: bad climo mean = 0
DEBUG 3: Rejected: bad climo stdev = 0
DEBUG 3: Rejected: duplicates = 0
For temperature, it looks like many points are rejected because the
quality control values are not in the list specified in the
PointStatConfig_wrapped file:
obs_quality = [ "1", "2", "3" ];
and here is what is in the ASCII2NC output file:
obs_qty_table =
"0",
"8.10000",
"7.40000",
"NA",
"MOBIL I",
"BIL",
"XX32 KW" ;
Those odd values in there suggest that there is bad data that went
into ASCII2NC, or that tool was configured incorrectly.
In the upcoming release it is very easy to override the value for
obs_quality in your MET config file by setting
POINT_STAT_MET_CONFIG_OVERRIDES in your METplus config file. However,
with the version you are running, you will have to copy the wrapped
file to your working area, modify the value, then update your METplus
config file to point to that file. Here is what I did on cheyenne:
cp /glade/p/ral/jntp/MET/METplus/METplus-4.0.0-
beta3/parm/met_config/PointStatConfig_wrapped .
# modify ~/PointStatConfig_wrapped to include:
obs_quality = [ ];
# modify PointStat3.conf to include:
POINT_STAT_CONFIG_FILE=/glade/u/home/mccabe/PointStatConfig_wrapped
Running with these settings produced some match pairs for TMP, but
still no pairs for VGRD. I would suggest trying to fix your point
observation so that ASCII2NC does not produce output with bad quality
control values. It is possible that fixing the input data would give
you better results.
The files I used to run are still on cheyenne if you'd like to refer
to them. The run only produces 1 line of output.
METplus Config: /glade/u/home/mccabe/PointStat3.conf
MET Config: /glade/u/home/mccabe/PointStatConfig_wrapped
Log file: /glade/u/home/mccabe/zhuming-
test/logs/master_metplus.log.20210323165014
Output: /glade/u/home/mccabe/zhuming-
test/point_stat3/point_stat_000000L_20150715_000000V.stat
I hope that helps. Let me know if you have any other questions.
Thanks,
George
On Mon Mar 22 10:51:55 2021, zhuming at ucar.edu wrote:
> Hello,
>
>
> Could you help me with this?
>
>
> Thanks a lot!!
>
>
> Zhuming
>
>
>
> I would like to verificate MPAS forecasts against obs. using MET
> "Point-Stat" tool.
>
>
> It looks METplus has successfully finished running, but when I
checked the
> results, point_stat*.txt are almost empty only with a head there.
>
>
> I already convert accii point obs data from little-r format to
NetCDF
> format using MET "ASCII2NC" tool, and
>
>
> convert MPAS NetCDF forecasts to MET compliant Climate and Forecast
(CF)
> NetCDF format.
>
>
> MET results: /glade/scratch/zhuming/metplus/point_stat3
>
>
> obs files : /glade/scratch/zhuming/metplus/ascii2nc
>
>
> mpas forecast: /glade/scratch/zhuming/test_mpas/r60-
3km/cf/2015071500/
> mpas_wrf_cf_2015071500.nc
>
>
> conf files: /gpfs/fs1/work/zhuming/METplus/PointStat3.conf
>
>
>
> cheyenne4:/gpfs/fs1/work/zhuming/METplus>master_metplus.py -c
> ./PointStat3.conf -c ./metplus_system.conf.cheyenne
>
> 03/22 10:08:58.723 METplus (met_util.py:97) INFO: Starting METplus
v3.1
>
> 03/22 10:08:58.723 metplus (config_metplus.py:76) INFO: Starting
METplus
> configuration setup.
>
> 03/22 10:08:58.725 metplus (config_launcher.py:204) INFO:
>
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_system.conf:
> Parsed this file
>
> 03/22 10:08:58.726 metplus (config_launcher.py:204) INFO:
>
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_data.conf:
> Parsed this file
>
> 03/22 10:08:58.726 metplus (config_launcher.py:204) INFO:
>
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_runtime.conf:
> Parsed this file
>
> 03/22 10:08:58.727 metplus (config_launcher.py:204) INFO:
>
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_logging.conf:
> Parsed this file
>
> 03/22 10:08:58.728 metplus (config_launcher.py:204) INFO:
> ./PointStat3.conf: Parsed this file
>
> 03/22 10:08:58.728 metplus (config_launcher.py:204) INFO:
> ./metplus_system.conf.cheyenne: Parsed this file
>
> 03/22 10:08:58.729 metplus (config_launcher.py:394) INFO: Adding:
> config.LOG_TIMESTAMP='20210322100858'
>
> 03/22 10:08:58.729 metplus (config_launcher.py:400) INFO: Replace
[config]
> LOG_METPLUS with
>
'/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210322100858'
>
> 03/22 10:08:58.730 metplus (config_launcher.py:272) INFO: Setting
[dir]
> METPLUS_BASE to /glade/p/ral/jntp/MET/METplus/METplus-3.1
>
> 03/22 10:08:58.730 metplus (config_launcher.py:272) INFO: Setting
[dir]
> PARM_BASE to /gpfs/fs1/work/zhuming/METplus/parm
>
> 03/22 10:08:58.730 metplus (config_launcher.py:279) INFO:
METPLUS_CONF:
> /glade/scratch/zhuming/metplus/metplus_final.conf written here.
>
> 03/22 10:08:58.730 metplus (config_metplus.py:147) INFO: Completed
METplus
> configuration setup.
>
> 03/22 10:08:58.731 metplus (met_util.py:107) INFO: Running METplus
v3.1
> called with command:
> /glade/p/ral/jntp/MET/METplus/METplus-3.1/ush/master_metplus.py -c
> ./PointStat3.conf -c ./metplus_system.conf.cheyenne
>
> 03/22 10:08:58.731 metplus (met_util.py:109) INFO: Log file:
>
/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210322100858
>
> 03/22 10:09:00.193 metplus (met_util.py:920) INFO:
> ****************************************
>
> 03/22 10:09:00.193 metplus (met_util.py:921) INFO: * Running METplus
>
> 03/22 10:09:00.193 metplus (met_util.py:925) INFO: * at valid time:
> 201507150000
>
> 03/22 10:09:00.193 metplus (met_util.py:926) INFO:
> ****************************************
>
> 03/22 10:09:00.194 metplus.PointStat
(compare_gridded_wrapper.py:176) INFO:
> Processing forecast lead 0 hours
>
> 03/22 10:09:00.200 metplus.PointStat (command_runner.py:89) INFO:
COMMAND:
> /glade/p/ral/jntp/MET/MET_releases/9.1/bin/point_stat -v 3
> /glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
> mpas_wrf_cf_2015071500.nc /glade/scratch/zhuming/metplus/ascii2nc/
> obs.2015071500.nc
>
/gpfs/fs1/work/zhuming/METplus/parm/met_config/PointStatConfig_wrapped
> -outdir /glade/scratch/zhuming/metplus/point_stat3
>
> 03/22 10:09:25.848 metplus (met_util.py:812) INFO:
> /glade/scratch/zhuming/metplus/metplus_final.conf: write
metplus.conf here
>
> 03/22 10:09:25.851 metplus (met_util.py:239) INFO: METplus has
successfully
> finished running.
------------------------------------------------
Subject: Verification on MPAS forecast using MET
From: zhuming at ucar.edu
Time: Wed Mar 24 10:11:41 2021
HI George,
Thanks for your quick response!
I will use the new beta version later.
So back to process the obs data using "MET ASCII2NC". I need 12 hour
obs
data to verify WRF and MPAS 12hr forecasts. I tried twice:
First using the global little-r format data, it failed. This data
puts 6
hour obs data in one file and located in
/glade/collections/rda/data/ds351.0/little_r/2015/OBS:*
The 2nd time I used hourly local little-r format data which got from
the
global data using WRFDA. it looks fine but still has problems when
using
"PointStat" as you said. The data are in
/glade/u/home/taosun/work/data/2015case/gts/
The ASCII2NC.conf and ASCII2NC2.conf are in
/gpfs/fs1/work/zhuming/METplus
Help me figure it out please!
One More question: How to set time for processing 12 hour obs data?
Thanks Thanks!
Zhuming
1.
cheyenne4:/gpfs/fs1/work/zhuming/METplus>master_metplus.py -c
ASCII2NC.conf
-c ./metplus_system.conf.cheyenne
03/24 09:41:44.130 METplus (met_util.py:97) INFO: Starting METplus
v3.1
03/24 09:41:44.130 metplus (config_metplus.py:76) INFO: Starting
METplus
configuration setup.
03/24 09:41:44.132 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_system.conf:
Parsed this file
03/24 09:41:44.132 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_data.conf:
Parsed this file
03/24 09:41:44.133 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_runtime.conf:
Parsed this file
03/24 09:41:44.133 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_logging.conf:
Parsed this file
03/24 09:41:44.134 metplus (config_launcher.py:204) INFO:
ASCII2NC.conf:
Parsed this file
03/24 09:41:44.135 metplus (config_launcher.py:204) INFO:
./metplus_system.conf.cheyenne: Parsed this file
03/24 09:41:44.136 metplus (config_launcher.py:394) INFO: Adding:
config.LOG_TIMESTAMP='20210324094144'
03/24 09:41:44.136 metplus (config_launcher.py:400) INFO: Replace
[config]
LOG_METPLUS with
'/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210324094144'
03/24 09:41:44.136 metplus (config_launcher.py:272) INFO: Setting
[dir]
METPLUS_BASE to /glade/p/ral/jntp/MET/METplus/METplus-3.1
03/24 09:41:44.136 metplus (config_launcher.py:272) INFO: Setting
[dir]
PARM_BASE to /gpfs/fs1/work/zhuming/METplus/parm
03/24 09:41:44.137 metplus (config_launcher.py:279) INFO:
METPLUS_CONF:
/glade/scratch/zhuming/metplus/metplus_final.conf written here.
03/24 09:41:44.137 metplus (config_metplus.py:147) INFO: Completed
METplus
configuration setup.
03/24 09:41:44.138 metplus (met_util.py:107) INFO: Running METplus
v3.1
called with command:
/glade/p/ral/jntp/MET/METplus/METplus-3.1/ush/master_metplus.py -c
ASCII2NC.conf -c ./metplus_system.conf.cheyenne
03/24 09:41:44.138 metplus (met_util.py:109) INFO: Log file:
/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210324094144
03/24 09:41:45.208 metplus (met_util.py:920) INFO:
****************************************
03/24 09:41:45.208 metplus (met_util.py:921) INFO: * Running METplus
03/24 09:41:45.209 metplus (met_util.py:925) INFO: * at valid time:
201507150000
03/24 09:41:45.209 metplus (met_util.py:926) INFO:
****************************************
03/24 09:41:45.210 metplus.ASCII2NC (command_runner.py:89) INFO:
COMMAND:
/glade/p/ral/jntp/MET/MET_releases/9.1/bin/ascii2nc
/glade/collections/rda/data/ds351.0/little_r/2015/OBS:2015071500
/glade/scratch/zhuming/metplus/ascii2nc/obs.2015071500.nc -config
/gpfs/fs1/work/zhuming/METplus/parm/met_config/Ascii2NcConfig_wrapped
-v 2
03/24 09:41:45.764 metplus.ASCII2NC (command_builder.py:122) ERROR:
(command_builder.py:1032) MET command returned a non-zero return code:
/glade/p/ral/jntp/MET/MET_releases/9.1/bin/ascii2nc
/glade/collections/rda/data/ds351.0/little_r/2015/OBS:2015071500
/glade/scratch/zhuming/metplus/ascii2nc/obs.2015071500.nc -config
/gpfs/fs1/work/zhuming/METplus/parm/met_config/Ascii2NcConfig_wrapped
-v 2
03/24 09:41:45Z run-METplus-metplus.ASCII2NC: ERROR:
(command_builder.py:1032)
MET command returned a non-zero return code:
/glade/p/ral/jntp/MET/MET_releases/9.1/bin/ascii2nc
/glade/collections/rda/data/ds351.0/little_r/2015/OBS:2015071500
/glade/scratch/zhuming/metplus/ascii2nc/obs.2015071500.nc -config
/gpfs/fs1/work/zhuming/METplus/parm/met_config/Ascii2NcConfig_wrapped
-v 2
03/24 09:41:45.765 metplus.ASCII2NC (command_builder.py:1033) INFO:
Check
the logfile for more information on why it failed:
/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210324094144
03/24 09:41:45.765 metplus.ASCII2NC (met_util.py:210) ERROR: ASCII2NC
had 1
error.
03/24 09:41:45Z run-METplus-metplus.ASCII2NC: ERROR: ASCII2NC had 1
error.
03/24 09:41:45.765 metplus (met_util.py:812) INFO:
/glade/scratch/zhuming/metplus/metplus_final.conf: write metplus.conf
here
03/24 09:41:45.768 metplus (met_util.py:245) ERROR: METplus has
finished
running but had 1 error.
03/24 09:41:45Z run-METplus-metplus: ERROR: METplus has finished
running
but had 1 error.
03/24 09:41:45.768 metplus (met_util.py:246) INFO: Check the log file
for
more information:
/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210324094144
2.
cheyenne4:/gpfs/fs1/work/zhuming/METplus>master_metplus.py -c
ASCII2NC2.conf -c ./metplus_system.conf.cheyenne
03/24 10:04:43.817 METplus (met_util.py:97) INFO: Starting METplus
v3.1
03/24 10:04:43.817 metplus (config_metplus.py:76) INFO: Starting
METplus
configuration setup.
03/24 10:04:43.821 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_system.conf:
Parsed this file
03/24 10:04:43.821 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_data.conf:
Parsed this file
03/24 10:04:43.821 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_runtime.conf:
Parsed this file
03/24 10:04:43.822 metplus (config_launcher.py:204) INFO:
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_logging.conf:
Parsed this file
03/24 10:04:43.823 metplus (config_launcher.py:204) INFO:
ASCII2NC2.conf:
Parsed this file
03/24 10:04:43.823 metplus (config_launcher.py:204) INFO:
./metplus_system.conf.cheyenne: Parsed this file
03/24 10:04:43.824 metplus (config_launcher.py:394) INFO: Adding:
config.LOG_TIMESTAMP='20210324100443'
03/24 10:04:43.824 metplus (config_launcher.py:400) INFO: Replace
[config]
LOG_METPLUS with
'/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210324100443'
03/24 10:04:43.825 metplus (config_launcher.py:272) INFO: Setting
[dir]
METPLUS_BASE to /glade/p/ral/jntp/MET/METplus/METplus-3.1
03/24 10:04:43.825 metplus (config_launcher.py:272) INFO: Setting
[dir]
PARM_BASE to /gpfs/fs1/work/zhuming/METplus/parm
03/24 10:04:43.825 metplus (config_launcher.py:279) INFO:
METPLUS_CONF:
/glade/scratch/zhuming/metplus/metplus_final.conf written here.
03/24 10:04:43.826 metplus (config_metplus.py:147) INFO: Completed
METplus
configuration setup.
03/24 10:04:43.826 metplus (met_util.py:107) INFO: Running METplus
v3.1
called with command:
/glade/p/ral/jntp/MET/METplus/METplus-3.1/ush/master_metplus.py -c
ASCII2NC2.conf -c ./metplus_system.conf.cheyenne
03/24 10:04:43.826 metplus (met_util.py:109) INFO: Log file:
/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210324100443
03/24 10:04:45.869 metplus (met_util.py:920) INFO:
****************************************
03/24 10:04:45.870 metplus (met_util.py:921) INFO: * Running METplus
03/24 10:04:45.870 metplus (met_util.py:925) INFO: * at valid time:
201507150000
03/24 10:04:45.870 metplus (met_util.py:926) INFO:
****************************************
03/24 10:04:45.875 metplus.ASCII2NC (command_runner.py:89) INFO:
COMMAND:
/glade/p/ral/jntp/MET/MET_releases/9.1/bin/ascii2nc
/glade/u/home/taosun/work/data/2015case/gts/obs.2015071500
/glade/scratch/zhuming/metplus/ascii2nc2/obs.2015071500.nc -format
"little_r" -config
/gpfs/fs1/work/zhuming/METplus/parm/met_config/Ascii2NcConfig_wrapped
-v 2
03/24 10:05:56.784 metplus (met_util.py:812) INFO:
/glade/scratch/zhuming/metplus/metplus_final.conf: write metplus.conf
here
03/24 10:05:56.788 metplus (met_util.py:239) INFO: METplus has
successfully
finished running.
On Tue, Mar 23, 2021 at 5:07 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Zhuming,
>
> I took a look at your log file and it looks like you have fields
specified
> in your config file that don't exist in the NetCDF forecast file:
>
> WARNING:
> WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find
> NetCDF variable "t" in file
> "/glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
> mpas_wrf_cf_2015071500.nc".
> WARNING:
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for tP925.
> WARNING:
> WARNING: process_fcst_climo_files() -> no fields matching tP925
found in
> file: /glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
> mpas_wrf_cf_2015071500.nc
> WARNING:
> WARNING:
> WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find
> NetCDF variable "t" in file
> "/glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
> mpas_wrf_cf_2015071500.nc".
> WARNING:
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for tP850.
> WARNING:
> WARNING: process_fcst_climo_files() -> no fields matching tP850
found in
> file: /glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
> mpas_wrf_cf_2015071500.nc
> WARNING:
> WARNING:
> WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find
> NetCDF variable "t" in file
> "/glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
> mpas_wrf_cf_2015071500.nc".
>
>
>
> I copied your config file to my directory and commented out the
variables
> that reference "t" since it does not exist. When I ran it again, it
found
> the t_2m and v_10m fields, but it looks like it rejected all of the
point
> observations:
>
>
>
> DEBUG 2: Processing t_2m(0,*,*) versus TMP/Z2, for observation type
> ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
> matched pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 1090016
> DEBUG 3: Rejected: station id = 0
> DEBUG 3: Rejected: obs var name = 898087
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 0
> DEBUG 3: Rejected: topography = 0
> DEBUG 3: Rejected: level mismatch = 0
> DEBUG 3: Rejected: quality marker = 191929
> DEBUG 3: Rejected: message type = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: bad climo mean = 0
> DEBUG 3: Rejected: bad climo stdev = 0
> DEBUG 3: Rejected: duplicates = 0
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing v_10m(0,*,*) versus VGRD/Z10, for observation
type
> ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
> matched pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 1090016
> DEBUG 3: Rejected: station id = 0
> DEBUG 3: Rejected: obs var name = 1090012
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 0
> DEBUG 3: Rejected: topography = 0
> DEBUG 3: Rejected: level mismatch = 0
> DEBUG 3: Rejected: quality marker = 4
> DEBUG 3: Rejected: message type = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: bad climo mean = 0
> DEBUG 3: Rejected: bad climo stdev = 0
> DEBUG 3: Rejected: duplicates = 0
>
>
> For temperature, it looks like many points are rejected because the
> quality control values are not in the list specified in the
> PointStatConfig_wrapped file:
>
> obs_quality = [ "1", "2", "3" ];
>
> and here is what is in the ASCII2NC output file:
>
> obs_qty_table =
> "0",
> "8.10000",
> "7.40000",
> "NA",
> "MOBIL I",
> "BIL",
> "XX32 KW" ;
>
> Those odd values in there suggest that there is bad data that went
into
> ASCII2NC, or that tool was configured incorrectly.
>
> In the upcoming release it is very easy to override the value for
> obs_quality in your MET config file by setting
> POINT_STAT_MET_CONFIG_OVERRIDES in your METplus config file.
However, with
> the version you are running, you will have to copy the wrapped file
to your
> working area, modify the value, then update your METplus config file
to
> point to that file. Here is what I did on cheyenne:
>
> cp
> /glade/p/ral/jntp/MET/METplus/METplus-4.0.0-
beta3/parm/met_config/PointStatConfig_wrapped
> .
>
> # modify ~/PointStatConfig_wrapped to include:
> obs_quality = [ ];
>
> # modify PointStat3.conf to include:
> POINT_STAT_CONFIG_FILE=/glade/u/home/mccabe/PointStatConfig_wrapped
>
> Running with these settings produced some match pairs for TMP, but
still
> no pairs for VGRD. I would suggest trying to fix your point
observation so
> that ASCII2NC does not produce output with bad quality control
values. It
> is possible that fixing the input data would give you better
results.
>
> The files I used to run are still on cheyenne if you'd like to refer
to
> them. The run only produces 1 line of output.
>
> METplus Config: /glade/u/home/mccabe/PointStat3.conf
> MET Config: /glade/u/home/mccabe/PointStatConfig_wrapped
> Log file:
> /glade/u/home/mccabe/zhuming-
test/logs/master_metplus.log.20210323165014
> Output:
> /glade/u/home/mccabe/zhuming-
test/point_stat3/point_stat_000000L_20150715_000000V.stat
>
> I hope that helps. Let me know if you have any other questions.
>
> Thanks,
> George
>
> On Mon Mar 22 10:51:55 2021, zhuming at ucar.edu wrote:
> > Hello,
> >
> >
> > Could you help me with this?
> >
> >
> > Thanks a lot!!
> >
> >
> > Zhuming
> >
> >
> >
> > I would like to verificate MPAS forecasts against obs. using MET
> > "Point-Stat" tool.
> >
> >
> > It looks METplus has successfully finished running, but when I
checked
> the
> > results, point_stat*.txt are almost empty only with a head there.
> >
> >
> > I already convert accii point obs data from little-r format to
NetCDF
> > format using MET "ASCII2NC" tool, and
> >
> >
> > convert MPAS NetCDF forecasts to MET compliant Climate and
Forecast (CF)
> > NetCDF format.
> >
> >
> > MET results: /glade/scratch/zhuming/metplus/point_stat3
> >
> >
> > obs files : /glade/scratch/zhuming/metplus/ascii2nc
> >
> >
> > mpas forecast: /glade/scratch/zhuming/test_mpas/r60-
3km/cf/2015071500/
> > mpas_wrf_cf_2015071500.nc
> >
> >
> > conf files: /gpfs/fs1/work/zhuming/METplus/PointStat3.conf
> >
> >
> >
> > cheyenne4:/gpfs/fs1/work/zhuming/METplus>master_metplus.py -c
> > ./PointStat3.conf -c ./metplus_system.conf.cheyenne
> >
> > 03/22 10:08:58.723 METplus (met_util.py:97) INFO: Starting METplus
v3.1
> >
> > 03/22 10:08:58.723 metplus (config_metplus.py:76) INFO: Starting
METplus
> > configuration setup.
> >
> > 03/22 10:08:58.725 metplus (config_launcher.py:204) INFO:
> >
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_system.conf:
> > Parsed this file
> >
> > 03/22 10:08:58.726 metplus (config_launcher.py:204) INFO:
> >
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_data.conf:
> > Parsed this file
> >
> > 03/22 10:08:58.726 metplus (config_launcher.py:204) INFO:
> >
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_runtime.conf:
> > Parsed this file
> >
> > 03/22 10:08:58.727 metplus (config_launcher.py:204) INFO:
> >
/gpfs/fs1/work/zhuming/METplus/parm/metplus_config/metplus_logging.conf:
> > Parsed this file
> >
> > 03/22 10:08:58.728 metplus (config_launcher.py:204) INFO:
> > ./PointStat3.conf: Parsed this file
> >
> > 03/22 10:08:58.728 metplus (config_launcher.py:204) INFO:
> > ./metplus_system.conf.cheyenne: Parsed this file
> >
> > 03/22 10:08:58.729 metplus (config_launcher.py:394) INFO: Adding:
> > config.LOG_TIMESTAMP='20210322100858'
> >
> > 03/22 10:08:58.729 metplus (config_launcher.py:400) INFO: Replace
> [config]
> > LOG_METPLUS with
> >
'/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210322100858'
> >
> > 03/22 10:08:58.730 metplus (config_launcher.py:272) INFO: Setting
[dir]
> > METPLUS_BASE to /glade/p/ral/jntp/MET/METplus/METplus-3.1
> >
> > 03/22 10:08:58.730 metplus (config_launcher.py:272) INFO: Setting
[dir]
> > PARM_BASE to /gpfs/fs1/work/zhuming/METplus/parm
> >
> > 03/22 10:08:58.730 metplus (config_launcher.py:279) INFO:
METPLUS_CONF:
> > /glade/scratch/zhuming/metplus/metplus_final.conf written here.
> >
> > 03/22 10:08:58.730 metplus (config_metplus.py:147) INFO: Completed
> METplus
> > configuration setup.
> >
> > 03/22 10:08:58.731 metplus (met_util.py:107) INFO: Running METplus
v3.1
> > called with command:
> > /glade/p/ral/jntp/MET/METplus/METplus-3.1/ush/master_metplus.py -c
> > ./PointStat3.conf -c ./metplus_system.conf.cheyenne
> >
> > 03/22 10:08:58.731 metplus (met_util.py:109) INFO: Log file:
> >
/glade/scratch/zhuming/metplus/logs/master_metplus.log.20210322100858
> >
> > 03/22 10:09:00.193 metplus (met_util.py:920) INFO:
> > ****************************************
> >
> > 03/22 10:09:00.193 metplus (met_util.py:921) INFO: * Running
METplus
> >
> > 03/22 10:09:00.193 metplus (met_util.py:925) INFO: * at valid
time:
> > 201507150000
> >
> > 03/22 10:09:00.193 metplus (met_util.py:926) INFO:
> > ****************************************
> >
> > 03/22 10:09:00.194 metplus.PointStat
(compare_gridded_wrapper.py:176)
> INFO:
> > Processing forecast lead 0 hours
> >
> > 03/22 10:09:00.200 metplus.PointStat (command_runner.py:89) INFO:
> COMMAND:
> > /glade/p/ral/jntp/MET/MET_releases/9.1/bin/point_stat -v 3
> > /glade/scratch/zhuming/test_mpas/r60-3km/cf/2015071500/
> > mpas_wrf_cf_2015071500.nc /glade/scratch/zhuming/metplus/ascii2nc/
> > obs.2015071500.nc
> >
/gpfs/fs1/work/zhuming/METplus/parm/met_config/PointStatConfig_wrapped
> > -outdir /glade/scratch/zhuming/metplus/point_stat3
> >
> > 03/22 10:09:25.848 metplus (met_util.py:812) INFO:
> > /glade/scratch/zhuming/metplus/metplus_final.conf: write
metplus.conf
> here
> >
> > 03/22 10:09:25.851 metplus (met_util.py:239) INFO: METplus has
> successfully
> > finished running.
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list