[Met_help] [rt.rap.ucar.edu #84827] History for comparing VSDB vs. MET stat output
John Halley Gotway via RT
met_help at ucar.edu
Wed May 16 09:41:28 MDT 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi, there,
I am having issues with loading MET .stat output and VSDB output into the
same database, and I was wondering how it was done. I have data in
/metviewer/staging/wd20ps, in the nammet and namvsdb directories on the
metviewer server if you want to take a look.
On the sudo directory in /usr1/metviewer, I used the
load_test_metvsvsdb_shafran.xml for loading and the output is the
load_metvsvsdb.log file.
If you note there are many messages regarding VCNT for the met files - why
are they there?
But also see that the .vsdb files were skipped:
/metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
** ERROR: caught class java.lang.Exception in processFile()
caught SQLException calling executeBatch: Column count doesn't match value
count at row 1
** WARNING: error(s) encountered loading file
/metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb - skipping file
java.lang.Exception: caught SQLException calling executeBatch: Column count
doesn't match value count at row 1
I'm wondering why that is. I haven't had trouble loading VSDB files
before.
Then when I go to the METViewer page, the only models I see are NAM/218,
NAM/242, and NAM/243, which is what I would expect if the VSDB files were
loaded, not the MET files.
Anyway, if you could let me know where I'm going wrong and how I can fix
it, that would be great. I know others have plotted VSDB vs. MET before.
Thanks!
Perry
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: Tatiana Burek
Time: Fri Apr 20 08:26:44 2018
Hi Perry,
The messages about VCNT line type are just warnings and do not affect
the loading. It means that all MET files were loaded.
I fixed the problem with loading vsdb files - we added new columns to
VL1L2 tables but the source code was not updated.
I rerun your XML for vsdb files and got them loaded to a database.
METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
Tatiana
On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> Hi, there,
>
> I am having issues with loading MET .stat output and VSDB output
into the
> same database, and I was wondering how it was done. I have data in
> /metviewer/staging/wd20ps, in the nammet and namvsdb directories on
the
> metviewer server if you want to take a look.
>
> On the sudo directory in /usr1/metviewer, I used the
> load_test_metvsvsdb_shafran.xml for loading and the output is the
> load_metvsvsdb.log file.
>
> If you note there are many messages regarding VCNT for the met files
- why
> are they there?
>
> But also see that the .vsdb files were skipped:
>
> /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> ** ERROR: caught class java.lang.Exception in processFile()
> caught SQLException calling executeBatch: Column count doesn't match
value
> count at row 1
> ** WARNING: error(s) encountered loading file
> /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb - skipping file
> java.lang.Exception: caught SQLException calling executeBatch:
Column count
> doesn't match value count at row 1
>
> I'm wondering why that is. I haven't had trouble loading VSDB files
> before.
>
> Then when I go to the METViewer page, the only models I see are
NAM/218,
> NAM/242, and NAM/243, which is what I would expect if the VSDB files
were
> loaded, not the MET files.
>
> Anyway, if you could let me know where I'm going wrong and how I can
fix
> it, that would be great. I know others have plotted VSDB vs. MET
before.
>
> Thanks!
>
> Perry
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Thu Apr 26 08:53:58 2018
Hi, Tatiana,
Which database did you load to? I'm trying to reload into the
existing
database (mv_emc_metvsvsdb), and I only see NAM/218, NAM/242, and
NAM/243.
No NAM alone.
I also see that you had edited my loading XML and I changed it back to
ensure I was loading from both directories. Was that a problem? I'm
not
sure what I'm doing wrong but I can't seem to duplicate what you did.
Thanks!
Perry
On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Hi Perry,
> The messages about VCNT line type are just warnings and do not
affect the
> loading. It means that all MET files were loaded.
> I fixed the problem with loading vsdb files - we added new columns
to
> VL1L2 tables but the source code was not updated.
> I rerun your XML for vsdb files and got them loaded to a database.
> METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
>
> Tatiana
>
> On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > Hi, there,
> >
> > I am having issues with loading MET .stat output and VSDB output
into the
> > same database, and I was wondering how it was done. I have data
in
> > /metviewer/staging/wd20ps, in the nammet and namvsdb directories
on the
> > metviewer server if you want to take a look.
> >
> > On the sudo directory in /usr1/metviewer, I used the
> > load_test_metvsvsdb_shafran.xml for loading and the output is the
> > load_metvsvsdb.log file.
> >
> > If you note there are many messages regarding VCNT for the met
files -
> why
> > are they there?
> >
> > But also see that the .vsdb files were skipped:
> >
> > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > ** ERROR: caught class java.lang.Exception in processFile()
> > caught SQLException calling executeBatch: Column count doesn't
match
> value
> > count at row 1
> > ** WARNING: error(s) encountered loading file
> > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb - skipping
file
> > java.lang.Exception: caught SQLException calling executeBatch:
Column
> count
> > doesn't match value count at row 1
> >
> > I'm wondering why that is. I haven't had trouble loading VSDB
files
> > before.
> >
> > Then when I go to the METViewer page, the only models I see are
NAM/218,
> > NAM/242, and NAM/243, which is what I would expect if the VSDB
files were
> > loaded, not the MET files.
> >
> > Anyway, if you could let me know where I'm going wrong and how I
can fix
> > it, that would be great. I know others have plotted VSDB vs. MET
before.
> >
> > Thanks!
> >
> > Perry
>
>
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: Tatiana Burek
Time: Thu Apr 26 14:27:45 2018
Hi Perry,
I changed your XML (commented one of the models ) because it was
loaded during your run and it was not necessary to reload it again.
I loaded the missing model into mv_emc_metvsvsdb database.
Currently this database contains data for four models:
mysql> select distinct model from stat_header;
+---------+
| model |
+---------+
| NAM |
| NAM/218 |
| NAM/242 |
| NAM/243 |
+---------+
But it doesn't mean that all variables have data for all models. For
example TMP has data for NAM only.
VGRD, UGRD and some others also have data for NAM but BCAPE doesn't.
Tatiana
On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> Hi, Tatiana,
>
> Which database did you load to? I'm trying to reload into the
> existing
> database (mv_emc_metvsvsdb), and I only see NAM/218, NAM/242, and
> NAM/243.
> No NAM alone.
>
> I also see that you had edited my loading XML and I changed it back
to
> ensure I was loading from both directories. Was that a problem?
I'm
> not
> sure what I'm doing wrong but I can't seem to duplicate what you
did.
>
> Thanks!
>
> Perry
>
> On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> <met_help at ucar.edu>
> wrote:
>
> > Hi Perry,
> > The messages about VCNT line type are just warnings and do not
affect
> > the
> > loading. It means that all MET files were loaded.
> > I fixed the problem with loading vsdb files - we added new
columns
> > to
> > VL1L2 tables but the source code was not updated.
> > I rerun your XML for vsdb files and got them loaded to a database.
> > METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
> >
> > Tatiana
> >
> > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > > Hi, there,
> > >
> > > I am having issues with loading MET .stat output and VSDB output
> > > into the
> > > same database, and I was wondering how it was done. I have data
in
> > > /metviewer/staging/wd20ps, in the nammet and namvsdb directories
on
> > > the
> > > metviewer server if you want to take a look.
> > >
> > > On the sudo directory in /usr1/metviewer, I used the
> > > load_test_metvsvsdb_shafran.xml for loading and the output is
the
> > > load_metvsvsdb.log file.
> > >
> > > If you note there are many messages regarding VCNT for the met
> > > files -
> > why
> > > are they there?
> > >
> > > But also see that the .vsdb files were skipped:
> > >
> > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > ** ERROR: caught class java.lang.Exception in processFile()
> > > caught SQLException calling executeBatch: Column count doesn't
> > > match
> > value
> > > count at row 1
> > > ** WARNING: error(s) encountered loading file
> > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb - skipping
file
> > > java.lang.Exception: caught SQLException calling executeBatch:
> > > Column
> > count
> > > doesn't match value count at row 1
> > >
> > > I'm wondering why that is. I haven't had trouble loading VSDB
> > > files
> > > before.
> > >
> > > Then when I go to the METViewer page, the only models I see are
> > > NAM/218,
> > > NAM/242, and NAM/243, which is what I would expect if the VSDB
> > > files were
> > > loaded, not the MET files.
> > >
> > > Anyway, if you could let me know where I'm going wrong and how I
> > > can fix
> > > it, that would be great. I know others have plotted VSDB vs.
MET
> > > before.
> > >
> > > Thanks!
> > >
> > > Perry
> >
> >
> >
> >
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Thu Apr 26 14:29:01 2018
Right. I know that people have compared VSDB to MET before, so is it
possible to compare TMP from the MET data vs. T from the vsdb data?
Perry
On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Hi Perry,
> I changed your XML (commented one of the models ) because it was
loaded
> during your run and it was not necessary to reload it again.
>
> I loaded the missing model into mv_emc_metvsvsdb database.
>
> Currently this database contains data for four models:
> mysql> select distinct model from stat_header;
> +---------+
> | model |
> +---------+
> | NAM |
> | NAM/218 |
> | NAM/242 |
> | NAM/243 |
> +---------+
>
> But it doesn't mean that all variables have data for all models. For
> example TMP has data for NAM only.
> VGRD, UGRD and some others also have data for NAM but BCAPE doesn't.
>
> Tatiana
> On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > Hi, Tatiana,
> >
> > Which database did you load to? I'm trying to reload into the
> > existing
> > database (mv_emc_metvsvsdb), and I only see NAM/218, NAM/242, and
> > NAM/243.
> > No NAM alone.
> >
> > I also see that you had edited my loading XML and I changed it
back to
> > ensure I was loading from both directories. Was that a problem?
I'm
> > not
> > sure what I'm doing wrong but I can't seem to duplicate what you
did.
> >
> > Thanks!
> >
> > Perry
> >
> > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > Hi Perry,
> > > The messages about VCNT line type are just warnings and do not
affect
> > > the
> > > loading. It means that all MET files were loaded.
> > > I fixed the problem with loading vsdb files - we added new
columns
> > > to
> > > VL1L2 tables but the source code was not updated.
> > > I rerun your XML for vsdb files and got them loaded to a
database.
> > > METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
> > >
> > > Tatiana
> > >
> > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > > > Hi, there,
> > > >
> > > > I am having issues with loading MET .stat output and VSDB
output
> > > > into the
> > > > same database, and I was wondering how it was done. I have
data in
> > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
directories on
> > > > the
> > > > metviewer server if you want to take a look.
> > > >
> > > > On the sudo directory in /usr1/metviewer, I used the
> > > > load_test_metvsvsdb_shafran.xml for loading and the output is
the
> > > > load_metvsvsdb.log file.
> > > >
> > > > If you note there are many messages regarding VCNT for the met
> > > > files -
> > > why
> > > > are they there?
> > > >
> > > > But also see that the .vsdb files were skipped:
> > > >
> > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > ** ERROR: caught class java.lang.Exception in processFile()
> > > > caught SQLException calling executeBatch: Column count doesn't
> > > > match
> > > value
> > > > count at row 1
> > > > ** WARNING: error(s) encountered loading file
> > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb - skipping
file
> > > > java.lang.Exception: caught SQLException calling executeBatch:
> > > > Column
> > > count
> > > > doesn't match value count at row 1
> > > >
> > > > I'm wondering why that is. I haven't had trouble loading VSDB
> > > > files
> > > > before.
> > > >
> > > > Then when I go to the METViewer page, the only models I see
are
> > > > NAM/218,
> > > > NAM/242, and NAM/243, which is what I would expect if the VSDB
> > > > files were
> > > > loaded, not the MET files.
> > > >
> > > > Anyway, if you could let me know where I'm going wrong and how
I
> > > > can fix
> > > > it, that would be great. I know others have plotted VSDB vs.
MET
> > > > before.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: Tatiana Burek
Time: Fri Apr 27 09:35:13 2018
John and I spent some time to create an example plot using your data.
This plot is for T - NAM/218 and TMP - NAM with RMSE stat calculated
using the summary method.
T is plotted on Y2 axis and TMP on Y1
We used SFC and Z2 levels and lead times 0,6,12,18,60000,120000,180000
with defined plot vals to compensate the differences in VSDB and MET
fcst_lead formats.
We also made some adjustments for Y1 and Y2 axis configurations.
The resulting plot is 20180427_151839.
Tatiana
On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> Right. I know that people have compared VSDB to MET before, so is
it
> possible to compare TMP from the MET data vs. T from the vsdb data?
>
> Perry
>
> On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Perry,
> > I changed your XML (commented one of the models ) because it was
loaded
> > during your run and it was not necessary to reload it again.
> >
> > I loaded the missing model into mv_emc_metvsvsdb database.
> >
> > Currently this database contains data for four models:
> > mysql> select distinct model from stat_header;
> > +---------+
> > | model |
> > +---------+
> > | NAM |
> > | NAM/218 |
> > | NAM/242 |
> > | NAM/243 |
> > +---------+
> >
> > But it doesn't mean that all variables have data for all models.
For
> > example TMP has data for NAM only.
> > VGRD, UGRD and some others also have data for NAM but BCAPE
doesn't.
> >
> > Tatiana
> > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > > Hi, Tatiana,
> > >
> > > Which database did you load to? I'm trying to reload into the
> > > existing
> > > database (mv_emc_metvsvsdb), and I only see NAM/218, NAM/242,
and
> > > NAM/243.
> > > No NAM alone.
> > >
> > > I also see that you had edited my loading XML and I changed it
back to
> > > ensure I was loading from both directories. Was that a problem?
I'm
> > > not
> > > sure what I'm doing wrong but I can't seem to duplicate what you
did.
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Perry,
> > > > The messages about VCNT line type are just warnings and do not
affect
> > > > the
> > > > loading. It means that all MET files were loaded.
> > > > I fixed the problem with loading vsdb files - we added new
columns
> > > > to
> > > > VL1L2 tables but the source code was not updated.
> > > > I rerun your XML for vsdb files and got them loaded to a
database.
> > > > METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
> > > >
> > > > Tatiana
> > > >
> > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > > > > Hi, there,
> > > > >
> > > > > I am having issues with loading MET .stat output and VSDB
output
> > > > > into the
> > > > > same database, and I was wondering how it was done. I have
data in
> > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
directories on
> > > > > the
> > > > > metviewer server if you want to take a look.
> > > > >
> > > > > On the sudo directory in /usr1/metviewer, I used the
> > > > > load_test_metvsvsdb_shafran.xml for loading and the output
is the
> > > > > load_metvsvsdb.log file.
> > > > >
> > > > > If you note there are many messages regarding VCNT for the
met
> > > > > files -
> > > > why
> > > > > are they there?
> > > > >
> > > > > But also see that the .vsdb files were skipped:
> > > > >
> > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > > ** ERROR: caught class java.lang.Exception in processFile()
> > > > > caught SQLException calling executeBatch: Column count
doesn't
> > > > > match
> > > > value
> > > > > count at row 1
> > > > > ** WARNING: error(s) encountered loading file
> > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
skipping file
> > > > > java.lang.Exception: caught SQLException calling
executeBatch:
> > > > > Column
> > > > count
> > > > > doesn't match value count at row 1
> > > > >
> > > > > I'm wondering why that is. I haven't had trouble loading
VSDB
> > > > > files
> > > > > before.
> > > > >
> > > > > Then when I go to the METViewer page, the only models I see
are
> > > > > NAM/218,
> > > > > NAM/242, and NAM/243, which is what I would expect if the
VSDB
> > > > > files were
> > > > > loaded, not the MET files.
> > > > >
> > > > > Anyway, if you could let me know where I'm going wrong and
how I
> > > > > can fix
> > > > > it, that would be great. I know others have plotted VSDB
vs. MET
> > > > > before.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> >
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Fri Apr 27 09:38:26 2018
OK, thanks! Haven't people compared MET with VSDB before? How did
they do
this?
Perry
On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> John and I spent some time to create an example plot using your
data.
> This plot is for T - NAM/218 and TMP - NAM with RMSE stat calculated
using
> the summary method.
> T is plotted on Y2 axis and TMP on Y1
> We used SFC and Z2 levels and lead times
0,6,12,18,60000,120000,180000
> with defined plot vals to compensate the differences in VSDB and MET
> fcst_lead formats.
> We also made some adjustments for Y1 and Y2 axis configurations.
> The resulting plot is 20180427_151839.
>
> Tatiana
>
> On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> > Right. I know that people have compared VSDB to MET before, so is
it
> > possible to compare TMP from the MET data vs. T from the vsdb
data?
> >
> > Perry
> >
> > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > > Hi Perry,
> > > I changed your XML (commented one of the models ) because it was
loaded
> > > during your run and it was not necessary to reload it again.
> > >
> > > I loaded the missing model into mv_emc_metvsvsdb database.
> > >
> > > Currently this database contains data for four models:
> > > mysql> select distinct model from stat_header;
> > > +---------+
> > > | model |
> > > +---------+
> > > | NAM |
> > > | NAM/218 |
> > > | NAM/242 |
> > > | NAM/243 |
> > > +---------+
> > >
> > > But it doesn't mean that all variables have data for all models.
For
> > > example TMP has data for NAM only.
> > > VGRD, UGRD and some others also have data for NAM but BCAPE
doesn't.
> > >
> > > Tatiana
> > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > > > Hi, Tatiana,
> > > >
> > > > Which database did you load to? I'm trying to reload into the
> > > > existing
> > > > database (mv_emc_metvsvsdb), and I only see NAM/218, NAM/242,
and
> > > > NAM/243.
> > > > No NAM alone.
> > > >
> > > > I also see that you had edited my loading XML and I changed it
back
> to
> > > > ensure I was loading from both directories. Was that a
problem? I'm
> > > > not
> > > > sure what I'm doing wrong but I can't seem to duplicate what
you did.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Perry,
> > > > > The messages about VCNT line type are just warnings and do
not
> affect
> > > > > the
> > > > > loading. It means that all MET files were loaded.
> > > > > I fixed the problem with loading vsdb files - we added new
columns
> > > > > to
> > > > > VL1L2 tables but the source code was not updated.
> > > > > I rerun your XML for vsdb files and got them loaded to a
database.
> > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
> > > > >
> > > > > Tatiana
> > > > >
> > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > > > > > Hi, there,
> > > > > >
> > > > > > I am having issues with loading MET .stat output and VSDB
output
> > > > > > into the
> > > > > > same database, and I was wondering how it was done. I
have data
> in
> > > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
directories
> on
> > > > > > the
> > > > > > metviewer server if you want to take a look.
> > > > > >
> > > > > > On the sudo directory in /usr1/metviewer, I used the
> > > > > > load_test_metvsvsdb_shafran.xml for loading and the output
is
> the
> > > > > > load_metvsvsdb.log file.
> > > > > >
> > > > > > If you note there are many messages regarding VCNT for the
met
> > > > > > files -
> > > > > why
> > > > > > are they there?
> > > > > >
> > > > > > But also see that the .vsdb files were skipped:
> > > > > >
> > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > > > ** ERROR: caught class java.lang.Exception in
processFile()
> > > > > > caught SQLException calling executeBatch: Column count
doesn't
> > > > > > match
> > > > > value
> > > > > > count at row 1
> > > > > > ** WARNING: error(s) encountered loading file
> > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
skipping
> file
> > > > > > java.lang.Exception: caught SQLException calling
executeBatch:
> > > > > > Column
> > > > > count
> > > > > > doesn't match value count at row 1
> > > > > >
> > > > > > I'm wondering why that is. I haven't had trouble loading
VSDB
> > > > > > files
> > > > > > before.
> > > > > >
> > > > > > Then when I go to the METViewer page, the only models I
see are
> > > > > > NAM/218,
> > > > > > NAM/242, and NAM/243, which is what I would expect if the
VSDB
> > > > > > files were
> > > > > > loaded, not the MET files.
> > > > > >
> > > > > > Anyway, if you could let me know where I'm going wrong and
how I
> > > > > > can fix
> > > > > > it, that would be great. I know others have plotted VSDB
vs. MET
> > > > > > before.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Fri Apr 27 09:39:58 2018
Plot 20180427_151839 says No Image Available. There also seems to be
a
blank XML template.
On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> John and I spent some time to create an example plot using your
data.
> This plot is for T - NAM/218 and TMP - NAM with RMSE stat calculated
using
> the summary method.
> T is plotted on Y2 axis and TMP on Y1
> We used SFC and Z2 levels and lead times
0,6,12,18,60000,120000,180000
> with defined plot vals to compensate the differences in VSDB and MET
> fcst_lead formats.
> We also made some adjustments for Y1 and Y2 axis configurations.
> The resulting plot is 20180427_151839.
>
> Tatiana
>
> On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> > Right. I know that people have compared VSDB to MET before, so is
it
> > possible to compare TMP from the MET data vs. T from the vsdb
data?
> >
> > Perry
> >
> > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > > Hi Perry,
> > > I changed your XML (commented one of the models ) because it was
loaded
> > > during your run and it was not necessary to reload it again.
> > >
> > > I loaded the missing model into mv_emc_metvsvsdb database.
> > >
> > > Currently this database contains data for four models:
> > > mysql> select distinct model from stat_header;
> > > +---------+
> > > | model |
> > > +---------+
> > > | NAM |
> > > | NAM/218 |
> > > | NAM/242 |
> > > | NAM/243 |
> > > +---------+
> > >
> > > But it doesn't mean that all variables have data for all models.
For
> > > example TMP has data for NAM only.
> > > VGRD, UGRD and some others also have data for NAM but BCAPE
doesn't.
> > >
> > > Tatiana
> > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > > > Hi, Tatiana,
> > > >
> > > > Which database did you load to? I'm trying to reload into the
> > > > existing
> > > > database (mv_emc_metvsvsdb), and I only see NAM/218, NAM/242,
and
> > > > NAM/243.
> > > > No NAM alone.
> > > >
> > > > I also see that you had edited my loading XML and I changed it
back
> to
> > > > ensure I was loading from both directories. Was that a
problem? I'm
> > > > not
> > > > sure what I'm doing wrong but I can't seem to duplicate what
you did.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Perry,
> > > > > The messages about VCNT line type are just warnings and do
not
> affect
> > > > > the
> > > > > loading. It means that all MET files were loaded.
> > > > > I fixed the problem with loading vsdb files - we added new
columns
> > > > > to
> > > > > VL1L2 tables but the source code was not updated.
> > > > > I rerun your XML for vsdb files and got them loaded to a
database.
> > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
> > > > >
> > > > > Tatiana
> > > > >
> > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > > > > > Hi, there,
> > > > > >
> > > > > > I am having issues with loading MET .stat output and VSDB
output
> > > > > > into the
> > > > > > same database, and I was wondering how it was done. I
have data
> in
> > > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
directories
> on
> > > > > > the
> > > > > > metviewer server if you want to take a look.
> > > > > >
> > > > > > On the sudo directory in /usr1/metviewer, I used the
> > > > > > load_test_metvsvsdb_shafran.xml for loading and the output
is
> the
> > > > > > load_metvsvsdb.log file.
> > > > > >
> > > > > > If you note there are many messages regarding VCNT for the
met
> > > > > > files -
> > > > > why
> > > > > > are they there?
> > > > > >
> > > > > > But also see that the .vsdb files were skipped:
> > > > > >
> > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > > > ** ERROR: caught class java.lang.Exception in
processFile()
> > > > > > caught SQLException calling executeBatch: Column count
doesn't
> > > > > > match
> > > > > value
> > > > > > count at row 1
> > > > > > ** WARNING: error(s) encountered loading file
> > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
skipping
> file
> > > > > > java.lang.Exception: caught SQLException calling
executeBatch:
> > > > > > Column
> > > > > count
> > > > > > doesn't match value count at row 1
> > > > > >
> > > > > > I'm wondering why that is. I haven't had trouble loading
VSDB
> > > > > > files
> > > > > > before.
> > > > > >
> > > > > > Then when I go to the METViewer page, the only models I
see are
> > > > > > NAM/218,
> > > > > > NAM/242, and NAM/243, which is what I would expect if the
VSDB
> > > > > > files were
> > > > > > loaded, not the MET files.
> > > > > >
> > > > > > Anyway, if you could let me know where I'm going wrong and
how I
> > > > > > can fix
> > > > > > it, that would be great. I know others have plotted VSDB
vs. MET
> > > > > > before.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: Tatiana Burek
Time: Fri Apr 27 09:45:56 2018
I rechecked METViewer and it shows the image and XML.
I attached the files t this ticket also.
Tatiana
On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
> Plot 20180427_151839 says No Image Available. There also seems to
be
> a
> blank XML template.
>
> On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
> <met_help at ucar.edu>
> wrote:
>
> > John and I spent some time to create an example plot using your
data.
> > This plot is for T - NAM/218 and TMP - NAM with RMSE stat
calculated
> > using
> > the summary method.
> > T is plotted on Y2 axis and TMP on Y1
> > We used SFC and Z2 levels and lead times
> > 0,6,12,18,60000,120000,180000
> > with defined plot vals to compensate the differences in VSDB and
MET
> > fcst_lead formats.
> > We also made some adjustments for Y1 and Y2 axis configurations.
> > The resulting plot is 20180427_151839.
> >
> > Tatiana
> >
> > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> > > Right. I know that people have compared VSDB to MET before, so
is
> > > it
> > > possible to compare TMP from the MET data vs. T from the vsdb
data?
> > >
> > > Perry
> > >
> > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
> > > <met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > > Hi Perry,
> > > > I changed your XML (commented one of the models ) because it
was
> > > > loaded
> > > > during your run and it was not necessary to reload it again.
> > > >
> > > > I loaded the missing model into mv_emc_metvsvsdb database.
> > > >
> > > > Currently this database contains data for four models:
> > > > mysql> select distinct model from stat_header;
> > > > +---------+
> > > > | model |
> > > > +---------+
> > > > | NAM |
> > > > | NAM/218 |
> > > > | NAM/242 |
> > > > | NAM/243 |
> > > > +---------+
> > > >
> > > > But it doesn't mean that all variables have data for all
models.
> > > > For
> > > > example TMP has data for NAM only.
> > > > VGRD, UGRD and some others also have data for NAM but BCAPE
> > > > doesn't.
> > > >
> > > > Tatiana
> > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > > > > Hi, Tatiana,
> > > > >
> > > > > Which database did you load to? I'm trying to reload into
the
> > > > > existing
> > > > > database (mv_emc_metvsvsdb), and I only see NAM/218,
NAM/242,
> > > > > and
> > > > > NAM/243.
> > > > > No NAM alone.
> > > > >
> > > > > I also see that you had edited my loading XML and I changed
it
> > > > > back
> > to
> > > > > ensure I was loading from both directories. Was that a
> > > > > problem? I'm
> > > > > not
> > > > > sure what I'm doing wrong but I can't seem to duplicate what
> > > > > you did.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > > >
> > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Perry,
> > > > > > The messages about VCNT line type are just warnings and do
> > > > > > not
> > affect
> > > > > > the
> > > > > > loading. It means that all MET files were loaded.
> > > > > > I fixed the problem with loading vsdb files - we added
new
> > > > > > columns
> > > > > > to
> > > > > > VL1L2 tables but the source code was not updated.
> > > > > > I rerun your XML for vsdb files and got them loaded to a
> > > > > > database.
> > > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
> > > > > >
> > > > > > Tatiana
> > > > > >
> > > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > > > > > > Hi, there,
> > > > > > >
> > > > > > > I am having issues with loading MET .stat output and
VSDB
> > > > > > > output
> > > > > > > into the
> > > > > > > same database, and I was wondering how it was done. I
have
> > > > > > > data
> > in
> > > > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
> > > > > > > directories
> > on
> > > > > > > the
> > > > > > > metviewer server if you want to take a look.
> > > > > > >
> > > > > > > On the sudo directory in /usr1/metviewer, I used the
> > > > > > > load_test_metvsvsdb_shafran.xml for loading and the
output
> > > > > > > is
> > the
> > > > > > > load_metvsvsdb.log file.
> > > > > > >
> > > > > > > If you note there are many messages regarding VCNT for
the
> > > > > > > met
> > > > > > > files -
> > > > > > why
> > > > > > > are they there?
> > > > > > >
> > > > > > > But also see that the .vsdb files were skipped:
> > > > > > >
> > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > > > > ** ERROR: caught class java.lang.Exception in
> > > > > > > processFile()
> > > > > > > caught SQLException calling executeBatch: Column count
> > > > > > > doesn't
> > > > > > > match
> > > > > > value
> > > > > > > count at row 1
> > > > > > > ** WARNING: error(s) encountered loading file
> > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
> > > > > > > skipping
> > file
> > > > > > > java.lang.Exception: caught SQLException calling
> > > > > > > executeBatch:
> > > > > > > Column
> > > > > > count
> > > > > > > doesn't match value count at row 1
> > > > > > >
> > > > > > > I'm wondering why that is. I haven't had trouble
loading
> > > > > > > VSDB
> > > > > > > files
> > > > > > > before.
> > > > > > >
> > > > > > > Then when I go to the METViewer page, the only models I
see
> > > > > > > are
> > > > > > > NAM/218,
> > > > > > > NAM/242, and NAM/243, which is what I would expect if
the
> > > > > > > VSDB
> > > > > > > files were
> > > > > > > loaded, not the MET files.
> > > > > > >
> > > > > > > Anyway, if you could let me know where I'm going wrong
and
> > > > > > > how I
> > > > > > > can fix
> > > > > > > it, that would be great. I know others have plotted
VSDB
> > > > > > > vs. MET
> > > > > > > before.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Perry
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> >
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Fri Apr 27 09:49:53 2018
For some reason I'm not getting that. I can download the XML but for
some
reason I can't get a plot nor can I see anything on the various tabs.
Can't say that I care for the differences between the traces either,
but
that's a different story.
Perry
On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> I rechecked METViewer and it shows the image and XML.
> I attached the files t this ticket also.
>
> Tatiana
> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
> > Plot 20180427_151839 says No Image Available. There also seems to
be
> > a
> > blank XML template.
> >
> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > John and I spent some time to create an example plot using your
data.
> > > This plot is for T - NAM/218 and TMP - NAM with RMSE stat
calculated
> > > using
> > > the summary method.
> > > T is plotted on Y2 axis and TMP on Y1
> > > We used SFC and Z2 levels and lead times
> > > 0,6,12,18,60000,120000,180000
> > > with defined plot vals to compensate the differences in VSDB and
MET
> > > fcst_lead formats.
> > > We also made some adjustments for Y1 and Y2 axis configurations.
> > > The resulting plot is 20180427_151839.
> > >
> > > Tatiana
> > >
> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> > > > Right. I know that people have compared VSDB to MET before,
so is
> > > > it
> > > > possible to compare TMP from the MET data vs. T from the vsdb
data?
> > > >
> > > > Perry
> > > >
> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
> > > > <met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > > > Hi Perry,
> > > > > I changed your XML (commented one of the models ) because it
was
> > > > > loaded
> > > > > during your run and it was not necessary to reload it again.
> > > > >
> > > > > I loaded the missing model into mv_emc_metvsvsdb database.
> > > > >
> > > > > Currently this database contains data for four models:
> > > > > mysql> select distinct model from stat_header;
> > > > > +---------+
> > > > > | model |
> > > > > +---------+
> > > > > | NAM |
> > > > > | NAM/218 |
> > > > > | NAM/242 |
> > > > > | NAM/243 |
> > > > > +---------+
> > > > >
> > > > > But it doesn't mean that all variables have data for all
models.
> > > > > For
> > > > > example TMP has data for NAM only.
> > > > > VGRD, UGRD and some others also have data for NAM but BCAPE
> > > > > doesn't.
> > > > >
> > > > > Tatiana
> > > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > > > > > Hi, Tatiana,
> > > > > >
> > > > > > Which database did you load to? I'm trying to reload into
the
> > > > > > existing
> > > > > > database (mv_emc_metvsvsdb), and I only see NAM/218,
NAM/242,
> > > > > > and
> > > > > > NAM/243.
> > > > > > No NAM alone.
> > > > > >
> > > > > > I also see that you had edited my loading XML and I
changed it
> > > > > > back
> > > to
> > > > > > ensure I was loading from both directories. Was that a
> > > > > > problem? I'm
> > > > > > not
> > > > > > sure what I'm doing wrong but I can't seem to duplicate
what
> > > > > > you did.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > > > > > <met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Perry,
> > > > > > > The messages about VCNT line type are just warnings and
do
> > > > > > > not
> > > affect
> > > > > > > the
> > > > > > > loading. It means that all MET files were loaded.
> > > > > > > I fixed the problem with loading vsdb files - we added
new
> > > > > > > columns
> > > > > > > to
> > > > > > > VL1L2 tables but the source code was not updated.
> > > > > > > I rerun your XML for vsdb files and got them loaded to a
> > > > > > > database.
> > > > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243,
NAM.
> > > > > > >
> > > > > > > Tatiana
> > > > > > >
> > > > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov
wrote:
> > > > > > > > Hi, there,
> > > > > > > >
> > > > > > > > I am having issues with loading MET .stat output and
VSDB
> > > > > > > > output
> > > > > > > > into the
> > > > > > > > same database, and I was wondering how it was done. I
have
> > > > > > > > data
> > > in
> > > > > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
> > > > > > > > directories
> > > on
> > > > > > > > the
> > > > > > > > metviewer server if you want to take a look.
> > > > > > > >
> > > > > > > > On the sudo directory in /usr1/metviewer, I used the
> > > > > > > > load_test_metvsvsdb_shafran.xml for loading and the
output
> > > > > > > > is
> > > the
> > > > > > > > load_metvsvsdb.log file.
> > > > > > > >
> > > > > > > > If you note there are many messages regarding VCNT for
the
> > > > > > > > met
> > > > > > > > files -
> > > > > > > why
> > > > > > > > are they there?
> > > > > > > >
> > > > > > > > But also see that the .vsdb files were skipped:
> > > > > > > >
> > > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > > > > > ** ERROR: caught class java.lang.Exception in
> > > > > > > > processFile()
> > > > > > > > caught SQLException calling executeBatch: Column count
> > > > > > > > doesn't
> > > > > > > > match
> > > > > > > value
> > > > > > > > count at row 1
> > > > > > > > ** WARNING: error(s) encountered loading file
> > > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
> > > > > > > > skipping
> > > file
> > > > > > > > java.lang.Exception: caught SQLException calling
> > > > > > > > executeBatch:
> > > > > > > > Column
> > > > > > > count
> > > > > > > > doesn't match value count at row 1
> > > > > > > >
> > > > > > > > I'm wondering why that is. I haven't had trouble
loading
> > > > > > > > VSDB
> > > > > > > > files
> > > > > > > > before.
> > > > > > > >
> > > > > > > > Then when I go to the METViewer page, the only models
I see
> > > > > > > > are
> > > > > > > > NAM/218,
> > > > > > > > NAM/242, and NAM/243, which is what I would expect if
the
> > > > > > > > VSDB
> > > > > > > > files were
> > > > > > > > loaded, not the MET files.
> > > > > > > >
> > > > > > > > Anyway, if you could let me know where I'm going wrong
and
> > > > > > > > how I
> > > > > > > > can fix
> > > > > > > > it, that would be great. I know others have plotted
VSDB
> > > > > > > > vs. MET
> > > > > > > > before.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Tue May 01 13:53:51 2018
Hi, Tatiana,
I just tried this using the instructions here and got No Plot
Available.
You can see the XML and the resulting plot in plot 20180501_195049.
Can
you take a look? What did I miss?
Thanks!
Perry
On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> John and I spent some time to create an example plot using your
data.
> This plot is for T - NAM/218 and TMP - NAM with RMSE stat calculated
using
> the summary method.
> T is plotted on Y2 axis and TMP on Y1
> We used SFC and Z2 levels and lead times
0,6,12,18,60000,120000,180000
> with defined plot vals to compensate the differences in VSDB and MET
> fcst_lead formats.
> We also made some adjustments for Y1 and Y2 axis configurations.
> The resulting plot is 20180427_151839.
>
> Tatiana
>
> On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> > Right. I know that people have compared VSDB to MET before, so is
it
> > possible to compare TMP from the MET data vs. T from the vsdb
data?
> >
> > Perry
> >
> > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > > Hi Perry,
> > > I changed your XML (commented one of the models ) because it was
loaded
> > > during your run and it was not necessary to reload it again.
> > >
> > > I loaded the missing model into mv_emc_metvsvsdb database.
> > >
> > > Currently this database contains data for four models:
> > > mysql> select distinct model from stat_header;
> > > +---------+
> > > | model |
> > > +---------+
> > > | NAM |
> > > | NAM/218 |
> > > | NAM/242 |
> > > | NAM/243 |
> > > +---------+
> > >
> > > But it doesn't mean that all variables have data for all models.
For
> > > example TMP has data for NAM only.
> > > VGRD, UGRD and some others also have data for NAM but BCAPE
doesn't.
> > >
> > > Tatiana
> > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > > > Hi, Tatiana,
> > > >
> > > > Which database did you load to? I'm trying to reload into the
> > > > existing
> > > > database (mv_emc_metvsvsdb), and I only see NAM/218, NAM/242,
and
> > > > NAM/243.
> > > > No NAM alone.
> > > >
> > > > I also see that you had edited my loading XML and I changed it
back
> to
> > > > ensure I was loading from both directories. Was that a
problem? I'm
> > > > not
> > > > sure what I'm doing wrong but I can't seem to duplicate what
you did.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Perry,
> > > > > The messages about VCNT line type are just warnings and do
not
> affect
> > > > > the
> > > > > loading. It means that all MET files were loaded.
> > > > > I fixed the problem with loading vsdb files - we added new
columns
> > > > > to
> > > > > VL1L2 tables but the source code was not updated.
> > > > > I rerun your XML for vsdb files and got them loaded to a
database.
> > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
> > > > >
> > > > > Tatiana
> > > > >
> > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > > > > > Hi, there,
> > > > > >
> > > > > > I am having issues with loading MET .stat output and VSDB
output
> > > > > > into the
> > > > > > same database, and I was wondering how it was done. I
have data
> in
> > > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
directories
> on
> > > > > > the
> > > > > > metviewer server if you want to take a look.
> > > > > >
> > > > > > On the sudo directory in /usr1/metviewer, I used the
> > > > > > load_test_metvsvsdb_shafran.xml for loading and the output
is
> the
> > > > > > load_metvsvsdb.log file.
> > > > > >
> > > > > > If you note there are many messages regarding VCNT for the
met
> > > > > > files -
> > > > > why
> > > > > > are they there?
> > > > > >
> > > > > > But also see that the .vsdb files were skipped:
> > > > > >
> > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > > > ** ERROR: caught class java.lang.Exception in
processFile()
> > > > > > caught SQLException calling executeBatch: Column count
doesn't
> > > > > > match
> > > > > value
> > > > > > count at row 1
> > > > > > ** WARNING: error(s) encountered loading file
> > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
skipping
> file
> > > > > > java.lang.Exception: caught SQLException calling
executeBatch:
> > > > > > Column
> > > > > count
> > > > > > doesn't match value count at row 1
> > > > > >
> > > > > > I'm wondering why that is. I haven't had trouble loading
VSDB
> > > > > > files
> > > > > > before.
> > > > > >
> > > > > > Then when I go to the METViewer page, the only models I
see are
> > > > > > NAM/218,
> > > > > > NAM/242, and NAM/243, which is what I would expect if the
VSDB
> > > > > > files were
> > > > > > loaded, not the MET files.
> > > > > >
> > > > > > Anyway, if you could let me know where I'm going wrong and
how I
> > > > > > can fix
> > > > > > it, that would be great. I know others have plotted VSDB
vs. MET
> > > > > > before.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: Tatiana Burek
Time: Tue May 01 14:47:52 2018
Perry,
You need to select Summary stat SL1L2 type so the values for all stats
are calculated.
It is also advisable to configure Y1 and Y2 axis bounds. I made them
the same with values (2.2, 3).
The resulting plot 20180501_204246.
Tatiana
On Tue May 01 13:53:51 2018, perry.shafran at noaa.gov wrote:
> Hi, Tatiana,
>
> I just tried this using the instructions here and got No Plot
> Available.
> You can see the XML and the resulting plot in plot 20180501_195049.
> Can
> you take a look? What did I miss?
>
> Thanks!
>
> Perry
>
> On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
> <met_help at ucar.edu>
> wrote:
>
> > John and I spent some time to create an example plot using your
data.
> > This plot is for T - NAM/218 and TMP - NAM with RMSE stat
calculated
> > using
> > the summary method.
> > T is plotted on Y2 axis and TMP on Y1
> > We used SFC and Z2 levels and lead times
> > 0,6,12,18,60000,120000,180000
> > with defined plot vals to compensate the differences in VSDB and
MET
> > fcst_lead formats.
> > We also made some adjustments for Y1 and Y2 axis configurations.
> > The resulting plot is 20180427_151839.
> >
> > Tatiana
> >
> > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> > > Right. I know that people have compared VSDB to MET before, so
is
> > > it
> > > possible to compare TMP from the MET data vs. T from the vsdb
data?
> > >
> > > Perry
> > >
> > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
> > > <met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > > Hi Perry,
> > > > I changed your XML (commented one of the models ) because it
was
> > > > loaded
> > > > during your run and it was not necessary to reload it again.
> > > >
> > > > I loaded the missing model into mv_emc_metvsvsdb database.
> > > >
> > > > Currently this database contains data for four models:
> > > > mysql> select distinct model from stat_header;
> > > > +---------+
> > > > | model |
> > > > +---------+
> > > > | NAM |
> > > > | NAM/218 |
> > > > | NAM/242 |
> > > > | NAM/243 |
> > > > +---------+
> > > >
> > > > But it doesn't mean that all variables have data for all
models.
> > > > For
> > > > example TMP has data for NAM only.
> > > > VGRD, UGRD and some others also have data for NAM but BCAPE
> > > > doesn't.
> > > >
> > > > Tatiana
> > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > > > > Hi, Tatiana,
> > > > >
> > > > > Which database did you load to? I'm trying to reload into
the
> > > > > existing
> > > > > database (mv_emc_metvsvsdb), and I only see NAM/218,
NAM/242,
> > > > > and
> > > > > NAM/243.
> > > > > No NAM alone.
> > > > >
> > > > > I also see that you had edited my loading XML and I changed
it
> > > > > back
> > to
> > > > > ensure I was loading from both directories. Was that a
> > > > > problem? I'm
> > > > > not
> > > > > sure what I'm doing wrong but I can't seem to duplicate what
> > > > > you did.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > > >
> > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Perry,
> > > > > > The messages about VCNT line type are just warnings and do
> > > > > > not
> > affect
> > > > > > the
> > > > > > loading. It means that all MET files were loaded.
> > > > > > I fixed the problem with loading vsdb files - we added
new
> > > > > > columns
> > > > > > to
> > > > > > VL1L2 tables but the source code was not updated.
> > > > > > I rerun your XML for vsdb files and got them loaded to a
> > > > > > database.
> > > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243, NAM.
> > > > > >
> > > > > > Tatiana
> > > > > >
> > > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov wrote:
> > > > > > > Hi, there,
> > > > > > >
> > > > > > > I am having issues with loading MET .stat output and
VSDB
> > > > > > > output
> > > > > > > into the
> > > > > > > same database, and I was wondering how it was done. I
have
> > > > > > > data
> > in
> > > > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
> > > > > > > directories
> > on
> > > > > > > the
> > > > > > > metviewer server if you want to take a look.
> > > > > > >
> > > > > > > On the sudo directory in /usr1/metviewer, I used the
> > > > > > > load_test_metvsvsdb_shafran.xml for loading and the
output
> > > > > > > is
> > the
> > > > > > > load_metvsvsdb.log file.
> > > > > > >
> > > > > > > If you note there are many messages regarding VCNT for
the
> > > > > > > met
> > > > > > > files -
> > > > > > why
> > > > > > > are they there?
> > > > > > >
> > > > > > > But also see that the .vsdb files were skipped:
> > > > > > >
> > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > > > > ** ERROR: caught class java.lang.Exception in
> > > > > > > processFile()
> > > > > > > caught SQLException calling executeBatch: Column count
> > > > > > > doesn't
> > > > > > > match
> > > > > > value
> > > > > > > count at row 1
> > > > > > > ** WARNING: error(s) encountered loading file
> > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
> > > > > > > skipping
> > file
> > > > > > > java.lang.Exception: caught SQLException calling
> > > > > > > executeBatch:
> > > > > > > Column
> > > > > > count
> > > > > > > doesn't match value count at row 1
> > > > > > >
> > > > > > > I'm wondering why that is. I haven't had trouble
loading
> > > > > > > VSDB
> > > > > > > files
> > > > > > > before.
> > > > > > >
> > > > > > > Then when I go to the METViewer page, the only models I
see
> > > > > > > are
> > > > > > > NAM/218,
> > > > > > > NAM/242, and NAM/243, which is what I would expect if
the
> > > > > > > VSDB
> > > > > > > files were
> > > > > > > loaded, not the MET files.
> > > > > > >
> > > > > > > Anyway, if you could let me know where I'm going wrong
and
> > > > > > > how I
> > > > > > > can fix
> > > > > > > it, that would be great. I know others have plotted
VSDB
> > > > > > > vs. MET
> > > > > > > before.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Perry
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> >
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Thu May 10 11:59:19 2018
Weird. I just downloaded this exact XML, loaded it into METViewer and
pressed generate plot, and got no plot available. I'm wondering if
something is wrong?
Thanks!
Perry
On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> I rechecked METViewer and it shows the image and XML.
> I attached the files t this ticket also.
>
> Tatiana
> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
> > Plot 20180427_151839 says No Image Available. There also seems to
be
> > a
> > blank XML template.
> >
> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > John and I spent some time to create an example plot using your
data.
> > > This plot is for T - NAM/218 and TMP - NAM with RMSE stat
calculated
> > > using
> > > the summary method.
> > > T is plotted on Y2 axis and TMP on Y1
> > > We used SFC and Z2 levels and lead times
> > > 0,6,12,18,60000,120000,180000
> > > with defined plot vals to compensate the differences in VSDB and
MET
> > > fcst_lead formats.
> > > We also made some adjustments for Y1 and Y2 axis configurations.
> > > The resulting plot is 20180427_151839.
> > >
> > > Tatiana
> > >
> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> > > > Right. I know that people have compared VSDB to MET before,
so is
> > > > it
> > > > possible to compare TMP from the MET data vs. T from the vsdb
data?
> > > >
> > > > Perry
> > > >
> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
> > > > <met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > > > Hi Perry,
> > > > > I changed your XML (commented one of the models ) because it
was
> > > > > loaded
> > > > > during your run and it was not necessary to reload it again.
> > > > >
> > > > > I loaded the missing model into mv_emc_metvsvsdb database.
> > > > >
> > > > > Currently this database contains data for four models:
> > > > > mysql> select distinct model from stat_header;
> > > > > +---------+
> > > > > | model |
> > > > > +---------+
> > > > > | NAM |
> > > > > | NAM/218 |
> > > > > | NAM/242 |
> > > > > | NAM/243 |
> > > > > +---------+
> > > > >
> > > > > But it doesn't mean that all variables have data for all
models.
> > > > > For
> > > > > example TMP has data for NAM only.
> > > > > VGRD, UGRD and some others also have data for NAM but BCAPE
> > > > > doesn't.
> > > > >
> > > > > Tatiana
> > > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
> > > > > > Hi, Tatiana,
> > > > > >
> > > > > > Which database did you load to? I'm trying to reload into
the
> > > > > > existing
> > > > > > database (mv_emc_metvsvsdb), and I only see NAM/218,
NAM/242,
> > > > > > and
> > > > > > NAM/243.
> > > > > > No NAM alone.
> > > > > >
> > > > > > I also see that you had edited my loading XML and I
changed it
> > > > > > back
> > > to
> > > > > > ensure I was loading from both directories. Was that a
> > > > > > problem? I'm
> > > > > > not
> > > > > > sure what I'm doing wrong but I can't seem to duplicate
what
> > > > > > you did.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> > > > > > <met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Perry,
> > > > > > > The messages about VCNT line type are just warnings and
do
> > > > > > > not
> > > affect
> > > > > > > the
> > > > > > > loading. It means that all MET files were loaded.
> > > > > > > I fixed the problem with loading vsdb files - we added
new
> > > > > > > columns
> > > > > > > to
> > > > > > > VL1L2 tables but the source code was not updated.
> > > > > > > I rerun your XML for vsdb files and got them loaded to a
> > > > > > > database.
> > > > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243,
NAM.
> > > > > > >
> > > > > > > Tatiana
> > > > > > >
> > > > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov
wrote:
> > > > > > > > Hi, there,
> > > > > > > >
> > > > > > > > I am having issues with loading MET .stat output and
VSDB
> > > > > > > > output
> > > > > > > > into the
> > > > > > > > same database, and I was wondering how it was done. I
have
> > > > > > > > data
> > > in
> > > > > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
> > > > > > > > directories
> > > on
> > > > > > > > the
> > > > > > > > metviewer server if you want to take a look.
> > > > > > > >
> > > > > > > > On the sudo directory in /usr1/metviewer, I used the
> > > > > > > > load_test_metvsvsdb_shafran.xml for loading and the
output
> > > > > > > > is
> > > the
> > > > > > > > load_metvsvsdb.log file.
> > > > > > > >
> > > > > > > > If you note there are many messages regarding VCNT for
the
> > > > > > > > met
> > > > > > > > files -
> > > > > > > why
> > > > > > > > are they there?
> > > > > > > >
> > > > > > > > But also see that the .vsdb files were skipped:
> > > > > > > >
> > > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > > > > > > ** ERROR: caught class java.lang.Exception in
> > > > > > > > processFile()
> > > > > > > > caught SQLException calling executeBatch: Column count
> > > > > > > > doesn't
> > > > > > > > match
> > > > > > > value
> > > > > > > > count at row 1
> > > > > > > > ** WARNING: error(s) encountered loading file
> > > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
> > > > > > > > skipping
> > > file
> > > > > > > > java.lang.Exception: caught SQLException calling
> > > > > > > > executeBatch:
> > > > > > > > Column
> > > > > > > count
> > > > > > > > doesn't match value count at row 1
> > > > > > > >
> > > > > > > > I'm wondering why that is. I haven't had trouble
loading
> > > > > > > > VSDB
> > > > > > > > files
> > > > > > > > before.
> > > > > > > >
> > > > > > > > Then when I go to the METViewer page, the only models
I see
> > > > > > > > are
> > > > > > > > NAM/218,
> > > > > > > > NAM/242, and NAM/243, which is what I would expect if
the
> > > > > > > > VSDB
> > > > > > > > files were
> > > > > > > > loaded, not the MET files.
> > > > > > > >
> > > > > > > > Anyway, if you could let me know where I'm going wrong
and
> > > > > > > > how I
> > > > > > > > can fix
> > > > > > > > it, that would be great. I know others have plotted
VSDB
> > > > > > > > vs. MET
> > > > > > > > before.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Tue May 15 08:27:44 2018
Just pinging in and was wondering if anyone saw this?
Thanks!
Perry
On Thu, May 10, 2018 at 1:59 PM, Perry Shafran - NOAA Affiliate <
perry.shafran at noaa.gov> wrote:
> Weird. I just downloaded this exact XML, loaded it into METViewer
and
> pressed generate plot, and got no plot available. I'm wondering if
> something is wrong?
>
> Thanks!
>
> Perry
>
> On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
>> I rechecked METViewer and it shows the image and XML.
>> I attached the files t this ticket also.
>>
>> Tatiana
>> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
>> > Plot 20180427_151839 says No Image Available. There also seems
to be
>> > a
>> > blank XML template.
>> >
>> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
>> > <met_help at ucar.edu>
>> > wrote:
>> >
>> > > John and I spent some time to create an example plot using your
data.
>> > > This plot is for T - NAM/218 and TMP - NAM with RMSE stat
calculated
>> > > using
>> > > the summary method.
>> > > T is plotted on Y2 axis and TMP on Y1
>> > > We used SFC and Z2 levels and lead times
>> > > 0,6,12,18,60000,120000,180000
>> > > with defined plot vals to compensate the differences in VSDB
and MET
>> > > fcst_lead formats.
>> > > We also made some adjustments for Y1 and Y2 axis
configurations.
>> > > The resulting plot is 20180427_151839.
>> > >
>> > > Tatiana
>> > >
>> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
>> > > > Right. I know that people have compared VSDB to MET before,
so is
>> > > > it
>> > > > possible to compare TMP from the MET data vs. T from the vsdb
data?
>> > > >
>> > > > Perry
>> > > >
>> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
>> > > > <met_help at ucar.edu
>> > > >
>> > > > wrote:
>> > > >
>> > > > > Hi Perry,
>> > > > > I changed your XML (commented one of the models ) because
it was
>> > > > > loaded
>> > > > > during your run and it was not necessary to reload it
again.
>> > > > >
>> > > > > I loaded the missing model into mv_emc_metvsvsdb database.
>> > > > >
>> > > > > Currently this database contains data for four models:
>> > > > > mysql> select distinct model from stat_header;
>> > > > > +---------+
>> > > > > | model |
>> > > > > +---------+
>> > > > > | NAM |
>> > > > > | NAM/218 |
>> > > > > | NAM/242 |
>> > > > > | NAM/243 |
>> > > > > +---------+
>> > > > >
>> > > > > But it doesn't mean that all variables have data for all
models.
>> > > > > For
>> > > > > example TMP has data for NAM only.
>> > > > > VGRD, UGRD and some others also have data for NAM but BCAPE
>> > > > > doesn't.
>> > > > >
>> > > > > Tatiana
>> > > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov wrote:
>> > > > > > Hi, Tatiana,
>> > > > > >
>> > > > > > Which database did you load to? I'm trying to reload
into the
>> > > > > > existing
>> > > > > > database (mv_emc_metvsvsdb), and I only see NAM/218,
NAM/242,
>> > > > > > and
>> > > > > > NAM/243.
>> > > > > > No NAM alone.
>> > > > > >
>> > > > > > I also see that you had edited my loading XML and I
changed it
>> > > > > > back
>> > > to
>> > > > > > ensure I was loading from both directories. Was that a
>> > > > > > problem? I'm
>> > > > > > not
>> > > > > > sure what I'm doing wrong but I can't seem to duplicate
what
>> > > > > > you did.
>> > > > > >
>> > > > > > Thanks!
>> > > > > >
>> > > > > > Perry
>> > > > > >
>> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
>> > > > > > <met_help at ucar.edu>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Perry,
>> > > > > > > The messages about VCNT line type are just warnings and
do
>> > > > > > > not
>> > > affect
>> > > > > > > the
>> > > > > > > loading. It means that all MET files were loaded.
>> > > > > > > I fixed the problem with loading vsdb files - we added
new
>> > > > > > > columns
>> > > > > > > to
>> > > > > > > VL1L2 tables but the source code was not updated.
>> > > > > > > I rerun your XML for vsdb files and got them loaded to
a
>> > > > > > > database.
>> > > > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243,
NAM.
>> > > > > > >
>> > > > > > > Tatiana
>> > > > > > >
>> > > > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov
wrote:
>> > > > > > > > Hi, there,
>> > > > > > > >
>> > > > > > > > I am having issues with loading MET .stat output and
VSDB
>> > > > > > > > output
>> > > > > > > > into the
>> > > > > > > > same database, and I was wondering how it was done.
I have
>> > > > > > > > data
>> > > in
>> > > > > > > > /metviewer/staging/wd20ps, in the nammet and namvsdb
>> > > > > > > > directories
>> > > on
>> > > > > > > > the
>> > > > > > > > metviewer server if you want to take a look.
>> > > > > > > >
>> > > > > > > > On the sudo directory in /usr1/metviewer, I used the
>> > > > > > > > load_test_metvsvsdb_shafran.xml for loading and the
output
>> > > > > > > > is
>> > > the
>> > > > > > > > load_metvsvsdb.log file.
>> > > > > > > >
>> > > > > > > > If you note there are many messages regarding VCNT
for the
>> > > > > > > > met
>> > > > > > > > files -
>> > > > > > > why
>> > > > > > > > are they there?
>> > > > > > > >
>> > > > > > > > But also see that the .vsdb files were skipped:
>> > > > > > > >
>> > > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
>> > > > > > > > ** ERROR: caught class java.lang.Exception in
>> > > > > > > > processFile()
>> > > > > > > > caught SQLException calling executeBatch: Column
count
>> > > > > > > > doesn't
>> > > > > > > > match
>> > > > > > > value
>> > > > > > > > count at row 1
>> > > > > > > > ** WARNING: error(s) encountered loading file
>> > > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
>> > > > > > > > skipping
>> > > file
>> > > > > > > > java.lang.Exception: caught SQLException calling
>> > > > > > > > executeBatch:
>> > > > > > > > Column
>> > > > > > > count
>> > > > > > > > doesn't match value count at row 1
>> > > > > > > >
>> > > > > > > > I'm wondering why that is. I haven't had trouble
loading
>> > > > > > > > VSDB
>> > > > > > > > files
>> > > > > > > > before.
>> > > > > > > >
>> > > > > > > > Then when I go to the METViewer page, the only models
I see
>> > > > > > > > are
>> > > > > > > > NAM/218,
>> > > > > > > > NAM/242, and NAM/243, which is what I would expect if
the
>> > > > > > > > VSDB
>> > > > > > > > files were
>> > > > > > > > loaded, not the MET files.
>> > > > > > > >
>> > > > > > > > Anyway, if you could let me know where I'm going
wrong and
>> > > > > > > > how I
>> > > > > > > > can fix
>> > > > > > > > it, that would be great. I know others have plotted
VSDB
>> > > > > > > > vs. MET
>> > > > > > > > before.
>> > > > > > > >
>> > > > > > > > Thanks!
>> > > > > > > >
>> > > > > > > > Perry
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > >
>> > >
>> > >
>> > >
>>
>>
>>
>>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: Tatiana Burek
Time: Tue May 15 11:59:31 2018
Perry,
I just repeated my steps and successfully generated the plot:
- open METViewer page
- Open History bar
- find the plot (you can use the one I mentioned before or the one I
generated today : 20180515_175025 )
- click on XML button and the plot XML will be displayed in the popup
window
- click on "Load to Page" button and METViewer will populate fields
- click on Generate Plot button and after a few seconds you should
have a plot
Tatiana
On Tue May 15 08:27:44 2018, perry.shafran at noaa.gov wrote:
> Just pinging in and was wondering if anyone saw this?
>
> Thanks!
>
> Perry
>
> On Thu, May 10, 2018 at 1:59 PM, Perry Shafran - NOAA Affiliate <
> perry.shafran at noaa.gov> wrote:
>
> > Weird. I just downloaded this exact XML, loaded it into METViewer
and
> > pressed generate plot, and got no plot available. I'm wondering
if
> > something is wrong?
> >
> > Thanks!
> >
> > Perry
> >
> > On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT
<met_help at ucar.edu>
> > wrote:
> >
> >> I rechecked METViewer and it shows the image and XML.
> >> I attached the files t this ticket also.
> >>
> >> Tatiana
> >> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
> >> > Plot 20180427_151839 says No Image Available. There also seems
to be
> >> > a
> >> > blank XML template.
> >> >
> >> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
> >> > <met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > John and I spent some time to create an example plot using
your data.
> >> > > This plot is for T - NAM/218 and TMP - NAM with RMSE stat
calculated
> >> > > using
> >> > > the summary method.
> >> > > T is plotted on Y2 axis and TMP on Y1
> >> > > We used SFC and Z2 levels and lead times
> >> > > 0,6,12,18,60000,120000,180000
> >> > > with defined plot vals to compensate the differences in VSDB
and MET
> >> > > fcst_lead formats.
> >> > > We also made some adjustments for Y1 and Y2 axis
configurations.
> >> > > The resulting plot is 20180427_151839.
> >> > >
> >> > > Tatiana
> >> > >
> >> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> >> > > > Right. I know that people have compared VSDB to MET
before, so is
> >> > > > it
> >> > > > possible to compare TMP from the MET data vs. T from the
vsdb data?
> >> > > >
> >> > > > Perry
> >> > > >
> >> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
> >> > > > <met_help at ucar.edu
> >> > > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Perry,
> >> > > > > I changed your XML (commented one of the models ) because
it was
> >> > > > > loaded
> >> > > > > during your run and it was not necessary to reload it
again.
> >> > > > >
> >> > > > > I loaded the missing model into mv_emc_metvsvsdb
database.
> >> > > > >
> >> > > > > Currently this database contains data for four models:
> >> > > > > mysql> select distinct model from stat_header;
> >> > > > > +---------+
> >> > > > > | model |
> >> > > > > +---------+
> >> > > > > | NAM |
> >> > > > > | NAM/218 |
> >> > > > > | NAM/242 |
> >> > > > > | NAM/243 |
> >> > > > > +---------+
> >> > > > >
> >> > > > > But it doesn't mean that all variables have data for all
models.
> >> > > > > For
> >> > > > > example TMP has data for NAM only.
> >> > > > > VGRD, UGRD and some others also have data for NAM but
BCAPE
> >> > > > > doesn't.
> >> > > > >
> >> > > > > Tatiana
> >> > > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov
wrote:
> >> > > > > > Hi, Tatiana,
> >> > > > > >
> >> > > > > > Which database did you load to? I'm trying to reload
into the
> >> > > > > > existing
> >> > > > > > database (mv_emc_metvsvsdb), and I only see NAM/218,
NAM/242,
> >> > > > > > and
> >> > > > > > NAM/243.
> >> > > > > > No NAM alone.
> >> > > > > >
> >> > > > > > I also see that you had edited my loading XML and I
changed it
> >> > > > > > back
> >> > > to
> >> > > > > > ensure I was loading from both directories. Was that a
> >> > > > > > problem? I'm
> >> > > > > > not
> >> > > > > > sure what I'm doing wrong but I can't seem to duplicate
what
> >> > > > > > you did.
> >> > > > > >
> >> > > > > > Thanks!
> >> > > > > >
> >> > > > > > Perry
> >> > > > > >
> >> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via RT
> >> > > > > > <met_help at ucar.edu>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Perry,
> >> > > > > > > The messages about VCNT line type are just warnings
and do
> >> > > > > > > not
> >> > > affect
> >> > > > > > > the
> >> > > > > > > loading. It means that all MET files were loaded.
> >> > > > > > > I fixed the problem with loading vsdb files - we
added new
> >> > > > > > > columns
> >> > > > > > > to
> >> > > > > > > VL1L2 tables but the source code was not updated.
> >> > > > > > > I rerun your XML for vsdb files and got them loaded
to a
> >> > > > > > > database.
> >> > > > > > > METViewer UI shows models NAM/218, NAM/242, NAM/243,
NAM.
> >> > > > > > >
> >> > > > > > > Tatiana
> >> > > > > > >
> >> > > > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov
wrote:
> >> > > > > > > > Hi, there,
> >> > > > > > > >
> >> > > > > > > > I am having issues with loading MET .stat output
and VSDB
> >> > > > > > > > output
> >> > > > > > > > into the
> >> > > > > > > > same database, and I was wondering how it was done.
I have
> >> > > > > > > > data
> >> > > in
> >> > > > > > > > /metviewer/staging/wd20ps, in the nammet and
namvsdb
> >> > > > > > > > directories
> >> > > on
> >> > > > > > > > the
> >> > > > > > > > metviewer server if you want to take a look.
> >> > > > > > > >
> >> > > > > > > > On the sudo directory in /usr1/metviewer, I used
the
> >> > > > > > > > load_test_metvsvsdb_shafran.xml for loading and the
output
> >> > > > > > > > is
> >> > > the
> >> > > > > > > > load_metvsvsdb.log file.
> >> > > > > > > >
> >> > > > > > > > If you note there are many messages regarding VCNT
for the
> >> > > > > > > > met
> >> > > > > > > > files -
> >> > > > > > > why
> >> > > > > > > > are they there?
> >> > > > > > > >
> >> > > > > > > > But also see that the .vsdb files were skipped:
> >> > > > > > > >
> >> > > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> >> > > > > > > > ** ERROR: caught class java.lang.Exception in
> >> > > > > > > > processFile()
> >> > > > > > > > caught SQLException calling executeBatch: Column
count
> >> > > > > > > > doesn't
> >> > > > > > > > match
> >> > > > > > > value
> >> > > > > > > > count at row 1
> >> > > > > > > > ** WARNING: error(s) encountered loading file
> >> > > > > > > > /metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
-
> >> > > > > > > > skipping
> >> > > file
> >> > > > > > > > java.lang.Exception: caught SQLException calling
> >> > > > > > > > executeBatch:
> >> > > > > > > > Column
> >> > > > > > > count
> >> > > > > > > > doesn't match value count at row 1
> >> > > > > > > >
> >> > > > > > > > I'm wondering why that is. I haven't had trouble
loading
> >> > > > > > > > VSDB
> >> > > > > > > > files
> >> > > > > > > > before.
> >> > > > > > > >
> >> > > > > > > > Then when I go to the METViewer page, the only
models I see
> >> > > > > > > > are
> >> > > > > > > > NAM/218,
> >> > > > > > > > NAM/242, and NAM/243, which is what I would expect
if the
> >> > > > > > > > VSDB
> >> > > > > > > > files were
> >> > > > > > > > loaded, not the MET files.
> >> > > > > > > >
> >> > > > > > > > Anyway, if you could let me know where I'm going
wrong and
> >> > > > > > > > how I
> >> > > > > > > > can fix
> >> > > > > > > > it, that would be great. I know others have
plotted VSDB
> >> > > > > > > > vs. MET
> >> > > > > > > > before.
> >> > > > > > > >
> >> > > > > > > > Thanks!
> >> > > > > > > >
> >> > > > > > > > Perry
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > >
> >> > >
> >> > >
> >> > >
> >>
> >>
> >>
> >>
> >
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Tue May 15 12:05:48 2018
When I click on the plot for 20180515_175025, I get this image.
Also the XML field is blank. But when I try to load to page and then
click
generate plot, I get the same blank plot.
I think that I personally have something wrong with my browser. If it
works for you and it doesn't work for me.
Perry
On Tue, May 15, 2018 at 1:59 PM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Perry,
>
> I just repeated my steps and successfully generated the plot:
>
> - open METViewer page
> - Open History bar
> - find the plot (you can use the one I mentioned before or the one I
> generated today : 20180515_175025 )
> - click on XML button and the plot XML will be displayed in the
popup
> window
> - click on "Load to Page" button and METViewer will populate fields
> - click on Generate Plot button and after a few seconds you should
have a
> plot
>
> Tatiana
> On Tue May 15 08:27:44 2018, perry.shafran at noaa.gov wrote:
> > Just pinging in and was wondering if anyone saw this?
> >
> > Thanks!
> >
> > Perry
> >
> > On Thu, May 10, 2018 at 1:59 PM, Perry Shafran - NOAA Affiliate <
> > perry.shafran at noaa.gov> wrote:
> >
> > > Weird. I just downloaded this exact XML, loaded it into
METViewer and
> > > pressed generate plot, and got no plot available. I'm wondering
if
> > > something is wrong?
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > > On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > >> I rechecked METViewer and it shows the image and XML.
> > >> I attached the files t this ticket also.
> > >>
> > >> Tatiana
> > >> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
> > >> > Plot 20180427_151839 says No Image Available. There also
seems to
> be
> > >> > a
> > >> > blank XML template.
> > >> >
> > >> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
> > >> > <met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> > > John and I spent some time to create an example plot using
your
> data.
> > >> > > This plot is for T - NAM/218 and TMP - NAM with RMSE stat
> calculated
> > >> > > using
> > >> > > the summary method.
> > >> > > T is plotted on Y2 axis and TMP on Y1
> > >> > > We used SFC and Z2 levels and lead times
> > >> > > 0,6,12,18,60000,120000,180000
> > >> > > with defined plot vals to compensate the differences in
VSDB and
> MET
> > >> > > fcst_lead formats.
> > >> > > We also made some adjustments for Y1 and Y2 axis
configurations.
> > >> > > The resulting plot is 20180427_151839.
> > >> > >
> > >> > > Tatiana
> > >> > >
> > >> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov wrote:
> > >> > > > Right. I know that people have compared VSDB to MET
before, so
> is
> > >> > > > it
> > >> > > > possible to compare TMP from the MET data vs. T from the
vsdb
> data?
> > >> > > >
> > >> > > > Perry
> > >> > > >
> > >> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
> > >> > > > <met_help at ucar.edu
> > >> > > >
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi Perry,
> > >> > > > > I changed your XML (commented one of the models )
because it
> was
> > >> > > > > loaded
> > >> > > > > during your run and it was not necessary to reload it
again.
> > >> > > > >
> > >> > > > > I loaded the missing model into mv_emc_metvsvsdb
database.
> > >> > > > >
> > >> > > > > Currently this database contains data for four models:
> > >> > > > > mysql> select distinct model from stat_header;
> > >> > > > > +---------+
> > >> > > > > | model |
> > >> > > > > +---------+
> > >> > > > > | NAM |
> > >> > > > > | NAM/218 |
> > >> > > > > | NAM/242 |
> > >> > > > > | NAM/243 |
> > >> > > > > +---------+
> > >> > > > >
> > >> > > > > But it doesn't mean that all variables have data for
all
> models.
> > >> > > > > For
> > >> > > > > example TMP has data for NAM only.
> > >> > > > > VGRD, UGRD and some others also have data for NAM but
BCAPE
> > >> > > > > doesn't.
> > >> > > > >
> > >> > > > > Tatiana
> > >> > > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov
wrote:
> > >> > > > > > Hi, Tatiana,
> > >> > > > > >
> > >> > > > > > Which database did you load to? I'm trying to reload
into
> the
> > >> > > > > > existing
> > >> > > > > > database (mv_emc_metvsvsdb), and I only see NAM/218,
> NAM/242,
> > >> > > > > > and
> > >> > > > > > NAM/243.
> > >> > > > > > No NAM alone.
> > >> > > > > >
> > >> > > > > > I also see that you had edited my loading XML and I
changed
> it
> > >> > > > > > back
> > >> > > to
> > >> > > > > > ensure I was loading from both directories. Was that
a
> > >> > > > > > problem? I'm
> > >> > > > > > not
> > >> > > > > > sure what I'm doing wrong but I can't seem to
duplicate what
> > >> > > > > > you did.
> > >> > > > > >
> > >> > > > > > Thanks!
> > >> > > > > >
> > >> > > > > > Perry
> > >> > > > > >
> > >> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via
RT
> > >> > > > > > <met_help at ucar.edu>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hi Perry,
> > >> > > > > > > The messages about VCNT line type are just warnings
and do
> > >> > > > > > > not
> > >> > > affect
> > >> > > > > > > the
> > >> > > > > > > loading. It means that all MET files were loaded.
> > >> > > > > > > I fixed the problem with loading vsdb files - we
added
> new
> > >> > > > > > > columns
> > >> > > > > > > to
> > >> > > > > > > VL1L2 tables but the source code was not updated.
> > >> > > > > > > I rerun your XML for vsdb files and got them loaded
to a
> > >> > > > > > > database.
> > >> > > > > > > METViewer UI shows models NAM/218, NAM/242,
NAM/243, NAM.
> > >> > > > > > >
> > >> > > > > > > Tatiana
> > >> > > > > > >
> > >> > > > > > > On Wed Apr 18 12:39:54 2018, perry.shafran at noaa.gov
> wrote:
> > >> > > > > > > > Hi, there,
> > >> > > > > > > >
> > >> > > > > > > > I am having issues with loading MET .stat output
and
> VSDB
> > >> > > > > > > > output
> > >> > > > > > > > into the
> > >> > > > > > > > same database, and I was wondering how it was
done. I
> have
> > >> > > > > > > > data
> > >> > > in
> > >> > > > > > > > /metviewer/staging/wd20ps, in the nammet and
namvsdb
> > >> > > > > > > > directories
> > >> > > on
> > >> > > > > > > > the
> > >> > > > > > > > metviewer server if you want to take a look.
> > >> > > > > > > >
> > >> > > > > > > > On the sudo directory in /usr1/metviewer, I used
the
> > >> > > > > > > > load_test_metvsvsdb_shafran.xml for loading and
the
> output
> > >> > > > > > > > is
> > >> > > the
> > >> > > > > > > > load_metvsvsdb.log file.
> > >> > > > > > > >
> > >> > > > > > > > If you note there are many messages regarding
VCNT for
> the
> > >> > > > > > > > met
> > >> > > > > > > > files -
> > >> > > > > > > why
> > >> > > > > > > > are they there?
> > >> > > > > > > >
> > >> > > > > > > > But also see that the .vsdb files were skipped:
> > >> > > > > > > >
> > >> > > > > > > >
/metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > >> > > > > > > > ** ERROR: caught class java.lang.Exception in
> > >> > > > > > > > processFile()
> > >> > > > > > > > caught SQLException calling executeBatch: Column
count
> > >> > > > > > > > doesn't
> > >> > > > > > > > match
> > >> > > > > > > value
> > >> > > > > > > > count at row 1
> > >> > > > > > > > ** WARNING: error(s) encountered loading file
> > >> > > > > > > >
/metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
> > >> > > > > > > > skipping
> > >> > > file
> > >> > > > > > > > java.lang.Exception: caught SQLException calling
> > >> > > > > > > > executeBatch:
> > >> > > > > > > > Column
> > >> > > > > > > count
> > >> > > > > > > > doesn't match value count at row 1
> > >> > > > > > > >
> > >> > > > > > > > I'm wondering why that is. I haven't had trouble
> loading
> > >> > > > > > > > VSDB
> > >> > > > > > > > files
> > >> > > > > > > > before.
> > >> > > > > > > >
> > >> > > > > > > > Then when I go to the METViewer page, the only
models I
> see
> > >> > > > > > > > are
> > >> > > > > > > > NAM/218,
> > >> > > > > > > > NAM/242, and NAM/243, which is what I would
expect if
> the
> > >> > > > > > > > VSDB
> > >> > > > > > > > files were
> > >> > > > > > > > loaded, not the MET files.
> > >> > > > > > > >
> > >> > > > > > > > Anyway, if you could let me know where I'm going
wrong
> and
> > >> > > > > > > > how I
> > >> > > > > > > > can fix
> > >> > > > > > > > it, that would be great. I know others have
plotted
> VSDB
> > >> > > > > > > > vs. MET
> > >> > > > > > > > before.
> > >> > > > > > > >
> > >> > > > > > > > Thanks!
> > >> > > > > > > >
> > >> > > > > > > > Perry
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >>
> > >>
> > >>
> > >>
> > >
>
>
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: John Halley Gotway
Time: Tue May 15 13:17:18 2018
Perry and Tatiana,
As a "neutral third party", I'll go through the steps Tatiana outlined
and
let you know what I find.
Thanks,
John
On Tue, May 15, 2018 at 12:05 PM, perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=84827 >
>
> When I click on the plot for 20180515_175025, I get this image.
>
> Also the XML field is blank. But when I try to load to page and
then click
> generate plot, I get the same blank plot.
>
> I think that I personally have something wrong with my browser. If
it
> works for you and it doesn't work for me.
>
> Perry
>
> On Tue, May 15, 2018 at 1:59 PM, Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
> > Perry,
> >
> > I just repeated my steps and successfully generated the plot:
> >
> > - open METViewer page
> > - Open History bar
> > - find the plot (you can use the one I mentioned before or the one
I
> > generated today : 20180515_175025 )
> > - click on XML button and the plot XML will be displayed in the
popup
> > window
> > - click on "Load to Page" button and METViewer will populate
fields
> > - click on Generate Plot button and after a few seconds you should
have a
> > plot
> >
> > Tatiana
> > On Tue May 15 08:27:44 2018, perry.shafran at noaa.gov wrote:
> > > Just pinging in and was wondering if anyone saw this?
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > > On Thu, May 10, 2018 at 1:59 PM, Perry Shafran - NOAA Affiliate
<
> > > perry.shafran at noaa.gov> wrote:
> > >
> > > > Weird. I just downloaded this exact XML, loaded it into
METViewer
> and
> > > > pressed generate plot, and got no plot available. I'm
wondering if
> > > > something is wrong?
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > > On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT <
> > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > >> I rechecked METViewer and it shows the image and XML.
> > > >> I attached the files t this ticket also.
> > > >>
> > > >> Tatiana
> > > >> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
> > > >> > Plot 20180427_151839 says No Image Available. There also
seems to
> > be
> > > >> > a
> > > >> > blank XML template.
> > > >> >
> > > >> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
> > > >> > <met_help at ucar.edu>
> > > >> > wrote:
> > > >> >
> > > >> > > John and I spent some time to create an example plot
using your
> > data.
> > > >> > > This plot is for T - NAM/218 and TMP - NAM with RMSE stat
> > calculated
> > > >> > > using
> > > >> > > the summary method.
> > > >> > > T is plotted on Y2 axis and TMP on Y1
> > > >> > > We used SFC and Z2 levels and lead times
> > > >> > > 0,6,12,18,60000,120000,180000
> > > >> > > with defined plot vals to compensate the differences in
VSDB and
> > MET
> > > >> > > fcst_lead formats.
> > > >> > > We also made some adjustments for Y1 and Y2 axis
configurations.
> > > >> > > The resulting plot is 20180427_151839.
> > > >> > >
> > > >> > > Tatiana
> > > >> > >
> > > >> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov
wrote:
> > > >> > > > Right. I know that people have compared VSDB to MET
before,
> so
> > is
> > > >> > > > it
> > > >> > > > possible to compare TMP from the MET data vs. T from
the vsdb
> > data?
> > > >> > > >
> > > >> > > > Perry
> > > >> > > >
> > > >> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
> > > >> > > > <met_help at ucar.edu
> > > >> > > >
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi Perry,
> > > >> > > > > I changed your XML (commented one of the models )
because it
> > was
> > > >> > > > > loaded
> > > >> > > > > during your run and it was not necessary to reload it
again.
> > > >> > > > >
> > > >> > > > > I loaded the missing model into mv_emc_metvsvsdb
database.
> > > >> > > > >
> > > >> > > > > Currently this database contains data for four
models:
> > > >> > > > > mysql> select distinct model from stat_header;
> > > >> > > > > +---------+
> > > >> > > > > | model |
> > > >> > > > > +---------+
> > > >> > > > > | NAM |
> > > >> > > > > | NAM/218 |
> > > >> > > > > | NAM/242 |
> > > >> > > > > | NAM/243 |
> > > >> > > > > +---------+
> > > >> > > > >
> > > >> > > > > But it doesn't mean that all variables have data for
all
> > models.
> > > >> > > > > For
> > > >> > > > > example TMP has data for NAM only.
> > > >> > > > > VGRD, UGRD and some others also have data for NAM but
BCAPE
> > > >> > > > > doesn't.
> > > >> > > > >
> > > >> > > > > Tatiana
> > > >> > > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov
wrote:
> > > >> > > > > > Hi, Tatiana,
> > > >> > > > > >
> > > >> > > > > > Which database did you load to? I'm trying to
reload into
> > the
> > > >> > > > > > existing
> > > >> > > > > > database (mv_emc_metvsvsdb), and I only see
NAM/218,
> > NAM/242,
> > > >> > > > > > and
> > > >> > > > > > NAM/243.
> > > >> > > > > > No NAM alone.
> > > >> > > > > >
> > > >> > > > > > I also see that you had edited my loading XML and I
> changed
> > it
> > > >> > > > > > back
> > > >> > > to
> > > >> > > > > > ensure I was loading from both directories. Was
that a
> > > >> > > > > > problem? I'm
> > > >> > > > > > not
> > > >> > > > > > sure what I'm doing wrong but I can't seem to
duplicate
> what
> > > >> > > > > > you did.
> > > >> > > > > >
> > > >> > > > > > Thanks!
> > > >> > > > > >
> > > >> > > > > > Perry
> > > >> > > > > >
> > > >> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek via
RT
> > > >> > > > > > <met_help at ucar.edu>
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Hi Perry,
> > > >> > > > > > > The messages about VCNT line type are just
warnings and
> do
> > > >> > > > > > > not
> > > >> > > affect
> > > >> > > > > > > the
> > > >> > > > > > > loading. It means that all MET files were loaded.
> > > >> > > > > > > I fixed the problem with loading vsdb files - we
added
> > new
> > > >> > > > > > > columns
> > > >> > > > > > > to
> > > >> > > > > > > VL1L2 tables but the source code was not updated.
> > > >> > > > > > > I rerun your XML for vsdb files and got them
loaded to a
> > > >> > > > > > > database.
> > > >> > > > > > > METViewer UI shows models NAM/218, NAM/242,
NAM/243,
> NAM.
> > > >> > > > > > >
> > > >> > > > > > > Tatiana
> > > >> > > > > > >
> > > >> > > > > > > On Wed Apr 18 12:39:54 2018,
perry.shafran at noaa.gov
> > wrote:
> > > >> > > > > > > > Hi, there,
> > > >> > > > > > > >
> > > >> > > > > > > > I am having issues with loading MET .stat
output and
> > VSDB
> > > >> > > > > > > > output
> > > >> > > > > > > > into the
> > > >> > > > > > > > same database, and I was wondering how it was
done. I
> > have
> > > >> > > > > > > > data
> > > >> > > in
> > > >> > > > > > > > /metviewer/staging/wd20ps, in the nammet and
namvsdb
> > > >> > > > > > > > directories
> > > >> > > on
> > > >> > > > > > > > the
> > > >> > > > > > > > metviewer server if you want to take a look.
> > > >> > > > > > > >
> > > >> > > > > > > > On the sudo directory in /usr1/metviewer, I
used the
> > > >> > > > > > > > load_test_metvsvsdb_shafran.xml for loading and
the
> > output
> > > >> > > > > > > > is
> > > >> > > the
> > > >> > > > > > > > load_metvsvsdb.log file.
> > > >> > > > > > > >
> > > >> > > > > > > > If you note there are many messages regarding
VCNT for
> > the
> > > >> > > > > > > > met
> > > >> > > > > > > > files -
> > > >> > > > > > > why
> > > >> > > > > > > > are they there?
> > > >> > > > > > > >
> > > >> > > > > > > > But also see that the .vsdb files were skipped:
> > > >> > > > > > > >
> > > >> > > > > > > >
/metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
> > > >> > > > > > > > ** ERROR: caught class java.lang.Exception in
> > > >> > > > > > > > processFile()
> > > >> > > > > > > > caught SQLException calling executeBatch:
Column count
> > > >> > > > > > > > doesn't
> > > >> > > > > > > > match
> > > >> > > > > > > value
> > > >> > > > > > > > count at row 1
> > > >> > > > > > > > ** WARNING: error(s) encountered loading
file
> > > >> > > > > > > >
/metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb -
> > > >> > > > > > > > skipping
> > > >> > > file
> > > >> > > > > > > > java.lang.Exception: caught SQLException
calling
> > > >> > > > > > > > executeBatch:
> > > >> > > > > > > > Column
> > > >> > > > > > > count
> > > >> > > > > > > > doesn't match value count at row 1
> > > >> > > > > > > >
> > > >> > > > > > > > I'm wondering why that is. I haven't had
trouble
> > loading
> > > >> > > > > > > > VSDB
> > > >> > > > > > > > files
> > > >> > > > > > > > before.
> > > >> > > > > > > >
> > > >> > > > > > > > Then when I go to the METViewer page, the only
models
> I
> > see
> > > >> > > > > > > > are
> > > >> > > > > > > > NAM/218,
> > > >> > > > > > > > NAM/242, and NAM/243, which is what I would
expect if
> > the
> > > >> > > > > > > > VSDB
> > > >> > > > > > > > files were
> > > >> > > > > > > > loaded, not the MET files.
> > > >> > > > > > > >
> > > >> > > > > > > > Anyway, if you could let me know where I'm
going wrong
> > and
> > > >> > > > > > > > how I
> > > >> > > > > > > > can fix
> > > >> > > > > > > > it, that would be great. I know others have
plotted
> > VSDB
> > > >> > > > > > > > vs. MET
> > > >> > > > > > > > before.
> > > >> > > > > > > >
> > > >> > > > > > > > Thanks!
> > > >> > > > > > > >
> > > >> > > > > > > > Perry
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >>
> > > >>
> > > >>
> > > >>
> > > >
> >
> >
> >
> >
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: John Halley Gotway
Time: Tue May 15 13:28:30 2018
OK, I did the following steps:
(1) Log on to NCEP VPN via 140.90.73.186
(2) Navigate to METViewer at
http://metviewerdev.ncep.noaa.gov/metviewer1.jsp
(3) Click the "Load XML" button in the top right corner, navigate to,
and
load: plot_20180427_151839.xml
(4) Click the "Generate Plot" button.
Looks like it works as Tatiana described. See attached image.
Perry, if you think there's some issue with your browser, you could
try
clearing the cache, as you suggested. The other differences is that
you're
sitting inside the firewall while we're using VPN to get inside.
That's a
difference... but there's no reason why that difference would explain
the
behavior you're describing.
As a sanity check, you could ask Ben or Mallory to go through the same
steps inside the firewall to see if they have more luck.
Thanks,
John
On Tue, May 15, 2018 at 1:16 PM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Perry and Tatiana,
>
> As a "neutral third party", I'll go through the steps Tatiana
outlined and
> let you know what I find.
>
> Thanks,
> John
>
> On Tue, May 15, 2018 at 12:05 PM, perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=84827 >
>>
>> When I click on the plot for 20180515_175025, I get this image.
>>
>> Also the XML field is blank. But when I try to load to page and
then
>> click
>> generate plot, I get the same blank plot.
>>
>> I think that I personally have something wrong with my browser. If
it
>> works for you and it doesn't work for me.
>>
>> Perry
>>
>> On Tue, May 15, 2018 at 1:59 PM, Tatiana Burek via RT
<met_help at ucar.edu>
>> wrote:
>>
>> > Perry,
>> >
>> > I just repeated my steps and successfully generated the plot:
>> >
>> > - open METViewer page
>> > - Open History bar
>> > - find the plot (you can use the one I mentioned before or the
one I
>> > generated today : 20180515_175025 )
>> > - click on XML button and the plot XML will be displayed in the
popup
>> > window
>> > - click on "Load to Page" button and METViewer will populate
fields
>> > - click on Generate Plot button and after a few seconds you
should have
>> a
>> > plot
>> >
>> > Tatiana
>> > On Tue May 15 08:27:44 2018, perry.shafran at noaa.gov wrote:
>> > > Just pinging in and was wondering if anyone saw this?
>> > >
>> > > Thanks!
>> > >
>> > > Perry
>> > >
>> > > On Thu, May 10, 2018 at 1:59 PM, Perry Shafran - NOAA Affiliate
<
>> > > perry.shafran at noaa.gov> wrote:
>> > >
>> > > > Weird. I just downloaded this exact XML, loaded it into
METViewer
>> and
>> > > > pressed generate plot, and got no plot available. I'm
wondering if
>> > > > something is wrong?
>> > > >
>> > > > Thanks!
>> > > >
>> > > > Perry
>> > > >
>> > > > On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT <
>> > met_help at ucar.edu>
>> > > > wrote:
>> > > >
>> > > >> I rechecked METViewer and it shows the image and XML.
>> > > >> I attached the files t this ticket also.
>> > > >>
>> > > >> Tatiana
>> > > >> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
>> > > >> > Plot 20180427_151839 says No Image Available. There also
seems
>> to
>> > be
>> > > >> > a
>> > > >> > blank XML template.
>> > > >> >
>> > > >> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
>> > > >> > <met_help at ucar.edu>
>> > > >> > wrote:
>> > > >> >
>> > > >> > > John and I spent some time to create an example plot
using your
>> > data.
>> > > >> > > This plot is for T - NAM/218 and TMP - NAM with RMSE
stat
>> > calculated
>> > > >> > > using
>> > > >> > > the summary method.
>> > > >> > > T is plotted on Y2 axis and TMP on Y1
>> > > >> > > We used SFC and Z2 levels and lead times
>> > > >> > > 0,6,12,18,60000,120000,180000
>> > > >> > > with defined plot vals to compensate the differences in
VSDB
>> and
>> > MET
>> > > >> > > fcst_lead formats.
>> > > >> > > We also made some adjustments for Y1 and Y2 axis
>> configurations.
>> > > >> > > The resulting plot is 20180427_151839.
>> > > >> > >
>> > > >> > > Tatiana
>> > > >> > >
>> > > >> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov
wrote:
>> > > >> > > > Right. I know that people have compared VSDB to MET
before,
>> so
>> > is
>> > > >> > > > it
>> > > >> > > > possible to compare TMP from the MET data vs. T from
the vsdb
>> > data?
>> > > >> > > >
>> > > >> > > > Perry
>> > > >> > > >
>> > > >> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via RT
>> > > >> > > > <met_help at ucar.edu
>> > > >> > > >
>> > > >> > > > wrote:
>> > > >> > > >
>> > > >> > > > > Hi Perry,
>> > > >> > > > > I changed your XML (commented one of the models )
because
>> it
>> > was
>> > > >> > > > > loaded
>> > > >> > > > > during your run and it was not necessary to reload
it
>> again.
>> > > >> > > > >
>> > > >> > > > > I loaded the missing model into mv_emc_metvsvsdb
database.
>> > > >> > > > >
>> > > >> > > > > Currently this database contains data for four
models:
>> > > >> > > > > mysql> select distinct model from stat_header;
>> > > >> > > > > +---------+
>> > > >> > > > > | model |
>> > > >> > > > > +---------+
>> > > >> > > > > | NAM |
>> > > >> > > > > | NAM/218 |
>> > > >> > > > > | NAM/242 |
>> > > >> > > > > | NAM/243 |
>> > > >> > > > > +---------+
>> > > >> > > > >
>> > > >> > > > > But it doesn't mean that all variables have data for
all
>> > models.
>> > > >> > > > > For
>> > > >> > > > > example TMP has data for NAM only.
>> > > >> > > > > VGRD, UGRD and some others also have data for NAM
but BCAPE
>> > > >> > > > > doesn't.
>> > > >> > > > >
>> > > >> > > > > Tatiana
>> > > >> > > > > On Thu Apr 26 08:53:58 2018, perry.shafran at noaa.gov
wrote:
>> > > >> > > > > > Hi, Tatiana,
>> > > >> > > > > >
>> > > >> > > > > > Which database did you load to? I'm trying to
reload
>> into
>> > the
>> > > >> > > > > > existing
>> > > >> > > > > > database (mv_emc_metvsvsdb), and I only see
NAM/218,
>> > NAM/242,
>> > > >> > > > > > and
>> > > >> > > > > > NAM/243.
>> > > >> > > > > > No NAM alone.
>> > > >> > > > > >
>> > > >> > > > > > I also see that you had edited my loading XML and
I
>> changed
>> > it
>> > > >> > > > > > back
>> > > >> > > to
>> > > >> > > > > > ensure I was loading from both directories. Was
that a
>> > > >> > > > > > problem? I'm
>> > > >> > > > > > not
>> > > >> > > > > > sure what I'm doing wrong but I can't seem to
duplicate
>> what
>> > > >> > > > > > you did.
>> > > >> > > > > >
>> > > >> > > > > > Thanks!
>> > > >> > > > > >
>> > > >> > > > > > Perry
>> > > >> > > > > >
>> > > >> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek
via RT
>> > > >> > > > > > <met_help at ucar.edu>
>> > > >> > > > > > wrote:
>> > > >> > > > > >
>> > > >> > > > > > > Hi Perry,
>> > > >> > > > > > > The messages about VCNT line type are just
warnings
>> and do
>> > > >> > > > > > > not
>> > > >> > > affect
>> > > >> > > > > > > the
>> > > >> > > > > > > loading. It means that all MET files were
loaded.
>> > > >> > > > > > > I fixed the problem with loading vsdb files -
we added
>> > new
>> > > >> > > > > > > columns
>> > > >> > > > > > > to
>> > > >> > > > > > > VL1L2 tables but the source code was not
updated.
>> > > >> > > > > > > I rerun your XML for vsdb files and got them
loaded to
>> a
>> > > >> > > > > > > database.
>> > > >> > > > > > > METViewer UI shows models NAM/218, NAM/242,
NAM/243,
>> NAM.
>> > > >> > > > > > >
>> > > >> > > > > > > Tatiana
>> > > >> > > > > > >
>> > > >> > > > > > > On Wed Apr 18 12:39:54 2018,
perry.shafran at noaa.gov
>> > wrote:
>> > > >> > > > > > > > Hi, there,
>> > > >> > > > > > > >
>> > > >> > > > > > > > I am having issues with loading MET .stat
output and
>> > VSDB
>> > > >> > > > > > > > output
>> > > >> > > > > > > > into the
>> > > >> > > > > > > > same database, and I was wondering how it was
done.
>> I
>> > have
>> > > >> > > > > > > > data
>> > > >> > > in
>> > > >> > > > > > > > /metviewer/staging/wd20ps, in the nammet and
namvsdb
>> > > >> > > > > > > > directories
>> > > >> > > on
>> > > >> > > > > > > > the
>> > > >> > > > > > > > metviewer server if you want to take a look.
>> > > >> > > > > > > >
>> > > >> > > > > > > > On the sudo directory in /usr1/metviewer, I
used the
>> > > >> > > > > > > > load_test_metvsvsdb_shafran.xml for loading
and the
>> > output
>> > > >> > > > > > > > is
>> > > >> > > the
>> > > >> > > > > > > > load_metvsvsdb.log file.
>> > > >> > > > > > > >
>> > > >> > > > > > > > If you note there are many messages regarding
VCNT
>> for
>> > the
>> > > >> > > > > > > > met
>> > > >> > > > > > > > files -
>> > > >> > > > > > > why
>> > > >> > > > > > > > are they there?
>> > > >> > > > > > > >
>> > > >> > > > > > > > But also see that the .vsdb files were
skipped:
>> > > >> > > > > > > >
>> > > >> > > > > > > >
/metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
>> > > >> > > > > > > > ** ERROR: caught class java.lang.Exception in
>> > > >> > > > > > > > processFile()
>> > > >> > > > > > > > caught SQLException calling executeBatch:
Column
>> count
>> > > >> > > > > > > > doesn't
>> > > >> > > > > > > > match
>> > > >> > > > > > > value
>> > > >> > > > > > > > count at row 1
>> > > >> > > > > > > > ** WARNING: error(s) encountered loading
file
>> > > >> > > > > > > >
/metviewer/staging/wd20ps/namvsdb/nam_20180414.vsdb
>> -
>> > > >> > > > > > > > skipping
>> > > >> > > file
>> > > >> > > > > > > > java.lang.Exception: caught SQLException
calling
>> > > >> > > > > > > > executeBatch:
>> > > >> > > > > > > > Column
>> > > >> > > > > > > count
>> > > >> > > > > > > > doesn't match value count at row 1
>> > > >> > > > > > > >
>> > > >> > > > > > > > I'm wondering why that is. I haven't had
trouble
>> > loading
>> > > >> > > > > > > > VSDB
>> > > >> > > > > > > > files
>> > > >> > > > > > > > before.
>> > > >> > > > > > > >
>> > > >> > > > > > > > Then when I go to the METViewer page, the only
>> models I
>> > see
>> > > >> > > > > > > > are
>> > > >> > > > > > > > NAM/218,
>> > > >> > > > > > > > NAM/242, and NAM/243, which is what I would
expect if
>> > the
>> > > >> > > > > > > > VSDB
>> > > >> > > > > > > > files were
>> > > >> > > > > > > > loaded, not the MET files.
>> > > >> > > > > > > >
>> > > >> > > > > > > > Anyway, if you could let me know where I'm
going
>> wrong
>> > and
>> > > >> > > > > > > > how I
>> > > >> > > > > > > > can fix
>> > > >> > > > > > > > it, that would be great. I know others have
plotted
>> > VSDB
>> > > >> > > > > > > > vs. MET
>> > > >> > > > > > > > before.
>> > > >> > > > > > > >
>> > > >> > > > > > > > Thanks!
>> > > >> > > > > > > >
>> > > >> > > > > > > > Perry
>> > > >> > > > > > >
>> > > >> > > > > > >
>> > > >> > > > > > >
>> > > >> > > > > > >
>> > > >> > > > >
>> > > >> > > > >
>> > > >> > > > >
>> > > >> > > > >
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >>
>> > > >>
>> > > >>
>> > > >>
>> > > >
>> >
>> >
>> >
>> >
>>
>>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Tue May 15 13:31:11 2018
I am also on a Linux machine and not Windows. But that shouldn't
matter as
METViewer has worked just fine before for me, at least using other
databases. This is so weird.
Perry
On Tue, May 15, 2018 at 3:28 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> OK, I did the following steps:
>
> (1) Log on to NCEP VPN via 140.90.73.186
> (2) Navigate to METViewer at
> http://metviewerdev.ncep.noaa.gov/metviewer1.jsp
> (3) Click the "Load XML" button in the top right corner, navigate
to, and
> load: plot_20180427_151839.xml
> (4) Click the "Generate Plot" button.
>
> Looks like it works as Tatiana described. See attached image.
>
> Perry, if you think there's some issue with your browser, you could
try
> clearing the cache, as you suggested. The other differences is that
you're
> sitting inside the firewall while we're using VPN to get inside.
That's a
> difference... but there's no reason why that difference would
explain the
> behavior you're describing.
>
> As a sanity check, you could ask Ben or Mallory to go through the
same
> steps inside the firewall to see if they have more luck.
>
> Thanks,
> John
>
> On Tue, May 15, 2018 at 1:16 PM, John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Perry and Tatiana,
> >
> > As a "neutral third party", I'll go through the steps Tatiana
outlined
> and
> > let you know what I find.
> >
> > Thanks,
> > John
> >
> > On Tue, May 15, 2018 at 12:05 PM, perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=84827 >
> >>
> >> When I click on the plot for 20180515_175025, I get this image.
> >>
> >> Also the XML field is blank. But when I try to load to page and
then
> >> click
> >> generate plot, I get the same blank plot.
> >>
> >> I think that I personally have something wrong with my browser.
If it
> >> works for you and it doesn't work for me.
> >>
> >> Perry
> >>
> >> On Tue, May 15, 2018 at 1:59 PM, Tatiana Burek via RT <
> met_help at ucar.edu>
> >> wrote:
> >>
> >> > Perry,
> >> >
> >> > I just repeated my steps and successfully generated the plot:
> >> >
> >> > - open METViewer page
> >> > - Open History bar
> >> > - find the plot (you can use the one I mentioned before or the
one I
> >> > generated today : 20180515_175025 )
> >> > - click on XML button and the plot XML will be displayed in the
popup
> >> > window
> >> > - click on "Load to Page" button and METViewer will populate
fields
> >> > - click on Generate Plot button and after a few seconds you
should
> have
> >> a
> >> > plot
> >> >
> >> > Tatiana
> >> > On Tue May 15 08:27:44 2018, perry.shafran at noaa.gov wrote:
> >> > > Just pinging in and was wondering if anyone saw this?
> >> > >
> >> > > Thanks!
> >> > >
> >> > > Perry
> >> > >
> >> > > On Thu, May 10, 2018 at 1:59 PM, Perry Shafran - NOAA
Affiliate <
> >> > > perry.shafran at noaa.gov> wrote:
> >> > >
> >> > > > Weird. I just downloaded this exact XML, loaded it into
METViewer
> >> and
> >> > > > pressed generate plot, and got no plot available. I'm
wondering
> if
> >> > > > something is wrong?
> >> > > >
> >> > > > Thanks!
> >> > > >
> >> > > > Perry
> >> > > >
> >> > > > On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT <
> >> > met_help at ucar.edu>
> >> > > > wrote:
> >> > > >
> >> > > >> I rechecked METViewer and it shows the image and XML.
> >> > > >> I attached the files t this ticket also.
> >> > > >>
> >> > > >> Tatiana
> >> > > >> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov wrote:
> >> > > >> > Plot 20180427_151839 says No Image Available. There
also seems
> >> to
> >> > be
> >> > > >> > a
> >> > > >> > blank XML template.
> >> > > >> >
> >> > > >> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
> >> > > >> > <met_help at ucar.edu>
> >> > > >> > wrote:
> >> > > >> >
> >> > > >> > > John and I spent some time to create an example plot
using
> your
> >> > data.
> >> > > >> > > This plot is for T - NAM/218 and TMP - NAM with RMSE
stat
> >> > calculated
> >> > > >> > > using
> >> > > >> > > the summary method.
> >> > > >> > > T is plotted on Y2 axis and TMP on Y1
> >> > > >> > > We used SFC and Z2 levels and lead times
> >> > > >> > > 0,6,12,18,60000,120000,180000
> >> > > >> > > with defined plot vals to compensate the differences
in VSDB
> >> and
> >> > MET
> >> > > >> > > fcst_lead formats.
> >> > > >> > > We also made some adjustments for Y1 and Y2 axis
> >> configurations.
> >> > > >> > > The resulting plot is 20180427_151839.
> >> > > >> > >
> >> > > >> > > Tatiana
> >> > > >> > >
> >> > > >> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov
wrote:
> >> > > >> > > > Right. I know that people have compared VSDB to MET
> before,
> >> so
> >> > is
> >> > > >> > > > it
> >> > > >> > > > possible to compare TMP from the MET data vs. T from
the
> vsdb
> >> > data?
> >> > > >> > > >
> >> > > >> > > > Perry
> >> > > >> > > >
> >> > > >> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via
RT
> >> > > >> > > > <met_help at ucar.edu
> >> > > >> > > >
> >> > > >> > > > wrote:
> >> > > >> > > >
> >> > > >> > > > > Hi Perry,
> >> > > >> > > > > I changed your XML (commented one of the models )
because
> >> it
> >> > was
> >> > > >> > > > > loaded
> >> > > >> > > > > during your run and it was not necessary to reload
it
> >> again.
> >> > > >> > > > >
> >> > > >> > > > > I loaded the missing model into mv_emc_metvsvsdb
> database.
> >> > > >> > > > >
> >> > > >> > > > > Currently this database contains data for four
models:
> >> > > >> > > > > mysql> select distinct model from stat_header;
> >> > > >> > > > > +---------+
> >> > > >> > > > > | model |
> >> > > >> > > > > +---------+
> >> > > >> > > > > | NAM |
> >> > > >> > > > > | NAM/218 |
> >> > > >> > > > > | NAM/242 |
> >> > > >> > > > > | NAM/243 |
> >> > > >> > > > > +---------+
> >> > > >> > > > >
> >> > > >> > > > > But it doesn't mean that all variables have data
for all
> >> > models.
> >> > > >> > > > > For
> >> > > >> > > > > example TMP has data for NAM only.
> >> > > >> > > > > VGRD, UGRD and some others also have data for NAM
but
> BCAPE
> >> > > >> > > > > doesn't.
> >> > > >> > > > >
> >> > > >> > > > > Tatiana
> >> > > >> > > > > On Thu Apr 26 08:53:58 2018,
perry.shafran at noaa.gov
> wrote:
> >> > > >> > > > > > Hi, Tatiana,
> >> > > >> > > > > >
> >> > > >> > > > > > Which database did you load to? I'm trying to
reload
> >> into
> >> > the
> >> > > >> > > > > > existing
> >> > > >> > > > > > database (mv_emc_metvsvsdb), and I only see
NAM/218,
> >> > NAM/242,
> >> > > >> > > > > > and
> >> > > >> > > > > > NAM/243.
> >> > > >> > > > > > No NAM alone.
> >> > > >> > > > > >
> >> > > >> > > > > > I also see that you had edited my loading XML
and I
> >> changed
> >> > it
> >> > > >> > > > > > back
> >> > > >> > > to
> >> > > >> > > > > > ensure I was loading from both directories. Was
that a
> >> > > >> > > > > > problem? I'm
> >> > > >> > > > > > not
> >> > > >> > > > > > sure what I'm doing wrong but I can't seem to
duplicate
> >> what
> >> > > >> > > > > > you did.
> >> > > >> > > > > >
> >> > > >> > > > > > Thanks!
> >> > > >> > > > > >
> >> > > >> > > > > > Perry
> >> > > >> > > > > >
> >> > > >> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek
via RT
> >> > > >> > > > > > <met_help at ucar.edu>
> >> > > >> > > > > > wrote:
> >> > > >> > > > > >
> >> > > >> > > > > > > Hi Perry,
> >> > > >> > > > > > > The messages about VCNT line type are just
warnings
> >> and do
> >> > > >> > > > > > > not
> >> > > >> > > affect
> >> > > >> > > > > > > the
> >> > > >> > > > > > > loading. It means that all MET files were
loaded.
> >> > > >> > > > > > > I fixed the problem with loading vsdb files -
we
> added
> >> > new
> >> > > >> > > > > > > columns
> >> > > >> > > > > > > to
> >> > > >> > > > > > > VL1L2 tables but the source code was not
updated.
> >> > > >> > > > > > > I rerun your XML for vsdb files and got them
loaded
> to
> >> a
> >> > > >> > > > > > > database.
> >> > > >> > > > > > > METViewer UI shows models NAM/218, NAM/242,
NAM/243,
> >> NAM.
> >> > > >> > > > > > >
> >> > > >> > > > > > > Tatiana
> >> > > >> > > > > > >
> >> > > >> > > > > > > On Wed Apr 18 12:39:54 2018,
perry.shafran at noaa.gov
> >> > wrote:
> >> > > >> > > > > > > > Hi, there,
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > I am having issues with loading MET .stat
output
> and
> >> > VSDB
> >> > > >> > > > > > > > output
> >> > > >> > > > > > > > into the
> >> > > >> > > > > > > > same database, and I was wondering how it
was done.
> >> I
> >> > have
> >> > > >> > > > > > > > data
> >> > > >> > > in
> >> > > >> > > > > > > > /metviewer/staging/wd20ps, in the nammet and
> namvsdb
> >> > > >> > > > > > > > directories
> >> > > >> > > on
> >> > > >> > > > > > > > the
> >> > > >> > > > > > > > metviewer server if you want to take a look.
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > On the sudo directory in /usr1/metviewer, I
used
> the
> >> > > >> > > > > > > > load_test_metvsvsdb_shafran.xml for loading
and
> the
> >> > output
> >> > > >> > > > > > > > is
> >> > > >> > > the
> >> > > >> > > > > > > > load_metvsvsdb.log file.
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > If you note there are many messages
regarding VCNT
> >> for
> >> > the
> >> > > >> > > > > > > > met
> >> > > >> > > > > > > > files -
> >> > > >> > > > > > > why
> >> > > >> > > > > > > > are they there?
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > But also see that the .vsdb files were
skipped:
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > /metviewer/staging/wd20ps/
> namvsdb/nam_20180414.vsdb
> >> > > >> > > > > > > > ** ERROR: caught class java.lang.Exception
in
> >> > > >> > > > > > > > processFile()
> >> > > >> > > > > > > > caught SQLException calling executeBatch:
Column
> >> count
> >> > > >> > > > > > > > doesn't
> >> > > >> > > > > > > > match
> >> > > >> > > > > > > value
> >> > > >> > > > > > > > count at row 1
> >> > > >> > > > > > > > ** WARNING: error(s) encountered loading
file
> >> > > >> > > > > > > > /metviewer/staging/wd20ps/
> namvsdb/nam_20180414.vsdb
> >> -
> >> > > >> > > > > > > > skipping
> >> > > >> > > file
> >> > > >> > > > > > > > java.lang.Exception: caught SQLException
calling
> >> > > >> > > > > > > > executeBatch:
> >> > > >> > > > > > > > Column
> >> > > >> > > > > > > count
> >> > > >> > > > > > > > doesn't match value count at row 1
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > I'm wondering why that is. I haven't had
trouble
> >> > loading
> >> > > >> > > > > > > > VSDB
> >> > > >> > > > > > > > files
> >> > > >> > > > > > > > before.
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > Then when I go to the METViewer page, the
only
> >> models I
> >> > see
> >> > > >> > > > > > > > are
> >> > > >> > > > > > > > NAM/218,
> >> > > >> > > > > > > > NAM/242, and NAM/243, which is what I would
expect
> if
> >> > the
> >> > > >> > > > > > > > VSDB
> >> > > >> > > > > > > > files were
> >> > > >> > > > > > > > loaded, not the MET files.
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > Anyway, if you could let me know where I'm
going
> >> wrong
> >> > and
> >> > > >> > > > > > > > how I
> >> > > >> > > > > > > > can fix
> >> > > >> > > > > > > > it, that would be great. I know others have
> plotted
> >> > VSDB
> >> > > >> > > > > > > > vs. MET
> >> > > >> > > > > > > > before.
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > Thanks!
> >> > > >> > > > > > > >
> >> > > >> > > > > > > > Perry
> >> > > >> > > > > > >
> >> > > >> > > > > > >
> >> > > >> > > > > > >
> >> > > >> > > > > > >
> >> > > >> > > > >
> >> > > >> > > > >
> >> > > >> > > > >
> >> > > >> > > > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >
>
>
------------------------------------------------
Subject: comparing VSDB vs. MET stat output
From: perry.shafran at noaa.gov
Time: Wed May 16 08:28:05 2018
OK, I think that I figured out the problem. I was using the wrong
weblink,
and because of that nothing is working. I found the proper weblink
and now
I see the plots and the XMLs and the like.
Sheesh.
Perry
On Tue, May 15, 2018 at 3:31 PM, Perry Shafran - NOAA Affiliate <
perry.shafran at noaa.gov> wrote:
> I am also on a Linux machine and not Windows. But that shouldn't
matter
> as METViewer has worked just fine before for me, at least using
other
> databases. This is so weird.
>
> Perry
>
> On Tue, May 15, 2018 at 3:28 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> OK, I did the following steps:
>>
>> (1) Log on to NCEP VPN via 140.90.73.186
>> (2) Navigate to METViewer at
>> http://metviewerdev.ncep.noaa.gov/metviewer1.jsp
>> (3) Click the "Load XML" button in the top right corner, navigate
to, and
>> load: plot_20180427_151839.xml
>> (4) Click the "Generate Plot" button.
>>
>> Looks like it works as Tatiana described. See attached image.
>>
>> Perry, if you think there's some issue with your browser, you could
try
>> clearing the cache, as you suggested. The other differences is
that
>> you're
>> sitting inside the firewall while we're using VPN to get inside.
That's a
>> difference... but there's no reason why that difference would
explain the
>> behavior you're describing.
>>
>> As a sanity check, you could ask Ben or Mallory to go through the
same
>> steps inside the firewall to see if they have more luck.
>>
>> Thanks,
>> John
>>
>> On Tue, May 15, 2018 at 1:16 PM, John Halley Gotway
<johnhg at ucar.edu>
>> wrote:
>>
>> > Perry and Tatiana,
>> >
>> > As a "neutral third party", I'll go through the steps Tatiana
outlined
>> and
>> > let you know what I find.
>> >
>> > Thanks,
>> > John
>> >
>> > On Tue, May 15, 2018 at 12:05 PM, perry.shafran at noaa.gov via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> >>
>> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=84827 >
>> >>
>> >> When I click on the plot for 20180515_175025, I get this image.
>> >>
>> >> Also the XML field is blank. But when I try to load to page and
then
>> >> click
>> >> generate plot, I get the same blank plot.
>> >>
>> >> I think that I personally have something wrong with my browser.
If it
>> >> works for you and it doesn't work for me.
>> >>
>> >> Perry
>> >>
>> >> On Tue, May 15, 2018 at 1:59 PM, Tatiana Burek via RT <
>> met_help at ucar.edu>
>> >> wrote:
>> >>
>> >> > Perry,
>> >> >
>> >> > I just repeated my steps and successfully generated the plot:
>> >> >
>> >> > - open METViewer page
>> >> > - Open History bar
>> >> > - find the plot (you can use the one I mentioned before or the
one I
>> >> > generated today : 20180515_175025 )
>> >> > - click on XML button and the plot XML will be displayed in
the popup
>> >> > window
>> >> > - click on "Load to Page" button and METViewer will populate
fields
>> >> > - click on Generate Plot button and after a few seconds you
should
>> have
>> >> a
>> >> > plot
>> >> >
>> >> > Tatiana
>> >> > On Tue May 15 08:27:44 2018, perry.shafran at noaa.gov wrote:
>> >> > > Just pinging in and was wondering if anyone saw this?
>> >> > >
>> >> > > Thanks!
>> >> > >
>> >> > > Perry
>> >> > >
>> >> > > On Thu, May 10, 2018 at 1:59 PM, Perry Shafran - NOAA
Affiliate <
>> >> > > perry.shafran at noaa.gov> wrote:
>> >> > >
>> >> > > > Weird. I just downloaded this exact XML, loaded it into
>> METViewer
>> >> and
>> >> > > > pressed generate plot, and got no plot available. I'm
wondering
>> if
>> >> > > > something is wrong?
>> >> > > >
>> >> > > > Thanks!
>> >> > > >
>> >> > > > Perry
>> >> > > >
>> >> > > > On Fri, Apr 27, 2018 at 11:45 AM, Tatiana Burek via RT <
>> >> > met_help at ucar.edu>
>> >> > > > wrote:
>> >> > > >
>> >> > > >> I rechecked METViewer and it shows the image and XML.
>> >> > > >> I attached the files t this ticket also.
>> >> > > >>
>> >> > > >> Tatiana
>> >> > > >> On Fri Apr 27 09:39:58 2018, perry.shafran at noaa.gov
wrote:
>> >> > > >> > Plot 20180427_151839 says No Image Available. There
also
>> seems
>> >> to
>> >> > be
>> >> > > >> > a
>> >> > > >> > blank XML template.
>> >> > > >> >
>> >> > > >> > On Fri, Apr 27, 2018 at 11:35 AM, Tatiana Burek via RT
>> >> > > >> > <met_help at ucar.edu>
>> >> > > >> > wrote:
>> >> > > >> >
>> >> > > >> > > John and I spent some time to create an example plot
using
>> your
>> >> > data.
>> >> > > >> > > This plot is for T - NAM/218 and TMP - NAM with RMSE
stat
>> >> > calculated
>> >> > > >> > > using
>> >> > > >> > > the summary method.
>> >> > > >> > > T is plotted on Y2 axis and TMP on Y1
>> >> > > >> > > We used SFC and Z2 levels and lead times
>> >> > > >> > > 0,6,12,18,60000,120000,180000
>> >> > > >> > > with defined plot vals to compensate the differences
in VSDB
>> >> and
>> >> > MET
>> >> > > >> > > fcst_lead formats.
>> >> > > >> > > We also made some adjustments for Y1 and Y2 axis
>> >> configurations.
>> >> > > >> > > The resulting plot is 20180427_151839.
>> >> > > >> > >
>> >> > > >> > > Tatiana
>> >> > > >> > >
>> >> > > >> > > On Thu Apr 26 14:29:01 2018, perry.shafran at noaa.gov
wrote:
>> >> > > >> > > > Right. I know that people have compared VSDB to
MET
>> before,
>> >> so
>> >> > is
>> >> > > >> > > > it
>> >> > > >> > > > possible to compare TMP from the MET data vs. T
from the
>> vsdb
>> >> > data?
>> >> > > >> > > >
>> >> > > >> > > > Perry
>> >> > > >> > > >
>> >> > > >> > > > On Thu, Apr 26, 2018 at 4:27 PM, Tatiana Burek via
RT
>> >> > > >> > > > <met_help at ucar.edu
>> >> > > >> > > >
>> >> > > >> > > > wrote:
>> >> > > >> > > >
>> >> > > >> > > > > Hi Perry,
>> >> > > >> > > > > I changed your XML (commented one of the models )
>> because
>> >> it
>> >> > was
>> >> > > >> > > > > loaded
>> >> > > >> > > > > during your run and it was not necessary to
reload it
>> >> again.
>> >> > > >> > > > >
>> >> > > >> > > > > I loaded the missing model into mv_emc_metvsvsdb
>> database.
>> >> > > >> > > > >
>> >> > > >> > > > > Currently this database contains data for four
models:
>> >> > > >> > > > > mysql> select distinct model from stat_header;
>> >> > > >> > > > > +---------+
>> >> > > >> > > > > | model |
>> >> > > >> > > > > +---------+
>> >> > > >> > > > > | NAM |
>> >> > > >> > > > > | NAM/218 |
>> >> > > >> > > > > | NAM/242 |
>> >> > > >> > > > > | NAM/243 |
>> >> > > >> > > > > +---------+
>> >> > > >> > > > >
>> >> > > >> > > > > But it doesn't mean that all variables have data
for all
>> >> > models.
>> >> > > >> > > > > For
>> >> > > >> > > > > example TMP has data for NAM only.
>> >> > > >> > > > > VGRD, UGRD and some others also have data for NAM
but
>> BCAPE
>> >> > > >> > > > > doesn't.
>> >> > > >> > > > >
>> >> > > >> > > > > Tatiana
>> >> > > >> > > > > On Thu Apr 26 08:53:58 2018,
perry.shafran at noaa.gov
>> wrote:
>> >> > > >> > > > > > Hi, Tatiana,
>> >> > > >> > > > > >
>> >> > > >> > > > > > Which database did you load to? I'm trying to
reload
>> >> into
>> >> > the
>> >> > > >> > > > > > existing
>> >> > > >> > > > > > database (mv_emc_metvsvsdb), and I only see
NAM/218,
>> >> > NAM/242,
>> >> > > >> > > > > > and
>> >> > > >> > > > > > NAM/243.
>> >> > > >> > > > > > No NAM alone.
>> >> > > >> > > > > >
>> >> > > >> > > > > > I also see that you had edited my loading XML
and I
>> >> changed
>> >> > it
>> >> > > >> > > > > > back
>> >> > > >> > > to
>> >> > > >> > > > > > ensure I was loading from both directories.
Was that
>> a
>> >> > > >> > > > > > problem? I'm
>> >> > > >> > > > > > not
>> >> > > >> > > > > > sure what I'm doing wrong but I can't seem to
>> duplicate
>> >> what
>> >> > > >> > > > > > you did.
>> >> > > >> > > > > >
>> >> > > >> > > > > > Thanks!
>> >> > > >> > > > > >
>> >> > > >> > > > > > Perry
>> >> > > >> > > > > >
>> >> > > >> > > > > > On Fri, Apr 20, 2018 at 10:26 AM, Tatiana Burek
via RT
>> >> > > >> > > > > > <met_help at ucar.edu>
>> >> > > >> > > > > > wrote:
>> >> > > >> > > > > >
>> >> > > >> > > > > > > Hi Perry,
>> >> > > >> > > > > > > The messages about VCNT line type are just
warnings
>> >> and do
>> >> > > >> > > > > > > not
>> >> > > >> > > affect
>> >> > > >> > > > > > > the
>> >> > > >> > > > > > > loading. It means that all MET files were
loaded.
>> >> > > >> > > > > > > I fixed the problem with loading vsdb files
- we
>> added
>> >> > new
>> >> > > >> > > > > > > columns
>> >> > > >> > > > > > > to
>> >> > > >> > > > > > > VL1L2 tables but the source code was not
updated.
>> >> > > >> > > > > > > I rerun your XML for vsdb files and got them
loaded
>> to
>> >> a
>> >> > > >> > > > > > > database.
>> >> > > >> > > > > > > METViewer UI shows models NAM/218, NAM/242,
NAM/243,
>> >> NAM.
>> >> > > >> > > > > > >
>> >> > > >> > > > > > > Tatiana
>> >> > > >> > > > > > >
>> >> > > >> > > > > > > On Wed Apr 18 12:39:54 2018,
perry.shafran at noaa.gov
>> >> > wrote:
>> >> > > >> > > > > > > > Hi, there,
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > I am having issues with loading MET .stat
output
>> and
>> >> > VSDB
>> >> > > >> > > > > > > > output
>> >> > > >> > > > > > > > into the
>> >> > > >> > > > > > > > same database, and I was wondering how it
was
>> done.
>> >> I
>> >> > have
>> >> > > >> > > > > > > > data
>> >> > > >> > > in
>> >> > > >> > > > > > > > /metviewer/staging/wd20ps, in the nammet
and
>> namvsdb
>> >> > > >> > > > > > > > directories
>> >> > > >> > > on
>> >> > > >> > > > > > > > the
>> >> > > >> > > > > > > > metviewer server if you want to take a
look.
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > On the sudo directory in /usr1/metviewer, I
used
>> the
>> >> > > >> > > > > > > > load_test_metvsvsdb_shafran.xml for loading
and
>> the
>> >> > output
>> >> > > >> > > > > > > > is
>> >> > > >> > > the
>> >> > > >> > > > > > > > load_metvsvsdb.log file.
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > If you note there are many messages
regarding VCNT
>> >> for
>> >> > the
>> >> > > >> > > > > > > > met
>> >> > > >> > > > > > > > files -
>> >> > > >> > > > > > > why
>> >> > > >> > > > > > > > are they there?
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > But also see that the .vsdb files were
skipped:
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > /metviewer/staging/wd20ps/namv
>> sdb/nam_20180414.vsdb
>> >> > > >> > > > > > > > ** ERROR: caught class java.lang.Exception
in
>> >> > > >> > > > > > > > processFile()
>> >> > > >> > > > > > > > caught SQLException calling executeBatch:
Column
>> >> count
>> >> > > >> > > > > > > > doesn't
>> >> > > >> > > > > > > > match
>> >> > > >> > > > > > > value
>> >> > > >> > > > > > > > count at row 1
>> >> > > >> > > > > > > > ** WARNING: error(s) encountered loading
file
>> >> > > >> > > > > > > > /metviewer/staging/wd20ps/namv
>> sdb/nam_20180414.vsdb
>> >> -
>> >> > > >> > > > > > > > skipping
>> >> > > >> > > file
>> >> > > >> > > > > > > > java.lang.Exception: caught SQLException
calling
>> >> > > >> > > > > > > > executeBatch:
>> >> > > >> > > > > > > > Column
>> >> > > >> > > > > > > count
>> >> > > >> > > > > > > > doesn't match value count at row 1
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > I'm wondering why that is. I haven't had
trouble
>> >> > loading
>> >> > > >> > > > > > > > VSDB
>> >> > > >> > > > > > > > files
>> >> > > >> > > > > > > > before.
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > Then when I go to the METViewer page, the
only
>> >> models I
>> >> > see
>> >> > > >> > > > > > > > are
>> >> > > >> > > > > > > > NAM/218,
>> >> > > >> > > > > > > > NAM/242, and NAM/243, which is what I would
>> expect if
>> >> > the
>> >> > > >> > > > > > > > VSDB
>> >> > > >> > > > > > > > files were
>> >> > > >> > > > > > > > loaded, not the MET files.
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > Anyway, if you could let me know where I'm
going
>> >> wrong
>> >> > and
>> >> > > >> > > > > > > > how I
>> >> > > >> > > > > > > > can fix
>> >> > > >> > > > > > > > it, that would be great. I know others
have
>> plotted
>> >> > VSDB
>> >> > > >> > > > > > > > vs. MET
>> >> > > >> > > > > > > > before.
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > Thanks!
>> >> > > >> > > > > > > >
>> >> > > >> > > > > > > > Perry
>> >> > > >> > > > > > >
>> >> > > >> > > > > > >
>> >> > > >> > > > > > >
>> >> > > >> > > > > > >
>> >> > > >> > > > >
>> >> > > >> > > > >
>> >> > > >> > > > >
>> >> > > >> > > > >
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >> > >
>> >> > > >>
>> >> > > >>
>> >> > > >>
>> >> > > >>
>> >> > > >
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >
>>
>>
>
------------------------------------------------
More information about the Met_help
mailing list