[Met_help] [rt.rap.ucar.edu #97515] History for error from mv_prune_on_aws.sh
Tatiana Burek via RT
met_help at ucar.edu
Thu Mar 18 11:23:26 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I tried to delete the model " Raikoke_31.0p25.p006 " from database "
mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref" using "mv_prune_on_aws.sh", but
I got error (see the attachment pls.)
Here is the script I used:
<prune_spec>
<connection>
<host>db_host:3306</host>
<database>mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref</database>
<user>rds_user</user>
<password>rds_pwd</password>
</connection>
<info_only>true</info_only>
<fields>
<field name="fcst_valid_beg">
<value_range>
<start>2019-06-20 06:00:00</start>
<end>2019-06-25 18:00:00</end>
</value_range>
</field>
<field name="model">
<value_list>
<value>Raikoke_31.0p25.p006</value>
</value_list>
</field>
<field name="fcst_var">
<value_list>
<value>VAFTD</value>
</value_list>
</field>
</fields>
</prune_spec>
-------------------------------------------------------------------------------------------
part of the error log file:
SEVERE: Unable to create initial connections of pool.
java.sql.SQLNonTransientConnectionException: Communications link failure
with primary. No active connection found for master.
at
org.mariadb.jdbc.internal.failover.AbstractMastersListener.throwFailoverMessage(AbstractMastersListener.java:534)
at
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.checkInitialConnection(MastersSlavesListener.java:336)
at
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:173)
at
org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
at
org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
at
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:174)
at org.mariadb.jdbc.Driver.connect(Driver.java:92)
Thank you for help.
Binyu
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: error from mv_prune_on_aws.sh
From: Tatiana Burek
Time: Thu Nov 19 07:41:54 2020
Binyu,
We had same problems with the server yesterday.
Could you try to run your process again and let me know if it still
doesn't work.
Tatiana
On Wed Nov 18 21:28:51 2020, binyu.wang at noaa.gov wrote:
> Hello,
>
> I tried to delete the model " Raikoke_31.0p25.p006 " from database "
> mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref" using
"mv_prune_on_aws.sh",
> but
> I got error (see the attachment pls.)
>
> Here is the script I used:
> <prune_spec>
> <connection>
> <host>db_host:3306</host>
> <database>mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref</database>
> <user>rds_user</user>
> <password>rds_pwd</password>
> </connection>
> <info_only>true</info_only>
>
> <fields>
> <field name="fcst_valid_beg">
> <value_range>
> <start>2019-06-20 06:00:00</start>
> <end>2019-06-25 18:00:00</end>
> </value_range>
> </field>
> <field name="model">
> <value_list>
> <value>Raikoke_31.0p25.p006</value>
> </value_list>
> </field>
> <field name="fcst_var">
> <value_list>
> <value>VAFTD</value>
> </value_list>
> </field>
> </fields>
> </prune_spec>
>
>
-------------------------------------------------------------------------------------------
> part of the error log file:
> SEVERE: Unable to create initial connections of pool.
> java.sql.SQLNonTransientConnectionException: Communications link
> failure
> with primary. No active connection found for master.
> at
>
org.mariadb.jdbc.internal.failover.AbstractMastersListener.throwFailoverMessage(AbstractMastersListener.java:534)
> at
>
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.checkInitialConnection(MastersSlavesListener.java:336)
> at
>
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:173)
> at
>
org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
> at
> org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
> at
>
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:174)
> at org.mariadb.jdbc.Driver.connect(Driver.java:92)
>
> Thank you for help.
> Binyu
------------------------------------------------
Subject: error from mv_prune_on_aws.sh
From: binyu.wang at noaa.gov
Time: Thu Nov 19 07:48:57 2020
Hello,
I just tried but the problem persists. So my script is correct, but
there
is a problem with the system? Thanks.
Binyu
On Thu, Nov 19, 2020 at 9:42 AM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Binyu,
> We had same problems with the server yesterday.
> Could you try to run your process again and let me know if it still
> doesn't work.
>
> Tatiana
>
> On Wed Nov 18 21:28:51 2020, binyu.wang at noaa.gov wrote:
> > Hello,
> >
> > I tried to delete the model " Raikoke_31.0p25.p006 " from database
"
> > mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref" using
"mv_prune_on_aws.sh",
> > but
> > I got error (see the attachment pls.)
> >
> > Here is the script I used:
> > <prune_spec>
> > <connection>
> > <host>db_host:3306</host>
> > <database>mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref</database>
> > <user>rds_user</user>
> > <password>rds_pwd</password>
> > </connection>
> > <info_only>true</info_only>
> >
> > <fields>
> > <field name="fcst_valid_beg">
> > <value_range>
> > <start>2019-06-20 06:00:00</start>
> > <end>2019-06-25 18:00:00</end>
> > </value_range>
> > </field>
> > <field name="model">
> > <value_list>
> > <value>Raikoke_31.0p25.p006</value>
> > </value_list>
> > </field>
> > <field name="fcst_var">
> > <value_list>
> > <value>VAFTD</value>
> > </value_list>
> > </field>
> > </fields>
> > </prune_spec>
> >
> >
>
-------------------------------------------------------------------------------------------
> > part of the error log file:
> > SEVERE: Unable to create initial connections of pool.
> > java.sql.SQLNonTransientConnectionException: Communications link
> > failure
> > with primary. No active connection found for master.
> > at
> >
>
org.mariadb.jdbc.internal.failover.AbstractMastersListener.throwFailoverMessage(AbstractMastersListener.java:534)
> > at
> >
>
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.checkInitialConnection(MastersSlavesListener.java:336)
> > at
> >
>
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:173)
> > at
> >
>
org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
> > at
> > org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
> > at
> >
>
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:174)
> > at org.mariadb.jdbc.Driver.connect(Driver.java:92)
> >
> > Thank you for help.
> > Binyu
>
>
>
>
------------------------------------------------
Subject: error from mv_prune_on_aws.sh
From: Tatiana Burek
Time: Thu Nov 19 08:46:33 2020
Binyu,
You have incorrect value in your XML.
Please change
<host>db_host:3306</host>
to
<host>rds_host:3306</host>
Tatiana
On Thu Nov 19 07:48:56 2020, binyu.wang at noaa.gov wrote:
> Hello,
>
> I just tried but the problem persists. So my script is correct, but
> there
> is a problem with the system? Thanks.
>
> Binyu
>
> On Thu, Nov 19, 2020 at 9:42 AM Tatiana Burek via RT
> <met_help at ucar.edu>
> wrote:
>
> > Binyu,
> > We had same problems with the server yesterday.
> > Could you try to run your process again and let me know if it
still
> > doesn't work.
> >
> > Tatiana
> >
> > On Wed Nov 18 21:28:51 2020, binyu.wang at noaa.gov wrote:
> > > Hello,
> > >
> > > I tried to delete the model " Raikoke_31.0p25.p006 " from
database
> > > "
> > > mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref" using
> > > "mv_prune_on_aws.sh",
> > > but
> > > I got error (see the attachment pls.)
> > >
> > > Here is the script I used:
> > > <prune_spec>
> > > <connection>
> > > <host>db_host:3306</host>
> > >
<database>mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref</database>
> > > <user>rds_user</user>
> > > <password>rds_pwd</password>
> > > </connection>
> > > <info_only>true</info_only>
> > >
> > > <fields>
> > > <field name="fcst_valid_beg">
> > > <value_range>
> > > <start>2019-06-20 06:00:00</start>
> > > <end>2019-06-25 18:00:00</end>
> > > </value_range>
> > > </field>
> > > <field name="model">
> > > <value_list>
> > > <value>Raikoke_31.0p25.p006</value>
> > > </value_list>
> > > </field>
> > > <field name="fcst_var">
> > > <value_list>
> > > <value>VAFTD</value>
> > > </value_list>
> > > </field>
> > > </fields>
> > > </prune_spec>
> > >
> > >
> >
-------------------------------------------------------------------------------------------
> > > part of the error log file:
> > > SEVERE: Unable to create initial connections of pool.
> > > java.sql.SQLNonTransientConnectionException: Communications link
> > > failure
> > > with primary. No active connection found for master.
> > > at
> > >
> >
org.mariadb.jdbc.internal.failover.AbstractMastersListener.throwFailoverMessage(AbstractMastersListener.java:534)
> > > at
> > >
> >
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.checkInitialConnection(MastersSlavesListener.java:336)
> > > at
> > >
> >
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:173)
> > > at
> > >
> >
org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
> > > at
> > >
org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
> > > at
> > >
> >
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:174)
> > > at org.mariadb.jdbc.Driver.connect(Driver.java:92)
> > >
> > > Thank you for help.
> > > Binyu
> >
> >
> >
> >
------------------------------------------------
Subject: error from mv_prune_on_aws.sh
From: binyu.wang at noaa.gov
Time: Thu Nov 19 09:55:04 2020
Thank you, Tatiana.
It seems it works now. I copied the log file below. But why do I get "
SyntaxErrorException: (conn=1465924) Unknown column 'model' in 'where
clause' ? I do have this model inside of the database.
BInyu
----------------------------------------------------------------------------------------------------------------------------------
[Binyu.Wang at v71a1 g2g]$ mv_prune_on_aws.sh binyu.wang
prune_db_dpec_file
CALLING: scp prune_db_dpec_file binyu.wang at 205.156.8.85:~
prune_db_dpec_file 100% 669
26.6KB/s 00:00
java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
'model' in
'where clause'
Field model is not in table 'data_file'. Skipping this table
>From table data_file would be deleted 0 records
>From table line_data_cnt would be deleted 0 records
>From table line_data_ctc would be deleted 0 records
>From table line_data_cts would be deleted 0 records
>From tables line_data_eclv and line_data_eclv_pnt would be deleted 0
records
>From table line_data_ecnt would be deleted 18 records
>From table line_data_enscnt would be deleted 0 records
>From table line_data_fho would be deleted 0 records
>From table line_data_grad would be deleted 0 records
>From table line_data_isc would be deleted 0 records
>From tables line_data_mctc and line_data_mctc_cnt would be deleted 0
records
>From table line_data_mcts would be deleted 0 records
>From table line_data_mpr would be deleted 0 records
>From table line_data_nbrcnt would be deleted 0 records
>From table line_data_nbrctc would be deleted 0 records
>From table line_data_nbrcts would be deleted 0 records
>From tables line_data_orank and line_data_orank_ens would be deleted 0
records
>From tables line_data_pct and line_data_pct_thresh would be deleted 0
records
>From tables line_data_phist and line_data_phist_bin would be deleted
378
records
>From tables line_data_pjc and line_data_pjc_thresh would be deleted 0
records
>From tables line_data_prc and line_data_prc_thresh would be deleted 0
records
>From tables line_data_pstd and line_data_pstd_thresh would be deleted
0
records
>From tables line_data_relp and line_data_relp_ens would be deleted 576
records
>From tables line_data_rhist and line_data_rhist_rank would be deleted
594
records
>From table line_data_sal1l2 would be deleted 0 records
>From table line_data_sl1l2 would be deleted 0 records
>From table line_data_ssvar would be deleted 18 records
>From table line_data_val1l2 would be deleted 0 records
>From table line_data_vcnt would be deleted 0 records
>From table line_data_vl1l2 would be deleted 0 records
java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
'fcst_valid_beg' in 'where clause'
Field fcst_valid_beg is not in table 'mode_cts'. Skipping this table
>From table mode_cts would be deleted 0 records
java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
'fcst_valid_beg' in 'where clause'
Field fcst_valid_beg is not in table 'mode_obj_pair'. Skipping this
table
>From table mode_obj_pair would be deleted 0 records
java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
'fcst_valid_beg' in 'where clause'
Field fcst_valid_beg is not in table 'mode_obj_single'. Skipping this
table
>From table mode_obj_single would be deleted 0 records
java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
'fcst_valid_beg' in 'where clause'
Field fcst_valid_beg is not in table 'mtd_2d_obj'. Skipping this table
>From table mtd_2d_obj would be deleted 0 records
java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
'fcst_valid_beg' in 'where clause'
Field fcst_valid_beg is not in table 'mtd_3d_obj_pair'. Skipping this
table
>From table mtd_3d_obj_pair would be deleted 0 records
java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
'fcst_valid_beg' in 'where clause'
Field fcst_valid_beg is not in table 'mtd_3d_obj_single'. Skipping
this
table
>From table mtd_3d_obj_single would be deleted 0 records
Total deleted 1584 records
---- MVPruneDB Done ----
On Thu, Nov 19, 2020 at 10:46 AM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Binyu,
> You have incorrect value in your XML.
> Please change
> <host>db_host:3306</host>
> to
> <host>rds_host:3306</host>
>
> Tatiana
> On Thu Nov 19 07:48:56 2020, binyu.wang at noaa.gov wrote:
> > Hello,
> >
> > I just tried but the problem persists. So my script is correct,
but
> > there
> > is a problem with the system? Thanks.
> >
> > Binyu
> >
> > On Thu, Nov 19, 2020 at 9:42 AM Tatiana Burek via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > Binyu,
> > > We had same problems with the server yesterday.
> > > Could you try to run your process again and let me know if it
still
> > > doesn't work.
> > >
> > > Tatiana
> > >
> > > On Wed Nov 18 21:28:51 2020, binyu.wang at noaa.gov wrote:
> > > > Hello,
> > > >
> > > > I tried to delete the model " Raikoke_31.0p25.p006 " from
database
> > > > "
> > > > mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref" using
> > > > "mv_prune_on_aws.sh",
> > > > but
> > > > I got error (see the attachment pls.)
> > > >
> > > > Here is the script I used:
> > > > <prune_spec>
> > > > <connection>
> > > > <host>db_host:3306</host>
> > > >
<database>mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref</database>
> > > > <user>rds_user</user>
> > > > <password>rds_pwd</password>
> > > > </connection>
> > > > <info_only>true</info_only>
> > > >
> > > > <fields>
> > > > <field name="fcst_valid_beg">
> > > > <value_range>
> > > > <start>2019-06-20 06:00:00</start>
> > > > <end>2019-06-25 18:00:00</end>
> > > > </value_range>
> > > > </field>
> > > > <field name="model">
> > > > <value_list>
> > > > <value>Raikoke_31.0p25.p006</value>
> > > > </value_list>
> > > > </field>
> > > > <field name="fcst_var">
> > > > <value_list>
> > > > <value>VAFTD</value>
> > > > </value_list>
> > > > </field>
> > > > </fields>
> > > > </prune_spec>
> > > >
> > > >
> > >
>
-------------------------------------------------------------------------------------------
> > > > part of the error log file:
> > > > SEVERE: Unable to create initial connections of pool.
> > > > java.sql.SQLNonTransientConnectionException: Communications
link
> > > > failure
> > > > with primary. No active connection found for master.
> > > > at
> > > >
> > >
>
org.mariadb.jdbc.internal.failover.AbstractMastersListener.throwFailoverMessage(AbstractMastersListener.java:534)
> > > > at
> > > >
> > >
>
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.checkInitialConnection(MastersSlavesListener.java:336)
> > > > at
> > > >
> > >
>
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:173)
> > > > at
> > > >
> > >
>
org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
> > > > at
> > > >
org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
> > > > at
> > > >
> > >
>
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:174)
> > > > at org.mariadb.jdbc.Driver.connect(Driver.java:92)
> > > >
> > > > Thank you for help.
> > > > Binyu
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: error from mv_prune_on_aws.sh
From: Tatiana Burek
Time: Thu Nov 19 13:14:06 2020
Binyu,
Please ignore this error.
METviewer looks for the records in all tables and some of them like
'data_file' doesn't have all columns. In this case METviewer ignores
these tables and prints an error.
Tatiana
On Thu Nov 19 09:55:04 2020, binyu.wang at noaa.gov wrote:
> Thank you, Tatiana.
> It seems it works now. I copied the log file below. But why do I get
"
> SyntaxErrorException: (conn=1465924) Unknown column 'model' in
'where
> clause' ? I do have this model inside of the database.
>
> BInyu
>
----------------------------------------------------------------------------------------------------------------------------------
>
> [Binyu.Wang at v71a1 g2g]$ mv_prune_on_aws.sh binyu.wang
> prune_db_dpec_file
> CALLING: scp prune_db_dpec_file binyu.wang at 205.156.8.85:~
> prune_db_dpec_file 100% 669
> 26.6KB/s 00:00
> java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> 'model' in
> 'where clause'
> Field model is not in table 'data_file'. Skipping this table
> From table data_file would be deleted 0 records
> From table line_data_cnt would be deleted 0 records
> From table line_data_ctc would be deleted 0 records
> From table line_data_cts would be deleted 0 records
> From tables line_data_eclv and line_data_eclv_pnt would be deleted 0
> records
> From table line_data_ecnt would be deleted 18 records
> From table line_data_enscnt would be deleted 0 records
> From table line_data_fho would be deleted 0 records
> From table line_data_grad would be deleted 0 records
> From table line_data_isc would be deleted 0 records
> From tables line_data_mctc and line_data_mctc_cnt would be deleted 0
> records
> From table line_data_mcts would be deleted 0 records
> From table line_data_mpr would be deleted 0 records
> From table line_data_nbrcnt would be deleted 0 records
> From table line_data_nbrctc would be deleted 0 records
> From table line_data_nbrcts would be deleted 0 records
> From tables line_data_orank and line_data_orank_ens would be deleted
0
> records
> From tables line_data_pct and line_data_pct_thresh would be deleted
0
> records
> From tables line_data_phist and line_data_phist_bin would be deleted
> 378
> records
> From tables line_data_pjc and line_data_pjc_thresh would be deleted
0
> records
> From tables line_data_prc and line_data_prc_thresh would be deleted
0
> records
> From tables line_data_pstd and line_data_pstd_thresh would be
deleted
> 0
> records
> From tables line_data_relp and line_data_relp_ens would be deleted
576
> records
> From tables line_data_rhist and line_data_rhist_rank would be
deleted
> 594
> records
> From table line_data_sal1l2 would be deleted 0 records
> From table line_data_sl1l2 would be deleted 0 records
> From table line_data_ssvar would be deleted 18 records
> From table line_data_val1l2 would be deleted 0 records
> From table line_data_vcnt would be deleted 0 records
> From table line_data_vl1l2 would be deleted 0 records
> java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> 'fcst_valid_beg' in 'where clause'
> Field fcst_valid_beg is not in table 'mode_cts'. Skipping this table
> From table mode_cts would be deleted 0 records
> java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> 'fcst_valid_beg' in 'where clause'
> Field fcst_valid_beg is not in table 'mode_obj_pair'. Skipping this
> table
> From table mode_obj_pair would be deleted 0 records
> java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> 'fcst_valid_beg' in 'where clause'
> Field fcst_valid_beg is not in table 'mode_obj_single'. Skipping
this
> table
> From table mode_obj_single would be deleted 0 records
> java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> 'fcst_valid_beg' in 'where clause'
> Field fcst_valid_beg is not in table 'mtd_2d_obj'. Skipping this
table
> From table mtd_2d_obj would be deleted 0 records
> java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> 'fcst_valid_beg' in 'where clause'
> Field fcst_valid_beg is not in table 'mtd_3d_obj_pair'. Skipping
this
> table
> From table mtd_3d_obj_pair would be deleted 0 records
> java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> 'fcst_valid_beg' in 'where clause'
> Field fcst_valid_beg is not in table 'mtd_3d_obj_single'. Skipping
> this
> table
> From table mtd_3d_obj_single would be deleted 0 records
> Total deleted 1584 records
> ---- MVPruneDB Done ----
>
> On Thu, Nov 19, 2020 at 10:46 AM Tatiana Burek via RT
> <met_help at ucar.edu>
> wrote:
>
> > Binyu,
> > You have incorrect value in your XML.
> > Please change
> > <host>db_host:3306</host>
> > to
> > <host>rds_host:3306</host>
> >
> > Tatiana
> > On Thu Nov 19 07:48:56 2020, binyu.wang at noaa.gov wrote:
> > > Hello,
> > >
> > > I just tried but the problem persists. So my script is correct,
but
> > > there
> > > is a problem with the system? Thanks.
> > >
> > > Binyu
> > >
> > > On Thu, Nov 19, 2020 at 9:42 AM Tatiana Burek via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > > Binyu,
> > > > We had same problems with the server yesterday.
> > > > Could you try to run your process again and let me know if it
> > > > still
> > > > doesn't work.
> > > >
> > > > Tatiana
> > > >
> > > > On Wed Nov 18 21:28:51 2020, binyu.wang at noaa.gov wrote:
> > > > > Hello,
> > > > >
> > > > > I tried to delete the model " Raikoke_31.0p25.p006 " from
> > > > > database
> > > > > "
> > > > > mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref" using
> > > > > "mv_prune_on_aws.sh",
> > > > > but
> > > > > I got error (see the attachment pls.)
> > > > >
> > > > > Here is the script I used:
> > > > > <prune_spec>
> > > > > <connection>
> > > > > <host>db_host:3306</host>
> > > > >
<database>mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref</database>
> > > > > <user>rds_user</user>
> > > > > <password>rds_pwd</password>
> > > > > </connection>
> > > > > <info_only>true</info_only>
> > > > >
> > > > > <fields>
> > > > > <field name="fcst_valid_beg">
> > > > > <value_range>
> > > > > <start>2019-06-20 06:00:00</start>
> > > > > <end>2019-06-25 18:00:00</end>
> > > > > </value_range>
> > > > > </field>
> > > > > <field name="model">
> > > > > <value_list>
> > > > > <value>Raikoke_31.0p25.p006</value>
> > > > > </value_list>
> > > > > </field>
> > > > > <field name="fcst_var">
> > > > > <value_list>
> > > > > <value>VAFTD</value>
> > > > > </value_list>
> > > > > </field>
> > > > > </fields>
> > > > > </prune_spec>
> > > > >
> > > > >
> > > >
> >
-------------------------------------------------------------------------------------------
> > > > > part of the error log file:
> > > > > SEVERE: Unable to create initial connections of pool.
> > > > > java.sql.SQLNonTransientConnectionException: Communications
> > > > > link
> > > > > failure
> > > > > with primary. No active connection found for master.
> > > > > at
> > > > >
> > > >
> >
org.mariadb.jdbc.internal.failover.AbstractMastersListener.throwFailoverMessage(AbstractMastersListener.java:534)
> > > > > at
> > > > >
> > > >
> >
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.checkInitialConnection(MastersSlavesListener.java:336)
> > > > > at
> > > > >
> > > >
> >
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:173)
> > > > > at
> > > > >
> > > >
> >
org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
> > > > > at
> > > > >
org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
> > > > > at
> > > > >
> > > >
> >
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:174)
> > > > > at org.mariadb.jdbc.Driver.connect(Driver.java:92)
> > > > >
> > > > > Thank you for help.
> > > > > Binyu
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> >
------------------------------------------------
Subject: error from mv_prune_on_aws.sh
From: binyu.wang at noaa.gov
Time: Thu Nov 19 20:22:23 2020
Ok, after I changed "true" to "false" and reran "
"mv_prune_on_aws.sh", it
seems the related tables are deleted. However, the model is still
there
although it can not be used to make plot (see the attachment). Is
there any
way to delete the model completely? Thank you.
Binyu
On Thu, Nov 19, 2020 at 3:14 PM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Binyu,
> Please ignore this error.
> METviewer looks for the records in all tables and some of them like
> 'data_file' doesn't have all columns. In this case METviewer ignores
these
> tables and prints an error.
>
> Tatiana
>
> On Thu Nov 19 09:55:04 2020, binyu.wang at noaa.gov wrote:
> > Thank you, Tatiana.
> > It seems it works now. I copied the log file below. But why do I
get "
> > SyntaxErrorException: (conn=1465924) Unknown column 'model' in
'where
> > clause' ? I do have this model inside of the database.
> >
> > BInyu
> >
>
----------------------------------------------------------------------------------------------------------------------------------
> >
> > [Binyu.Wang at v71a1 g2g]$ mv_prune_on_aws.sh binyu.wang
> > prune_db_dpec_file
> > CALLING: scp prune_db_dpec_file binyu.wang at 205.156.8.85:~
> > prune_db_dpec_file 100%
669
> > 26.6KB/s 00:00
> > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > 'model' in
> > 'where clause'
> > Field model is not in table 'data_file'. Skipping this table
> > From table data_file would be deleted 0 records
> > From table line_data_cnt would be deleted 0 records
> > From table line_data_ctc would be deleted 0 records
> > From table line_data_cts would be deleted 0 records
> > From tables line_data_eclv and line_data_eclv_pnt would be deleted
0
> > records
> > From table line_data_ecnt would be deleted 18 records
> > From table line_data_enscnt would be deleted 0 records
> > From table line_data_fho would be deleted 0 records
> > From table line_data_grad would be deleted 0 records
> > From table line_data_isc would be deleted 0 records
> > From tables line_data_mctc and line_data_mctc_cnt would be deleted
0
> > records
> > From table line_data_mcts would be deleted 0 records
> > From table line_data_mpr would be deleted 0 records
> > From table line_data_nbrcnt would be deleted 0 records
> > From table line_data_nbrctc would be deleted 0 records
> > From table line_data_nbrcts would be deleted 0 records
> > From tables line_data_orank and line_data_orank_ens would be
deleted 0
> > records
> > From tables line_data_pct and line_data_pct_thresh would be
deleted 0
> > records
> > From tables line_data_phist and line_data_phist_bin would be
deleted
> > 378
> > records
> > From tables line_data_pjc and line_data_pjc_thresh would be
deleted 0
> > records
> > From tables line_data_prc and line_data_prc_thresh would be
deleted 0
> > records
> > From tables line_data_pstd and line_data_pstd_thresh would be
deleted
> > 0
> > records
> > From tables line_data_relp and line_data_relp_ens would be deleted
576
> > records
> > From tables line_data_rhist and line_data_rhist_rank would be
deleted
> > 594
> > records
> > From table line_data_sal1l2 would be deleted 0 records
> > From table line_data_sl1l2 would be deleted 0 records
> > From table line_data_ssvar would be deleted 18 records
> > From table line_data_val1l2 would be deleted 0 records
> > From table line_data_vcnt would be deleted 0 records
> > From table line_data_vl1l2 would be deleted 0 records
> > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > 'fcst_valid_beg' in 'where clause'
> > Field fcst_valid_beg is not in table 'mode_cts'. Skipping this
table
> > From table mode_cts would be deleted 0 records
> > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > 'fcst_valid_beg' in 'where clause'
> > Field fcst_valid_beg is not in table 'mode_obj_pair'. Skipping
this
> > table
> > From table mode_obj_pair would be deleted 0 records
> > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > 'fcst_valid_beg' in 'where clause'
> > Field fcst_valid_beg is not in table 'mode_obj_single'. Skipping
this
> > table
> > From table mode_obj_single would be deleted 0 records
> > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > 'fcst_valid_beg' in 'where clause'
> > Field fcst_valid_beg is not in table 'mtd_2d_obj'. Skipping this
table
> > From table mtd_2d_obj would be deleted 0 records
> > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > 'fcst_valid_beg' in 'where clause'
> > Field fcst_valid_beg is not in table 'mtd_3d_obj_pair'. Skipping
this
> > table
> > From table mtd_3d_obj_pair would be deleted 0 records
> > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > 'fcst_valid_beg' in 'where clause'
> > Field fcst_valid_beg is not in table 'mtd_3d_obj_single'. Skipping
> > this
> > table
> > From table mtd_3d_obj_single would be deleted 0 records
> > Total deleted 1584 records
> > ---- MVPruneDB Done ----
> >
> > On Thu, Nov 19, 2020 at 10:46 AM Tatiana Burek via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > Binyu,
> > > You have incorrect value in your XML.
> > > Please change
> > > <host>db_host:3306</host>
> > > to
> > > <host>rds_host:3306</host>
> > >
> > > Tatiana
> > > On Thu Nov 19 07:48:56 2020, binyu.wang at noaa.gov wrote:
> > > > Hello,
> > > >
> > > > I just tried but the problem persists. So my script is
correct, but
> > > > there
> > > > is a problem with the system? Thanks.
> > > >
> > > > Binyu
> > > >
> > > > On Thu, Nov 19, 2020 at 9:42 AM Tatiana Burek via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Binyu,
> > > > > We had same problems with the server yesterday.
> > > > > Could you try to run your process again and let me know if
it
> > > > > still
> > > > > doesn't work.
> > > > >
> > > > > Tatiana
> > > > >
> > > > > On Wed Nov 18 21:28:51 2020, binyu.wang at noaa.gov wrote:
> > > > > > Hello,
> > > > > >
> > > > > > I tried to delete the model " Raikoke_31.0p25.p006 " from
> > > > > > database
> > > > > > "
> > > > > > mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref" using
> > > > > > "mv_prune_on_aws.sh",
> > > > > > but
> > > > > > I got error (see the attachment pls.)
> > > > > >
> > > > > > Here is the script I used:
> > > > > > <prune_spec>
> > > > > > <connection>
> > > > > > <host>db_host:3306</host>
> > > > > >
> <database>mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref</database>
> > > > > > <user>rds_user</user>
> > > > > > <password>rds_pwd</password>
> > > > > > </connection>
> > > > > > <info_only>true</info_only>
> > > > > >
> > > > > > <fields>
> > > > > > <field name="fcst_valid_beg">
> > > > > > <value_range>
> > > > > > <start>2019-06-20 06:00:00</start>
> > > > > > <end>2019-06-25 18:00:00</end>
> > > > > > </value_range>
> > > > > > </field>
> > > > > > <field name="model">
> > > > > > <value_list>
> > > > > > <value>Raikoke_31.0p25.p006</value>
> > > > > > </value_list>
> > > > > > </field>
> > > > > > <field name="fcst_var">
> > > > > > <value_list>
> > > > > > <value>VAFTD</value>
> > > > > > </value_list>
> > > > > > </field>
> > > > > > </fields>
> > > > > > </prune_spec>
> > > > > >
> > > > > >
> > > > >
> > >
>
-------------------------------------------------------------------------------------------
> > > > > > part of the error log file:
> > > > > > SEVERE: Unable to create initial connections of pool.
> > > > > > java.sql.SQLNonTransientConnectionException:
Communications
> > > > > > link
> > > > > > failure
> > > > > > with primary. No active connection found for master.
> > > > > > at
> > > > > >
> > > > >
> > >
>
org.mariadb.jdbc.internal.failover.AbstractMastersListener.throwFailoverMessage(AbstractMastersListener.java:534)
> > > > > > at
> > > > > >
> > > > >
> > >
>
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.checkInitialConnection(MastersSlavesListener.java:336)
> > > > > > at
> > > > > >
> > > > >
> > >
>
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:173)
> > > > > > at
> > > > > >
> > > > >
> > >
>
org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
> > > > > > at
> > > > > >
> org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
> > > > > > at
> > > > > >
> > > > >
> > >
>
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:174)
> > > > > > at org.mariadb.jdbc.Driver.connect(Driver.java:92)
> > > > > >
> > > > > > Thank you for help.
> > > > > > Binyu
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: error from mv_prune_on_aws.sh
From: Tatiana Burek
Time: Fri Nov 20 08:15:49 2020
Binyu
prune script only deletes the statistical data but not the data's
header that includes the model.
In other words, METviewer doesn't check if any data that has a
particular header left in the database.
We don't have this feature because of the the large size of some
databases and the length of time this operation would take to
complete.
In your case, all the data for 'Raikoke_31.0p25.p006' database was
deleted and if you select this database you will mot be able to make a
plot.
Tatiana
On Thu Nov 19 20:22:23 2020, binyu.wang at noaa.gov wrote:
> Ok, after I changed "true" to "false" and reran "
> "mv_prune_on_aws.sh", it
> seems the related tables are deleted. However, the model is still
> there
> although it can not be used to make plot (see the attachment). Is
> there any
> way to delete the model completely? Thank you.
>
> Binyu
>
> On Thu, Nov 19, 2020 at 3:14 PM Tatiana Burek via RT
> <met_help at ucar.edu>
> wrote:
>
> > Binyu,
> > Please ignore this error.
> > METviewer looks for the records in all tables and some of them
like
> > 'data_file' doesn't have all columns. In this case METviewer
ignores
> > these
> > tables and prints an error.
> >
> > Tatiana
> >
> > On Thu Nov 19 09:55:04 2020, binyu.wang at noaa.gov wrote:
> > > Thank you, Tatiana.
> > > It seems it works now. I copied the log file below. But why do I
> > > get "
> > > SyntaxErrorException: (conn=1465924) Unknown column 'model' in
> > > 'where
> > > clause' ? I do have this model inside of the database.
> > >
> > > BInyu
> > >
> >
----------------------------------------------------------------------------------------------------------------------------------
> > >
> > > [Binyu.Wang at v71a1 g2g]$ mv_prune_on_aws.sh binyu.wang
> > > prune_db_dpec_file
> > > CALLING: scp prune_db_dpec_file binyu.wang at 205.156.8.85:~
> > > prune_db_dpec_file 100%
> > > 669
> > > 26.6KB/s 00:00
> > > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > > 'model' in
> > > 'where clause'
> > > Field model is not in table 'data_file'. Skipping this table
> > > From table data_file would be deleted 0 records
> > > From table line_data_cnt would be deleted 0 records
> > > From table line_data_ctc would be deleted 0 records
> > > From table line_data_cts would be deleted 0 records
> > > From tables line_data_eclv and line_data_eclv_pnt would be
deleted
> > > 0
> > > records
> > > From table line_data_ecnt would be deleted 18 records
> > > From table line_data_enscnt would be deleted 0 records
> > > From table line_data_fho would be deleted 0 records
> > > From table line_data_grad would be deleted 0 records
> > > From table line_data_isc would be deleted 0 records
> > > From tables line_data_mctc and line_data_mctc_cnt would be
deleted
> > > 0
> > > records
> > > From table line_data_mcts would be deleted 0 records
> > > From table line_data_mpr would be deleted 0 records
> > > From table line_data_nbrcnt would be deleted 0 records
> > > From table line_data_nbrctc would be deleted 0 records
> > > From table line_data_nbrcts would be deleted 0 records
> > > From tables line_data_orank and line_data_orank_ens would be
> > > deleted 0
> > > records
> > > From tables line_data_pct and line_data_pct_thresh would be
deleted
> > > 0
> > > records
> > > From tables line_data_phist and line_data_phist_bin would be
> > > deleted
> > > 378
> > > records
> > > From tables line_data_pjc and line_data_pjc_thresh would be
deleted
> > > 0
> > > records
> > > From tables line_data_prc and line_data_prc_thresh would be
deleted
> > > 0
> > > records
> > > From tables line_data_pstd and line_data_pstd_thresh would be
> > > deleted
> > > 0
> > > records
> > > From tables line_data_relp and line_data_relp_ens would be
deleted
> > > 576
> > > records
> > > From tables line_data_rhist and line_data_rhist_rank would be
> > > deleted
> > > 594
> > > records
> > > From table line_data_sal1l2 would be deleted 0 records
> > > From table line_data_sl1l2 would be deleted 0 records
> > > From table line_data_ssvar would be deleted 18 records
> > > From table line_data_val1l2 would be deleted 0 records
> > > From table line_data_vcnt would be deleted 0 records
> > > From table line_data_vl1l2 would be deleted 0 records
> > > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > > 'fcst_valid_beg' in 'where clause'
> > > Field fcst_valid_beg is not in table 'mode_cts'. Skipping this
> > > table
> > > From table mode_cts would be deleted 0 records
> > > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > > 'fcst_valid_beg' in 'where clause'
> > > Field fcst_valid_beg is not in table 'mode_obj_pair'. Skipping
this
> > > table
> > > From table mode_obj_pair would be deleted 0 records
> > > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > > 'fcst_valid_beg' in 'where clause'
> > > Field fcst_valid_beg is not in table 'mode_obj_single'. Skipping
> > > this
> > > table
> > > From table mode_obj_single would be deleted 0 records
> > > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > > 'fcst_valid_beg' in 'where clause'
> > > Field fcst_valid_beg is not in table 'mtd_2d_obj'. Skipping this
> > > table
> > > From table mtd_2d_obj would be deleted 0 records
> > > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > > 'fcst_valid_beg' in 'where clause'
> > > Field fcst_valid_beg is not in table 'mtd_3d_obj_pair'. Skipping
> > > this
> > > table
> > > From table mtd_3d_obj_pair would be deleted 0 records
> > > java.sql.SQLSyntaxErrorException: (conn=1465924) Unknown column
> > > 'fcst_valid_beg' in 'where clause'
> > > Field fcst_valid_beg is not in table 'mtd_3d_obj_single'.
Skipping
> > > this
> > > table
> > > From table mtd_3d_obj_single would be deleted 0 records
> > > Total deleted 1584 records
> > > ---- MVPruneDB Done ----
> > >
> > > On Thu, Nov 19, 2020 at 10:46 AM Tatiana Burek via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > > Binyu,
> > > > You have incorrect value in your XML.
> > > > Please change
> > > > <host>db_host:3306</host>
> > > > to
> > > > <host>rds_host:3306</host>
> > > >
> > > > Tatiana
> > > > On Thu Nov 19 07:48:56 2020, binyu.wang at noaa.gov wrote:
> > > > > Hello,
> > > > >
> > > > > I just tried but the problem persists. So my script is
correct,
> > > > > but
> > > > > there
> > > > > is a problem with the system? Thanks.
> > > > >
> > > > > Binyu
> > > > >
> > > > > On Thu, Nov 19, 2020 at 9:42 AM Tatiana Burek via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Binyu,
> > > > > > We had same problems with the server yesterday.
> > > > > > Could you try to run your process again and let me know if
it
> > > > > > still
> > > > > > doesn't work.
> > > > > >
> > > > > > Tatiana
> > > > > >
> > > > > > On Wed Nov 18 21:28:51 2020, binyu.wang at noaa.gov wrote:
> > > > > > > Hello,
> > > > > > >
> > > > > > > I tried to delete the model " Raikoke_31.0p25.p006 "
from
> > > > > > > database
> > > > > > > "
> > > > > > > mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref" using
> > > > > > > "mv_prune_on_aws.sh",
> > > > > > > but
> > > > > > > I got error (see the attachment pls.)
> > > > > > >
> > > > > > > Here is the script I used:
> > > > > > > <prune_spec>
> > > > > > > <connection>
> > > > > > > <host>db_host:3306</host>
> > > > > > >
> > <database>mv_g2g_met_verf_Raikoke31_0p25_ths0p1_ref</database>
> > > > > > > <user>rds_user</user>
> > > > > > > <password>rds_pwd</password>
> > > > > > > </connection>
> > > > > > > <info_only>true</info_only>
> > > > > > >
> > > > > > > <fields>
> > > > > > > <field name="fcst_valid_beg">
> > > > > > > <value_range>
> > > > > > > <start>2019-06-20 06:00:00</start>
> > > > > > > <end>2019-06-25 18:00:00</end>
> > > > > > > </value_range>
> > > > > > > </field>
> > > > > > > <field name="model">
> > > > > > > <value_list>
> > > > > > > <value>Raikoke_31.0p25.p006</value>
> > > > > > > </value_list>
> > > > > > > </field>
> > > > > > > <field name="fcst_var">
> > > > > > > <value_list>
> > > > > > > <value>VAFTD</value>
> > > > > > > </value_list>
> > > > > > > </field>
> > > > > > > </fields>
> > > > > > > </prune_spec>
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> >
-------------------------------------------------------------------------------------------
> > > > > > > part of the error log file:
> > > > > > > SEVERE: Unable to create initial connections of pool.
> > > > > > > java.sql.SQLNonTransientConnectionException:
Communications
> > > > > > > link
> > > > > > > failure
> > > > > > > with primary. No active connection found for master.
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
org.mariadb.jdbc.internal.failover.AbstractMastersListener.throwFailoverMessage(AbstractMastersListener.java:534)
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.checkInitialConnection(MastersSlavesListener.java:336)
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:173)
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
> > > > > > > at
> > > > > > >
> > org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:174)
> > > > > > > at org.mariadb.jdbc.Driver.connect(Driver.java:92)
> > > > > > >
> > > > > > > Thank you for help.
> > > > > > > Binyu
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> >
------------------------------------------------
More information about the Met_help
mailing list