[Met_help] [rt.rap.ucar.edu #92346] History for upload database can not being seen in AWS Metviewer
Tatiana Burek via RT
met_help at ucar.edu
Fri Sep 27 09:20:40 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
To Whom It May Concern:
I upload a new database to AWS but can not find it in MetViewer. The load
xml and AWS database inquiry is shown below
The scp files seems working,
Thank you for your help.
Ho-Chun
<load_spec>
<connection>
<host>
metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-1.rds.amazonaws.com:3306
</host>
<database>mv_g2g_met_verf_aod</database>
<user>rds_user</user>
<password>rds_pwd</password>
<management_system>aurora</management_system>
</connection>
<met_version>V8.1.2</met_version>
<verbose>true</verbose>
<insert_size>1</insert_size>
<mode_header_db_check>true</mode_header_db_check>
<stat_header_db_check>true</stat_header_db_check>
<drop_indexes>false</drop_indexes>
<apply_indexes>true</apply_indexes>
<load_stat>true</load_stat>
<load_mode>true</load_mode>
<load_mpr>true</load_mpr>
<load_orank>true</load_orank>
<force_dup_file>false</force_dup_file>
<group>NCEP_hcHuang</group>
<description>Loading mv_g2g_met_verf_aod</description>
<folder_tmpl>/base_dir/{type}</folder_tmpl>
<load_val>
<field name="type">
<val>stat</val>
</field>
</load_val>
<load_xml>true</load_xml>
<load_note></load_note>
</load_spec>
>mv_db_size_on_aws.sh ho-chun.huang
:::::::::::::::::::::::::::::::::::::::::
:::::::::::::::::::::::::::::::::::::::::
mv_agibbs_nam_hi_dng_FAA annette.gibbs 2.28
mv_agibbs_nam_pr_dng_FAA annette.gibbs 2.28
emc_cmaq_g2o perry.shafran 2.02
emc_cmaq_g2o2 perry.shafran 2.02
mv_vh venita.hagerty 2.02
testvh4 venita.hagerty 2.02
testvh5 venita.hagerty 2.02
mv_g2g_met_verf_aod ho-chun.huang 1.97
mats_metadata kenneth.sperow 1.28
_______________________________
TOTAL_SIZE_GB
203.36
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Tara Jensen
Time: Thu Sep 26 13:24:01 2019
Ho-Chun,
I see you are trying to view a new database up on AWS. After adding a
new
database to METviewer, you will need to click on Reload databases
above the
plotting area. If you still don't see it, consider clearing your
cache and
hitting the refresh button. If you are using METexpress, you may not
have
access to it until tomorrow. I'm not sure how often they are running
metadata, which is what is needed to see a new database.
[image: image.png]
Let us know if you have tried these things and still don't see it. We
can
pass it over to Tatiana at that point.
Cheers, Tara
On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> Thu Sep 26 12:40:26 2019: Request 92346 was acted upon.
> Transaction: Ticket created by ho-chun.huang at noaa.gov
> Queue: met_help
> Subject: upload database can not being seen in AWS Metviewer
> Owner: Nobody
> Requestors: ho-chun.huang at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
>
>
> To Whom It May Concern:
>
> I upload a new database to AWS but can not find it in MetViewer.
The load
> xml and AWS database inquiry is shown below
>
> The scp files seems working,
>
> Thank you for your help.
>
> Ho-Chun
>
> <load_spec>
> <connection>
> <host>
> metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
1.rds.amazonaws.com:3306
> </host>
> <database>mv_g2g_met_verf_aod</database>
> <user>rds_user</user>
> <password>rds_pwd</password>
> <management_system>aurora</management_system>
> </connection>
>
> <met_version>V8.1.2</met_version>
>
> <verbose>true</verbose>
> <insert_size>1</insert_size>
> <mode_header_db_check>true</mode_header_db_check>
> <stat_header_db_check>true</stat_header_db_check>
> <drop_indexes>false</drop_indexes>
> <apply_indexes>true</apply_indexes>
> <load_stat>true</load_stat>
> <load_mode>true</load_mode>
> <load_mpr>true</load_mpr>
> <load_orank>true</load_orank>
> <force_dup_file>false</force_dup_file>
> <group>NCEP_hcHuang</group>
> <description>Loading mv_g2g_met_verf_aod</description>
> <folder_tmpl>/base_dir/{type}</folder_tmpl>
> <load_val>
> <field name="type">
> <val>stat</val>
> </field>
> </load_val>
> <load_xml>true</load_xml>
> <load_note></load_note>
> </load_spec>
>
>
>
> >mv_db_size_on_aws.sh ho-chun.huang
> :::::::::::::::::::::::::::::::::::::::::
> :::::::::::::::::::::::::::::::::::::::::
> mv_agibbs_nam_hi_dng_FAA annette.gibbs 2.28
> mv_agibbs_nam_pr_dng_FAA annette.gibbs 2.28
> emc_cmaq_g2o perry.shafran 2.02
> emc_cmaq_g2o2 perry.shafran 2.02
> mv_vh venita.hagerty 2.02
> testvh4 venita.hagerty 2.02
> testvh5 venita.hagerty 2.02
> mv_g2g_met_verf_aod ho-chun.huang 1.97
> mats_metadata kenneth.sperow 1.28
> _______________________________
> TOTAL_SIZE_GB
> 203.36
>
>
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tara Jensen
Project Manager II
NCAR RAL and DTC
PO Box 3000, Boulder, Colorado 80307 USA
+1 303-497-8479 jensen at ucar.edu
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Ho-Chun Huang - NOAA Affiliate
Time: Thu Sep 26 13:49:57 2019
Hi, Tara:
I tried
(1) hit reload database
(2) refresh button
(3) clear all web browser data
(4) close the web browser and open it again
None of it works.
Can you get on the AWS MEtviewer and be able to see
mv_g2g_met_verf_aod under NCEP_hcHuang?
Ho-Chun
On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT <met_help at ucar.edu>
wrote:
> Ho-Chun,
>
> I see you are trying to view a new database up on AWS. After adding
a new
> database to METviewer, you will need to click on Reload databases
above the
> plotting area. If you still don't see it, consider clearing your
cache and
> hitting the refresh button. If you are using METexpress, you may
not have
> access to it until tomorrow. I'm not sure how often they are
running
> metadata, which is what is needed to see a new database.
>
> [image: image.png]
> Let us know if you have tried these things and still don't see it.
We can
> pass it over to Tatiana at that point.
> Cheers, Tara
>
> On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Sep 26 12:40:26 2019: Request 92346 was acted upon.
> > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > Queue: met_help
> > Subject: upload database can not being seen in AWS Metviewer
> > Owner: Nobody
> > Requestors: ho-chun.huang at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
> >
> >
> > To Whom It May Concern:
> >
> > I upload a new database to AWS but can not find it in MetViewer.
The
> load
> > xml and AWS database inquiry is shown below
> >
> > The scp files seems working,
> >
> > Thank you for your help.
> >
> > Ho-Chun
> >
> > <load_spec>
> > <connection>
> > <host>
> >
> metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
1.rds.amazonaws.com:3306
> > </host>
> > <database>mv_g2g_met_verf_aod</database>
> > <user>rds_user</user>
> > <password>rds_pwd</password>
> > <management_system>aurora</management_system>
> > </connection>
> >
> > <met_version>V8.1.2</met_version>
> >
> > <verbose>true</verbose>
> > <insert_size>1</insert_size>
> > <mode_header_db_check>true</mode_header_db_check>
> > <stat_header_db_check>true</stat_header_db_check>
> > <drop_indexes>false</drop_indexes>
> > <apply_indexes>true</apply_indexes>
> > <load_stat>true</load_stat>
> > <load_mode>true</load_mode>
> > <load_mpr>true</load_mpr>
> > <load_orank>true</load_orank>
> > <force_dup_file>false</force_dup_file>
> > <group>NCEP_hcHuang</group>
> > <description>Loading mv_g2g_met_verf_aod</description>
> > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > <load_val>
> > <field name="type">
> > <val>stat</val>
> > </field>
> > </load_val>
> > <load_xml>true</load_xml>
> > <load_note></load_note>
> > </load_spec>
> >
> >
> >
> > >mv_db_size_on_aws.sh ho-chun.huang
> > :::::::::::::::::::::::::::::::::::::::::
> > :::::::::::::::::::::::::::::::::::::::::
> > mv_agibbs_nam_hi_dng_FAA annette.gibbs 2.28
> > mv_agibbs_nam_pr_dng_FAA annette.gibbs 2.28
> > emc_cmaq_g2o perry.shafran 2.02
> > emc_cmaq_g2o2 perry.shafran 2.02
> > mv_vh venita.hagerty 2.02
> > testvh4 venita.hagerty 2.02
> > testvh5 venita.hagerty 2.02
> > mv_g2g_met_verf_aod ho-chun.huang 1.97
> > mats_metadata kenneth.sperow 1.28
> > _______________________________
> > TOTAL_SIZE_GB
> > 203.36
> >
> >
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Tara Jensen
> Project Manager II
> NCAR RAL and DTC
> PO Box 3000, Boulder, Colorado 80307 USA
> +1 303-497-8479 jensen at ucar.edu
>
>
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Tatiana Burek
Time: Thu Sep 26 14:08:22 2019
Ho-Chun
mv_g2g_met_verf_aod database exists on AWS but it is empty.
Do you have a copy of the log file from the data loading?
Tatiana
On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov wrote:
> Hi, Tara:
>
> I tried
>
> (1) hit reload database
> (2) refresh button
> (3) clear all web browser data
> (4) close the web browser and open it again
>
> None of it works.
>
>
> Can you get on the AWS MEtviewer and be able to see
>
> mv_g2g_met_verf_aod under NCEP_hcHuang?
>
> Ho-Chun
>
>
> On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT
<met_help at ucar.edu>
> wrote:
>
> > Ho-Chun,
> >
> > I see you are trying to view a new database up on AWS. After
adding a
> > new
> > database to METviewer, you will need to click on Reload databases
> > above the
> > plotting area. If you still don't see it, consider clearing your
> > cache and
> > hitting the refresh button. If you are using METexpress, you may
not
> > have
> > access to it until tomorrow. I'm not sure how often they are
running
> > metadata, which is what is needed to see a new database.
> >
> > [image: image.png]
> > Let us know if you have tried these things and still don't see it.
> > We can
> > pass it over to Tatiana at that point.
> > Cheers, Tara
> >
> > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA Affiliate
via
> > RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Thu Sep 26 12:40:26 2019: Request 92346 was acted upon.
> > > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > > Queue: met_help
> > > Subject: upload database can not being seen in AWS
Metviewer
> > > Owner: Nobody
> > > Requestors: ho-chun.huang at noaa.gov
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
> > >
> > >
> > > To Whom It May Concern:
> > >
> > > I upload a new database to AWS but can not find it in MetViewer.
> > > The
> > load
> > > xml and AWS database inquiry is shown below
> > >
> > > The scp files seems working,
> > >
> > > Thank you for your help.
> > >
> > > Ho-Chun
> > >
> > > <load_spec>
> > > <connection>
> > > <host>
> > >
> > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
> > 1.rds.amazonaws.com:3306
> > > </host>
> > > <database>mv_g2g_met_verf_aod</database>
> > > <user>rds_user</user>
> > > <password>rds_pwd</password>
> > > <management_system>aurora</management_system>
> > > </connection>
> > >
> > > <met_version>V8.1.2</met_version>
> > >
> > > <verbose>true</verbose>
> > > <insert_size>1</insert_size>
> > > <mode_header_db_check>true</mode_header_db_check>
> > > <stat_header_db_check>true</stat_header_db_check>
> > > <drop_indexes>false</drop_indexes>
> > > <apply_indexes>true</apply_indexes>
> > > <load_stat>true</load_stat>
> > > <load_mode>true</load_mode>
> > > <load_mpr>true</load_mpr>
> > > <load_orank>true</load_orank>
> > > <force_dup_file>false</force_dup_file>
> > > <group>NCEP_hcHuang</group>
> > > <description>Loading mv_g2g_met_verf_aod</description>
> > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > <load_val>
> > > <field name="type">
> > > <val>stat</val>
> > > </field>
> > > </load_val>
> > > <load_xml>true</load_xml>
> > > <load_note></load_note>
> > > </load_spec>
> > >
> > >
> > >
> > > > mv_db_size_on_aws.sh ho-chun.huang
> > > :::::::::::::::::::::::::::::::::::::::::
> > > :::::::::::::::::::::::::::::::::::::::::
> > > mv_agibbs_nam_hi_dng_FAA annette.gibbs 2.28
> > > mv_agibbs_nam_pr_dng_FAA annette.gibbs 2.28
> > > emc_cmaq_g2o perry.shafran 2.02
> > > emc_cmaq_g2o2 perry.shafran 2.02
> > > mv_vh venita.hagerty 2.02
> > > testvh4 venita.hagerty 2.02
> > > testvh5 venita.hagerty 2.02
> > > mv_g2g_met_verf_aod ho-chun.huang 1.97
> > > mats_metadata kenneth.sperow 1.28
> > > _______________________________
> > > TOTAL_SIZE_GB
> > > 203.36
> > >
> > >
> >
> > --
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > Tara Jensen
> > Project Manager II
> > NCAR RAL and DTC
> > PO Box 3000, Boulder, Colorado 80307 USA
> > +1 303-497-8479 jensen at ucar.edu
> >
> >
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Ho-Chun Huang - NOAA Affiliate
Time: Thu Sep 26 14:28:00 2019
Hi, Tara:
I remove the database and start again
The log file is in
https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
When I run it without piping it to run.log, I did see the scp working
transferring files. For some reason, I did not see it in the logfile.
Ho-Chun
On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Ho-Chun
>
> mv_g2g_met_verf_aod database exists on AWS but it is empty.
> Do you have a copy of the log file from the data loading?
>
> Tatiana
>
> On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov wrote:
> > Hi, Tara:
> >
> > I tried
> >
> > (1) hit reload database
> > (2) refresh button
> > (3) clear all web browser data
> > (4) close the web browser and open it again
> >
> > None of it works.
> >
> >
> > Can you get on the AWS MEtviewer and be able to see
> >
> > mv_g2g_met_verf_aod under NCEP_hcHuang?
> >
> > Ho-Chun
> >
> >
> > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Ho-Chun,
> > >
> > > I see you are trying to view a new database up on AWS. After
adding a
> > > new
> > > database to METviewer, you will need to click on Reload
databases
> > > above the
> > > plotting area. If you still don't see it, consider clearing
your
> > > cache and
> > > hitting the refresh button. If you are using METexpress, you
may not
> > > have
> > > access to it until tomorrow. I'm not sure how often they are
running
> > > metadata, which is what is needed to see a new database.
> > >
> > > [image: image.png]
> > > Let us know if you have tried these things and still don't see
it.
> > > We can
> > > pass it over to Tatiana at that point.
> > > Cheers, Tara
> > >
> > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA Affiliate
via
> > > RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted upon.
> > > > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > > > Queue: met_help
> > > > Subject: upload database can not being seen in AWS
Metviewer
> > > > Owner: Nobody
> > > > Requestors: ho-chun.huang at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
> > > >
> > > >
> > > > To Whom It May Concern:
> > > >
> > > > I upload a new database to AWS but can not find it in
MetViewer.
> > > > The
> > > load
> > > > xml and AWS database inquiry is shown below
> > > >
> > > > The scp files seems working,
> > > >
> > > > Thank you for your help.
> > > >
> > > > Ho-Chun
> > > >
> > > > <load_spec>
> > > > <connection>
> > > > <host>
> > > >
> > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
> > > 1.rds.amazonaws.com:3306
> > > > </host>
> > > > <database>mv_g2g_met_verf_aod</database>
> > > > <user>rds_user</user>
> > > > <password>rds_pwd</password>
> > > > <management_system>aurora</management_system>
> > > > </connection>
> > > >
> > > > <met_version>V8.1.2</met_version>
> > > >
> > > > <verbose>true</verbose>
> > > > <insert_size>1</insert_size>
> > > > <mode_header_db_check>true</mode_header_db_check>
> > > > <stat_header_db_check>true</stat_header_db_check>
> > > > <drop_indexes>false</drop_indexes>
> > > > <apply_indexes>true</apply_indexes>
> > > > <load_stat>true</load_stat>
> > > > <load_mode>true</load_mode>
> > > > <load_mpr>true</load_mpr>
> > > > <load_orank>true</load_orank>
> > > > <force_dup_file>false</force_dup_file>
> > > > <group>NCEP_hcHuang</group>
> > > > <description>Loading mv_g2g_met_verf_aod</description>
> > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > > <load_val>
> > > > <field name="type">
> > > > <val>stat</val>
> > > > </field>
> > > > </load_val>
> > > > <load_xml>true</load_xml>
> > > > <load_note></load_note>
> > > > </load_spec>
> > > >
> > > >
> > > >
> > > > > mv_db_size_on_aws.sh ho-chun.huang
> > > > :::::::::::::::::::::::::::::::::::::::::
> > > > :::::::::::::::::::::::::::::::::::::::::
> > > > mv_agibbs_nam_hi_dng_FAA annette.gibbs 2.28
> > > > mv_agibbs_nam_pr_dng_FAA annette.gibbs 2.28
> > > > emc_cmaq_g2o perry.shafran 2.02
> > > > emc_cmaq_g2o2 perry.shafran 2.02
> > > > mv_vh venita.hagerty 2.02
> > > > testvh4 venita.hagerty 2.02
> > > > testvh5 venita.hagerty 2.02
> > > > mv_g2g_met_verf_aod ho-chun.huang 1.97
> > > > mats_metadata kenneth.sperow 1.28
> > > > _______________________________
> > > > TOTAL_SIZE_GB
> > > > 203.36
> > > >
> > > >
> > >
> > > --
> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > Tara Jensen
> > > Project Manager II
> > > NCAR RAL and DTC
> > > PO Box 3000, Boulder, Colorado 80307 USA
> > > +1 303-497-8479 jensen at ucar.edu
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Tatiana Burek
Time: Thu Sep 26 14:37:07 2019
Ho-Chun
The log file shows that 0 records was inserted into the database.
What is the path to your data on WCOSS and how do you call the load
command?
Tatiana
On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov wrote:
> Hi, Tara:
>
>
> I remove the database and start again
>
> The log file is in
> https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
>
> When I run it without piping it to run.log, I did see the scp
working
> transferring files. For some reason, I did not see it in the
logfile.
>
> Ho-Chun
>
> On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
> > Ho-Chun
> >
> > mv_g2g_met_verf_aod database exists on AWS but it is empty.
> > Do you have a copy of the log file from the data loading?
> >
> > Tatiana
> >
> > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov wrote:
> > > Hi, Tara:
> > >
> > > I tried
> > >
> > > (1) hit reload database
> > > (2) refresh button
> > > (3) clear all web browser data
> > > (4) close the web browser and open it again
> > >
> > > None of it works.
> > >
> > >
> > > Can you get on the AWS MEtviewer and be able to see
> > >
> > > mv_g2g_met_verf_aod under NCEP_hcHuang?
> > >
> > > Ho-Chun
> > >
> > >
> > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT
<met_help at ucar.edu>
> > > wrote:
> > >
> > > > Ho-Chun,
> > > >
> > > > I see you are trying to view a new database up on AWS. After
adding a
> > > > new
> > > > database to METviewer, you will need to click on Reload
databases
> > > > above the
> > > > plotting area. If you still don't see it, consider clearing
your
> > > > cache and
> > > > hitting the refresh button. If you are using METexpress, you
may not
> > > > have
> > > > access to it until tomorrow. I'm not sure how often they are
running
> > > > metadata, which is what is needed to see a new database.
> > > >
> > > > [image: image.png]
> > > > Let us know if you have tried these things and still don't see
it.
> > > > We can
> > > > pass it over to Tatiana at that point.
> > > > Cheers, Tara
> > > >
> > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA
Affiliate via
> > > > RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted upon.
> > > > > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: upload database can not being seen in AWS
Metviewer
> > > > > Owner: Nobody
> > > > > Requestors: ho-chun.huang at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
> > > > >
> > > > >
> > > > > To Whom It May Concern:
> > > > >
> > > > > I upload a new database to AWS but can not find it in
MetViewer.
> > > > > The
> > > > load
> > > > > xml and AWS database inquiry is shown below
> > > > >
> > > > > The scp files seems working,
> > > > >
> > > > > Thank you for your help.
> > > > >
> > > > > Ho-Chun
> > > > >
> > > > > <load_spec>
> > > > > <connection>
> > > > > <host>
> > > > >
> > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
> > > > 1.rds.amazonaws.com:3306
> > > > > </host>
> > > > > <database>mv_g2g_met_verf_aod</database>
> > > > > <user>rds_user</user>
> > > > > <password>rds_pwd</password>
> > > > > <management_system>aurora</management_system>
> > > > > </connection>
> > > > >
> > > > > <met_version>V8.1.2</met_version>
> > > > >
> > > > > <verbose>true</verbose>
> > > > > <insert_size>1</insert_size>
> > > > > <mode_header_db_check>true</mode_header_db_check>
> > > > > <stat_header_db_check>true</stat_header_db_check>
> > > > > <drop_indexes>false</drop_indexes>
> > > > > <apply_indexes>true</apply_indexes>
> > > > > <load_stat>true</load_stat>
> > > > > <load_mode>true</load_mode>
> > > > > <load_mpr>true</load_mpr>
> > > > > <load_orank>true</load_orank>
> > > > > <force_dup_file>false</force_dup_file>
> > > > > <group>NCEP_hcHuang</group>
> > > > > <description>Loading mv_g2g_met_verf_aod</description>
> > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > > > <load_val>
> > > > > <field name="type">
> > > > > <val>stat</val>
> > > > > </field>
> > > > > </load_val>
> > > > > <load_xml>true</load_xml>
> > > > > <load_note></load_note>
> > > > > </load_spec>
> > > > >
> > > > >
> > > > >
> > > > > > mv_db_size_on_aws.sh ho-chun.huang
> > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > mv_agibbs_nam_hi_dng_FAA annette.gibbs
2.28
> > > > > mv_agibbs_nam_pr_dng_FAA annette.gibbs
2.28
> > > > > emc_cmaq_g2o perry.shafran
2.02
> > > > > emc_cmaq_g2o2 perry.shafran
2.02
> > > > > mv_vh venita.hagerty
2.02
> > > > > testvh4 venita.hagerty
2.02
> > > > > testvh5 venita.hagerty
2.02
> > > > > mv_g2g_met_verf_aod ho-chun.huang
1.97
> > > > > mats_metadata kenneth.sperow
1.28
> > > > > _______________________________
> > > > > TOTAL_SIZE_GB
> > > > > 203.36
> > > > >
> > > > >
> > > >
> > > > --
> > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > Tara Jensen
> > > > Project Manager II
> > > > NCAR RAL and DTC
> > > > PO Box 3000, Boulder, Colorado 80307 USA
> > > > +1 303-497-8479 jensen at ucar.edu
> > > >
> > > >
> >
> >
> >
> >
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Ho-Chun Huang - NOAA Affiliate
Time: Thu Sep 26 14:45:53 2019
Hi, Tatiana:
Dev machine is down. I have access to operational machine and you do
not.
So you can not check my data directory, but as in the log, it
is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
I do have files in the directory, and the scp did show it was
transferring
files.
/gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
This is when I try it again to different database name
mv_g2g_met_verf_test
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
100% 3514 251.1KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
100% 3503 411.8KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
100% 3515 457.7KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
100% 3538 465.2KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
100% 3514 248.3KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
100% 3537 370.5KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
100% 3538 175.2KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
100% 3516 416.8KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
100% 3504 461.5KB/s 00:00
CALLING: scp
/gpfs/dell2/emc/modeling/noscrub/Ho-
Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
ho-chun.huang at 205.156.8.85:~
load_g2g_met_verf_aod.xml
100% 1019 136.7KB/s 00:00
---- MVLoad ----
Begin time: 2019-09-26 20:30:35
Parsing: load_g2g_met_verf_aod.xml
Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
==== grid_stat ====
stat_header table time total: 0:00:00.000
stat header records: 0
stat header inserts: 0
line data records: 0
line data inserts: 0
stat group records: 0
stat group inserts: 0
var length records: 0
var length inserts: 0
total lines: 0
insert size: 1
lines / msec: 0.000
num files: 0
==== mode ====
mode_header search time total: 0:00:00.000
mtd_header search time total: 0:00:00.000
stat_header search time total: 0:00:00.000
mode_header inserts: 0
mode_cts inserts: 0
mode_obj_single inserts: 0
mode_obj_pair inserts: 0
==== mtd ====
mtd_header inserts: 0
mtd_3d_obj_single inserts: 0
mtd_3d_obj_pair inserts: 0
mtd_2d_obj inserts: 0
total lines: 0
num files: 0
num files: 0
==== indexes ====creating
Indexes creating 0:00:01.002
End time: 2019-09-26 20:30:36
Load total: 0:00:00.010
---- MVLoad Done ----
Each file does have data in there
/gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END
OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV
OB
S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS
FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=0.5
>=0.5 NA NA CTC 17941 0 5 1
17935
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=1.0
>=1.0 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=1.5
>=1.5 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=2.0
>=2.0 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=2.5
>=2.5 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=3.0
>=3.0 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=3.5
>=3.5 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=4.0
>=4.0 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=4.5
>=4.5 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>=5.0
>=5.0 NA NA CTC 17941 0 0 0
17941
V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
000000
20190820_120236 20190820_120236 AOTK Numeric L1 AO
D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1 NA
NA NA NA SL1L2 17941 0.070913 0.1156 0.
011134 0.008766 0.018307 0.054144
On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Ho-Chun
>
> The log file shows that 0 records was inserted into the database.
> What is the path to your data on WCOSS and how do you call the load
> command?
>
> Tatiana
>
> On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov wrote:
> > Hi, Tara:
> >
> >
> > I remove the database and start again
> >
> > The log file is in
> > https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
> >
> > When I run it without piping it to run.log, I did see the scp
working
> > transferring files. For some reason, I did not see it in the
logfile.
> >
> > Ho-Chun
> >
> > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Ho-Chun
> > >
> > > mv_g2g_met_verf_aod database exists on AWS but it is empty.
> > > Do you have a copy of the log file from the data loading?
> > >
> > > Tatiana
> > >
> > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov wrote:
> > > > Hi, Tara:
> > > >
> > > > I tried
> > > >
> > > > (1) hit reload database
> > > > (2) refresh button
> > > > (3) clear all web browser data
> > > > (4) close the web browser and open it again
> > > >
> > > > None of it works.
> > > >
> > > >
> > > > Can you get on the AWS MEtviewer and be able to see
> > > >
> > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
> > > >
> > > > Ho-Chun
> > > >
> > > >
> > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT <
> met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Ho-Chun,
> > > > >
> > > > > I see you are trying to view a new database up on AWS. After
> adding a
> > > > > new
> > > > > database to METviewer, you will need to click on Reload
databases
> > > > > above the
> > > > > plotting area. If you still don't see it, consider clearing
your
> > > > > cache and
> > > > > hitting the refresh button. If you are using METexpress,
you may
> not
> > > > > have
> > > > > access to it until tomorrow. I'm not sure how often they
are
> running
> > > > > metadata, which is what is needed to see a new database.
> > > > >
> > > > > [image: image.png]
> > > > > Let us know if you have tried these things and still don't
see it.
> > > > > We can
> > > > > pass it over to Tatiana at that point.
> > > > > Cheers, Tara
> > > > >
> > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA
Affiliate via
> > > > > RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted upon.
> > > > > > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > > > > > Queue: met_help
> > > > > > Subject: upload database can not being seen in AWS
Metviewer
> > > > > > Owner: Nobody
> > > > > > Requestors: ho-chun.huang at noaa.gov
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
> > > > > >
> > > > > >
> > > > > > To Whom It May Concern:
> > > > > >
> > > > > > I upload a new database to AWS but can not find it in
MetViewer.
> > > > > > The
> > > > > load
> > > > > > xml and AWS database inquiry is shown below
> > > > > >
> > > > > > The scp files seems working,
> > > > > >
> > > > > > Thank you for your help.
> > > > > >
> > > > > > Ho-Chun
> > > > > >
> > > > > > <load_spec>
> > > > > > <connection>
> > > > > > <host>
> > > > > >
> > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
> > > > > 1.rds.amazonaws.com:3306
> > > > > > </host>
> > > > > > <database>mv_g2g_met_verf_aod</database>
> > > > > > <user>rds_user</user>
> > > > > > <password>rds_pwd</password>
> > > > > > <management_system>aurora</management_system>
> > > > > > </connection>
> > > > > >
> > > > > > <met_version>V8.1.2</met_version>
> > > > > >
> > > > > > <verbose>true</verbose>
> > > > > > <insert_size>1</insert_size>
> > > > > > <mode_header_db_check>true</mode_header_db_check>
> > > > > > <stat_header_db_check>true</stat_header_db_check>
> > > > > > <drop_indexes>false</drop_indexes>
> > > > > > <apply_indexes>true</apply_indexes>
> > > > > > <load_stat>true</load_stat>
> > > > > > <load_mode>true</load_mode>
> > > > > > <load_mpr>true</load_mpr>
> > > > > > <load_orank>true</load_orank>
> > > > > > <force_dup_file>false</force_dup_file>
> > > > > > <group>NCEP_hcHuang</group>
> > > > > > <description>Loading mv_g2g_met_verf_aod</description>
> > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > > > > <load_val>
> > > > > > <field name="type">
> > > > > > <val>stat</val>
> > > > > > </field>
> > > > > > </load_val>
> > > > > > <load_xml>true</load_xml>
> > > > > > <load_note></load_note>
> > > > > > </load_spec>
> > > > > >
> > > > > >
> > > > > >
> > > > > > > mv_db_size_on_aws.sh ho-chun.huang
> > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > mv_agibbs_nam_hi_dng_FAA annette.gibbs
2.28
> > > > > > mv_agibbs_nam_pr_dng_FAA annette.gibbs
2.28
> > > > > > emc_cmaq_g2o perry.shafran
2.02
> > > > > > emc_cmaq_g2o2 perry.shafran
2.02
> > > > > > mv_vh venita.hagerty
2.02
> > > > > > testvh4 venita.hagerty
2.02
> > > > > > testvh5 venita.hagerty
2.02
> > > > > > mv_g2g_met_verf_aod ho-chun.huang
1.97
> > > > > > mats_metadata kenneth.sperow
1.28
> > > > > > _______________________________
> > > > > > TOTAL_SIZE_GB
> > > > > > 203.36
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > Tara Jensen
> > > > > Project Manager II
> > > > > NCAR RAL and DTC
> > > > > PO Box 3000, Boulder, Colorado 80307 USA
> > > > > +1 303-497-8479 jensen at ucar.edu
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Tatiana Burek
Time: Thu Sep 26 14:52:40 2019
In your XML you specified that your data directory contains 'stats":
<folder_tmpl>/base_dir/{type}</folder_tmpl>
<load_val>
<field name="type">
<val>stat</val>
</field>
</load_val>
But your actual path doesn't have this directory. Is it correct?
Tatiana
On Thu Sep 26 14:45:53 2019, ho-chun.huang at noaa.gov wrote:
> Hi, Tatiana:
>
> Dev machine is down. I have access to operational machine and you
do
> not.
> So you can not check my data directory, but as in the log, it
> is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
>
> I do have files in the directory, and the scp did show it was
> transferring
> files.
> /gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>
>
> This is when I try it again to different database name
> mv_g2g_met_verf_test
>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> 100% 3514 251.1KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> 100% 3503 411.8KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> 100% 3515 457.7KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> 100% 3538 465.2KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> 100% 3514 248.3KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> 100% 3537 370.5KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> 100% 3538 175.2KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> 100% 3516 416.8KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> 100% 3504 461.5KB/s 00:00
> CALLING: scp
> /gpfs/dell2/emc/modeling/noscrub/Ho-
> Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
> ho-chun.huang at 205.156.8.85:~
> load_g2g_met_verf_aod.xml
> 100% 1019 136.7KB/s 00:00
> ---- MVLoad ----
>
> Begin time: 2019-09-26 20:30:35
> Parsing: load_g2g_met_verf_aod.xml
>
> Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
> ==== grid_stat ====
>
> stat_header table time total: 0:00:00.000
> stat header records: 0
> stat header inserts: 0
> line data records: 0
> line data inserts: 0
> stat group records: 0
> stat group inserts: 0
> var length records: 0
> var length inserts: 0
> total lines: 0
> insert size: 1
> lines / msec: 0.000
> num files: 0
>
> ==== mode ====
>
> mode_header search time total: 0:00:00.000
> mtd_header search time total: 0:00:00.000
> stat_header search time total: 0:00:00.000
> mode_header inserts: 0
> mode_cts inserts: 0
> mode_obj_single inserts: 0
> mode_obj_pair inserts: 0
> ==== mtd ====
>
> mtd_header inserts: 0
> mtd_3d_obj_single inserts: 0
> mtd_3d_obj_pair inserts: 0
> mtd_2d_obj inserts: 0
> total lines: 0
> num files: 0
> num files: 0
>
> ==== indexes ====creating
> Indexes creating 0:00:01.002
>
> End time: 2019-09-26 20:30:36
> Load total: 0:00:00.010
>
> ---- MVLoad Done ----
>
> Each file does have data in there
>
> /gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END
> OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS
FCST_LEV
> OB
> S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS
> FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=0.5
> > =0.5 NA NA CTC 17941 0 5 1
> 17935
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=1.0
> > =1.0 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=1.5
> > =1.5 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=2.0
> > =2.0 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=2.5
> > =2.5 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=3.0
> > =3.0 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=3.5
> > =3.5 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=4.0
> > =4.0 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=4.5
> > =4.5 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >=5.0
> > =5.0 NA NA CTC 17941 0 0 0
> 17941
> V8.1.1 cmaqaod_prod NA 480000 20190820_120000 20190820_120000
> 000000
> 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1 NA
> NA NA NA SL1L2 17941 0.070913 0.1156 0.
> 011134 0.008766 0.018307 0.054144
>
> On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
> <met_help at ucar.edu>
> wrote:
>
> > Ho-Chun
> >
> > The log file shows that 0 records was inserted into the database.
> > What is the path to your data on WCOSS and how do you call the
load
> > command?
> >
> > Tatiana
> >
> > On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov wrote:
> > > Hi, Tara:
> > >
> > >
> > > I remove the database and start again
> > >
> > > The log file is in
> > > https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
> > >
> > > When I run it without piping it to run.log, I did see the scp
> > > working
> > > transferring files. For some reason, I did not see it in the
> > > logfile.
> > >
> > > Ho-Chun
> > >
> > > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > > Ho-Chun
> > > >
> > > > mv_g2g_met_verf_aod database exists on AWS but it is empty.
> > > > Do you have a copy of the log file from the data loading?
> > > >
> > > > Tatiana
> > > >
> > > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov wrote:
> > > > > Hi, Tara:
> > > > >
> > > > > I tried
> > > > >
> > > > > (1) hit reload database
> > > > > (2) refresh button
> > > > > (3) clear all web browser data
> > > > > (4) close the web browser and open it again
> > > > >
> > > > > None of it works.
> > > > >
> > > > >
> > > > > Can you get on the AWS MEtviewer and be able to see
> > > > >
> > > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
> > > > >
> > > > > Ho-Chun
> > > > >
> > > > >
> > > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT <
> > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Ho-Chun,
> > > > > >
> > > > > > I see you are trying to view a new database up on AWS.
After
> > adding a
> > > > > > new
> > > > > > database to METviewer, you will need to click on Reload
> > > > > > databases
> > > > > > above the
> > > > > > plotting area. If you still don't see it, consider
clearing
> > > > > > your
> > > > > > cache and
> > > > > > hitting the refresh button. If you are using METexpress,
you
> > > > > > may
> > not
> > > > > > have
> > > > > > access to it until tomorrow. I'm not sure how often they
are
> > running
> > > > > > metadata, which is what is needed to see a new database.
> > > > > >
> > > > > > [image: image.png]
> > > > > > Let us know if you have tried these things and still don't
> > > > > > see it.
> > > > > > We can
> > > > > > pass it over to Tatiana at that point.
> > > > > > Cheers, Tara
> > > > > >
> > > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA
> > > > > > Affiliate via
> > > > > > RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted upon.
> > > > > > > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > > > > > > Queue: met_help
> > > > > > > Subject: upload database can not being seen in AWS
> > > > > > > Metviewer
> > > > > > > Owner: Nobody
> > > > > > > Requestors: ho-chun.huang at noaa.gov
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346
>
> > > > > > >
> > > > > > >
> > > > > > > To Whom It May Concern:
> > > > > > >
> > > > > > > I upload a new database to AWS but can not find it in
> > > > > > > MetViewer.
> > > > > > > The
> > > > > > load
> > > > > > > xml and AWS database inquiry is shown below
> > > > > > >
> > > > > > > The scp files seems working,
> > > > > > >
> > > > > > > Thank you for your help.
> > > > > > >
> > > > > > > Ho-Chun
> > > > > > >
> > > > > > > <load_spec>
> > > > > > > <connection>
> > > > > > > <host>
> > > > > > >
> > > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
> > > > > > 1.rds.amazonaws.com:3306
> > > > > > > </host>
> > > > > > > <database>mv_g2g_met_verf_aod</database>
> > > > > > > <user>rds_user</user>
> > > > > > > <password>rds_pwd</password>
> > > > > > > <management_system>aurora</management_system>
> > > > > > > </connection>
> > > > > > >
> > > > > > > <met_version>V8.1.2</met_version>
> > > > > > >
> > > > > > > <verbose>true</verbose>
> > > > > > > <insert_size>1</insert_size>
> > > > > > > <mode_header_db_check>true</mode_header_db_check>
> > > > > > > <stat_header_db_check>true</stat_header_db_check>
> > > > > > > <drop_indexes>false</drop_indexes>
> > > > > > > <apply_indexes>true</apply_indexes>
> > > > > > > <load_stat>true</load_stat>
> > > > > > > <load_mode>true</load_mode>
> > > > > > > <load_mpr>true</load_mpr>
> > > > > > > <load_orank>true</load_orank>
> > > > > > > <force_dup_file>false</force_dup_file>
> > > > > > > <group>NCEP_hcHuang</group>
> > > > > > > <description>Loading mv_g2g_met_verf_aod</description>
> > > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > > > > > <load_val>
> > > > > > > <field name="type">
> > > > > > > <val>stat</val>
> > > > > > > </field>
> > > > > > > </load_val>
> > > > > > > <load_xml>true</load_xml>
> > > > > > > <load_note></load_note>
> > > > > > > </load_spec>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > mv_db_size_on_aws.sh ho-chun.huang
> > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > > mv_agibbs_nam_hi_dng_FAA annette.gibbs
> > > > > > > 2.28
> > > > > > > mv_agibbs_nam_pr_dng_FAA annette.gibbs
> > > > > > > 2.28
> > > > > > > emc_cmaq_g2o perry.shafran
> > > > > > > 2.02
> > > > > > > emc_cmaq_g2o2 perry.shafran
> > > > > > > 2.02
> > > > > > > mv_vh venita.hagerty
> > > > > > > 2.02
> > > > > > > testvh4 venita.hagerty
> > > > > > > 2.02
> > > > > > > testvh5 venita.hagerty
> > > > > > > 2.02
> > > > > > > mv_g2g_met_verf_aod ho-chun.huang
> > > > > > > 1.97
> > > > > > > mats_metadata kenneth.sperow
> > > > > > > 1.28
> > > > > > > _______________________________
> > > > > > > TOTAL_SIZE_GB
> > > > > > > 203.36
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > Tara Jensen
> > > > > > Project Manager II
> > > > > > NCAR RAL and DTC
> > > > > > PO Box 3000, Boulder, Colorado 80307 USA
> > > > > > +1 303-497-8479 jensen at ucar.edu
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> >
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Ho-Chun Huang - NOAA Affiliate
Time: Thu Sep 26 14:57:56 2019
Hi, Tatiana:
It is the filename in the base_dir contain "stat" /base_dir/{type}
My filename look like
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
in base_dir as /gpfs/dell2/stmp/Ho-Chun.Huang/verif
I do not quite understand you question. The directory exist and the
filename contain "stat"
I will leave office in 10 mins to catch the shuttle. If necessary, I
can
tar the stat file and load xml to ftp directory and you can try it
from
NCAR machine. Please let me know do you want to do it this way.
Ho-Chun
On Thu, Sep 26, 2019 at 4:52 PM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
>
> In your XML you specified that your data directory contains 'stats":
>
> <folder_tmpl>/base_dir/{type}</folder_tmpl>
> <load_val>
> <field name="type">
> <val>stat</val>
> </field>
> </load_val>
> But your actual path doesn't have this directory. Is it correct?
>
>
> Tatiana
>
> On Thu Sep 26 14:45:53 2019, ho-chun.huang at noaa.gov wrote:
> > Hi, Tatiana:
> >
> > Dev machine is down. I have access to operational machine and you
do
> > not.
> > So you can not check my data directory, but as in the log, it
> > is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
> >
> > I do have files in the directory, and the scp did show it was
> > transferring
> > files.
> > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
> >
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> >
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> >
>
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> >
> >
> > This is when I try it again to different database name
> > mv_g2g_met_verf_test
> >
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> > 100% 3514 251.1KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> > 100% 3503 411.8KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> > 100% 3515 457.7KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> > 100% 3538 465.2KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> > 100% 3514 248.3KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> > 100% 3537 370.5KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> > 100% 3538 175.2KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> > 100% 3516 416.8KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > 100% 3504 461.5KB/s 00:00
> > CALLING: scp
> > /gpfs/dell2/emc/modeling/noscrub/Ho-
> > Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
> > ho-chun.huang at 205.156.8.85:~
> > load_g2g_met_verf_aod.xml
> > 100% 1019 136.7KB/s 00:00
> > ---- MVLoad ----
> >
> > Begin time: 2019-09-26 20:30:35
> > Parsing: load_g2g_met_verf_aod.xml
> >
> > Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
> > ==== grid_stat ====
> >
> > stat_header table time total: 0:00:00.000
> > stat header records: 0
> > stat header inserts: 0
> > line data records: 0
> > line data inserts: 0
> > stat group records: 0
> > stat group inserts: 0
> > var length records: 0
> > var length inserts: 0
> > total lines: 0
> > insert size: 1
> > lines / msec: 0.000
> > num files: 0
> >
> > ==== mode ====
> >
> > mode_header search time total: 0:00:00.000
> > mtd_header search time total: 0:00:00.000
> > stat_header search time total: 0:00:00.000
> > mode_header inserts: 0
> > mode_cts inserts: 0
> > mode_obj_single inserts: 0
> > mode_obj_pair inserts: 0
> > ==== mtd ====
> >
> > mtd_header inserts: 0
> > mtd_3d_obj_single inserts: 0
> > mtd_3d_obj_pair inserts: 0
> > mtd_2d_obj inserts: 0
> > total lines: 0
> > num files: 0
> > num files: 0
> >
> > ==== indexes ====creating
> > Indexes creating 0:00:01.002
> >
> > End time: 2019-09-26 20:30:36
> > Load total: 0:00:00.010
> >
> > ---- MVLoad Done ----
> >
> > Each file does have data in there
> >
> > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END
> > OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS
FCST_LEV
> > OB
> > S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS
> > FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=0.5
> > > =0.5 NA NA CTC 17941 0 5 1
> > 17935
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=1.0
> > > =1.0 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=1.5
> > > =1.5 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=2.0
> > > =2.0 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=2.5
> > > =2.5 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=3.0
> > > =3.0 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=3.5
> > > =3.5 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=4.0
> > > =4.0 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=4.5
> > > =4.5 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >=5.0
> > > =5.0 NA NA CTC 17941 0 0 0
> > 17941
> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > 000000
> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
NA
> > NA NA NA SL1L2 17941 0.070913 0.1156 0.
> > 011134 0.008766 0.018307 0.054144
> >
> > On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > Ho-Chun
> > >
> > > The log file shows that 0 records was inserted into the
database.
> > > What is the path to your data on WCOSS and how do you call the
load
> > > command?
> > >
> > > Tatiana
> > >
> > > On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov wrote:
> > > > Hi, Tara:
> > > >
> > > >
> > > > I remove the database and start again
> > > >
> > > > The log file is in
> > > > https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
> > > >
> > > > When I run it without piping it to run.log, I did see the scp
> > > > working
> > > > transferring files. For some reason, I did not see it in the
> > > > logfile.
> > > >
> > > > Ho-Chun
> > > >
> > > > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Ho-Chun
> > > > >
> > > > > mv_g2g_met_verf_aod database exists on AWS but it is empty.
> > > > > Do you have a copy of the log file from the data loading?
> > > > >
> > > > > Tatiana
> > > > >
> > > > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov wrote:
> > > > > > Hi, Tara:
> > > > > >
> > > > > > I tried
> > > > > >
> > > > > > (1) hit reload database
> > > > > > (2) refresh button
> > > > > > (3) clear all web browser data
> > > > > > (4) close the web browser and open it again
> > > > > >
> > > > > > None of it works.
> > > > > >
> > > > > >
> > > > > > Can you get on the AWS MEtviewer and be able to see
> > > > > >
> > > > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
> > > > > >
> > > > > > Ho-Chun
> > > > > >
> > > > > >
> > > > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT <
> > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Ho-Chun,
> > > > > > >
> > > > > > > I see you are trying to view a new database up on AWS.
After
> > > adding a
> > > > > > > new
> > > > > > > database to METviewer, you will need to click on Reload
> > > > > > > databases
> > > > > > > above the
> > > > > > > plotting area. If you still don't see it, consider
clearing
> > > > > > > your
> > > > > > > cache and
> > > > > > > hitting the refresh button. If you are using
METexpress, you
> > > > > > > may
> > > not
> > > > > > > have
> > > > > > > access to it until tomorrow. I'm not sure how often
they are
> > > running
> > > > > > > metadata, which is what is needed to see a new database.
> > > > > > >
> > > > > > > [image: image.png]
> > > > > > > Let us know if you have tried these things and still
don't
> > > > > > > see it.
> > > > > > > We can
> > > > > > > pass it over to Tatiana at that point.
> > > > > > > Cheers, Tara
> > > > > > >
> > > > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA
> > > > > > > Affiliate via
> > > > > > > RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted
upon.
> > > > > > > > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > > > > > > > Queue: met_help
> > > > > > > > Subject: upload database can not being seen in
AWS
> > > > > > > > Metviewer
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: ho-chun.huang at noaa.gov
> > > > > > > > Status: new
> > > > > > > > Ticket <URL:
> > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
> > > > > > > >
> > > > > > > >
> > > > > > > > To Whom It May Concern:
> > > > > > > >
> > > > > > > > I upload a new database to AWS but can not find it in
> > > > > > > > MetViewer.
> > > > > > > > The
> > > > > > > load
> > > > > > > > xml and AWS database inquiry is shown below
> > > > > > > >
> > > > > > > > The scp files seems working,
> > > > > > > >
> > > > > > > > Thank you for your help.
> > > > > > > >
> > > > > > > > Ho-Chun
> > > > > > > >
> > > > > > > > <load_spec>
> > > > > > > > <connection>
> > > > > > > > <host>
> > > > > > > >
> > > > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
> > > > > > > 1.rds.amazonaws.com:3306
> > > > > > > > </host>
> > > > > > > > <database>mv_g2g_met_verf_aod</database>
> > > > > > > > <user>rds_user</user>
> > > > > > > > <password>rds_pwd</password>
> > > > > > > > <management_system>aurora</management_system>
> > > > > > > > </connection>
> > > > > > > >
> > > > > > > > <met_version>V8.1.2</met_version>
> > > > > > > >
> > > > > > > > <verbose>true</verbose>
> > > > > > > > <insert_size>1</insert_size>
> > > > > > > > <mode_header_db_check>true</mode_header_db_check>
> > > > > > > > <stat_header_db_check>true</stat_header_db_check>
> > > > > > > > <drop_indexes>false</drop_indexes>
> > > > > > > > <apply_indexes>true</apply_indexes>
> > > > > > > > <load_stat>true</load_stat>
> > > > > > > > <load_mode>true</load_mode>
> > > > > > > > <load_mpr>true</load_mpr>
> > > > > > > > <load_orank>true</load_orank>
> > > > > > > > <force_dup_file>false</force_dup_file>
> > > > > > > > <group>NCEP_hcHuang</group>
> > > > > > > > <description>Loading mv_g2g_met_verf_aod</description>
> > > > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > > > > > > <load_val>
> > > > > > > > <field name="type">
> > > > > > > > <val>stat</val>
> > > > > > > > </field>
> > > > > > > > </load_val>
> > > > > > > > <load_xml>true</load_xml>
> > > > > > > > <load_note></load_note>
> > > > > > > > </load_spec>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > mv_db_size_on_aws.sh ho-chun.huang
> > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > > > mv_agibbs_nam_hi_dng_FAA
annette.gibbs
> > > > > > > > 2.28
> > > > > > > > mv_agibbs_nam_pr_dng_FAA
annette.gibbs
> > > > > > > > 2.28
> > > > > > > > emc_cmaq_g2o
perry.shafran
> > > > > > > > 2.02
> > > > > > > > emc_cmaq_g2o2
perry.shafran
> > > > > > > > 2.02
> > > > > > > > mv_vh
venita.hagerty
> > > > > > > > 2.02
> > > > > > > > testvh4
venita.hagerty
> > > > > > > > 2.02
> > > > > > > > testvh5
venita.hagerty
> > > > > > > > 2.02
> > > > > > > > mv_g2g_met_verf_aod ho-
chun.huang
> > > > > > > > 1.97
> > > > > > > > mats_metadata
kenneth.sperow
> > > > > > > > 1.28
> > > > > > > > _______________________________
> > > > > > > > TOTAL_SIZE_GB
> > > > > > > > 203.36
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > Tara Jensen
> > > > > > > Project Manager II
> > > > > > > NCAR RAL and DTC
> > > > > > > PO Box 3000, Boulder, Colorado 80307 USA
> > > > > > > +1 303-497-8479 jensen at ucar.edu
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Ho-Chun Huang - NOAA Affiliate
Time: Thu Sep 26 15:00:38 2019
Hi, Tatiana::
If I want to upload grid_stat file with filename "*.stat" in
base-directory, how should I write for
<folder_tmpl>?????</folder_tmpl> ?
Ho-Chun
On Thu, Sep 26, 2019 at 4:57 PM Ho-Chun Huang - NOAA Affiliate <
ho-chun.huang at noaa.gov> wrote:
> Hi, Tatiana:
>
> It is the filename in the base_dir contain "stat" /base_dir/{type}
>
> My filename look like
>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>
>
> in base_dir as /gpfs/dell2/stmp/Ho-Chun.Huang/verif
>
> I do not quite understand you question. The directory exist and the
> filename contain "stat"
>
>
> I will leave office in 10 mins to catch the shuttle. If necessary,
I can
> tar the stat file and load xml to ftp directory and you can try it
from
> NCAR machine. Please let me know do you want to do it this way.
>
>
> Ho-Chun
>
> On Thu, Sep 26, 2019 at 4:52 PM Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
>>
>> In your XML you specified that your data directory contains
'stats":
>>
>> <folder_tmpl>/base_dir/{type}</folder_tmpl>
>> <load_val>
>> <field name="type">
>> <val>stat</val>
>> </field>
>> </load_val>
>> But your actual path doesn't have this directory. Is it correct?
>>
>>
>> Tatiana
>>
>> On Thu Sep 26 14:45:53 2019, ho-chun.huang at noaa.gov wrote:
>> > Hi, Tatiana:
>> >
>> > Dev machine is down. I have access to operational machine and
you do
>> > not.
>> > So you can not check my data directory, but as in the log, it
>> > is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
>> >
>> > I do have files in the directory, and the scp did show it was
>> > transferring
>> > files.
>> > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
>> >
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>> >
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>> >
>>
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>> >
>> >
>> > This is when I try it again to different database name
>> > mv_g2g_met_verf_test
>> >
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
>> > 100% 3514 251.1KB/s 00:00
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
>> > 100% 3503 411.8KB/s 00:00
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
>> > 100% 3515 457.7KB/s 00:00
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
>> > 100% 3538 465.2KB/s 00:00
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
>> > 100% 3514 248.3KB/s 00:00
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
>> > 100% 3537 370.5KB/s 00:00
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
>> > 100% 3538 175.2KB/s 00:00
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
>> > 100% 3516 416.8KB/s 00:00
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>> > 100% 3504 461.5KB/s 00:00
>> > CALLING: scp
>> > /gpfs/dell2/emc/modeling/noscrub/Ho-
>> > Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
>> > ho-chun.huang at 205.156.8.85:~
>> > load_g2g_met_verf_aod.xml
>> > 100% 1019 136.7KB/s 00:00
>> > ---- MVLoad ----
>> >
>> > Begin time: 2019-09-26 20:30:35
>> > Parsing: load_g2g_met_verf_aod.xml
>> >
>> > Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
>> > ==== grid_stat ====
>> >
>> > stat_header table time total: 0:00:00.000
>> > stat header records: 0
>> > stat header inserts: 0
>> > line data records: 0
>> > line data inserts: 0
>> > stat group records: 0
>> > stat group inserts: 0
>> > var length records: 0
>> > var length inserts: 0
>> > total lines: 0
>> > insert size: 1
>> > lines / msec: 0.000
>> > num files: 0
>> >
>> > ==== mode ====
>> >
>> > mode_header search time total: 0:00:00.000
>> > mtd_header search time total: 0:00:00.000
>> > stat_header search time total: 0:00:00.000
>> > mode_header inserts: 0
>> > mode_cts inserts: 0
>> > mode_obj_single inserts: 0
>> > mode_obj_pair inserts: 0
>> > ==== mtd ====
>> >
>> > mtd_header inserts: 0
>> > mtd_3d_obj_single inserts: 0
>> > mtd_3d_obj_pair inserts: 0
>> > mtd_2d_obj inserts: 0
>> > total lines: 0
>> > num files: 0
>> > num files: 0
>> >
>> > ==== indexes ====creating
>> > Indexes creating 0:00:01.002
>> >
>> > End time: 2019-09-26 20:30:36
>> > Load total: 0:00:00.010
>> >
>> > ---- MVLoad Done ----
>> >
>> > Each file does have data in there
>> >
>> > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>> > VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
FCST_VALID_END
>> > OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS
FCST_LEV
>> > OB
>> > S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS
>> > FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=0.5
>> > > =0.5 NA NA CTC 17941 0 5 1
>> > 17935
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=1.0
>> > > =1.0 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=1.5
>> > > =1.5 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=2.0
>> > > =2.0 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=2.5
>> > > =2.5 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=3.0
>> > > =3.0 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=3.5
>> > > =3.5 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=4.0
>> > > =4.0 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=4.5
>> > > =4.5 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > >=5.0
>> > > =5.0 NA NA CTC 17941 0 0 0
>> > 17941
>> > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > 000000
>> > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
>> > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
NA
>> > NA NA NA SL1L2 17941 0.070913 0.1156 0.
>> > 011134 0.008766 0.018307 0.054144
>> >
>> > On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
>> > <met_help at ucar.edu>
>> > wrote:
>> >
>> > > Ho-Chun
>> > >
>> > > The log file shows that 0 records was inserted into the
database.
>> > > What is the path to your data on WCOSS and how do you call the
load
>> > > command?
>> > >
>> > > Tatiana
>> > >
>> > > On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov wrote:
>> > > > Hi, Tara:
>> > > >
>> > > >
>> > > > I remove the database and start again
>> > > >
>> > > > The log file is in
>> > > > https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
>> > > >
>> > > > When I run it without piping it to run.log, I did see the
scp
>> > > > working
>> > > > transferring files. For some reason, I did not see it in the
>> > > > logfile.
>> > > >
>> > > > Ho-Chun
>> > > >
>> > > > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
>> > > > <met_help at ucar.edu>
>> > > > wrote:
>> > > >
>> > > > > Ho-Chun
>> > > > >
>> > > > > mv_g2g_met_verf_aod database exists on AWS but it is empty.
>> > > > > Do you have a copy of the log file from the data loading?
>> > > > >
>> > > > > Tatiana
>> > > > >
>> > > > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov wrote:
>> > > > > > Hi, Tara:
>> > > > > >
>> > > > > > I tried
>> > > > > >
>> > > > > > (1) hit reload database
>> > > > > > (2) refresh button
>> > > > > > (3) clear all web browser data
>> > > > > > (4) close the web browser and open it again
>> > > > > >
>> > > > > > None of it works.
>> > > > > >
>> > > > > >
>> > > > > > Can you get on the AWS MEtviewer and be able to see
>> > > > > >
>> > > > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
>> > > > > >
>> > > > > > Ho-Chun
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT <
>> > > met_help at ucar.edu>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Ho-Chun,
>> > > > > > >
>> > > > > > > I see you are trying to view a new database up on AWS.
After
>> > > adding a
>> > > > > > > new
>> > > > > > > database to METviewer, you will need to click on Reload
>> > > > > > > databases
>> > > > > > > above the
>> > > > > > > plotting area. If you still don't see it, consider
clearing
>> > > > > > > your
>> > > > > > > cache and
>> > > > > > > hitting the refresh button. If you are using
METexpress, you
>> > > > > > > may
>> > > not
>> > > > > > > have
>> > > > > > > access to it until tomorrow. I'm not sure how often
they are
>> > > running
>> > > > > > > metadata, which is what is needed to see a new
database.
>> > > > > > >
>> > > > > > > [image: image.png]
>> > > > > > > Let us know if you have tried these things and still
don't
>> > > > > > > see it.
>> > > > > > > We can
>> > > > > > > pass it over to Tatiana at that point.
>> > > > > > > Cheers, Tara
>> > > > > > >
>> > > > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA
>> > > > > > > Affiliate via
>> > > > > > > RT <
>> > > > > > > met_help at ucar.edu> wrote:
>> > > > > > >
>> > > > > > > >
>> > > > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted
upon.
>> > > > > > > > Transaction: Ticket created by ho-chun.huang at noaa.gov
>> > > > > > > > Queue: met_help
>> > > > > > > > Subject: upload database can not being seen in
AWS
>> > > > > > > > Metviewer
>> > > > > > > > Owner: Nobody
>> > > > > > > > Requestors: ho-chun.huang at noaa.gov
>> > > > > > > > Status: new
>> > > > > > > > Ticket <URL:
>> > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > To Whom It May Concern:
>> > > > > > > >
>> > > > > > > > I upload a new database to AWS but can not find it in
>> > > > > > > > MetViewer.
>> > > > > > > > The
>> > > > > > > load
>> > > > > > > > xml and AWS database inquiry is shown below
>> > > > > > > >
>> > > > > > > > The scp files seems working,
>> > > > > > > >
>> > > > > > > > Thank you for your help.
>> > > > > > > >
>> > > > > > > > Ho-Chun
>> > > > > > > >
>> > > > > > > > <load_spec>
>> > > > > > > > <connection>
>> > > > > > > > <host>
>> > > > > > > >
>> > > > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
>> > > > > > > 1.rds.amazonaws.com:3306
>> > > > > > > > </host>
>> > > > > > > > <database>mv_g2g_met_verf_aod</database>
>> > > > > > > > <user>rds_user</user>
>> > > > > > > > <password>rds_pwd</password>
>> > > > > > > > <management_system>aurora</management_system>
>> > > > > > > > </connection>
>> > > > > > > >
>> > > > > > > > <met_version>V8.1.2</met_version>
>> > > > > > > >
>> > > > > > > > <verbose>true</verbose>
>> > > > > > > > <insert_size>1</insert_size>
>> > > > > > > > <mode_header_db_check>true</mode_header_db_check>
>> > > > > > > > <stat_header_db_check>true</stat_header_db_check>
>> > > > > > > > <drop_indexes>false</drop_indexes>
>> > > > > > > > <apply_indexes>true</apply_indexes>
>> > > > > > > > <load_stat>true</load_stat>
>> > > > > > > > <load_mode>true</load_mode>
>> > > > > > > > <load_mpr>true</load_mpr>
>> > > > > > > > <load_orank>true</load_orank>
>> > > > > > > > <force_dup_file>false</force_dup_file>
>> > > > > > > > <group>NCEP_hcHuang</group>
>> > > > > > > > <description>Loading
mv_g2g_met_verf_aod</description>
>> > > > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
>> > > > > > > > <load_val>
>> > > > > > > > <field name="type">
>> > > > > > > > <val>stat</val>
>> > > > > > > > </field>
>> > > > > > > > </load_val>
>> > > > > > > > <load_xml>true</load_xml>
>> > > > > > > > <load_note></load_note>
>> > > > > > > > </load_spec>
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > > mv_db_size_on_aws.sh ho-chun.huang
>> > > > > > > > :::::::::::::::::::::::::::::::::::::::::
>> > > > > > > > :::::::::::::::::::::::::::::::::::::::::
>> > > > > > > > mv_agibbs_nam_hi_dng_FAA
annette.gibbs
>> > > > > > > > 2.28
>> > > > > > > > mv_agibbs_nam_pr_dng_FAA
annette.gibbs
>> > > > > > > > 2.28
>> > > > > > > > emc_cmaq_g2o
perry.shafran
>> > > > > > > > 2.02
>> > > > > > > > emc_cmaq_g2o2
perry.shafran
>> > > > > > > > 2.02
>> > > > > > > > mv_vh
venita.hagerty
>> > > > > > > > 2.02
>> > > > > > > > testvh4
venita.hagerty
>> > > > > > > > 2.02
>> > > > > > > > testvh5
venita.hagerty
>> > > > > > > > 2.02
>> > > > > > > > mv_g2g_met_verf_aod ho-
chun.huang
>> > > > > > > > 1.97
>> > > > > > > > mats_metadata
kenneth.sperow
>> > > > > > > > 1.28
>> > > > > > > > _______________________________
>> > > > > > > > TOTAL_SIZE_GB
>> > > > > > > > 203.36
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > > > > > > Tara Jensen
>> > > > > > > Project Manager II
>> > > > > > > NCAR RAL and DTC
>> > > > > > > PO Box 3000, Boulder, Colorado 80307 USA
>> > > > > > > +1 303-497-8479 jensen at ucar.edu
>> > > > > > >
>> > > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > >
>> > >
>> > >
>> > >
>>
>>
>>
>>
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Tatiana Burek
Time: Thu Sep 26 15:05:26 2019
<folder_tmpl> tag in XML describes the path to data and not the file
name or extension.
In your case it should look like this:
<folder_tmpl>/base_dir/{type}</folder_tmpl>
<load_val>
<field name="type">
<val>verif</val>
</field>
</load_val>
and base_dir is /gpfs/dell2/stmp/Ho-Chun.Huang/
Tatiana
On Thu Sep 26 14:57:56 2019, ho-chun.huang at noaa.gov wrote:
> Hi, Tatiana:
>
> It is the filename in the base_dir contain "stat" /base_dir/{type}
>
> My filename look like
>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>
>
> in base_dir as /gpfs/dell2/stmp/Ho-Chun.Huang/verif
>
> I do not quite understand you question. The directory exist and the
> filename contain "stat"
>
>
> I will leave office in 10 mins to catch the shuttle. If necessary,
I can
> tar the stat file and load xml to ftp directory and you can try it
from
> NCAR machine. Please let me know do you want to do it this way.
>
>
> Ho-Chun
>
> On Thu, Sep 26, 2019 at 4:52 PM Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > In your XML you specified that your data directory contains
'stats":
> >
> > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > <load_val>
> > <field name="type">
> > <val>stat</val>
> > </field>
> > </load_val>
> > But your actual path doesn't have this directory. Is it correct?
> >
> >
> > Tatiana
> >
> > On Thu Sep 26 14:45:53 2019, ho-chun.huang at noaa.gov wrote:
> > > Hi, Tatiana:
> > >
> > > Dev machine is down. I have access to operational machine and
you do
> > > not.
> > > So you can not check my data directory, but as in the log, it
> > > is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
> > >
> > > I do have files in the directory, and the scp did show it was
> > > transferring
> > > files.
> > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
> > >
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> > >
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> > >
> >
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > >
> > >
> > > This is when I try it again to different database name
> > > mv_g2g_met_verf_test
> > >
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> > > 100% 3514 251.1KB/s 00:00
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> > > 100% 3503 411.8KB/s 00:00
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> > > 100% 3515 457.7KB/s 00:00
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> > > 100% 3538 465.2KB/s 00:00
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> > > 100% 3514 248.3KB/s 00:00
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> > > 100% 3537 370.5KB/s 00:00
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> > > 100% 3538 175.2KB/s 00:00
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> > > 100% 3516 416.8KB/s 00:00
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > > 100% 3504 461.5KB/s 00:00
> > > CALLING: scp
> > > /gpfs/dell2/emc/modeling/noscrub/Ho-
> > > Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
> > > ho-chun.huang at 205.156.8.85:~
> > > load_g2g_met_verf_aod.xml
> > > 100% 1019 136.7KB/s 00:00
> > > ---- MVLoad ----
> > >
> > > Begin time: 2019-09-26 20:30:35
> > > Parsing: load_g2g_met_verf_aod.xml
> > >
> > > Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
> > > ==== grid_stat ====
> > >
> > > stat_header table time total: 0:00:00.000
> > > stat header records: 0
> > > stat header inserts: 0
> > > line data records: 0
> > > line data inserts: 0
> > > stat group records: 0
> > > stat group inserts: 0
> > > var length records: 0
> > > var length inserts: 0
> > > total lines: 0
> > > insert size: 1
> > > lines / msec: 0.000
> > > num files: 0
> > >
> > > ==== mode ====
> > >
> > > mode_header search time total: 0:00:00.000
> > > mtd_header search time total: 0:00:00.000
> > > stat_header search time total: 0:00:00.000
> > > mode_header inserts: 0
> > > mode_cts inserts: 0
> > > mode_obj_single inserts: 0
> > > mode_obj_pair inserts: 0
> > > ==== mtd ====
> > >
> > > mtd_header inserts: 0
> > > mtd_3d_obj_single inserts: 0
> > > mtd_3d_obj_pair inserts: 0
> > > mtd_2d_obj inserts: 0
> > > total lines: 0
> > > num files: 0
> > > num files: 0
> > >
> > > ==== indexes ====creating
> > > Indexes creating 0:00:01.002
> > >
> > > End time: 2019-09-26 20:30:36
> > > Load total: 0:00:00.010
> > >
> > > ---- MVLoad Done ----
> > >
> > > Each file does have data in there
> > >
> > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
> > >
> >
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > > VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
FCST_VALID_END
> > > OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS
FCST_LEV
> > > OB
> > > S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD
INTERP_PNTS
> > > FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=0.5
> > > > =0.5 NA NA CTC 17941 0 5 1
> > > 17935
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=1.0
> > > > =1.0 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=1.5
> > > > =1.5 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=2.0
> > > > =2.0 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=2.5
> > > > =2.5 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=3.0
> > > > =3.0 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=3.5
> > > > =3.5 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=4.0
> > > > =4.0 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=4.5
> > > > =4.5 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > >=5.0
> > > > =5.0 NA NA CTC 17941 0 0 0
> > > 17941
> > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > 000000
> > > 20190820_120236 20190820_120236 AOTK Numeric L1 AO
> > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
NA
> > > NA NA NA SL1L2 17941 0.070913 0.1156 0.
> > > 011134 0.008766 0.018307 0.054144
> > >
> > > On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > > Ho-Chun
> > > >
> > > > The log file shows that 0 records was inserted into the
database.
> > > > What is the path to your data on WCOSS and how do you call the
load
> > > > command?
> > > >
> > > > Tatiana
> > > >
> > > > On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov wrote:
> > > > > Hi, Tara:
> > > > >
> > > > >
> > > > > I remove the database and start again
> > > > >
> > > > > The log file is in
> > > > > https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
> > > > >
> > > > > When I run it without piping it to run.log, I did see the
scp
> > > > > working
> > > > > transferring files. For some reason, I did not see it in
the
> > > > > logfile.
> > > > >
> > > > > Ho-Chun
> > > > >
> > > > > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Ho-Chun
> > > > > >
> > > > > > mv_g2g_met_verf_aod database exists on AWS but it is
empty.
> > > > > > Do you have a copy of the log file from the data loading?
> > > > > >
> > > > > > Tatiana
> > > > > >
> > > > > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov wrote:
> > > > > > > Hi, Tara:
> > > > > > >
> > > > > > > I tried
> > > > > > >
> > > > > > > (1) hit reload database
> > > > > > > (2) refresh button
> > > > > > > (3) clear all web browser data
> > > > > > > (4) close the web browser and open it again
> > > > > > >
> > > > > > > None of it works.
> > > > > > >
> > > > > > >
> > > > > > > Can you get on the AWS MEtviewer and be able to see
> > > > > > >
> > > > > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
> > > > > > >
> > > > > > > Ho-Chun
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT <
> > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Ho-Chun,
> > > > > > > >
> > > > > > > > I see you are trying to view a new database up on AWS.
After
> > > > adding a
> > > > > > > > new
> > > > > > > > database to METviewer, you will need to click on
Reload
> > > > > > > > databases
> > > > > > > > above the
> > > > > > > > plotting area. If you still don't see it, consider
clearing
> > > > > > > > your
> > > > > > > > cache and
> > > > > > > > hitting the refresh button. If you are using
METexpress, you
> > > > > > > > may
> > > > not
> > > > > > > > have
> > > > > > > > access to it until tomorrow. I'm not sure how often
they are
> > > > running
> > > > > > > > metadata, which is what is needed to see a new
database.
> > > > > > > >
> > > > > > > > [image: image.png]
> > > > > > > > Let us know if you have tried these things and still
don't
> > > > > > > > see it.
> > > > > > > > We can
> > > > > > > > pass it over to Tatiana at that point.
> > > > > > > > Cheers, Tara
> > > > > > > >
> > > > > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang - NOAA
> > > > > > > > Affiliate via
> > > > > > > > RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted
upon.
> > > > > > > > > Transaction: Ticket created by ho-
chun.huang at noaa.gov
> > > > > > > > > Queue: met_help
> > > > > > > > > Subject: upload database can not being seen in
AWS
> > > > > > > > > Metviewer
> > > > > > > > > Owner: Nobody
> > > > > > > > > Requestors: ho-chun.huang at noaa.gov
> > > > > > > > > Status: new
> > > > > > > > > Ticket <URL:
> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346 >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > To Whom It May Concern:
> > > > > > > > >
> > > > > > > > > I upload a new database to AWS but can not find it
in
> > > > > > > > > MetViewer.
> > > > > > > > > The
> > > > > > > > load
> > > > > > > > > xml and AWS database inquiry is shown below
> > > > > > > > >
> > > > > > > > > The scp files seems working,
> > > > > > > > >
> > > > > > > > > Thank you for your help.
> > > > > > > > >
> > > > > > > > > Ho-Chun
> > > > > > > > >
> > > > > > > > > <load_spec>
> > > > > > > > > <connection>
> > > > > > > > > <host>
> > > > > > > > >
> > > > > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
> > > > > > > > 1.rds.amazonaws.com:3306
> > > > > > > > > </host>
> > > > > > > > > <database>mv_g2g_met_verf_aod</database>
> > > > > > > > > <user>rds_user</user>
> > > > > > > > > <password>rds_pwd</password>
> > > > > > > > > <management_system>aurora</management_system>
> > > > > > > > > </connection>
> > > > > > > > >
> > > > > > > > > <met_version>V8.1.2</met_version>
> > > > > > > > >
> > > > > > > > > <verbose>true</verbose>
> > > > > > > > > <insert_size>1</insert_size>
> > > > > > > > > <mode_header_db_check>true</mode_header_db_check>
> > > > > > > > > <stat_header_db_check>true</stat_header_db_check>
> > > > > > > > > <drop_indexes>false</drop_indexes>
> > > > > > > > > <apply_indexes>true</apply_indexes>
> > > > > > > > > <load_stat>true</load_stat>
> > > > > > > > > <load_mode>true</load_mode>
> > > > > > > > > <load_mpr>true</load_mpr>
> > > > > > > > > <load_orank>true</load_orank>
> > > > > > > > > <force_dup_file>false</force_dup_file>
> > > > > > > > > <group>NCEP_hcHuang</group>
> > > > > > > > > <description>Loading
mv_g2g_met_verf_aod</description>
> > > > > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > > > > > > > <load_val>
> > > > > > > > > <field name="type">
> > > > > > > > > <val>stat</val>
> > > > > > > > > </field>
> > > > > > > > > </load_val>
> > > > > > > > > <load_xml>true</load_xml>
> > > > > > > > > <load_note></load_note>
> > > > > > > > > </load_spec>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > mv_db_size_on_aws.sh ho-chun.huang
> > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > > > > mv_agibbs_nam_hi_dng_FAA
annette.gibbs
> > > > > > > > > 2.28
> > > > > > > > > mv_agibbs_nam_pr_dng_FAA
annette.gibbs
> > > > > > > > > 2.28
> > > > > > > > > emc_cmaq_g2o
perry.shafran
> > > > > > > > > 2.02
> > > > > > > > > emc_cmaq_g2o2
perry.shafran
> > > > > > > > > 2.02
> > > > > > > > > mv_vh
venita.hagerty
> > > > > > > > > 2.02
> > > > > > > > > testvh4
venita.hagerty
> > > > > > > > > 2.02
> > > > > > > > > testvh5
venita.hagerty
> > > > > > > > > 2.02
> > > > > > > > > mv_g2g_met_verf_aod ho-
chun.huang
> > > > > > > > > 1.97
> > > > > > > > > mats_metadata
kenneth.sperow
> > > > > > > > > 1.28
> > > > > > > > > _______________________________
> > > > > > > > > TOTAL_SIZE_GB
> > > > > > > > > 203.36
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > > Tara Jensen
> > > > > > > > Project Manager II
> > > > > > > > NCAR RAL and DTC
> > > > > > > > PO Box 3000, Boulder, Colorado 80307 USA
> > > > > > > > +1 303-497-8479 jensen at ucar.edu
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> >
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Ho-Chun Huang - NOAA Affiliate
Time: Thu Sep 26 15:07:01 2019
Tatiana:
Thanks. I will try it first thing tomorrow
On Thu, Sep 26, 2019 at 5:05 PM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> <folder_tmpl> tag in XML describes the path to data and not the file
name
> or extension.
> In your case it should look like this:
>
> <folder_tmpl>/base_dir/{type}</folder_tmpl>
> <load_val>
> <field name="type">
> <val>verif</val>
> </field>
> </load_val>
>
> and base_dir is /gpfs/dell2/stmp/Ho-Chun.Huang/
>
> Tatiana
>
> On Thu Sep 26 14:57:56 2019, ho-chun.huang at noaa.gov wrote:
> > Hi, Tatiana:
> >
> > It is the filename in the base_dir contain "stat"
/base_dir/{type}
> >
> > My filename look like
> >
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> >
> >
> > in base_dir as /gpfs/dell2/stmp/Ho-Chun.Huang/verif
> >
> > I do not quite understand you question. The directory exist and
the
> > filename contain "stat"
> >
> >
> > I will leave office in 10 mins to catch the shuttle. If
necessary, I can
> > tar the stat file and load xml to ftp directory and you can try it
from
> > NCAR machine. Please let me know do you want to do it this way.
> >
> >
> > Ho-Chun
> >
> > On Thu, Sep 26, 2019 at 4:52 PM Tatiana Burek via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > In your XML you specified that your data directory contains
'stats":
> > >
> > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > <load_val>
> > > <field name="type">
> > > <val>stat</val>
> > > </field>
> > > </load_val>
> > > But your actual path doesn't have this directory. Is it correct?
> > >
> > >
> > > Tatiana
> > >
> > > On Thu Sep 26 14:45:53 2019, ho-chun.huang at noaa.gov wrote:
> > > > Hi, Tatiana:
> > > >
> > > > Dev machine is down. I have access to operational machine and
you do
> > > > not.
> > > > So you can not check my data directory, but as in the log, it
> > > > is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
> > > >
> > > > I do have files in the directory, and the scp did show it was
> > > > transferring
> > > > files.
> > > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
> > > >
>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> > > >
>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> > > >
> > >
>
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > > >
> > > >
> > > > This is when I try it again to different database name
> > > > mv_g2g_met_verf_test
> > > >
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> > > > 100% 3514 251.1KB/s 00:00
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> > > > 100% 3503 411.8KB/s 00:00
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> > > > 100% 3515 457.7KB/s 00:00
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> > > > 100% 3538 465.2KB/s 00:00
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> > > > 100% 3514 248.3KB/s 00:00
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> > > > 100% 3537 370.5KB/s 00:00
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> > > > 100% 3538 175.2KB/s 00:00
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> > > > 100% 3516 416.8KB/s 00:00
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > > > 100% 3504 461.5KB/s 00:00
> > > > CALLING: scp
> > > > /gpfs/dell2/emc/modeling/noscrub/Ho-
> > > > Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
> > > > ho-chun.huang at 205.156.8.85:~
> > > > load_g2g_met_verf_aod.xml
> > > > 100% 1019 136.7KB/s 00:00
> > > > ---- MVLoad ----
> > > >
> > > > Begin time: 2019-09-26 20:30:35
> > > > Parsing: load_g2g_met_verf_aod.xml
> > > >
> > > > Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
> > > > ==== grid_stat ====
> > > >
> > > > stat_header table time total: 0:00:00.000
> > > > stat header records: 0
> > > > stat header inserts: 0
> > > > line data records: 0
> > > > line data inserts: 0
> > > > stat group records: 0
> > > > stat group inserts: 0
> > > > var length records: 0
> > > > var length inserts: 0
> > > > total lines: 0
> > > > insert size: 1
> > > > lines / msec: 0.000
> > > > num files: 0
> > > >
> > > > ==== mode ====
> > > >
> > > > mode_header search time total: 0:00:00.000
> > > > mtd_header search time total: 0:00:00.000
> > > > stat_header search time total: 0:00:00.000
> > > > mode_header inserts: 0
> > > > mode_cts inserts: 0
> > > > mode_obj_single inserts: 0
> > > > mode_obj_pair inserts: 0
> > > > ==== mtd ====
> > > >
> > > > mtd_header inserts: 0
> > > > mtd_3d_obj_single inserts: 0
> > > > mtd_3d_obj_pair inserts: 0
> > > > mtd_2d_obj inserts: 0
> > > > total lines: 0
> > > > num files: 0
> > > > num files: 0
> > > >
> > > > ==== indexes ====creating
> > > > Indexes creating 0:00:01.002
> > > >
> > > > End time: 2019-09-26 20:30:36
> > > > Load total: 0:00:00.010
> > > >
> > > > ---- MVLoad Done ----
> > > >
> > > > Each file does have data in there
> > > >
> > > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
> > > >
> > >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > > > VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
FCST_VALID_END
> > > > OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS
> FCST_LEV
> > > > OB
> > > > S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD
INTERP_PNTS
> > > > FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=0.5
> > > > > =0.5 NA NA CTC 17941 0 5 1
> > > > 17935
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=1.0
> > > > > =1.0 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=1.5
> > > > > =1.5 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=2.0
> > > > > =2.0 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=2.5
> > > > > =2.5 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=3.0
> > > > > =3.0 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=3.5
> > > > > =3.5 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=4.0
> > > > > =4.0 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=4.5
> > > > > =4.5 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > > > >=5.0
> > > > > =5.0 NA NA CTC 17941 0 0 0
> > > > 17941
> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
> > > > 000000
> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
NA
> > > > NA NA NA SL1L2 17941 0.070913 0.1156 0.
> > > > 011134 0.008766 0.018307 0.054144
> > > >
> > > > On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Ho-Chun
> > > > >
> > > > > The log file shows that 0 records was inserted into the
database.
> > > > > What is the path to your data on WCOSS and how do you call
the load
> > > > > command?
> > > > >
> > > > > Tatiana
> > > > >
> > > > > On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov wrote:
> > > > > > Hi, Tara:
> > > > > >
> > > > > >
> > > > > > I remove the database and start again
> > > > > >
> > > > > > The log file is in
> > > > > > https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
> > > > > >
> > > > > > When I run it without piping it to run.log, I did see the
scp
> > > > > > working
> > > > > > transferring files. For some reason, I did not see it in
the
> > > > > > logfile.
> > > > > >
> > > > > > Ho-Chun
> > > > > >
> > > > > > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
> > > > > > <met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Ho-Chun
> > > > > > >
> > > > > > > mv_g2g_met_verf_aod database exists on AWS but it is
empty.
> > > > > > > Do you have a copy of the log file from the data
loading?
> > > > > > >
> > > > > > > Tatiana
> > > > > > >
> > > > > > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov
wrote:
> > > > > > > > Hi, Tara:
> > > > > > > >
> > > > > > > > I tried
> > > > > > > >
> > > > > > > > (1) hit reload database
> > > > > > > > (2) refresh button
> > > > > > > > (3) clear all web browser data
> > > > > > > > (4) close the web browser and open it again
> > > > > > > >
> > > > > > > > None of it works.
> > > > > > > >
> > > > > > > >
> > > > > > > > Can you get on the AWS MEtviewer and be able to see
> > > > > > > >
> > > > > > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
> > > > > > > >
> > > > > > > > Ho-Chun
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT <
> > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Ho-Chun,
> > > > > > > > >
> > > > > > > > > I see you are trying to view a new database up on
AWS.
> After
> > > > > adding a
> > > > > > > > > new
> > > > > > > > > database to METviewer, you will need to click on
Reload
> > > > > > > > > databases
> > > > > > > > > above the
> > > > > > > > > plotting area. If you still don't see it, consider
> clearing
> > > > > > > > > your
> > > > > > > > > cache and
> > > > > > > > > hitting the refresh button. If you are using
METexpress,
> you
> > > > > > > > > may
> > > > > not
> > > > > > > > > have
> > > > > > > > > access to it until tomorrow. I'm not sure how often
they
> are
> > > > > running
> > > > > > > > > metadata, which is what is needed to see a new
database.
> > > > > > > > >
> > > > > > > > > [image: image.png]
> > > > > > > > > Let us know if you have tried these things and still
don't
> > > > > > > > > see it.
> > > > > > > > > We can
> > > > > > > > > pass it over to Tatiana at that point.
> > > > > > > > > Cheers, Tara
> > > > > > > > >
> > > > > > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang -
NOAA
> > > > > > > > > Affiliate via
> > > > > > > > > RT <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted
upon.
> > > > > > > > > > Transaction: Ticket created by ho-
chun.huang at noaa.gov
> > > > > > > > > > Queue: met_help
> > > > > > > > > > Subject: upload database can not being seen
in AWS
> > > > > > > > > > Metviewer
> > > > > > > > > > Owner: Nobody
> > > > > > > > > > Requestors: ho-chun.huang at noaa.gov
> > > > > > > > > > Status: new
> > > > > > > > > > Ticket <URL:
> > > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346
> >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > To Whom It May Concern:
> > > > > > > > > >
> > > > > > > > > > I upload a new database to AWS but can not find it
in
> > > > > > > > > > MetViewer.
> > > > > > > > > > The
> > > > > > > > > load
> > > > > > > > > > xml and AWS database inquiry is shown below
> > > > > > > > > >
> > > > > > > > > > The scp files seems working,
> > > > > > > > > >
> > > > > > > > > > Thank you for your help.
> > > > > > > > > >
> > > > > > > > > > Ho-Chun
> > > > > > > > > >
> > > > > > > > > > <load_spec>
> > > > > > > > > > <connection>
> > > > > > > > > > <host>
> > > > > > > > > >
> > > > > > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
> > > > > > > > > 1.rds.amazonaws.com:3306
> > > > > > > > > > </host>
> > > > > > > > > > <database>mv_g2g_met_verf_aod</database>
> > > > > > > > > > <user>rds_user</user>
> > > > > > > > > > <password>rds_pwd</password>
> > > > > > > > > > <management_system>aurora</management_system>
> > > > > > > > > > </connection>
> > > > > > > > > >
> > > > > > > > > > <met_version>V8.1.2</met_version>
> > > > > > > > > >
> > > > > > > > > > <verbose>true</verbose>
> > > > > > > > > > <insert_size>1</insert_size>
> > > > > > > > > > <mode_header_db_check>true</mode_header_db_check>
> > > > > > > > > > <stat_header_db_check>true</stat_header_db_check>
> > > > > > > > > > <drop_indexes>false</drop_indexes>
> > > > > > > > > > <apply_indexes>true</apply_indexes>
> > > > > > > > > > <load_stat>true</load_stat>
> > > > > > > > > > <load_mode>true</load_mode>
> > > > > > > > > > <load_mpr>true</load_mpr>
> > > > > > > > > > <load_orank>true</load_orank>
> > > > > > > > > > <force_dup_file>false</force_dup_file>
> > > > > > > > > > <group>NCEP_hcHuang</group>
> > > > > > > > > > <description>Loading
mv_g2g_met_verf_aod</description>
> > > > > > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > > > > > > > > > <load_val>
> > > > > > > > > > <field name="type">
> > > > > > > > > > <val>stat</val>
> > > > > > > > > > </field>
> > > > > > > > > > </load_val>
> > > > > > > > > > <load_xml>true</load_xml>
> > > > > > > > > > <load_note></load_note>
> > > > > > > > > > </load_spec>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > mv_db_size_on_aws.sh ho-chun.huang
> > > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > > > > > > > > > mv_agibbs_nam_hi_dng_FAA
annette.gibbs
> > > > > > > > > > 2.28
> > > > > > > > > > mv_agibbs_nam_pr_dng_FAA
annette.gibbs
> > > > > > > > > > 2.28
> > > > > > > > > > emc_cmaq_g2o
perry.shafran
> > > > > > > > > > 2.02
> > > > > > > > > > emc_cmaq_g2o2
perry.shafran
> > > > > > > > > > 2.02
> > > > > > > > > > mv_vh
venita.hagerty
> > > > > > > > > > 2.02
> > > > > > > > > > testvh4
venita.hagerty
> > > > > > > > > > 2.02
> > > > > > > > > > testvh5
venita.hagerty
> > > > > > > > > > 2.02
> > > > > > > > > > mv_g2g_met_verf_aod ho-
chun.huang
> > > > > > > > > > 1.97
> > > > > > > > > > mats_metadata
kenneth.sperow
> > > > > > > > > > 1.28
> > > > > > > > > > _______________________________
> > > > > > > > > > TOTAL_SIZE_GB
> > > > > > > > > > 203.36
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > > > Tara Jensen
> > > > > > > > > Project Manager II
> > > > > > > > > NCAR RAL and DTC
> > > > > > > > > PO Box 3000, Boulder, Colorado 80307 USA
> > > > > > > > > +1 303-497-8479 jensen at ucar.edu
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Ho-Chun Huang - NOAA Affiliate
Time: Fri Sep 27 05:32:24 2019
Hi, Tatiana:
Good morning, I tried and result is the same.
I put latest run_script, load_xml, and log message on
https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/
Please ignore the file type of run_g2g_met_verf_aod.sh. I need to
change
it for it to be viewed.
Any suggestion of modification? Thank you for the help.
Ho-Chun
p.s. during execution I did see file been transferred (although not
in
run.log that I used ./run_g2g_met_verf_aod.sh > run.log 2>&1)
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
100% 3514 255.5KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
100% 3537 259.1KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
100% 3538 463.4KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
100% 3516 247.3KB/s 00:00
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
100% 3504 466.9KB/s 00:00
CALLING: scp
/gpfs/dell2/emc/modeling/noscrub/Ho-
Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
ho-chun.huang at 205.156.8.85:~
load_g2g_met_verf_aod.xml
100% 1019 106.6KB/s 00:00
I wonder whether you can tell whether there are files
in //data/mv_data/ho-chun.huang//verif ?
On Thu, Sep 26, 2019 at 5:06 PM Ho-Chun Huang - NOAA Affiliate <
ho-chun.huang at noaa.gov> wrote:
> Tatiana:
>
> Thanks. I will try it first thing tomorrow
>
> On Thu, Sep 26, 2019 at 5:05 PM Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
>> <folder_tmpl> tag in XML describes the path to data and not the
file name
>> or extension.
>> In your case it should look like this:
>>
>> <folder_tmpl>/base_dir/{type}</folder_tmpl>
>> <load_val>
>> <field name="type">
>> <val>verif</val>
>> </field>
>> </load_val>
>>
>> and base_dir is /gpfs/dell2/stmp/Ho-Chun.Huang/
>>
>> Tatiana
>>
>> On Thu Sep 26 14:57:56 2019, ho-chun.huang at noaa.gov wrote:
>> > Hi, Tatiana:
>> >
>> > It is the filename in the base_dir contain "stat"
/base_dir/{type}
>> >
>> > My filename look like
>> >
>> >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>> >
>> >
>> > in base_dir as /gpfs/dell2/stmp/Ho-Chun.Huang/verif
>> >
>> > I do not quite understand you question. The directory exist and
the
>> > filename contain "stat"
>> >
>> >
>> > I will leave office in 10 mins to catch the shuttle. If
necessary, I
>> can
>> > tar the stat file and load xml to ftp directory and you can try
it from
>> > NCAR machine. Please let me know do you want to do it this way.
>> >
>> >
>> > Ho-Chun
>> >
>> > On Thu, Sep 26, 2019 at 4:52 PM Tatiana Burek via RT
<met_help at ucar.edu
>> >
>> > wrote:
>> >
>> > >
>> > > In your XML you specified that your data directory contains
'stats":
>> > >
>> > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
>> > > <load_val>
>> > > <field name="type">
>> > > <val>stat</val>
>> > > </field>
>> > > </load_val>
>> > > But your actual path doesn't have this directory. Is it
correct?
>> > >
>> > >
>> > > Tatiana
>> > >
>> > > On Thu Sep 26 14:45:53 2019, ho-chun.huang at noaa.gov wrote:
>> > > > Hi, Tatiana:
>> > > >
>> > > > Dev machine is down. I have access to operational machine
and you
>> do
>> > > > not.
>> > > > So you can not check my data directory, but as in the log, it
>> > > > is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
>> > > >
>> > > > I do have files in the directory, and the scp did show it was
>> > > > transferring
>> > > > files.
>> > > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
>> > > >
>>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>> > > >
>>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>> > > >
>> > > >
>> > > > This is when I try it again to different database name
>> > > > mv_g2g_met_verf_test
>> > > >
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
>> > > > 100% 3514 251.1KB/s 00:00
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
>> > > > 100% 3503 411.8KB/s 00:00
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
>> > > > 100% 3515 457.7KB/s 00:00
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
>> > > > 100% 3538 465.2KB/s 00:00
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
>> > > > 100% 3514 248.3KB/s 00:00
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
>> > > > 100% 3537 370.5KB/s 00:00
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
>> > > > 100% 3538 175.2KB/s 00:00
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
>> > > > 100% 3516 416.8KB/s 00:00
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>> > > > 100% 3504 461.5KB/s 00:00
>> > > > CALLING: scp
>> > > > /gpfs/dell2/emc/modeling/noscrub/Ho-
>> > > > Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
>> > > > ho-chun.huang at 205.156.8.85:~
>> > > > load_g2g_met_verf_aod.xml
>> > > > 100% 1019 136.7KB/s 00:00
>> > > > ---- MVLoad ----
>> > > >
>> > > > Begin time: 2019-09-26 20:30:35
>> > > > Parsing: load_g2g_met_verf_aod.xml
>> > > >
>> > > > Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
>> > > > ==== grid_stat ====
>> > > >
>> > > > stat_header table time total: 0:00:00.000
>> > > > stat header records: 0
>> > > > stat header inserts: 0
>> > > > line data records: 0
>> > > > line data inserts: 0
>> > > > stat group records: 0
>> > > > stat group inserts: 0
>> > > > var length records: 0
>> > > > var length inserts: 0
>> > > > total lines: 0
>> > > > insert size: 1
>> > > > lines / msec: 0.000
>> > > > num files: 0
>> > > >
>> > > > ==== mode ====
>> > > >
>> > > > mode_header search time total: 0:00:00.000
>> > > > mtd_header search time total: 0:00:00.000
>> > > > stat_header search time total: 0:00:00.000
>> > > > mode_header inserts: 0
>> > > > mode_cts inserts: 0
>> > > > mode_obj_single inserts: 0
>> > > > mode_obj_pair inserts: 0
>> > > > ==== mtd ====
>> > > >
>> > > > mtd_header inserts: 0
>> > > > mtd_3d_obj_single inserts: 0
>> > > > mtd_3d_obj_pair inserts: 0
>> > > > mtd_2d_obj inserts: 0
>> > > > total lines: 0
>> > > > num files: 0
>> > > > num files: 0
>> > > >
>> > > > ==== indexes ====creating
>> > > > Indexes creating 0:00:01.002
>> > > >
>> > > > End time: 2019-09-26 20:30:36
>> > > > Load total: 0:00:00.010
>> > > >
>> > > > ---- MVLoad Done ----
>> > > >
>> > > > Each file does have data in there
>> > > >
>> > > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
>> > > >
>> > >
>>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
>> > > > VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
FCST_VALID_END
>> > > > OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS
>> FCST_LEV
>> > > > OB
>> > > > S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD
INTERP_PNTS
>> > > > FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=0.5
>> > > > > =0.5 NA NA CTC 17941 0 5
1
>> > > > 17935
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=1.0
>> > > > > =1.0 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=1.5
>> > > > > =1.5 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=2.0
>> > > > > =2.0 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=2.5
>> > > > > =2.5 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=3.0
>> > > > > =3.0 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=3.5
>> > > > > =3.5 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=4.0
>> > > > > =4.0 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=4.5
>> > > > > =4.5 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
>> > > > >=5.0
>> > > > > =5.0 NA NA CTC 17941 0 0
0
>> > > > 17941
>> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
20190820_120000
>> > > > 000000
>> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
AO
>> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
NA
>> > > > NA NA NA SL1L2 17941 0.070913 0.1156
0.
>> > > > 011134 0.008766 0.018307 0.054144
>> > > >
>> > > > On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
>> > > > <met_help at ucar.edu>
>> > > > wrote:
>> > > >
>> > > > > Ho-Chun
>> > > > >
>> > > > > The log file shows that 0 records was inserted into the
database.
>> > > > > What is the path to your data on WCOSS and how do you call
the
>> load
>> > > > > command?
>> > > > >
>> > > > > Tatiana
>> > > > >
>> > > > > On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov wrote:
>> > > > > > Hi, Tara:
>> > > > > >
>> > > > > >
>> > > > > > I remove the database and start again
>> > > > > >
>> > > > > > The log file is in
>> > > > > >
https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
>> > > > > >
>> > > > > > When I run it without piping it to run.log, I did see
the scp
>> > > > > > working
>> > > > > > transferring files. For some reason, I did not see it in
the
>> > > > > > logfile.
>> > > > > >
>> > > > > > Ho-Chun
>> > > > > >
>> > > > > > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
>> > > > > > <met_help at ucar.edu>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Ho-Chun
>> > > > > > >
>> > > > > > > mv_g2g_met_verf_aod database exists on AWS but it is
empty.
>> > > > > > > Do you have a copy of the log file from the data
loading?
>> > > > > > >
>> > > > > > > Tatiana
>> > > > > > >
>> > > > > > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov
wrote:
>> > > > > > > > Hi, Tara:
>> > > > > > > >
>> > > > > > > > I tried
>> > > > > > > >
>> > > > > > > > (1) hit reload database
>> > > > > > > > (2) refresh button
>> > > > > > > > (3) clear all web browser data
>> > > > > > > > (4) close the web browser and open it again
>> > > > > > > >
>> > > > > > > > None of it works.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Can you get on the AWS MEtviewer and be able to see
>> > > > > > > >
>> > > > > > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
>> > > > > > > >
>> > > > > > > > Ho-Chun
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT <
>> > > > > met_help at ucar.edu>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Ho-Chun,
>> > > > > > > > >
>> > > > > > > > > I see you are trying to view a new database up on
AWS.
>> After
>> > > > > adding a
>> > > > > > > > > new
>> > > > > > > > > database to METviewer, you will need to click on
Reload
>> > > > > > > > > databases
>> > > > > > > > > above the
>> > > > > > > > > plotting area. If you still don't see it, consider
>> clearing
>> > > > > > > > > your
>> > > > > > > > > cache and
>> > > > > > > > > hitting the refresh button. If you are using
METexpress,
>> you
>> > > > > > > > > may
>> > > > > not
>> > > > > > > > > have
>> > > > > > > > > access to it until tomorrow. I'm not sure how
often they
>> are
>> > > > > running
>> > > > > > > > > metadata, which is what is needed to see a new
database.
>> > > > > > > > >
>> > > > > > > > > [image: image.png]
>> > > > > > > > > Let us know if you have tried these things and
still don't
>> > > > > > > > > see it.
>> > > > > > > > > We can
>> > > > > > > > > pass it over to Tatiana at that point.
>> > > > > > > > > Cheers, Tara
>> > > > > > > > >
>> > > > > > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang -
NOAA
>> > > > > > > > > Affiliate via
>> > > > > > > > > RT <
>> > > > > > > > > met_help at ucar.edu> wrote:
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was acted
upon.
>> > > > > > > > > > Transaction: Ticket created by ho-
chun.huang at noaa.gov
>> > > > > > > > > > Queue: met_help
>> > > > > > > > > > Subject: upload database can not being seen
in AWS
>> > > > > > > > > > Metviewer
>> > > > > > > > > > Owner: Nobody
>> > > > > > > > > > Requestors: ho-chun.huang at noaa.gov
>> > > > > > > > > > Status: new
>> > > > > > > > > > Ticket <URL:
>> > > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346
>> >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > To Whom It May Concern:
>> > > > > > > > > >
>> > > > > > > > > > I upload a new database to AWS but can not find
it in
>> > > > > > > > > > MetViewer.
>> > > > > > > > > > The
>> > > > > > > > > load
>> > > > > > > > > > xml and AWS database inquiry is shown below
>> > > > > > > > > >
>> > > > > > > > > > The scp files seems working,
>> > > > > > > > > >
>> > > > > > > > > > Thank you for your help.
>> > > > > > > > > >
>> > > > > > > > > > Ho-Chun
>> > > > > > > > > >
>> > > > > > > > > > <load_spec>
>> > > > > > > > > > <connection>
>> > > > > > > > > > <host>
>> > > > > > > > > >
>> > > > > > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-east-
>> > > > > > > > > 1.rds.amazonaws.com:3306
>> > > > > > > > > > </host>
>> > > > > > > > > > <database>mv_g2g_met_verf_aod</database>
>> > > > > > > > > > <user>rds_user</user>
>> > > > > > > > > > <password>rds_pwd</password>
>> > > > > > > > > > <management_system>aurora</management_system>
>> > > > > > > > > > </connection>
>> > > > > > > > > >
>> > > > > > > > > > <met_version>V8.1.2</met_version>
>> > > > > > > > > >
>> > > > > > > > > > <verbose>true</verbose>
>> > > > > > > > > > <insert_size>1</insert_size>
>> > > > > > > > > > <mode_header_db_check>true</mode_header_db_check>
>> > > > > > > > > > <stat_header_db_check>true</stat_header_db_check>
>> > > > > > > > > > <drop_indexes>false</drop_indexes>
>> > > > > > > > > > <apply_indexes>true</apply_indexes>
>> > > > > > > > > > <load_stat>true</load_stat>
>> > > > > > > > > > <load_mode>true</load_mode>
>> > > > > > > > > > <load_mpr>true</load_mpr>
>> > > > > > > > > > <load_orank>true</load_orank>
>> > > > > > > > > > <force_dup_file>false</force_dup_file>
>> > > > > > > > > > <group>NCEP_hcHuang</group>
>> > > > > > > > > > <description>Loading
mv_g2g_met_verf_aod</description>
>> > > > > > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
>> > > > > > > > > > <load_val>
>> > > > > > > > > > <field name="type">
>> > > > > > > > > > <val>stat</val>
>> > > > > > > > > > </field>
>> > > > > > > > > > </load_val>
>> > > > > > > > > > <load_xml>true</load_xml>
>> > > > > > > > > > <load_note></load_note>
>> > > > > > > > > > </load_spec>
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > mv_db_size_on_aws.sh ho-chun.huang
>> > > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
>> > > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
>> > > > > > > > > > mv_agibbs_nam_hi_dng_FAA
annette.gibbs
>> > > > > > > > > > 2.28
>> > > > > > > > > > mv_agibbs_nam_pr_dng_FAA
annette.gibbs
>> > > > > > > > > > 2.28
>> > > > > > > > > > emc_cmaq_g2o
perry.shafran
>> > > > > > > > > > 2.02
>> > > > > > > > > > emc_cmaq_g2o2
perry.shafran
>> > > > > > > > > > 2.02
>> > > > > > > > > > mv_vh
venita.hagerty
>> > > > > > > > > > 2.02
>> > > > > > > > > > testvh4
venita.hagerty
>> > > > > > > > > > 2.02
>> > > > > > > > > > testvh5
venita.hagerty
>> > > > > > > > > > 2.02
>> > > > > > > > > > mv_g2g_met_verf_aod ho-
chun.huang
>> > > > > > > > > > 1.97
>> > > > > > > > > > mats_metadata
kenneth.sperow
>> > > > > > > > > > 1.28
>> > > > > > > > > > _______________________________
>> > > > > > > > > > TOTAL_SIZE_GB
>> > > > > > > > > > 203.36
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > > > > > > > > Tara Jensen
>> > > > > > > > > Project Manager II
>> > > > > > > > > NCAR RAL and DTC
>> > > > > > > > > PO Box 3000, Boulder, Colorado 80307 USA
>> > > > > > > > > +1 303-497-8479 jensen at ucar.edu
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > >
>> > >
>> > >
>> > >
>>
>>
>>
>>
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Tatiana Burek
Time: Fri Sep 27 08:31:03 2019
Hi Ho-Chun
I see that your BASE_DIR parameterr is
BASE_DIR=/gpfs/dell2/stmp/Ho-Chun.Huang/verif
In your case it should be
BASE_DIR=/gpfs/dell2/stmp/Ho-Chun.Huang/
Tatiana
On Fri Sep 27 05:32:24 2019, ho-chun.huang at noaa.gov wrote:
> Hi, Tatiana:
>
> Good morning, I tried and result is the same.
>
> I put latest run_script, load_xml, and log message on
> https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/
>
> Please ignore the file type of run_g2g_met_verf_aod.sh. I need to
> change
> it for it to be viewed.
>
> Any suggestion of modification? Thank you for the help.
>
> Ho-Chun
>
> p.s. during execution I did see file been transferred (although not
> in
> run.log that I used ./run_g2g_met_verf_aod.sh > run.log 2>&1)
>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> 100% 3514 255.5KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> 100% 3537 259.1KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> 100% 3538 463.4KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> 100% 3516 247.3KB/s 00:00
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> 100% 3504 466.9KB/s 00:00
> CALLING: scp
> /gpfs/dell2/emc/modeling/noscrub/Ho-
> Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
> ho-chun.huang at 205.156.8.85:~
> load_g2g_met_verf_aod.xml
> 100% 1019 106.6KB/s 00:00
>
> I wonder whether you can tell whether there are files
> in //data/mv_data/ho-chun.huang//verif ?
>
> On Thu, Sep 26, 2019 at 5:06 PM Ho-Chun Huang - NOAA Affiliate <
> ho-chun.huang at noaa.gov> wrote:
>
> > Tatiana:
> >
> > Thanks. I will try it first thing tomorrow
> >
> > On Thu, Sep 26, 2019 at 5:05 PM Tatiana Burek via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> >> <folder_tmpl> tag in XML describes the path to data and not the
file
> >> name
> >> or extension.
> >> In your case it should look like this:
> >>
> >> <folder_tmpl>/base_dir/{type}</folder_tmpl>
> >> <load_val>
> >> <field name="type">
> >> <val>verif</val>
> >> </field>
> >> </load_val>
> >>
> >> and base_dir is /gpfs/dell2/stmp/Ho-Chun.Huang/
> >>
> >> Tatiana
> >>
> >> On Thu Sep 26 14:57:56 2019, ho-chun.huang at noaa.gov wrote:
> >> > Hi, Tatiana:
> >> >
> >> > It is the filename in the base_dir contain "stat"
> >> > /base_dir/{type}
> >> >
> >> > My filename look like
> >> >
> >> >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> >> >
> >> >
> >> > in base_dir as /gpfs/dell2/stmp/Ho-Chun.Huang/verif
> >> >
> >> > I do not quite understand you question. The directory exist
and
> >> > the
> >> > filename contain "stat"
> >> >
> >> >
> >> > I will leave office in 10 mins to catch the shuttle. If
> >> > necessary, I
> >> can
> >> > tar the stat file and load xml to ftp directory and you can try
it
> >> > from
> >> > NCAR machine. Please let me know do you want to do it this
way.
> >> >
> >> >
> >> > Ho-Chun
> >> >
> >> > On Thu, Sep 26, 2019 at 4:52 PM Tatiana Burek via RT
> >> > <met_help at ucar.edu
> >> >
> >> > wrote:
> >> >
> >> > >
> >> > > In your XML you specified that your data directory contains
> >> > > 'stats":
> >> > >
> >> > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> >> > > <load_val>
> >> > > <field name="type">
> >> > > <val>stat</val>
> >> > > </field>
> >> > > </load_val>
> >> > > But your actual path doesn't have this directory. Is it
correct?
> >> > >
> >> > >
> >> > > Tatiana
> >> > >
> >> > > On Thu Sep 26 14:45:53 2019, ho-chun.huang at noaa.gov wrote:
> >> > > > Hi, Tatiana:
> >> > > >
> >> > > > Dev machine is down. I have access to operational machine
and
> >> > > > you
> >> do
> >> > > > not.
> >> > > > So you can not check my data directory, but as in the log,
it
> >> > > > is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
> >> > > >
> >> > > > I do have files in the directory, and the scp did show it
was
> >> > > > transferring
> >> > > > files.
> >> > > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
> >> > > >
> >>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> >> > > >
> >>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> >> > > >
> >> > > >
> >> > > > This is when I try it again to different database name
> >> > > > mv_g2g_met_verf_test
> >> > > >
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> >> > > > 100% 3514 251.1KB/s 00:00
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> >> > > > 100% 3503 411.8KB/s 00:00
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> >> > > > 100% 3515 457.7KB/s 00:00
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> >> > > > 100% 3538 465.2KB/s 00:00
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> >> > > > 100% 3514 248.3KB/s 00:00
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> >> > > > 100% 3537 370.5KB/s 00:00
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> >> > > > 100% 3538 175.2KB/s 00:00
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> >> > > > 100% 3516 416.8KB/s 00:00
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> >> > > > 100% 3504 461.5KB/s 00:00
> >> > > > CALLING: scp
> >> > > > /gpfs/dell2/emc/modeling/noscrub/Ho-
> >> > > > Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
> >> > > > ho-chun.huang at 205.156.8.85:~
> >> > > > load_g2g_met_verf_aod.xml
> >> > > > 100% 1019 136.7KB/s 00:00
> >> > > > ---- MVLoad ----
> >> > > >
> >> > > > Begin time: 2019-09-26 20:30:35
> >> > > > Parsing: load_g2g_met_verf_aod.xml
> >> > > >
> >> > > > Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
> >> > > > ==== grid_stat ====
> >> > > >
> >> > > > stat_header table time total: 0:00:00.000
> >> > > > stat header records: 0
> >> > > > stat header inserts: 0
> >> > > > line data records: 0
> >> > > > line data inserts: 0
> >> > > > stat group records: 0
> >> > > > stat group inserts: 0
> >> > > > var length records: 0
> >> > > > var length inserts: 0
> >> > > > total lines: 0
> >> > > > insert size: 1
> >> > > > lines / msec: 0.000
> >> > > > num files: 0
> >> > > >
> >> > > > ==== mode ====
> >> > > >
> >> > > > mode_header search time total: 0:00:00.000
> >> > > > mtd_header search time total: 0:00:00.000
> >> > > > stat_header search time total: 0:00:00.000
> >> > > > mode_header inserts: 0
> >> > > > mode_cts inserts: 0
> >> > > > mode_obj_single inserts: 0
> >> > > > mode_obj_pair inserts: 0
> >> > > > ==== mtd ====
> >> > > >
> >> > > > mtd_header inserts: 0
> >> > > > mtd_3d_obj_single inserts: 0
> >> > > > mtd_3d_obj_pair inserts: 0
> >> > > > mtd_2d_obj inserts: 0
> >> > > > total lines: 0
> >> > > > num files: 0
> >> > > > num files: 0
> >> > > >
> >> > > > ==== indexes ====creating
> >> > > > Indexes creating 0:00:01.002
> >> > > >
> >> > > > End time: 2019-09-26 20:30:36
> >> > > > Load total: 0:00:00.010
> >> > > >
> >> > > > ---- MVLoad Done ----
> >> > > >
> >> > > > Each file does have data in there
> >> > > >
> >> > > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
> >> > > >
> >> > >
> >>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> >> > > > VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
> >> > > > FCST_VALID_END
> >> > > > OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR
FCST_UNITS
> >> FCST_LEV
> >> > > > OB
> >> > > > S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD
> >> > > > INTERP_PNTS
> >> > > > FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =0.5
> >> > > > > =0.5 NA NA CTC 17941 0 5
> >> > > > > 1
> >> > > > 17935
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =1.0
> >> > > > > =1.0 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =1.5
> >> > > > > =1.5 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =2.0
> >> > > > > =2.0 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =2.5
> >> > > > > =2.5 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =3.0
> >> > > > > =3.0 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =3.5
> >> > > > > =3.5 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =4.0
> >> > > > > =4.0 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =4.5
> >> > > > > =4.5 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > > =5.0
> >> > > > > =5.0 NA NA CTC 17941 0 0
> >> > > > > 0
> >> > > > 17941
> >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> >> > > > 20190820_120000
> >> > > > 000000
> >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> >> > > > AO
> >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> >> > > > NA
> >> > > > NA NA NA SL1L2 17941 0.070913 0.1156
0.
> >> > > > 011134 0.008766 0.018307 0.054144
> >> > > >
> >> > > > On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
> >> > > > <met_help at ucar.edu>
> >> > > > wrote:
> >> > > >
> >> > > > > Ho-Chun
> >> > > > >
> >> > > > > The log file shows that 0 records was inserted into the
> >> > > > > database.
> >> > > > > What is the path to your data on WCOSS and how do you
call
> >> > > > > the
> >> load
> >> > > > > command?
> >> > > > >
> >> > > > > Tatiana
> >> > > > >
> >> > > > > On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov
wrote:
> >> > > > > > Hi, Tara:
> >> > > > > >
> >> > > > > >
> >> > > > > > I remove the database and start again
> >> > > > > >
> >> > > > > > The log file is in
> >> > > > > >
https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
> >> > > > > >
> >> > > > > > When I run it without piping it to run.log, I did see
the
> >> > > > > > scp
> >> > > > > > working
> >> > > > > > transferring files. For some reason, I did not see it
in
> >> > > > > > the
> >> > > > > > logfile.
> >> > > > > >
> >> > > > > > Ho-Chun
> >> > > > > >
> >> > > > > > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
> >> > > > > > <met_help at ucar.edu>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Ho-Chun
> >> > > > > > >
> >> > > > > > > mv_g2g_met_verf_aod database exists on AWS but it is
> >> > > > > > > empty.
> >> > > > > > > Do you have a copy of the log file from the data
> >> > > > > > > loading?
> >> > > > > > >
> >> > > > > > > Tatiana
> >> > > > > > >
> >> > > > > > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov
> >> > > > > > > wrote:
> >> > > > > > > > Hi, Tara:
> >> > > > > > > >
> >> > > > > > > > I tried
> >> > > > > > > >
> >> > > > > > > > (1) hit reload database
> >> > > > > > > > (2) refresh button
> >> > > > > > > > (3) clear all web browser data
> >> > > > > > > > (4) close the web browser and open it again
> >> > > > > > > >
> >> > > > > > > > None of it works.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Can you get on the AWS MEtviewer and be able to see
> >> > > > > > > >
> >> > > > > > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
> >> > > > > > > >
> >> > > > > > > > Ho-Chun
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via RT
<
> >> > > > > met_help at ucar.edu>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Ho-Chun,
> >> > > > > > > > >
> >> > > > > > > > > I see you are trying to view a new database up on
> >> > > > > > > > > AWS.
> >> After
> >> > > > > adding a
> >> > > > > > > > > new
> >> > > > > > > > > database to METviewer, you will need to click on
> >> > > > > > > > > Reload
> >> > > > > > > > > databases
> >> > > > > > > > > above the
> >> > > > > > > > > plotting area. If you still don't see it,
consider
> >> clearing
> >> > > > > > > > > your
> >> > > > > > > > > cache and
> >> > > > > > > > > hitting the refresh button. If you are using
> >> > > > > > > > > METexpress,
> >> you
> >> > > > > > > > > may
> >> > > > > not
> >> > > > > > > > > have
> >> > > > > > > > > access to it until tomorrow. I'm not sure how
often
> >> > > > > > > > > they
> >> are
> >> > > > > running
> >> > > > > > > > > metadata, which is what is needed to see a new
> >> > > > > > > > > database.
> >> > > > > > > > >
> >> > > > > > > > > [image: image.png]
> >> > > > > > > > > Let us know if you have tried these things and
still
> >> > > > > > > > > don't
> >> > > > > > > > > see it.
> >> > > > > > > > > We can
> >> > > > > > > > > pass it over to Tatiana at that point.
> >> > > > > > > > > Cheers, Tara
> >> > > > > > > > >
> >> > > > > > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang -
> >> > > > > > > > > NOAA
> >> > > > > > > > > Affiliate via
> >> > > > > > > > > RT <
> >> > > > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was
acted
> >> > > > > > > > > > upon.
> >> > > > > > > > > > Transaction: Ticket created by ho-
> >> > > > > > > > > > chun.huang at noaa.gov
> >> > > > > > > > > > Queue: met_help
> >> > > > > > > > > > Subject: upload database can not being
seen
> >> > > > > > > > > > in AWS
> >> > > > > > > > > > Metviewer
> >> > > > > > > > > > Owner: Nobody
> >> > > > > > > > > > Requestors: ho-chun.huang at noaa.gov
> >> > > > > > > > > > Status: new
> >> > > > > > > > > > Ticket <URL:
> >> > > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346
> >> >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > To Whom It May Concern:
> >> > > > > > > > > >
> >> > > > > > > > > > I upload a new database to AWS but can not find
it
> >> > > > > > > > > > in
> >> > > > > > > > > > MetViewer.
> >> > > > > > > > > > The
> >> > > > > > > > > load
> >> > > > > > > > > > xml and AWS database inquiry is shown below
> >> > > > > > > > > >
> >> > > > > > > > > > The scp files seems working,
> >> > > > > > > > > >
> >> > > > > > > > > > Thank you for your help.
> >> > > > > > > > > >
> >> > > > > > > > > > Ho-Chun
> >> > > > > > > > > >
> >> > > > > > > > > > <load_spec>
> >> > > > > > > > > > <connection>
> >> > > > > > > > > > <host>
> >> > > > > > > > > >
> >> > > > > > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-
east-
> >> > > > > > > > > 1.rds.amazonaws.com:3306
> >> > > > > > > > > > </host>
> >> > > > > > > > > > <database>mv_g2g_met_verf_aod</database>
> >> > > > > > > > > > <user>rds_user</user>
> >> > > > > > > > > > <password>rds_pwd</password>
> >> > > > > > > > > >
<management_system>aurora</management_system>
> >> > > > > > > > > > </connection>
> >> > > > > > > > > >
> >> > > > > > > > > > <met_version>V8.1.2</met_version>
> >> > > > > > > > > >
> >> > > > > > > > > > <verbose>true</verbose>
> >> > > > > > > > > > <insert_size>1</insert_size>
> >> > > > > > > > > >
<mode_header_db_check>true</mode_header_db_check>
> >> > > > > > > > > >
<stat_header_db_check>true</stat_header_db_check>
> >> > > > > > > > > > <drop_indexes>false</drop_indexes>
> >> > > > > > > > > > <apply_indexes>true</apply_indexes>
> >> > > > > > > > > > <load_stat>true</load_stat>
> >> > > > > > > > > > <load_mode>true</load_mode>
> >> > > > > > > > > > <load_mpr>true</load_mpr>
> >> > > > > > > > > > <load_orank>true</load_orank>
> >> > > > > > > > > > <force_dup_file>false</force_dup_file>
> >> > > > > > > > > > <group>NCEP_hcHuang</group>
> >> > > > > > > > > > <description>Loading
> >> > > > > > > > > > mv_g2g_met_verf_aod</description>
> >> > > > > > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> >> > > > > > > > > > <load_val>
> >> > > > > > > > > > <field name="type">
> >> > > > > > > > > > <val>stat</val>
> >> > > > > > > > > > </field>
> >> > > > > > > > > > </load_val>
> >> > > > > > > > > > <load_xml>true</load_xml>
> >> > > > > > > > > > <load_note></load_note>
> >> > > > > > > > > > </load_spec>
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > mv_db_size_on_aws.sh ho-chun.huang
> >> > > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> >> > > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> >> > > > > > > > > > mv_agibbs_nam_hi_dng_FAA
> >> > > > > > > > > > annette.gibbs
> >> > > > > > > > > > 2.28
> >> > > > > > > > > > mv_agibbs_nam_pr_dng_FAA
> >> > > > > > > > > > annette.gibbs
> >> > > > > > > > > > 2.28
> >> > > > > > > > > > emc_cmaq_g2o
> >> > > > > > > > > > perry.shafran
> >> > > > > > > > > > 2.02
> >> > > > > > > > > > emc_cmaq_g2o2
> >> > > > > > > > > > perry.shafran
> >> > > > > > > > > > 2.02
> >> > > > > > > > > > mv_vh
> >> > > > > > > > > > venita.hagerty
> >> > > > > > > > > > 2.02
> >> > > > > > > > > > testvh4
> >> > > > > > > > > > venita.hagerty
> >> > > > > > > > > > 2.02
> >> > > > > > > > > > testvh5
> >> > > > > > > > > > venita.hagerty
> >> > > > > > > > > > 2.02
> >> > > > > > > > > > mv_g2g_met_verf_aod ho-
> >> > > > > > > > > > chun.huang
> >> > > > > > > > > > 1.97
> >> > > > > > > > > > mats_metadata
> >> > > > > > > > > > kenneth.sperow
> >> > > > > > > > > > 1.28
> >> > > > > > > > > > _______________________________
> >> > > > > > > > > > TOTAL_SIZE_GB
> >> > > > > > > > > > 203.36
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > --
> >> > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >> > > > > > > > > Tara Jensen
> >> > > > > > > > > Project Manager II
> >> > > > > > > > > NCAR RAL and DTC
> >> > > > > > > > > PO Box 3000, Boulder, Colorado 80307 USA
> >> > > > > > > > > +1 303-497-8479 jensen at ucar.edu
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > >
> >> > >
> >> > >
> >> > >
> >>
> >>
> >>
> >>
------------------------------------------------
Subject: upload database can not being seen in AWS Metviewer
From: Ho-Chun Huang - NOAA Affiliate
Time: Fri Sep 27 09:12:47 2019
Hi, Tatiana:
I can now see it on AWS METviewer now. Appreciated your help.
Please
close the ticket.
Ho-Chun
On Fri, Sep 27, 2019 at 10:31 AM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Hi Ho-Chun
>
> I see that your BASE_DIR parameterr is
> BASE_DIR=/gpfs/dell2/stmp/Ho-Chun.Huang/verif
> In your case it should be
> BASE_DIR=/gpfs/dell2/stmp/Ho-Chun.Huang/
>
> Tatiana
>
> On Fri Sep 27 05:32:24 2019, ho-chun.huang at noaa.gov wrote:
> > Hi, Tatiana:
> >
> > Good morning, I tried and result is the same.
> >
> > I put latest run_script, load_xml, and log message on
> > https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/
> >
> > Please ignore the file type of run_g2g_met_verf_aod.sh. I need to
> > change
> > it for it to be viewed.
> >
> > Any suggestion of modification? Thank you for the help.
> >
> > Ho-Chun
> >
> > p.s. during execution I did see file been transferred (although
not
> > in
> > run.log that I used ./run_g2g_met_verf_aod.sh > run.log 2>&1)
> >
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> > 100% 3514 255.5KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> > 100% 3537 259.1KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> > 100% 3538 463.4KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> > 100% 3516 247.3KB/s 00:00
> >
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > 100% 3504 466.9KB/s 00:00
> > CALLING: scp
> > /gpfs/dell2/emc/modeling/noscrub/Ho-
> > Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
> > ho-chun.huang at 205.156.8.85:~
> > load_g2g_met_verf_aod.xml
> > 100% 1019 106.6KB/s 00:00
> >
> > I wonder whether you can tell whether there are files
> > in //data/mv_data/ho-chun.huang//verif ?
> >
> > On Thu, Sep 26, 2019 at 5:06 PM Ho-Chun Huang - NOAA Affiliate <
> > ho-chun.huang at noaa.gov> wrote:
> >
> > > Tatiana:
> > >
> > > Thanks. I will try it first thing tomorrow
> > >
> > > On Thu, Sep 26, 2019 at 5:05 PM Tatiana Burek via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > >> <folder_tmpl> tag in XML describes the path to data and not the
file
> > >> name
> > >> or extension.
> > >> In your case it should look like this:
> > >>
> > >> <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > >> <load_val>
> > >> <field name="type">
> > >> <val>verif</val>
> > >> </field>
> > >> </load_val>
> > >>
> > >> and base_dir is /gpfs/dell2/stmp/Ho-Chun.Huang/
> > >>
> > >> Tatiana
> > >>
> > >> On Thu Sep 26 14:57:56 2019, ho-chun.huang at noaa.gov wrote:
> > >> > Hi, Tatiana:
> > >> >
> > >> > It is the filename in the base_dir contain "stat"
> > >> > /base_dir/{type}
> > >> >
> > >> > My filename look like
> > >> >
> > >> >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > >> >
> > >> >
> > >> > in base_dir as /gpfs/dell2/stmp/Ho-Chun.Huang/verif
> > >> >
> > >> > I do not quite understand you question. The directory exist
and
> > >> > the
> > >> > filename contain "stat"
> > >> >
> > >> >
> > >> > I will leave office in 10 mins to catch the shuttle. If
> > >> > necessary, I
> > >> can
> > >> > tar the stat file and load xml to ftp directory and you can
try it
> > >> > from
> > >> > NCAR machine. Please let me know do you want to do it this
way.
> > >> >
> > >> >
> > >> > Ho-Chun
> > >> >
> > >> > On Thu, Sep 26, 2019 at 4:52 PM Tatiana Burek via RT
> > >> > <met_help at ucar.edu
> > >> >
> > >> > wrote:
> > >> >
> > >> > >
> > >> > > In your XML you specified that your data directory contains
> > >> > > 'stats":
> > >> > >
> > >> > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > >> > > <load_val>
> > >> > > <field name="type">
> > >> > > <val>stat</val>
> > >> > > </field>
> > >> > > </load_val>
> > >> > > But your actual path doesn't have this directory. Is it
correct?
> > >> > >
> > >> > >
> > >> > > Tatiana
> > >> > >
> > >> > > On Thu Sep 26 14:45:53 2019, ho-chun.huang at noaa.gov wrote:
> > >> > > > Hi, Tatiana:
> > >> > > >
> > >> > > > Dev machine is down. I have access to operational
machine and
> > >> > > > you
> > >> do
> > >> > > > not.
> > >> > > > So you can not check my data directory, but as in the
log, it
> > >> > > > is /gpfs/dell2/stmp/Ho-Chun.Huang/verif.
> > >> > > >
> > >> > > > I do have files in the directory, and the scp did show it
was
> > >> > > > transferring
> > >> > > > files.
> > >> > > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > ls
> > >> > > >
> > >>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> > >> > > >
> > >>
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F47_CONUS_g16_CMAQAOD_470000L_20190820_110000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190804_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190805_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190806_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190807_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190808_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190809_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190810_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190811_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > >> > > >
> > >> > > >
> > >> > > > This is when I try it again to different database name
> > >> > > > mv_g2g_met_verf_test
> > >> > > >
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190812_120000V.stat
> > >> > > > 100% 3514 251.1KB/s 00:00
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190813_120000V.stat
> > >> > > > 100% 3503 411.8KB/s 00:00
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190814_120000V.stat
> > >> > > > 100% 3515 457.7KB/s 00:00
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190815_120000V.stat
> > >> > > > 100% 3538 465.2KB/s 00:00
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190816_120000V.stat
> > >> > > > 100% 3514 248.3KB/s 00:00
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190817_120000V.stat
> > >> > > > 100% 3537 370.5KB/s 00:00
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190818_120000V.stat
> > >> > > > 100% 3538 175.2KB/s 00:00
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190819_120000V.stat
> > >> > > > 100% 3516 416.8KB/s 00:00
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > >> > > > 100% 3504 461.5KB/s 00:00
> > >> > > > CALLING: scp
> > >> > > > /gpfs/dell2/emc/modeling/noscrub/Ho-
> > >> > > > Chun.Huang/METviewer_AWS/XML/load_g2g_met_verf_aod.xml
> > >> > > > ho-chun.huang at 205.156.8.85:~
> > >> > > > load_g2g_met_verf_aod.xml
> > >> > > > 100% 1019 136.7KB/s 00:00
> > >> > > > ---- MVLoad ----
> > >> > > >
> > >> > > > Begin time: 2019-09-26 20:30:35
> > >> > > > Parsing: load_g2g_met_verf_aod.xml
> > >> > > >
> > >> > > > Permutation 1 of 1 - //data/mv_data/ho-chun.huang//stat
> > >> > > > ==== grid_stat ====
> > >> > > >
> > >> > > > stat_header table time total: 0:00:00.000
> > >> > > > stat header records: 0
> > >> > > > stat header inserts: 0
> > >> > > > line data records: 0
> > >> > > > line data inserts: 0
> > >> > > > stat group records: 0
> > >> > > > stat group inserts: 0
> > >> > > > var length records: 0
> > >> > > > var length inserts: 0
> > >> > > > total lines: 0
> > >> > > > insert size: 1
> > >> > > > lines / msec: 0.000
> > >> > > > num files: 0
> > >> > > >
> > >> > > > ==== mode ====
> > >> > > >
> > >> > > > mode_header search time total: 0:00:00.000
> > >> > > > mtd_header search time total: 0:00:00.000
> > >> > > > stat_header search time total: 0:00:00.000
> > >> > > > mode_header inserts: 0
> > >> > > > mode_cts inserts: 0
> > >> > > > mode_obj_single inserts: 0
> > >> > > > mode_obj_pair inserts: 0
> > >> > > > ==== mtd ====
> > >> > > >
> > >> > > > mtd_header inserts: 0
> > >> > > > mtd_3d_obj_single inserts: 0
> > >> > > > mtd_3d_obj_pair inserts: 0
> > >> > > > mtd_2d_obj inserts: 0
> > >> > > > total lines: 0
> > >> > > > num files: 0
> > >> > > > num files: 0
> > >> > > >
> > >> > > > ==== indexes ====creating
> > >> > > > Indexes creating 0:00:01.002
> > >> > > >
> > >> > > > End time: 2019-09-26 20:30:36
> > >> > > > Load total: 0:00:00.010
> > >> > > >
> > >> > > > ---- MVLoad Done ----
> > >> > > >
> > >> > > > Each file does have data in there
> > >> > > >
> > >> > > > /gpfs/dell2/stmp/Ho-Chun.Huang/verif > more
> > >> > > >
> > >> > >
> > >>
>
grid_stat_cmaqaod_prod_F48_CONUS_g16_CMAQAOD_480000L_20190820_120000V.stat
> > >> > > > VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
> > >> > > > FCST_VALID_END
> > >> > > > OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR
FCST_UNITS
> > >> FCST_LEV
> > >> > > > OB
> > >> > > > S_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD
> > >> > > > INTERP_PNTS
> > >> > > > FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =0.5
> > >> > > > > =0.5 NA NA CTC 17941 0 5
> > >> > > > > 1
> > >> > > > 17935
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =1.0
> > >> > > > > =1.0 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =1.5
> > >> > > > > =1.5 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =2.0
> > >> > > > > =2.0 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =2.5
> > >> > > > > =2.5 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =3.0
> > >> > > > > =3.0 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =3.5
> > >> > > > > =3.5 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =4.0
> > >> > > > > =4.0 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =4.5
> > >> > > > > =4.5 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > > =5.0
> > >> > > > > =5.0 NA NA CTC 17941 0 0
> > >> > > > > 0
> > >> > > > 17941
> > >> > > > V8.1.1 cmaqaod_prod NA 480000 20190820_120000
> > >> > > > 20190820_120000
> > >> > > > 000000
> > >> > > > 20190820_120236 20190820_120236 AOTK Numeric L1
> > >> > > > AO
> > >> > > > D 1 *,* ANALYS AOD_*,*>=0 NEAREST 1
> > >> > > > NA
> > >> > > > NA NA NA SL1L2 17941 0.070913
0.1156 0.
> > >> > > > 011134 0.008766 0.018307 0.054144
> > >> > > >
> > >> > > > On Thu, Sep 26, 2019 at 4:37 PM Tatiana Burek via RT
> > >> > > > <met_help at ucar.edu>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Ho-Chun
> > >> > > > >
> > >> > > > > The log file shows that 0 records was inserted into the
> > >> > > > > database.
> > >> > > > > What is the path to your data on WCOSS and how do you
call
> > >> > > > > the
> > >> load
> > >> > > > > command?
> > >> > > > >
> > >> > > > > Tatiana
> > >> > > > >
> > >> > > > > On Thu Sep 26 14:28:00 2019, ho-chun.huang at noaa.gov
wrote:
> > >> > > > > > Hi, Tara:
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > I remove the database and start again
> > >> > > > > >
> > >> > > > > > The log file is in
> > >> > > > > >
https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/run.log
> > >> > > > > >
> > >> > > > > > When I run it without piping it to run.log, I did
see the
> > >> > > > > > scp
> > >> > > > > > working
> > >> > > > > > transferring files. For some reason, I did not see
it in
> > >> > > > > > the
> > >> > > > > > logfile.
> > >> > > > > >
> > >> > > > > > Ho-Chun
> > >> > > > > >
> > >> > > > > > On Thu, Sep 26, 2019 at 4:08 PM Tatiana Burek via RT
> > >> > > > > > <met_help at ucar.edu>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Ho-Chun
> > >> > > > > > >
> > >> > > > > > > mv_g2g_met_verf_aod database exists on AWS but it
is
> > >> > > > > > > empty.
> > >> > > > > > > Do you have a copy of the log file from the data
> > >> > > > > > > loading?
> > >> > > > > > >
> > >> > > > > > > Tatiana
> > >> > > > > > >
> > >> > > > > > > On Thu Sep 26 13:49:57 2019, ho-chun.huang at noaa.gov
> > >> > > > > > > wrote:
> > >> > > > > > > > Hi, Tara:
> > >> > > > > > > >
> > >> > > > > > > > I tried
> > >> > > > > > > >
> > >> > > > > > > > (1) hit reload database
> > >> > > > > > > > (2) refresh button
> > >> > > > > > > > (3) clear all web browser data
> > >> > > > > > > > (4) close the web browser and open it again
> > >> > > > > > > >
> > >> > > > > > > > None of it works.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Can you get on the AWS MEtviewer and be able to
see
> > >> > > > > > > >
> > >> > > > > > > > mv_g2g_met_verf_aod under NCEP_hcHuang?
> > >> > > > > > > >
> > >> > > > > > > > Ho-Chun
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > On Thu, Sep 26, 2019 at 3:24 PM Tara Jensen via
RT <
> > >> > > > > met_help at ucar.edu>
> > >> > > > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > > Ho-Chun,
> > >> > > > > > > > >
> > >> > > > > > > > > I see you are trying to view a new database up
on
> > >> > > > > > > > > AWS.
> > >> After
> > >> > > > > adding a
> > >> > > > > > > > > new
> > >> > > > > > > > > database to METviewer, you will need to click
on
> > >> > > > > > > > > Reload
> > >> > > > > > > > > databases
> > >> > > > > > > > > above the
> > >> > > > > > > > > plotting area. If you still don't see it,
consider
> > >> clearing
> > >> > > > > > > > > your
> > >> > > > > > > > > cache and
> > >> > > > > > > > > hitting the refresh button. If you are using
> > >> > > > > > > > > METexpress,
> > >> you
> > >> > > > > > > > > may
> > >> > > > > not
> > >> > > > > > > > > have
> > >> > > > > > > > > access to it until tomorrow. I'm not sure how
often
> > >> > > > > > > > > they
> > >> are
> > >> > > > > running
> > >> > > > > > > > > metadata, which is what is needed to see a new
> > >> > > > > > > > > database.
> > >> > > > > > > > >
> > >> > > > > > > > > [image: image.png]
> > >> > > > > > > > > Let us know if you have tried these things and
still
> > >> > > > > > > > > don't
> > >> > > > > > > > > see it.
> > >> > > > > > > > > We can
> > >> > > > > > > > > pass it over to Tatiana at that point.
> > >> > > > > > > > > Cheers, Tara
> > >> > > > > > > > >
> > >> > > > > > > > > On Thu, Sep 26, 2019 at 12:40 PM Ho-Chun Huang
-
> > >> > > > > > > > > NOAA
> > >> > > > > > > > > Affiliate via
> > >> > > > > > > > > RT <
> > >> > > > > > > > > met_help at ucar.edu> wrote:
> > >> > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > Thu Sep 26 12:40:26 2019: Request 92346 was
acted
> > >> > > > > > > > > > upon.
> > >> > > > > > > > > > Transaction: Ticket created by ho-
> > >> > > > > > > > > > chun.huang at noaa.gov
> > >> > > > > > > > > > Queue: met_help
> > >> > > > > > > > > > Subject: upload database can not being
seen
> > >> > > > > > > > > > in AWS
> > >> > > > > > > > > > Metviewer
> > >> > > > > > > > > > Owner: Nobody
> > >> > > > > > > > > > Requestors: ho-chun.huang at noaa.gov
> > >> > > > > > > > > > Status: new
> > >> > > > > > > > > > Ticket <URL:
> > >> > > > > > > > > >
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92346
> > >> >
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > To Whom It May Concern:
> > >> > > > > > > > > >
> > >> > > > > > > > > > I upload a new database to AWS but can not
find it
> > >> > > > > > > > > > in
> > >> > > > > > > > > > MetViewer.
> > >> > > > > > > > > > The
> > >> > > > > > > > > load
> > >> > > > > > > > > > xml and AWS database inquiry is shown below
> > >> > > > > > > > > >
> > >> > > > > > > > > > The scp files seems working,
> > >> > > > > > > > > >
> > >> > > > > > > > > > Thank you for your help.
> > >> > > > > > > > > >
> > >> > > > > > > > > > Ho-Chun
> > >> > > > > > > > > >
> > >> > > > > > > > > > <load_spec>
> > >> > > > > > > > > > <connection>
> > >> > > > > > > > > > <host>
> > >> > > > > > > > > >
> > >> > > > > > > > > metviewer-dev-cluster.cluster-czbts4gd2wm2.us-
east-
> > >> > > > > > > > > 1.rds.amazonaws.com:3306
> > >> > > > > > > > > > </host>
> > >> > > > > > > > > > <database>mv_g2g_met_verf_aod</database>
> > >> > > > > > > > > > <user>rds_user</user>
> > >> > > > > > > > > > <password>rds_pwd</password>
> > >> > > > > > > > > >
<management_system>aurora</management_system>
> > >> > > > > > > > > > </connection>
> > >> > > > > > > > > >
> > >> > > > > > > > > > <met_version>V8.1.2</met_version>
> > >> > > > > > > > > >
> > >> > > > > > > > > > <verbose>true</verbose>
> > >> > > > > > > > > > <insert_size>1</insert_size>
> > >> > > > > > > > > >
<mode_header_db_check>true</mode_header_db_check>
> > >> > > > > > > > > >
<stat_header_db_check>true</stat_header_db_check>
> > >> > > > > > > > > > <drop_indexes>false</drop_indexes>
> > >> > > > > > > > > > <apply_indexes>true</apply_indexes>
> > >> > > > > > > > > > <load_stat>true</load_stat>
> > >> > > > > > > > > > <load_mode>true</load_mode>
> > >> > > > > > > > > > <load_mpr>true</load_mpr>
> > >> > > > > > > > > > <load_orank>true</load_orank>
> > >> > > > > > > > > > <force_dup_file>false</force_dup_file>
> > >> > > > > > > > > > <group>NCEP_hcHuang</group>
> > >> > > > > > > > > > <description>Loading
> > >> > > > > > > > > > mv_g2g_met_verf_aod</description>
> > >> > > > > > > > > > <folder_tmpl>/base_dir/{type}</folder_tmpl>
> > >> > > > > > > > > > <load_val>
> > >> > > > > > > > > > <field name="type">
> > >> > > > > > > > > > <val>stat</val>
> > >> > > > > > > > > > </field>
> > >> > > > > > > > > > </load_val>
> > >> > > > > > > > > > <load_xml>true</load_xml>
> > >> > > > > > > > > > <load_note></load_note>
> > >> > > > > > > > > > </load_spec>
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > > mv_db_size_on_aws.sh ho-chun.huang
> > >> > > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > >> > > > > > > > > > :::::::::::::::::::::::::::::::::::::::::
> > >> > > > > > > > > > mv_agibbs_nam_hi_dng_FAA
> > >> > > > > > > > > > annette.gibbs
> > >> > > > > > > > > > 2.28
> > >> > > > > > > > > > mv_agibbs_nam_pr_dng_FAA
> > >> > > > > > > > > > annette.gibbs
> > >> > > > > > > > > > 2.28
> > >> > > > > > > > > > emc_cmaq_g2o
> > >> > > > > > > > > > perry.shafran
> > >> > > > > > > > > > 2.02
> > >> > > > > > > > > > emc_cmaq_g2o2
> > >> > > > > > > > > > perry.shafran
> > >> > > > > > > > > > 2.02
> > >> > > > > > > > > > mv_vh
> > >> > > > > > > > > > venita.hagerty
> > >> > > > > > > > > > 2.02
> > >> > > > > > > > > > testvh4
> > >> > > > > > > > > > venita.hagerty
> > >> > > > > > > > > > 2.02
> > >> > > > > > > > > > testvh5
> > >> > > > > > > > > > venita.hagerty
> > >> > > > > > > > > > 2.02
> > >> > > > > > > > > > mv_g2g_met_verf_aod ho-
> > >> > > > > > > > > > chun.huang
> > >> > > > > > > > > > 1.97
> > >> > > > > > > > > > mats_metadata
> > >> > > > > > > > > > kenneth.sperow
> > >> > > > > > > > > > 1.28
> > >> > > > > > > > > > _______________________________
> > >> > > > > > > > > > TOTAL_SIZE_GB
> > >> > > > > > > > > > 203.36
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > --
> > >> > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > > > > > > > > Tara Jensen
> > >> > > > > > > > > Project Manager II
> > >> > > > > > > > > NCAR RAL and DTC
> > >> > > > > > > > > PO Box 3000, Boulder, Colorado 80307 USA
> > >> > > > > > > > > +1 303-497-8479 jensen at ucar.edu
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >>
> > >>
> > >>
> > >>
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list