[Met_help] [rt.rap.ucar.edu #90094] History for mv_load_to_aws.sh: delete data directory on AWS after loading?

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello.  When using mv_load_to_aws.sh to load additional data to an 
existing data base (after setting <drop_indexes>true</drop_indexes> in 
the xml), I'm getting messages like

   **  WARNING: file already present in table data_file
   /data/mv_data/ying.lin/vsdb.erly/rap_20190426.vsdb

${model}_20190426.vsdb was what I loaded to the database previously.  I 
can do an "ssh ying.lin at ... 'rm /data/mv_data ...' to clear out the old 
data, but if possible/feasible, would you add a final step to the 
official mv_load_to_aws.sh to remove the data directory after the "run 
load"?

Thank you -

Ying

-- 
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov




----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: mv_load_to_aws.sh: delete data directory on AWS after loading?
From: John Halley Gotway
Time: Mon May 06 17:04:07 2019

Hi Ying,

I'll go ahead and assign this ticket to Tatiana.  You should hear back
from
her in the next couple of days.  Hopefully she can help diagnose this
behavior.

Thanks,
John

On Mon, May 6, 2019 at 4:55 PM Ying Lin via RT <met_help at ucar.edu>
wrote:

>
> Mon May 06 16:55:09 2019: Request 90094 was acted upon.
> Transaction: Ticket created by ying.lin at noaa.gov
>        Queue: met_help
>      Subject: mv_load_to_aws.sh: delete data directory on AWS after
> loading?
>        Owner: Nobody
>   Requestors: ying.lin at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90094 >
>
>
> Hello.  When using mv_load_to_aws.sh to load additional data to an
> existing data base (after setting <drop_indexes>true</drop_indexes>
in
> the xml), I'm getting messages like
>
>    **  WARNING: file already present in table data_file
>    /data/mv_data/ying.lin/vsdb.erly/rap_20190426.vsdb
>
> ${model}_20190426.vsdb was what I loaded to the database previously.
I
> can do an "ssh ying.lin at ... 'rm /data/mv_data ...' to clear out the
old
> data, but if possible/feasible, would you add a final step to the
> official mv_load_to_aws.sh to remove the data directory after the
"run
> load"?
>
> Thank you -
>
> Ying
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>

------------------------------------------------
Subject: mv_load_to_aws.sh: delete data directory on AWS after loading?
From: Tatiana Burek
Time: Tue May 07 08:29:25 2019

Hi Ying,
I previously commented out the data removing step in the loading
script for the debugging reasons.
I uncommitted this step.  After the loading is done the script should
remove all data files.

Tatiana

On Mon May 06 17:04:07 2019, johnhg wrote:
> Hi Ying,
>
> I'll go ahead and assign this ticket to Tatiana.  You should hear
back from
> her in the next couple of days.  Hopefully she can help diagnose
this
> behavior.
>
> Thanks,
> John
>
> On Mon, May 6, 2019 at 4:55 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> >
> > Mon May 06 16:55:09 2019: Request 90094 was acted upon.
> > Transaction: Ticket created by ying.lin at noaa.gov
> >        Queue: met_help
> >      Subject: mv_load_to_aws.sh: delete data directory on AWS
after
> > loading?
> >        Owner: Nobody
> >   Requestors: ying.lin at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90094 >
> >
> >
> > Hello.  When using mv_load_to_aws.sh to load additional data to an
> > existing data base (after setting
<drop_indexes>true</drop_indexes> in
> > the xml), I'm getting messages like
> >
> >    **  WARNING: file already present in table data_file
> >    /data/mv_data/ying.lin/vsdb.erly/rap_20190426.vsdb
> >
> > ${model}_20190426.vsdb was what I loaded to the database
previously.  I
> > can do an "ssh ying.lin at ... 'rm /data/mv_data ...' to clear out
the old
> > data, but if possible/feasible, would you add a final step to the
> > official mv_load_to_aws.sh to remove the data directory after the
"run
> > load"?
> >
> > Thank you -
> >
> > Ying
> >
> > --
> > Ying Lin
> > NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> > NCWCP Cubicle No. 2015
> > Ying.Lin at noaa.gov
> >
> >
> >
> >



------------------------------------------------