[Met_help] [rt.rap.ucar.edu #94306] History for Re: add user fillmore to tcgp group on eyewall
Tor Mohling via RT
met_help at ucar.edu
Tue Mar 3 15:56:41 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi David,
Do you need on-going access to the real-time data, or just to a chunk of
sample data?
Because of SNAT's new security paradigm, the TCGP project has its own
dedicated UNIX group to enable a multi-developer environment. But I'm
not sure adding you to the group is the right way to add access. I'm
checking with SNAT to see if there is a more elegant solution. Another
easy thing would just be to copy some of the data over to somewhere
where you can access it.
The TCGP data are generally available on the public web site as well - I
can give you tips on where to find stuff.
Jonathan
On 2020-02-25 16:46, David Fillmore wrote:
> Hi -
> Can I get added to the tcgp group on eyewall?
> I want to do some TC tool testing on data which resides in /d1/TCGP ...
> I think Jonathon Vigh can approve (cc'ed).
> thanks,
> David
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: David Fillmore
Time: Tue Feb 25 10:18:31 2020
Hi Jonathan -
Copying the data elsewhere should be fine for now,
though I do want to process a large volume of TC model output (global,
multiple storms over multiple years), the more the better ...
Let's come up with a storage space estimate and see if copying is
feasible.
thanks,
David
On Tue, Feb 25, 2020 at 10:00 AM jvigh via RT <met_help at ucar.edu>
wrote:
>
> Tue Feb 25 10:00:09 2020: Request 94306 was acted upon.
> Transaction: Ticket created by jvigh
> Queue: met_help
> Subject: Re: add user fillmore to tcgp group on eyewall
> Owner: Nobody
> Requestors: jvigh at ucar.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94306 >
>
>
> Hi David,
>
> Do you need on-going access to the real-time data, or just to a
chunk of
> sample data?
>
> Because of SNAT's new security paradigm, the TCGP project has its
own
> dedicated UNIX group to enable a multi-developer environment. But
I'm
> not sure adding you to the group is the right way to add access. I'm
> checking with SNAT to see if there is a more elegant solution.
Another
> easy thing would just be to copy some of the data over to somewhere
> where you can access it.
>
> The TCGP data are generally available on the public web site as well
- I
> can give you tips on where to find stuff.
>
> Jonathan
>
>
> On 2020-02-25 16:46, David Fillmore wrote:
> > Hi -
> > Can I get added to the tcgp group on eyewall?
> > I want to do some TC tool testing on data which resides in
/d1/TCGP ...
> > I think Jonathon Vigh can approve (cc'ed).
> > thanks,
> > David
>
>
>
>
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: Tor Mohling
Time: Tue Feb 25 10:24:50 2020
Copying this data seems like a bad idea.. it's 4+ TB !
Can it be made read-only for "world"? Then David can see it for
processing (I assume).
-tor
--
Tor Mohling -- UCAR/RAL System Administrator
Phone: 303/497-2867 -- Email: tor at ucar.edu
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: David Fillmore
Time: Tue Feb 25 10:26:52 2020
I have access to mohawk.rap.ucar.edu, I think we can withdraw and
close
this ticket.
thanks,
David
On Tue, Feb 25, 2020 at 10:18 AM David Fillmore <fillmore at ucar.edu>
wrote:
> Hi Jonathan -
> Copying the data elsewhere should be fine for now,
> though I do want to process a large volume of TC model output
(global,
> multiple storms over multiple years), the more the better ...
> Let's come up with a storage space estimate and see if copying is
feasible.
> thanks,
> David
>
>
> On Tue, Feb 25, 2020 at 10:00 AM jvigh via RT <met_help at ucar.edu>
wrote:
>
>>
>> Tue Feb 25 10:00:09 2020: Request 94306 was acted upon.
>> Transaction: Ticket created by jvigh
>> Queue: met_help
>> Subject: Re: add user fillmore to tcgp group on eyewall
>> Owner: Nobody
>> Requestors: jvigh at ucar.edu
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94306 >
>>
>>
>> Hi David,
>>
>> Do you need on-going access to the real-time data, or just to a
chunk of
>> sample data?
>>
>> Because of SNAT's new security paradigm, the TCGP project has its
own
>> dedicated UNIX group to enable a multi-developer environment. But
I'm
>> not sure adding you to the group is the right way to add access.
I'm
>> checking with SNAT to see if there is a more elegant solution.
Another
>> easy thing would just be to copy some of the data over to somewhere
>> where you can access it.
>>
>> The TCGP data are generally available on the public web site as
well - I
>> can give you tips on where to find stuff.
>>
>> Jonathan
>>
>>
>> On 2020-02-25 16:46, David Fillmore wrote:
>> > Hi -
>> > Can I get added to the tcgp group on eyewall?
>> > I want to do some TC tool testing on data which resides in
/d1/TCGP ...
>> > I think Jonathon Vigh can approve (cc'ed).
>> > thanks,
>> > David
>>
>>
>>
>>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #94306] Re: add user fillmore to tcgp group on eyewall
From: jvigh
Time: Tue Feb 25 12:06:07 2020
Hi Tor,
I just met with David. I didn't realize before he also needs the full
fields. Yes, we don't want to copy that.
I've opened up the portions of TCGP's ATCF repository space that he
needs:
(base) jvigh at eyewall:/d1/TCGP/data/data_realtime/ATCF$ ls -la
total 136
drwxrwsr--+ 21 jvigh tcgp 4096 Feb 25 18:47 ./
drwxrwsr--+ 12 jvigh tcgp 36864 Jan 5 08:59 ../
drwxrwsr--+ 2 jvigh tcgp 12288 Feb 25 18:47 adecks_open/
drwxrwsr--+ 2 jvigh tcgp 4096 Feb 24 00:49 bdecks_open/
drwxrwsr--+ 2 jvigh tcgp 4096 Feb 24 00:49 fdecks_open/
The gefs directory contains the GEFS fields (the 4 TB) that he needs,
but it's owned by Kerry's PhD student, Jonathan Lin (jzlin).
(base) jvigh at eyewall:/d1/TCGP/data/data_realtime$ ls -la
total 88
...
drwxrwsr--+ 21 jvigh tcgp 4096 Feb 25 18:47 ATCF/
drwxrws---+ 5 jzlin tcgp 4096 Jan 1 06:00 gefs/
...
I'm not able to open these up because I'm not the owner:
(base) jvigh at eyewall:/d1/TCGP/data/data_realtime$ chmod a+r gefs/
chmod: changing permissions of 'gefs/': Operation not permitted
Is this something a sysadmin needs to do? Or is there a way for me to
do
this?
Thanks,
Jonathan
On 2020-02-25 17:26, David Fillmore via RT wrote:
> I have access to mohawk.rap.ucar.edu, I think we can withdraw and
close
> this ticket.
> thanks,
> David
>
> On Tue, Feb 25, 2020 at 10:18 AM David Fillmore <fillmore at ucar.edu>
wrote:
>
>> Hi Jonathan -
>> Copying the data elsewhere should be fine for now,
>> though I do want to process a large volume of TC model output
(global,
>> multiple storms over multiple years), the more the better ...
>> Let's come up with a storage space estimate and see if copying is
feasible.
>> thanks,
>> David
>>
>>
>> On Tue, Feb 25, 2020 at 10:00 AM jvigh via RT <met_help at ucar.edu>
wrote:
>>
>>> Tue Feb 25 10:00:09 2020: Request 94306 was acted upon.
>>> Transaction: Ticket created by jvigh
>>> Queue: met_help
>>> Subject: Re: add user fillmore to tcgp group on eyewall
>>> Owner: Nobody
>>> Requestors: jvigh at ucar.edu
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94306 >
>>>
>>>
>>> Hi David,
>>>
>>> Do you need on-going access to the real-time data, or just to a
chunk of
>>> sample data?
>>>
>>> Because of SNAT's new security paradigm, the TCGP project has its
own
>>> dedicated UNIX group to enable a multi-developer environment. But
I'm
>>> not sure adding you to the group is the right way to add access.
I'm
>>> checking with SNAT to see if there is a more elegant solution.
Another
>>> easy thing would just be to copy some of the data over to
somewhere
>>> where you can access it.
>>>
>>> The TCGP data are generally available on the public web site as
well - I
>>> can give you tips on where to find stuff.
>>>
>>> Jonathan
>>>
>>>
>>> On 2020-02-25 16:46, David Fillmore wrote:
>>>> Hi -
>>>> Can I get added to the tcgp group on eyewall?
>>>> I want to do some TC tool testing on data which resides in
/d1/TCGP ...
>>>> I think Jonathon Vigh can approve (cc'ed).
>>>> thanks,
>>>> David
>>>
>>>
>>>
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: Tor Mohling
Time: Tue Feb 25 13:35:33 2020
Hi Jonathan,
> I just met with David. I didn't realize before he also needs the
full
> fields. Yes, we don't want to copy that.
Yeah, it's big enough to not want to do that..
>
> I've opened up the portions of TCGP's ATCF repository space that he
> needs:
>
> (base) jvigh at eyewall:/d1/TCGP/data/data_realtime/ATCF$ ls -la
> total 136
> drwxrwsr--+ 21 jvigh tcgp 4096 Feb 25 18:47 ./
> drwxrwsr--+ 12 jvigh tcgp 36864 Jan 5 08:59 ../
> drwxrwsr--+ 2 jvigh tcgp 12288 Feb 25 18:47 adecks_open/
> drwxrwsr--+ 2 jvigh tcgp 4096 Feb 24 00:49 bdecks_open/
> drwxrwsr--+ 2 jvigh tcgp 4096 Feb 24 00:49 fdecks_open/
Actually, the directories themselves need to have the eXecute bit set
too.
I'll show you an example below.
> ...
>
> Is this something a sysadmin needs to do? Or is there a way for me
to do
> this?
Here is a find(1) command that can do this all at once (run as root
since jzlin owns
some directories):
eyewall:/d1/TCGP# find ./data -xdev -type d -exec chmod o+rx \{\} \;
-type f -exec o+r \{\} \;
eyewall:/d1/TCGP#
The toplevel dir should be opened up as well so that David can descend
into the data subdir:
eyewall:/d1/TCGP# chmod o+rx .
eyewall:/d1/TCGP#
Please LMK if there are issues..
Thanks!
-tor
--
Tor Mohling -- UCAR/RAL System Administrator
Phone: 303/497-2867 -- Email: tor at ucar.edu
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: jvigh
Time: Wed Feb 26 11:22:19 2020
On 2020-02-25 20:35, Tor Mohling via RT wrote:
> find ./data -xdev -type d -exec chmod o+rx \{\} \; -type f -exec o+r
\{\} \;
Hi Tor,
Thanks for the tips.
I've opened up the top-level directory, data, and data_realtime, but
it
won't let me modify any permissions for directories jzlin owns:
(base) jvigh at eyewall:/d1/TCGP/data/data_realtime$ chmod o+rx ./gefs/
chmod: changing permissions of './gefs/': Operation not permitted
(base) jvigh at eyewall:/d1/TCGP/data/data_realtime$ ls -la
total 88
drwxrwsr-x+ 12 jvigh tcgp 36864 Jan 5 08:59 ./
drwxrwsr-x+ 3 jvigh tcgp 4096 Nov 3 06:36 ../
drwxrwsr-x+ 11 jzlin tcgp 4096 Aug 30 15:48 atcf/
drwxrwsr-x+ 21 jvigh tcgp 4096 Feb 26 15:19 ATCF/
drwxrwsr-x+ 7 jzlin tcgp 4096 Jan 5 13:49 cxml/
drwxrwsr-x+ 5 jvigh tcgp 4096 Nov 8 17:31 DIAGNOSTICS/
drwxrwsr-x+ 13 jzlin tcgp 4096 Jan 5 08:51 ecmwf/
*drwxrwsr-x+ 5 jzlin tcgp 4096 Jan 1 06:00 gefs/*
drwxr-sr-x+ 2 jzlin tcgp 4096 Dec 28 02:31 .locs/
-rw-rw---- 1 jzlin tcgp 400 Dec 28 04:22 move_files.py
-rw-rw---- 1 jzlin tcgp 399 Dec 28 03:00 move_files.py~
drwxrwsr-x+ 4 jzlin tcgp 4096 Jan 20 17:12 ocean/
drwxrwsr-x+ 2 jzlin tcgp 4096 Sep 15 17:32 thermo/
drwxr-sr-x+ 2 jzlin tcgp 4096 Dec 28 02:31 .tmp/
I note that the directory already had o+rx permissions, but just for
kicks:
(base) jvigh at eyewall:/d1/TCGP/data/data_realtime$ chmod o-r gefs/
chmod: changing permissions of 'gefs/': Operation not permitted
So when I try running the find command on the ./gefs directory, I get
the following:
(base) jvigh at eyewall:/d1/TCGP/data/data_realtime$ find ./gefs -xdev
-type d -exec chmod o+rx \{\} \; -type f -exec o+r \{\} \;
chmod: changing permissions of './gefs': Operation not permitted
chmod: changing permissions of './gefs/2019': Operation not permitted
chmod: changing permissions of './gefs/2019/10': Operation not
permitted
chmod: changing permissions of './gefs/2019/10/29': Operation not
permitted
chmod: changing permissions of './gefs/2019/10/29/06z': Operation not
permitted
chmod: changing permissions of './gefs/2019/10/29/18z': Operation not
permitted
chmod: changing permissions of './gefs/2019/10/29/12z': Operation not
permitted
chmod: changing permissions of './gefs/2019/10/29/00z': Operation not
permitted
chmod: changing permissions of './gefs/2019/10/23': Operation not
permitted
chmod: changing permissions of './gefs/2019/10/23/06z': Operation not
permitted
chmod: changing permissions of './gefs/2019/10/23/18z': Operation not
permitted
It still can't change any permissions on the subdirectories, of which
there are many.
I'm not sure what to try next, apart from asking Jonathan Lin open up
the permissions on these.
It seems like there should be a way for me to manage this. If not,
could
this be a problem with the paradigm of using the shared group for
multi-development environments?
Jonathan
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: Tor Mohling
Time: Wed Feb 26 12:43:04 2020
> Here is a find(1) command that can do this all at once (run as root
> since jzlin owns
> some directories):
>
> eyewall:/d1/TCGP# find ./data -xdev -type d -exec chmod o+rx \{\} \;
> -type f -exec o+r \{\} \;
> eyewall:/d1/TCGP#
There is a bug in my find command. Here is the correct one:
eyewall:/d1/TCGP# find ./data -xdev -type d -exec chmod o+rx \{\} \; ,
-type f -exec chmod o+r \{\} \;
The second half of the find that changes FILES was not triggering.
-tor
--
Tor Mohling -- UCAR/RAL System Administrator
Phone: 303/497-2867 -- Email: tor at ucar.edu
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: Tor Mohling
Time: Wed Feb 26 12:46:04 2020
> won't let me modify any permissions for directories jzlin owns:
Yeah.. my find was supposed to fix the perms for these files. :(
(so it is re-running now - and taking a lot longer as it is now
actually
touching each file)
>
> ...
> It still can't change any permissions on the subdirectories, of
which
> there are many.
>
> I'm not sure what to try next, apart from asking Jonathan Lin open
up
> the permissions on these.
You can always ask RAL IT.. ;)
>
> It seems like there should be a way for me to manage this. If not,
> could
> this be a problem with the paradigm of using the shared group for
> multi-development environments?
You mean "multi-developER" environments? Yes. Advanced posix ACLs
can help
here but they are a PITA to manage.
-tor
--
Tor Mohling -- UCAR/RAL System Administrator
Phone: 303/497-2867 -- Email: tor at ucar.edu
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: jvigh
Time: Wed Feb 26 15:14:55 2020
Hi Tor,
Okay, I tried your revised command. I'm at home, so I can't copy the
result
here, but see the attached screenshot. The 2nd part is still not
working.
Jonathan
Jonathan Vigh
Project Scientist I, Joint Numerical Testbed
Research Applications Laboratory (RAL)
National Center for Atmospheric Research (NCAR)
P.O. Box 3000 tel: +1 (303) 497-8205
Boulder, CO 80307-3000 fax: +1 (303)
497-
8171http://www.ral.ucar.edu/staff/jvigh/http://hurricanes.ral.ucar.edu/
https://verif.rap.ucar.edu/tcdata/
On Wed, Feb 26, 2020 at 12:43 PM Tor Mohling via RT
<met_help at ucar.edu>
wrote:
>
> > Here is a find(1) command that can do this all at once (run as
root
> > since jzlin owns
> > some directories):
> >
> > eyewall:/d1/TCGP# find ./data -xdev -type d -exec chmod o+rx \{\}
\;
> > -type f -exec o+r \{\} \;
> > eyewall:/d1/TCGP#
>
> There is a bug in my find command. Here is the correct one:
>
> eyewall:/d1/TCGP# find ./data -xdev -type d -exec chmod o+rx \{\} \;
,
> -type f -exec chmod o+r \{\} \;
>
>
> The second half of the find that changes FILES was not triggering.
>
> -tor
> --
> Tor Mohling -- UCAR/RAL System Administrator
> Phone: 303/497-2867 -- Email: tor at ucar.edu
>
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: Tor Mohling
Time: Wed Feb 26 15:23:08 2020
Hi Jonathan,
Okay, I tried your revised command. I'm at home, so I can't copy the
result
> here, but see the attached screenshot. The 2nd part is still not
working.
>
I apologize for not being clear. It was not meant to work for you. I
was
just
wanting to let you know that the command was not syntactically
correct.
It seems that the users creating files in this directory hierarchy
should
simply
change their umask to 002 during the creation process. So that world-
read
gets enabled automatically. That would solve the problem in this
instance.
Thanks,
-tor
--
Tor Mohling -- System Administrator
National Center for Atmospheric Research
Research Application Laboratories
303.497.2867 tor at ucar.edu
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #94306] Re: add user fillmore to tcgp group on eyewall
From: jvigh
Time: Fri Feb 28 12:04:29 2020
Hi Tor,
Thanks. I had wondered if you had already run it, based on something
you
said in your e-mail.
I'll ask Jonathan Lin to create the files with umask 002.
David - just wanted to check - are you able to access the files now?
Thanks,
Jonathan
On 2020-02-26 22:23, Tor Mohling via RT wrote:
> Hi Jonathan,
>
> Okay, I tried your revised command. I'm at home, so I can't copy the
result
>> here, but see the attached screenshot. The 2nd part is still not
working.
>>
> I apologize for not being clear. It was not meant to work for you.
I was
> just
> wanting to let you know that the command was not syntactically
correct.
>
> It seems that the users creating files in this directory hierarchy
should
> simply
> change their umask to 002 during the creation process. So that
world-read
> gets enabled automatically. That would solve the problem in this
instance.
>
> Thanks,
> -tor
------------------------------------------------
Subject: Re: add user fillmore to tcgp group on eyewall
From: Tor Mohling
Time: Tue Mar 03 15:56:40 2020
Hi folks,
I'm going to go ahead and resolve this ticket but feel free to reply
and re-open it if you have additional questions or concerns.
Thanks!
-tor
--
Tor Mohling -- UCAR/RAL System Administrator
Phone: 303/497-2867 -- Email: tor at ucar.edu
------------------------------------------------
More information about the Met_help
mailing list