Subject: [ESP] asking for feedback on LAS

From: Steve Hankin <[email-masked]>
Date: Thu, 13 Mar 2003 11:19:53 -0800

Hi Bryan (et. al.),

At the December ESP meeting in Boulder I said I would follow
up with a request for feedback on LAS: what needs to be
added or modified in order to fill the needs that you see in
the ESP portal design.   Our next LAS design meeting will be
in early April so feedback in the next couple of weeks would
be very timely

Of course, this request for input applies to everyone in ESP
-- not just Bryan.

    thanks in advance - steve

--

Steve Hankin, NOAA/PMEL -- [email-masked]
7600 Sand Point Way NE, Seattle, WA 98115-0070
ph. (206) 526-6080, FAX (206) 526-6744

---

From: Bryan Lawrence <[email-masked]>
Date: Fri, 28 Mar 2003 17:30:30 +0000

Hi Steve

Sorry about the slow reply; in part because I wanted to get some feedback 
locally, and in part because I hoped Andrew Woolf would get a chance for a 
chat with you during the opendap meeting ...

> in early April so feedback in the next couple of weeks would

Anyway I think for us the key issues for LAS are web-services, modularisation, 
security, and graceful support for java-challenged browsers. I know that most 
of these are things you are thinking about (or actually doing) anyway, so I 
suspect there will be no surprises here.

In slightly more detail, from my perspective the modularisation and 
web-service issues are nearly the same, in so far as it would be nice to 
break up the components into browse, visualise, deliver, with independent 
web-services with well-defined (simple) interfaces. OK, so that's probably a 
little unfair as I know that you have got a large degree of modularisation 
but I understand that it's not as clean as it could be (you understand that I 
haven't done any of this myself, so feel free to shoot me as the messenger if 
I'm getting it wrong). (And apologies if this sounds carping, it's very 
obvious that a lot of work has gone into making it as good as it is now, not 
only in what it can do, but in terms of the existing modularisation).

I have heard that you are planning on making it possible to put IDL in the 
backend, this would be a good thing (TM), and hopefully the process of doing 
that might make the back-end visualisation separation even cleaner.

The security issue is still important to us; currently we use cookies to do 
this, but it's clumsy ... if you do go down the web-service route, then a 
certificate-based approach would be very helpful for us ... (although the 
sand isn't very firm underfoot for doing this as far as I can tell).

Our last issue is java related. We have a number of folk who are mac based and 
have in the past grumbled, it might be that things are better now (it seems 
that time may solve this and you may not have to do anything). However, a 
clean failure mode is important. I haven't tested this myself, it maybe that 
there is one at V6 and later?

Well, I had hoped to provide you with more info. We may do that over the next 
six months as we do have an internal project to try and use LAS V6+ (&CDAT) 
as one interface to most of our numerical model data, but it rumbles on as we 
have relatively little staff effort for that.

Why web-services and modularisation? In the slightly longer term, clearly we 
want to be able to have metadata services which return a URI which points to 
a LAS enabled service for a single dataset. (I think this is really the only 
way to avoid the existing browsing interface which is a very big problem for 
large systems). This might require the ability to dynamically generate a LAS 
service and populate the metadata backend with templated visualisation 
information and dynamic data information. If this were possible then it would 
be possible for different organisations to deal with their own metadata 
idiosnycracies and browsing concepts without imposing the LAS standard. 
We could also offer a wider variety of delivery mechanisms.

Oh, and while I'm on pie-in-the-sky, OGC Web-Mapping Server compatability?

OK, enough. Thanks for the opportunity to comment, and thanks also for a great 
piece of software as it is. People will always want more from a good thing! 

Bryan
-- 
Bryan Lawrence, Head NCAS/British Atmospheric Data Centre
web: www.badc.nerc.ac.uk  phone: +44 1235 445012
CLRC: Rutherford Appleton Laboratory, Chilton, Didcot, OX110QX, UK

---

From: Steve Hankin <[email-masked]>
Date: Thu, 15 May 2003 10:18:27 -0700

Hi Bryan,

Sorry for the slow response.  The LAS planning meeting that I spoke of had to be
postponed and only just occurred two weeks ago.  For the last two weeks of delay
you can blame me :-}

Responses below ...

> only in what it can do, but in terms of the existing modularisation).

LAS does have an XML protocol by which you can query the server and can request
products.  My question to you:  how great is the importance that this protocol
conform strictly to the SOAP specification? The current XML protocol is documented
at http://www.ferret.noaa.gov/Ferret/LAS/Documentation/manual/reqapi.html.  It is
used by the standard LAS UI and also used to provide the "batch" Unix command line
access to LAS (see
http://tmap.pmel.noaa.gov/~hankin/GODAE/Biarritz/LAS/batch.html)

Both visualization of data and delivery of formatted data subsets utilize the same
protocol, but I don't see this as a limitation.  Visualizations and files are both
regarded as "operations".  The LAS request protocol allows you to specify
customization options that may be distinct for each operation -- for example
setting the contour levels on a plot, or specifying the delimiter character on an
ASCII output file.  So I suspect there is enough generality to meet your needs.

On the topic of modularization (or "modularisation" to be more international) a
further consideration is the adaptability of the standard LAS UI for different
applications.  Version 6.2 has added a lot of new flexibility there.  You can
customize the look and feel of how data sets and variables are presented (e.g. the
collection of model runs at http://www.ferret.noaa.gov/GODAE/servlets/dataset).
You can also enter the user interface either at the top (where the user first
picks the data set of interest); at a named data set (where the user first picks
the variable of interest); or at a named variable (where the user is ready to
specify a region and request output).  This allows you to link into LAS from, say,
a document that discusses a particular satellite SST field.  A further "nice" idea
would be to be able to enter LAS with a pre-specified region and time, but no one
has given this as a strict requirement, so it has not risen onto our top
priorities.

> that might make the back-end visualisation separation even cleaner.

This idea has come up on several occasions but is not yet a "requirement" in our
projects.  It would definitely enhance the ability of LAS to handle high-res
satellite fields.  If you have an IDL expert in your group and you'd like to work
in partnership on this lets definitely follow up!

> sand isn't very firm underfoot for doing this as far as I can tell).

I assume that when you mention security your focus is on access control, rather
that non-specific concerns that LAS may have security holes through which a system
break-in might be possible, right?  Access control is another of those topics
that, while often discussed, is not a top priority for any of the projects that
pay our bills.  Is there a way that we can approach this in partnership?  We would
be happy to make (reasonable) changes in the LAS architecture to help support an
access control layer.  Can we work something out from that starting point?

> there is one at V6 and later?

Improving robustness in funky browsers is a top priority for us.  Here's where LAS
V6.2 is today, and where we are headed:

Today: As of V6.1 LAS has a "graceful fail-over" mode.  On a funky browser LAS
still works, but it lacks a clickable map interface.  That's an improvement at
least over V5 ...

Pretty soon: We are just beginning work a server-side support for a clickable
map.  That should provide a nice solution -- not quite as nice as the Java map,
but pretty good.  At this time I'd say that functionality is still 4 to 6 months
away.

> We could also offer a wider variety of delivery mechanisms.

I think that you can do all of this with the pieces that already exist today.

> Oh, and while I'm on pie-in-the-sky, OGC Web-Mapping Server compatability?

Ah, that would be nice, wouldn't it?  From my reading on the subject the WMS image
output and WCS coverage outputs should be quite straightforward -- essentially LAS
just needs to understand a slightly different (alternative) request protocol and
respond to some new queries and the LAS installer needs to configure in operations
(products) that conform to OGC expectations.  The WFS feature output will require
that we learn our way around the GML specification.  Discussions of OGC always
come up against a chicken-and-egg problem:  where are the client applications that
are going to *use* these servers?  So as a practical matter it is another of those
topics that don't quite make it to the "requirements" list.

> piece of software as it is. People will always want more from a good thing!

I have my fingers crossed that you may take us up on offers for partnerships --
esp. access control and IDL back end.

    - steve (for the LAS development group)

--

---

From: Bryan Lawrence <[email-masked]>
Date: Fri, 23 May 2003 17:32:21 +0100

Hi Steve

Andrew and I had a quick chat about your email. Thanks very much for the 
positive response, and the detailed answer to some of my points. Much for us 
to think about.

By way of a placeholder response, we need to think carefully about what we are 
going to do ourselves over the next few months, but there are two areas where 
we could be interested to work with you  ...

Firstly, the access control issue (yes that's what i mean by security). Andrew 
had a solution working in LAS5 that might work in LAS6+. In some ways it was 
a bit of a hack (I hope he wont mind me putting it like that), but it worked, 
and it might provide the vehicle for a more elegant solution using a local 
user database (what we need) or flat files (what others might want) ..

We'll try and have a look at the feasibility of that and get back to you.

The second area is that we might be able to work on is the OGC services area. 
We are certainly very interested in that on a number of fronts. This would be 
a slightly longer term goal, but providing an appropriate filter on the front 
end, and working through the products required issue would be a good exercise 
for us.

Regrettably, IDL is on my wish list, rather than something we want to do 
ourselves any time soon ... so maybe we'll both carrying on wishing.

There are no timescales above, as I say, we need to think this through. When 
Andrew comes up for air from his current set of tasks, he'll get back to you 
with some details about both issues, and all of us at this end will think 
about timescales etc.

However, we are committed to using LAS for a big demo in September, so we will 
at least have cracked something in the access-control area by then ...

Regards,
Bryan

On Thursday 15 May 2003 18:18, Steve Hankin wrote:
> Hi Bryan,
>
> Sorry for the slow response.  The LAS planning meeting that I spoke of had
> to be postponed and only just occurred two weeks ago.  For the last two
> weeks of delay you can blame me :-}
>
> Responses below ...
>
> Bryan Lawrence wrote:
> > Hi Steve
> >
> > locally, and in part because I hoped Andrew Woolf would get a chance for
> > a chat with you during the opendap meeting ...
> >
> > Anyway I think for us the key issues for LAS are web-services,
> > modularisation, security, and graceful support for java-challenged
> > browsers. I know that most of these are things you are thinking about (or
> > actually doing) anyway, so I suspect there will be no surprises here.
> >
> > web-services with well-defined (simple) interfaces. OK, so that's
> > probably a little unfair as I know that you have got a large degree of
> > modularisation but I understand that it's not as clean as it could be
> > (you understand that I haven't done any of this myself, so feel free to
> > shoot me as the messenger if I'm getting it wrong). (And apologies if
> > this sounds carping, it's very obvious that a lot of work has gone into
> > making it as good as it is now, not only in what it can do, but in terms
> > of the existing modularisation).
>
> LAS does have an XML protocol by which you can query the server and can
> request products.  My question to you:  how great is the importance that
> this protocol conform strictly to the SOAP specification? The current XML
> protocol is documented at
> http://www.ferret.noaa.gov/Ferret/LAS/Documentation/manual/reqapi.html.  It
> is used by the standard LAS UI and also used to provide the "batch" Unix
> command line access to LAS (see
> Both visualization of data and delivery of formatted data subsets utilize
> the same protocol, but I don't see this as a limitation.  Visualizations
> and files are both regarded as "operations".  The LAS request protocol
> allows you to specify customization options that may be distinct for each
> operation -- for example setting the contour levels on a plot, or
> specifying the delimiter character on an ASCII output file.  So I suspect
> there is enough generality to meet your needs.
>
> On the topic of modularization (or "modularisation" to be more
> international) a further consideration is the adaptability of the standard
> LAS UI for different applications.  Version 6.2 has added a lot of new
> flexibility there.  You can customize the look and feel of how data sets
> and variables are presented (e.g. the collection of model runs at
> http://www.ferret.noaa.gov/GODAE/servlets/dataset). You can also enter the
> user interface either at the top (where the user first picks the data set
> of interest); at a named data set (where the user first picks the variable
> of interest); or at a named variable (where the user is ready to specify a
> region and request output).  This allows you to link into LAS from, say, a
> document that discusses a particular satellite SST field.  A further "nice"
> idea would be to be able to enter LAS with a pre-specified region and time,
> but no one has given this as a strict requirement, so it has not risen onto
> our top priorities.
>
> > I have heard that you are planning on making it possible to put IDL in
> > the backend, this would be a good thing (TM), and hopefully the process
> > of doing that might make the back-end visualisation separation even
> > cleaner.
>
> This idea has come up on several occasions but is not yet a "requirement"
> in our projects.  It would definitely enhance the ability of LAS to handle
> high-res satellite fields.  If you have an IDL expert in your group and
> you'd like to work in partnership on this lets definitely follow up!
>
> > The security issue is still important to us; currently we use cookies to
> > do this, but it's clumsy ... if you do go down the web-service route,
> > then a certificate-based approach would be very helpful for us ...
> > (although the sand isn't very firm underfoot for doing this as far as I
> > can tell).
>
> I assume that when you mention security your focus is on access control,
> rather that non-specific concerns that LAS may have security holes through
> which a system break-in might be possible, right?  Access control is
> another of those topics that, while often discussed, is not a top priority
> for any of the projects that pay our bills.  Is there a way that we can
> approach this in partnership?  We would be happy to make (reasonable)
> changes in the LAS architecture to help support an access control layer. 
> Can we work something out from that starting point?
>
> > Our last issue is java related. We have a number of folk who are mac
> > based and have in the past grumbled, it might be that things are better
> > now (it seems that time may solve this and you may not have to do
> > anything). However, a clean failure mode is important. I haven't tested
> > this myself, it maybe that there is one at V6 and later?
>
> Improving robustness in funky browsers is a top priority for us.  Here's
> where LAS V6.2 is today, and where we are headed:
>
> Today: As of V6.1 LAS has a "graceful fail-over" mode.  On a funky browser
> LAS still works, but it lacks a clickable map interface.  That's an
> improvement at least over V5 ...
>
> Pretty soon: We are just beginning work a server-side support for a
> clickable map.  That should provide a nice solution -- not quite as nice as
> the Java map, but pretty good.  At this time I'd say that functionality is
> still 4 to 6 months away.
>
> > Well, I had hoped to provide you with more info. We may do that over the
> > next six months as we do have an internal project to try and use LAS V6+
> > (&CDAT) as one interface to most of our numerical model data, but it
> > rumbles on as we have relatively little staff effort for that.
> >
> > Why web-services and modularisation? In the slightly longer term, clearly
> > we want to be able to have metadata services which return a URI which
> > points to a LAS enabled service for a single dataset. (I think this is
> > really the only way to avoid the existing browsing interface which is a
> > very big problem for large systems). This might require the ability to
> > dynamically generate a LAS service and populate the metadata backend with
> > templated visualisation information and dynamic data information. If this
> > were possible then it would be possible for different organisations to
> > deal with their own metadata idiosnycracies and browsing concepts without
> > imposing the LAS standard. We could also offer a wider variety of
> > delivery mechanisms.
>
> Ah, that would be nice, wouldn't it?  From my reading on the subject the
> WMS image output and WCS coverage outputs should be quite straightforward
> -- essentially LAS just needs to understand a slightly different
> (alternative) request protocol and respond to some new queries and the LAS
> installer needs to configure in operations (products) that conform to OGC
> expectations.  The WFS feature output will require that we learn our way
> around the GML specification.  Discussions of OGC always come up against a
> chicken-and-egg problem:  where are the client applications that are going
> to *use* these servers?  So as a practical matter it is another of those
> > OK, enough. Thanks for the opportunity to comment, and thanks also for a
> > great piece of software as it is. People will always want more from a
> > good thing!
>
> I have my fingers crossed that you may take us up on offers for
> partnerships -- esp. access control and IDL back end.
>