[Go-essp-tech] [esg-node-dev] Re: A CMIP5 FAQ Application

Tue Aug 16 03:43:08 MDT 2011

Hi Martin,

> The problem with tagging users or labelling their contributions as spam is that you may get letters from their lawyers, so I would start out with a
> system we know how to control - there will be enough chaos if we just open it to contributions from the 40 or so groups who are running
> models or data nodes.

My fear is that the alternative to not having a system like this is that communication within the CMIP5 science and data management community will remain poor.  I understand where you are coming from, because I occasionally follow the climate-debate blogosphere, but I think we should run with this system and see where it works for us where it doesn't.  I think we have no hope of designing a system up-front that does the job nearly as well.
I believe the application has the power to control what content is placed on it.  At the moment it's configured to be very open but cmip5-help can give permissions to users based on their reputation score.  For instance your post is moderated if your score is below a certain value.  Similarly we can set the reputation required to answer/comment/vote/flag/etc.  Once a question is accepted it can be flagged as "misleading", "not-relevant", "inappropriate" or anything you want to configure.  Questions can be closed as "Duplicate", "off-topic", "too subjective and argumentative", "outdated".  Anything that we need that it can't do could be developed (we have the code) but we won't know we need it until we have used it for a while.

> We want to have some system for keeping the content up to date -- perhaps this could be done by tagging answers which are specific to a
> current software version with that version (e.g. "Gateway 1.2.x") and then reviewing them as older versions are, hopefully, eliminated from the
> operational federation,

In the end this tool would be *part of* our procedures for communication and issue tracking.  Someone would have to make sure answers are accurately tagged (maybe with software versions) and answered if relevant.  Closing questions that are out of date, flagging them, etc. etc.

Stephen.

---
Stephen Pascoe  +44 (0)1235 445980
Centre of Environmental Data Archival
STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK

From: Juckes, Martin (STFC,RAL,RALSP)
Sent: 16 August 2011 10:01
To: Pascoe, Stephen (STFC,RAL,RALSP); 'Don Middleton'; Karl Taylor
Cc: go-essp-tech at ucar.edu; esg-gateway-dev at earthsystemgrid.org; esg-node-dev at lists.llnl.gov
Subject: RE: [Go-essp-tech] [esg-node-dev] Re: A CMIP5 FAQ Application

Hi Stephen,

The problem with tagging users or labelling their contributions as spam is that you may get letters from their lawyers, so I would start out with a system we know how to control - there will be enough chaos if we just open it to contributions from the 40 or so groups who are running models or data nodes.

We want to have some system for keeping the content up to date -- perhaps this could be done by tagging answers which are specific to a current software version with that version (e.g. "Gateway 1.2.x") and then reviewing them as older versions are, hopefully, eliminated from the operational federation,

Cheers,
Martin

From: Pascoe, Stephen (STFC,RAL,RALSP)
Sent: 16 August 2011 09:49
To: Juckes, Martin (STFC,RAL,RALSP); 'Don Middleton'; Karl Taylor
Cc: go-essp-tech at ucar.edu; esg-gateway-dev at earthsystemgrid.org; esg-node-dev at lists.llnl.gov
Subject: RE: [Go-essp-tech] [esg-node-dev] Re: A CMIP5 FAQ Application

I'm confident the software could be configured to require a login to ask a question and we could constrain logins to those with ESGF OpenIDs.  However I would put this in the category of "premature optimisation".  If we get spurious questions we can tag them as "spam" down-vote them or even delete them.  Users can be given a bad reputation score or you can even ban users.

The system is designed to be open and, to some extent, chaotic.  This encourages people to get involved whilst allowing the community to assign value to users, questions and answers.  Therefore there isn't a workflow as such like you get in a bug tracker.  If we want that we should link questions to bug tracker tickets so that closing a ticket triggers updating the question.  However, I believe too much process could get in the way here.

I'm not thinking the questions I bootstrapped the system with will be typical -- they are just what was in my mind at the time.

Stephen.

---
Stephen Pascoe  +44 (0)1235 445980
Centre of Environmental Data Archival
STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK

From: Juckes, Martin (STFC,RAL,RALSP)
Sent: 16 August 2011 09:33
To: 'Don Middleton'; Karl Taylor; Pascoe, Stephen (STFC,RAL,RALSP)
Cc: go-essp-tech at ucar.edu; esg-gateway-dev at earthsystemgrid.org; esg-node-dev at lists.llnl.gov
Subject: RE: [Go-essp-tech] [esg-node-dev] Re: A CMIP5 FAQ Application

Hello All,

On the FAQ issue - I think we do need an FAQ system that many people can contribute to, but I'd be worried about using an open one. At the moment we are dealing with a handful of developers, but we will have thousands of people looking at the data soon. One web sites work well in co-operative communities, but I think we would be flooded with superficial comments claiming to show that the data is worthless. On the other hand, as Stephen says, there is a clear need for something of this kind to allow us to gather the answers which we are all providing to users.

As Karl says, there is also a need to make the system work better. I notice that some of the answers in the mock-up Stephen has created refer to known bugs which are being worked on - and will presumably be fixed at some point. When we have thousands of questions and answers referring to known bugs, will there be a way of systematically ensuring that answers are updated when bugs are fixed?

Can we use the "stackoverflow" system and configure it so that only a restricted (but large) group of people can contribute?

Cheers,
Martin

From: go-essp-tech-bounces at ucar.edu<mailto:go-essp-tech-bounces at ucar.edu> [mailto:go-essp-tech-bounces at ucar.edu]<mailto:[mailto:go-essp-tech-bounces at ucar.edu]> On Behalf Of Don Middleton
Sent: 16 August 2011 00:27
To: Karl Taylor
Cc: go-essp-tech at ucar.edu<mailto:go-essp-tech at ucar.edu>; esg-gateway-dev at earthsystemgrid.org<mailto:esg-gateway-dev at earthsystemgrid.org>; esg-node-dev at lists.llnl.gov<mailto:esg-node-dev at lists.llnl.gov>
Subject: Re: [Go-essp-tech] [esg-node-dev] Re: A CMIP5 FAQ Application

Hi again, Karl - catching up on email pileup after a bit of vacation. First of all, I really like what Stephen has put together for collaboratively addressing user problems, and think this could help us quite a lot. I personally like using environments like this when dealing with app software problems.

I think there's pretty good agreement that #1 below is a primary problem area. Getting consistent deployments is expected to help some, so the sooner you can upgrade to 1.3.1, the better. The attribute service being down may have been another issue here. Site configuration can be a problem as well. The problem with users hitting something that looked like dual registration interfaces involved several issues. The work is pretty much done on improving that a lot, and that's in 1.3.2 which we'll post more on shortly.

Regarding #2, the community SOLR-search preview and now the 2.0 alpha are expected to solve many problems that have plagued us for quite some time. The sooner that folks can deploy that, the better for testing and soon making things better for our users.

cheers - don

On Aug 4, 2011, at 10:11 AM, Karl Taylor wrote:

Hi all,

I agree with Stephen that our ability to quickly respond to the increasing number of requests seems likely to become overwhelmed.  My cursory impression is, however, that the most common problems reported will be difficult for the user community to respond to; they seem to be associated with flaws in ESG and/or gateway hardware.  Am I correct that most frequent queries to the help desk are related to:

1.  getting error messages, no response, or incorrect notice that a user doesn't have the proper permission to successfully download data

2. having problems finding data using the search capability (because sometimes this fails to return all datasets that it should).

I think NCAR is making progress on the search problems (2 above), but I'm not sure anyone understands whether there are bugs or just confusion that's causing all the problems listed in 1 above.  Perhaps along with thinking about alternatives to the current help desk, immediate attention needs to be paid to reducing the real problems encountered by the user by modifying the ESG software, possibly modifying the ESG user interface, so users won't be so easily confused.

The first step might be to try to confirm that 1 above is causing most of the problems and to pin down exactly why the problems are being encountered.  I think someone with a bit more complete technical understanding of ESG should be able to find the common issues that are being raised and suggest ways they might be addressed.

Best regards,
Karl

On 8/4/11 7:55 AM, stephen.pascoe at stfc.ac.uk<mailto:stephen.pascoe at stfc.ac.uk> wrote:
Hi everyone,

I am increasingly convinced that the CMIP5 HelpDesk isn't scaling to the number of queries we are getting and definitely won't scale as CMIP5 gets more use.  This is partly our fault at BADC for not managing to open up the HelpDesk sufficiently to make it easy for ESGF developers to contribute.  This will improve in the near future (honest!) but I still think we need a more agile way to communicate with CMIP5 users.

Therefore I want to float an alternative solution.  A community-driven, interactive FAQ in the style of stackoverflow.com<http://stackoverflow.com>.  The idea is that users, administrators and developers can collaborate on asking and answering questions in an open forum.  There is a reputation system and a mechanism for voting for answers/questions which enables common questions and good answers to be highlighted.  There is also a tagging system for classification.

I have created a prototype service with a few questions in it at http://esg-dev1.badc.rl.ac.uk/.  Please take a look and give me some feedback.  Even better create an account and start asking and answering questions.  I have deliberately answered only some of the questions to encourage people to get involved.  If you don't like my answer add another one or comment on mine.

If this appears to work for us the site can be moved to a production server easily without losing the questions.  We could then link to it from the "Contact Us" page in the gateways and ask people to email cmip5-helpdesk only for questions they don't want to share with the community.  I would like to discuss this at the ESGF telco on Tuesday.

One note.  You can use ESGF OpenIDs to create an account but CEDA OpenIDs don't work right now.  I need to get Phil to fix that.

Cheers,
Stephen.

---
Stephen Pascoe  +44 (0)1235 445980
Centre of Environmental Data Archival
STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK

--
Scanned by iCritical.

-- 
Scanned by iCritical.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110816/4088f567/attachment-0001.html 