<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Am 20.01.2012 14:12, schrieb Sylvia Murphy:
<blockquote
cite="mid:CAKRbQhwEJ2Z86uhDEpkKODRF-_KvN8MMTy5K-ryJ1480L7ka6w@mail.gmail.com"
type="cite">Hi Estani, <br>
<br>
comments inline<br>
<br>
<div class="gmail_quote">On Fri, Jan 20, 2012 at 5:49 AM,
Estanislao Gonzalez <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:gonzalez@dkrz.de">gonzalez@dkrz.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hi Sylvia,<br>
<br>
I was referring to the trackback only. Also
questionnaire/CIM metadata.<br>
The only "data" reference was the implication that if both
use the same xml parser (i.e. if it's used as a singleton)
then the same problem will arise at the metadata harvesting
end as what I've seen at the harvesting end.<br>
</div>
</blockquote>
<div><br>
There is no parsing for data, which was my point. The only
XML in the system that requires parsing is the CIM XML. Data
metadata is harvests from the TDS, which has nothing to do
with XML. <br>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <br>
Indeed we don't have any GFDL data, but we do have MPI data.
In both cases the system says there's no data at the moment,
which is simply not true for MPI.<br>
Just see for yourself at the link I've provided:<br>
<a moz-do-not-send="true"
href="http://ipcc-ar5.dkrz.de/trackback/query.htm?id=esg%3amodel_mpiom_decadal1960-lrbasesimulation&session=true&fromSearch=true"
target="_blank">http://ipcc-ar5.dkrz.de/trackback/query.htm?id=esg%3amodel_mpiom_decadal1960-lrbasesimulation&session=true&fromSearch=true</a><br>
go to outputs: "Output data for this simulation is not yet
available in the archive."<br>
</div>
</blockquote>
<div><br>
<br>
Estani, recall that I already identified this issue with you
over the holidays. There is most likely a mismatch between
the model name in the questionnaire and the one in the data.
There are two mapping files that need to be updated to make
that connection work. Nate sent out the information on how to
update these files, we have been waiting for you to give this
a try. <br>
</div>
</div>
</blockquote>
Then I'm sorry for not answering sooner. Could you please send the
advisory again as I can't find anything in my email box.<br>
<blockquote
cite="mid:CAKRbQhwEJ2Z86uhDEpkKODRF-_KvN8MMTy5K-ryJ1480L7ka6w@mail.gmail.com"
type="cite">
<div class="gmail_quote">
<div>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <br>
We only have data at 1.3.4 so I guess we should keep waiting
until G2.0 comes out and is ready for production (we do have
a G2.0-RC1 installed, but we can't update more often nor
publish data to it at this time because we are short on
resources for both tasks). The current situation is a
problem for our users and to those of us that answer at the
help-desk.<br>
<br>
So basically, can't say if it's corrected in G2.0, but in
1.3.4 if the data-metadata connection was broken, it
resulted in a message saying there's no data.<br>
My point: if we can't be sure that's the case, then don't
tell the user it is.<br>
</div>
</blockquote>
<div><br>
A quick note about the message on the outputs tab.
Originally, when no data, metadata connection was made we just
did not have that tab but we found that in testing we could
not tell if its absence was due to an error or because there
was no data. Now the system does a check and if it does not
find data the tab is present and the error message exists.
After implementation, we realized that method had its own
downside and put in a ticket to make it better. We are not
sure when that work will be programmed, however. Would you
recommend going back to the original behavior? <br>
</div>
</div>
</blockquote>
Yes. It's better.<br>
<br>
Thanks,<br>
Estani<br>
<blockquote
cite="mid:CAKRbQhwEJ2Z86uhDEpkKODRF-_KvN8MMTy5K-ryJ1480L7ka6w@mail.gmail.com"
type="cite">
<div class="gmail_quote">
<div>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <br>
Thanks,<br>
Estani<br>
Am 19.01.2012 18:04, schrieb Sylvia Murphy:
<div>
<div class="h5">
<blockquote type="cite">
<div>Hi Estani,</div>
<div> </div>
<div>I am on military duty this week and have only
been able to follow your concerns from afar. </div>
<div> </div>
<div>In your latest email, I am having trouble
determining whether your issues are for searching
for DATA or METADATA. </div>
<div> </div>
<div>Here are some key points:</div>
<div> </div>
<div>* How the information about data and metadata
gets into the system is different for each type.
Metadata as you know comes from the questionnaire.
Data metadata comes from the harvesting of the TDS
system. </div>
<div> </div>
<div>* The path for data and metadata is different
such that there may be metadata before data or vice
versa. We actually have very limited cases right
now with both existing at the same time.</div>
<div> </div>
<div>* ESG 1.3.x requires that data and metadata be
co-located on the same system for the output tab to
show any information. ESG 2.0 is federated and this
requirement goes away. Note that the GFDL example
you point to below is a sample metadata instance
that I created a long time ago for demonstration
purposes. There is no data associated with it.
This instance and all references to it have been
removed in 2.0. The data connection is working at
BADC for those hadgem-2 instances they have local
data for. </div>
<div> </div>
<div> </div>
<div>I am happy to create some tickets in the system
for you, but I need to better understand what
searches you are trying to conduct. It would also
be useful to determine if the issues you are seeing
in 1.3.x have been fixed in 2.0. </div>
<div> </div>
<div>Cheers,</div>
<div> </div>
<div>Sylvia</div>
<div> </div>
<div> </div>
<div class="gmail_quote">On Thu, Jan 19, 2012 at 3:18
AM, Estanislao Gonzalez <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:gonzalez@dkrz.de" target="_blank">gonzalez@dkrz.de</a>></span>
wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px
solid;MARGIN:0px 0px 0px 0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi,<br>
<br>
I think this is a good opportunity to describe the
problem with more detail.<br>
<br>
The GFDL results are different for DKRZ and BADC.
Neither Gateway has local GFDL data and both are
exactly the same version.<br>
So the only difference between them is the
harvesting time.<br>
<br>
I already mentioned a problem while harvesting the
OAI from other gateways (a concurrency problem),
not sure this got solved. If the trackback is
ussing, yet again, the same xml parser in the same
thread-unsafe way, then it might be potentially
breaking in the middle of the transaction leaving
the DB/3Store (whatever it's writing to) in an
incomplete state.<br>
In any case if the harvesting were deterministic,
this shouldn't really matter and both DKRZ and
BADC should present the same results (whatever
that might be).<br>
<br>
Searching for GFDL at DKRZ breaks the system, this
is something that needs urgent attention in my
opinion. How could a query of a specific term
result in an internal error?<br>
Apparently if the user searches for anything that
would result in a broken document (perhaps some
GFDL ones?) then it breaks with an "internal
error".<br>
We can't tell them to search for good ones only.<br>
<br>
Furthermore the output tab says: "Output data for
this simulation is not yet available in the
archive."<br>
(see <a moz-do-not-send="true"
href="http://ipcc-ar5.dkrz.de/trackback/query.htm?id=esg%3amodel_gfdl_lm2_control_1860_r1i1&session=true&fromSearch=true"
target="_blank">http://ipcc-ar5.dkrz.de/trackback/query.htm?id=esg%3amodel_gfdl_lm2_control_1860_r1i1&session=true&fromSearch=true</a><br>
<a moz-do-not-send="true"
href="http://ipcc-ar5.dkrz.de/trackback/query.htm?id=esg%3amodel_mpiom_decadal1960-lrbasesimulation&session=true&fromSearch=true"
target="_blank">http://ipcc-ar5.dkrz.de/trackback/query.htm?id=esg%3amodel_mpiom_decadal1960-lrbasesimulation&session=true&fromSearch=true</a>)<br>
Which isn't correct and is misleading users to
think institutions haven't provide any data. I
think that if we can't get it right, we should
better right that "we don't know" than assuring
the user there's no data at all...<br>
<br>
My 2c,<br>
Estani<br>
<br>
Am 18.01.2012 22:44, schrieb Cecelia DeLuca:
<div>
<div><br>
<blockquote style="BORDER-LEFT:#ccc 1px
solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex" class="gmail_quote">Hi
Serguei and all,<br>
<br>
Thanks, I see the error now too.<br>
<br>
From BADC I tried a Datasets text search
for GFDL and it reliably hangs<br>
for me in "Loading, please<br>
wait." If I click the x of that "Loading
..." box to cancel it and try<br>
to go to the second (or third ...) page<br>
of those results I get an internal server
error. Seeing that, I'm not<br>
sure that the problem lies with the<br>
metadata or the metadata integration with
search but maybe with the<br>
keyword search itself. Our team<br>
doesn't have expertise in that part of the
system so maybe someone else<br>
can comment if this is<br>
a known bug.<br>
<br>
It would be nice to have an organized list
of known bugs that may affect<br>
users for the CMIP5<br>
production system that users could see ...<br>
<br>
Best,<br>
Cecelia<br>
<br>
<br>
On 1/18/2012 11:30 AM, Serguei Nikonov
wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px
solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi Cecelia,<br>
<br>
thanks for your help. I made a search on
"CMIP5 simulation metadata"<br>
page you pointed out and didn't find any
GFDL simulations there. As<br>
you told it means that we should not
expect to get any simulations in<br>
gateway search.<br>
<br>
Regarding the stability of search error on
BADC gateway, I tested it<br>
today again several times and it's
persistent. I chose "Simulations"<br>
type of search and put "GFDL" into search
field. It returned 23<br>
results split into 3 pages. When I tried
to go to 2nd page "An<br>
internal server error..." happened.<br>
<br>
Thanks,<br>
Sergey<br>
<br>
<br>
On 01/17/2012 07:42 PM, Cecelia DeLuca
wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px
solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi Serguei,<br>
<br>
Sylvia is out this week but I just spoke
to her about your questions<br>
and relay the<br>
responses below. Please look here:<br>
<a moz-do-not-send="true"
href="http://q.cmip5.ceda.ac.uk/feeds/cmip5/simulation/"
target="_blank">http://q.cmip5.ceda.ac.uk/feeds/cmip5/simulation/</a><br>
<br>
You can see the simulations entered
through the CMIP5 questionnaire and<br>
available for<br>
ingestion into the metadata trackback.
If you don't see the ones you<br>
expect<br>
from GFDL,<br>
it means that the process of submitting
these simulations through the<br>
questionnaire isn't completed.<br>
<br>
The only one I see in the trackback is
an early synthetic example<br>
from GFDL that we<br>
used for testing (Control-1860).<br>
<br>
Our team doesn't handle the
questionnaire input, the METAFOR group<br>
does. Ron and<br>
Charlotte Pascoe from METAFOR have a
dialogue going about how to<br>
complete the GFDL<br>
submissions. When that works more
simulations should show up at the<br>
URL above, and<br>
will then be ingested and shown in the
metadata trackback.<br>
<br>
It would be good to test in a current
(1.3.4) version of the<br>
gateway. PCMDI is<br>
still at an earlier<br>
version (1.3.2) and may not show the
same results as others. I was<br>
testing<br>
earlier today<br>
and saw erratic errors there, either
hanging while loading or getting<br>
the internal<br>
server error, in all kinds of search
situations. I didn't see these<br>
issues<br>
testing at BADC<br>
and could not reproduce the internal
server error you got when going<br>
to the second<br>
page of your search results. Does that
reliably fail for you?<br>
<br>
There are a number of other issues with
metadata being tracked as<br>
known bugs.<br>
Sylvia<br>
recently sent out a summary. Her mail
is included below my<br>
response. I added some<br>
updates.<br>
<br>
In summary, to begin to understand and
fix the issues you see:<br>
- ensure GFDL simulations are submitted,
and only expect to see<br>
simulations that are<br>
available at the URL above<br>
- test on gateways that have 1.3.4
installed<br>
- remaining errors should be analyzed
and checked against known bugs<br>
below<br>
<br>
Please write if you have further
questions.<br>
<br>
Best,<br>
Cecelia<br>
<br>
<br>
-------- Original Message --------<br>
Subject: [Curator] Status report on
metadata display issues<br>
Date: Mon, 9 Jan 2012 09:10:06 -0700<br>
From: Sylvia Murphy<<a
moz-do-not-send="true"
href="mailto:sylvia.murphy@noaa.gov"
target="_blank">sylvia.murphy@noaa.gov</a>><br>
To: <a moz-do-not-send="true"
href="mailto:go-essp-tech@ucar.edu"
target="_blank">go-essp-tech@ucar.edu</a><br>
CC: Metafor List<<a
moz-do-not-send="true"
href="mailto:metafor@lists.enes.org"
target="_blank">metafor@lists.enes.org</a>>,
<a moz-do-not-send="true"
href="mailto:curator@list.woc.noaa.gov"
target="_blank">curator@list.woc.noaa.gov</a><br>
<br>
<br>
<br>
Hi Everyone,<br>
<br>
Since we have not had a go-essp-tech
call in quite a while, I thought<br>
I would<br>
send out an email summarizing the
metadata-display operational issues<br>
going<br>
across the federation:<br>
<br>
<br>
STATUS OF BUGS IN THE SOFTWARE: These
issues affect every<br>
installation and all<br>
users...<br>
<br>
1) CIM instances (that contain unicode
characters) are not displaying<br>
properly.<br>
This includes instances that contain %
signs, "(", or umlauts. This<br>
has been<br>
permanently fixed in 2.0 RC1, but will
still be an issue for users<br>
until 2.0 is<br>
finalized and installed across the
federation. [CD - I don't know<br>
about across<br>
the federation, but if anybody does end
up installing 2.0, it should<br>
be fixed<br>
there. Goes for the next item too.]<br>
<br>
2) Users are presented with a blurb on
the simulation search page<br>
indicating<br>
that "Much of the 5th Coupled Model
Intercomparison Project (CMIP5)<br>
metadata<br>
that will be accessible from this
gateway is not yet available". This<br>
has been<br>
fixed in 2.0 RC1, but will still be an
issue for users until 2.0 is<br>
finalized<br>
and installed across the federation.<br>
<br>
CENTER SPECIFIC ISSUES:<br>
<br>
There are some gateways with technical
issues that also affect users:<br>
<br>
1) PCMDI: Users get an internal server
error when they click on the<br>
results of<br>
any metadata search, and see no model
metadata at all. This fix<br>
requires that<br>
PCMDI update their postgres database.
[CD: Looks like the postgres<br>
database was<br>
updated by PCMDI in the last week, so
metadata is working in the<br>
PCMDI gateway,<br>
the first time since September. Yay!!
However, PCMDI is still at<br>
gateway<br>
version 1.3.2, and behavior may not be
the same as behavior of<br>
gateways with the<br>
latest production version installed.]<br>
<br>
2) NCI: Users see most (37/61) but not
all metadata records because<br>
NCI needs<br>
to reharvest their triple store as part
of their upgrade to 1.3.4.<br>
We are in<br>
contact with NCI and walking them
through this. Note that triple store<br>
reharvests will be significantly less
painful with ESG 2.0.<br>
<br>
3) DKRZ: Users will not see data links
in the trackback for local<br>
DKRZ data.<br>
This problem has been identified as a
mismatch between the names in<br>
the data and<br>
what is coming out of the CIM. Such
mismatches were anticipated and<br>
a mapping<br>
file created to translate between the
two. This file just needs to<br>
be updated<br>
at DKRZ. This is being worked.<br>
<br>
As always, you may check the status of
metadata across the federation<br>
at:<br>
<a moz-do-not-send="true"
href="http://esgf.org/wiki/Cmip5Status"
target="_blank">http://esgf.org/wiki/Cmip5Status</a><br>
I last checked all the sites on 6
January and recheck every Friday.<br>
<br>
<br>
<br>
<br>
On 1/17/2012 12:51 PM, Serguei Nikonov
wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px
solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi Sylvia,<br>
<br>
Ron Stouffer found the serious issue
with GFDL simulations - they<br>
are not seen<br>
all through gateway search interface.
I tried it on different<br>
gateways (PCMDI,<br>
BADC, DKRZ), the results are not good:<br>
<br>
1. "Simulations" search by keywords
does not work normally. On PCMDI<br>
and DKRZ<br>
gateways "GFDL" keyword in search
generates message "An internal<br>
server error<br>
has occurred. The problem has been
logged and the administrator<br>
notified". BADC<br>
is little bit better but the search
returns 23 results (on 3 pages)<br>
which are<br>
only 6 is GFDL; further, going to the
2nd page of returned results<br>
gives the<br>
same internal server error.<br>
<br>
2. Searching without keywords returns
only 6 GFDL results (PCMDI,<br>
BADC, DKRZ).<br>
<br>
Other issue is that the tab "Outputs"
in chosen simulation does not<br>
contain any<br>
datasets. I am not sure whether it's
working functionality currently<br>
cause I<br>
could find only one good working
example on BADC gateway for<br>
HadGEM2-ES model.<br>
<br>
Estani recommended me to ask you about
help. Can you advise where to<br>
start with<br>
to fix it? Is it GFDL problem or it's
general issue.<br>
<br>
Thanks,<br>
Sergey Nikonov<br>
GFDL Data Portal<br>
<br>
On 01/16/2012 12:05 PM, Estanislao
Gonzalez wrote:<br>
<blockquote style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi Sergei,<br>
<br>
that's a whole different issue... I
think it's best to start a new<br>
thread for<br>
that and ask Sylvia Murphy<<a
moz-do-not-send="true"
href="mailto:Sylvia.Murphy@noaa.gov"
target="_blank">Sylvia.Murphy@noaa.gov</a>>
directly.<br>
There are many issues regarding
Simulations right now... I don't<br>
think this will<br>
affect that at all, since
simulations has no dataset version
concept.<br>
<br>
If simulations are not appearing as
they should, or the link in<br>
trackback to the<br>
datasets is not being displayed,
this normally has to do with a<br>
problem in the<br>
DB of the Gateway and has no
relation to the data nodes
whatsoever.<br>
So If you<br>
find any problems related to
Simulations, you should contact Bob<br>
and Sylvia<br>
directly.<br>
As of this time there are still some
unresolved issues though.<br>
<br>
thanks,<br>
Estani<br>
Am 16.01.2012 17:39, schrieb Serguei
Nikonov:<br>
<blockquote style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi Estani,<br>
<br>
thanks for you help. I realize
that I need to republish all<br>
datasets. My main<br>
point right now is what should be
done else to make simulations<br>
visible<br>
(datasets are OK) in "Simulation"
search. This is the main issue<br>
currently. Or<br>
"harmonizing" versions in datasets
on gateway with physical<br>
version in DRS<br>
will be enough and simulations
will appear in "Simulations"
search<br>
after that<br>
at once?<br>
<br>
Thanks,<br>
Sergey<br>
<br>
On 01/16/2012 11:28 AM, Estanislao
Gonzalez wrote:<br>
<blockquote
style="BORDER-LEFT:#ccc 1px
solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi Sergey,<br>
<br>
sorry if this got confusing but
there's a small problem with
what<br>
Hans said.<br>
Setting "version_by_date=true"
won't help you here, since AFAIK<br>
it will generate<br>
a version from the current date,
which wont match what you<br>
already have in you<br>
directory path.<br>
<br>
To publish a version that's not
the current date, you should
pass<br>
it to the<br>
esgpublisher "somehow". One
option is to use the
--new-version<br>
flag as I said,<br>
another would be to provide a
list of datasets and versions
via the<br>
--version-list flag (check the
--help)<br>
<br>
I' CCing Bob, maybe he has a
better approach. But I'd say,
you<br>
have to<br>
re-publish everything with the
proper version.<br>
<br>
If you use mapfiles, the
simplest way to do it I can
think of is<br>
to extract the<br>
version and dataset from the map
file using an sed command:<br>
sed
's#^\([^|].*\)|.*/v\(20[0-9]*\)/.*#\1|\2#'<_your_map_file><br>
|sort -u<br>
<br>
If you use bash you could do
(after unpublishing):<br>
map=<path_to_map_file><br>
esgpublish --map $map
--version-list<(sed<br>
's#^\([^|].*\)|.*/v\(20[0-9]*\)/.*#\1|\2#'
$map |sort -u)<br>
....[and the rest as<br>
usual]<br>
<br>
This ways you could publish
mapfiles which contains datasets
from<br>
different<br>
versions. If all files in the
mapfile are from the same
version<br>
(might be from<br>
different datasets, no problem)
then it's easier to use<br>
--new-version 2011xxxx<br>
It depends on what your
environment looks like.<br>
<br>
Hope this helps,<br>
Estani<br>
<br>
<br>
Am 16.01.2012 16:58, schrieb
Hans Ramthun:<br>
<blockquote
style="BORDER-LEFT:#ccc 1px
solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hallo
Sergey,<br>
<br>
What I wanted is described
here were Estani pointed to in
one of<br>
his mails:<br>
<a moz-do-not-send="true"
href="http://esg-pcmdi.llnl.gov/internal/esg-data-node-documentation/cmip5-best-practices"
target="_blank">http://esg-pcmdi.llnl.gov/internal/esg-data-node-documentation/cmip5-best-practices</a><br>
<br>
<br>
<br>
If you publish the datasets
with the option
'version_by_date =<br>
true' in the<br>
project section of the esg.ini
file then the discrepancy
should<br>
be reversed<br>
and the view in the tds should
be correct.<br>
<br>
The gateway search will always
show the most recent version
of<br>
the data and in<br>
the history tab other versions
of the found data.<br>
<br>
So the only thing you have to
do is to insert the above
option<br>
in the esg.ini<br>
file of your data node before
publishing the data.<br>
<br>
Hope that clarified this.<br>
<br>
Regards<br>
Hans<br>
<br>
<br>
<br>
Am 16.01.2012 16:36, schrieb
Serguei Nikonov:<br>
<blockquote
style="BORDER-LEFT:#ccc 1px
solid;MARGIN:0px 0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi Hans,<br>
<br>
so, if I change version of
dataset on gateway to have
it the<br>
same as what we<br>
have in physical path of
files then "Simulations"
search in<br>
gateway (e.g.<br>
<a moz-do-not-send="true"
href="http://cmip-gw.badc.rl.ac.uk/query/advanced.htm?product=ConfiguredModelwill"
target="_blank">http://cmip-gw.badc.rl.ac.uk/query/advanced.htm?product=ConfiguredModelwill</a>)<br>
<br>
gives all published
experiments visible in
"Datasets" search?<br>
And also<br>
"Output" tab of simulation
will contain links to
datasets? Do I<br>
understand<br>
correctly what the final
result should be achieved?<br>
<br>
Thanks,<br>
Sergey<br>
<br>
On 01/16/2012 02:56 AM, Hans
Ramthun wrote:<br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px 0px
0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hallo
Estani,<br>
<br>
Correct that discrepancy
is what I meant.<br>
<br>
Thanks for investigating
and clarifying this<br>
Hans<br>
<br>
Am 14.01.2012 20:28,
schrieb Estanislao
Gonzalez:<br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px 0px
0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi,<br>
<br>
I've CCed Has, for some
reason it wasn't in the
list.<br>
<br>
Anyway, the problem was
the version number of
the datasets,<br>
e.g.:<br>
This is the url of the
metadata to file:<br>
<a
moz-do-not-send="true"
href="http://esgdata.gfdl.noaa.gov/thredds/esgcet/1/cmip5.output1.NOAA-GFDL.GFDL-CM3.historical.mon.atmos.Amon.r1i1p1.v1.html?dataset=cmip5.output1.NOAA-GFDL.GFDL-CM3.historical.mon.atmos.Amon.r1i1p1.v1.ps_Amon_GFDL-CM3_historical_r1i1p1_186001-186412.nc"
target="_blank">http://esgdata.gfdl.noaa.gov/thredds/esgcet/1/cmip5.output1.NOAA-GFDL.GFDL-CM3.historical.mon.atmos.Amon.r1i1p1.v1.html?dataset=cmip5.output1.NOAA-GFDL.GFDL-CM3.historical.mon.atmos.Amon.r1i1p1.v1.ps_Amon_GFDL-CM3_historical_r1i1p1_186001-186412.nc</a><br>
<br>
<br>
<br>
<br>
<br>
says the dataset it's v1
as you note from the
url. Now the<br>
download path is:<br>
<a
moz-do-not-send="true"
href="http://esgdata.gfdl.noaa.gov/thredds/fileServer/gfdl_dataroot/NOAA-GFDL/GFDL-CM3/historical/mon/atmos/Amon/r1i1p1/v20110601/ps/ps_Amon_GFDL-CM3_historical_r1i1p1_186001-186412.nc"
target="_blank">http://esgdata.gfdl.noaa.gov/thredds/fileServer/gfdl_dataroot/NOAA-GFDL/GFDL-CM3/historical/mon/atmos/Amon/r1i1p1/v20110601/ps/ps_Amon_GFDL-CM3_historical_r1i1p1_186001-186412.nc</a><br>
<br>
<br>
<br>
<br>
<br>
where you see that the
version in the DRS path
says the<br>
version is 20110601.<br>
<br>
The problem is the
discrepancy between the
two versions. The<br>
required<br>
step was<br>
to publish the dataset
using --new-version
20110601 or any of<br>
the others<br>
possibilities the
publisher offers.<br>
<br>
Hans, that's what you've
meant right?<br>
<br>
Cheers,<br>
Estani<br>
<br>
Am 13.01.2012 21:18,
schrieb Serguei Nikonov:<br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi
Hans,<br>
<br>
Ron asked me to follow
up this issue. As I
understand<br>
primary problem is<br>
with<br>
visibility of GFDL
metafor metadata from
different gateways,<br>
isn't it?<br>
<br>
As I checked GFDL
data, it's accessible
from European<br>
gateways. I am<br>
wondering how type of
versioning (numbers or
date) in<br>
datasets can<br>
affect on<br>
this primary problem?
May be I am not clear
understand where<br>
is the<br>
problem,<br>
can you explain me,
please.<br>
<br>
Thanks,<br>
Sergey Nikonov,<br>
GFDL Data Portal<br>
<br>
<br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Am
13.01.2012 09:05,
schrieb Hans
Ramthun:<br>
Hallo Estani,<br>
<br>
How can Ron fix this
problem with the
version numbers like<br>
v1,v2,...on<br>
</blockquote>
the<br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">GFDL
thredds server<br>
(<a
moz-do-not-send="true"
href="http://esgdata.gfdl.noaa.gov/thredds/esgcet/catalog.html"
target="_blank">http://esgdata.gfdl.noaa.gov/thredds/esgcet/catalog.html</a>)?<br>
<br>
Could you please
guide him to get the
correct ones like<br>
v20120113,...?<br>
<br>
Thanks<br>
Hans<br>
<br>
<br>
--------
Original-Nachricht
--------<br>
Betreff: Re:
[metafor] metafor
questionnaire<br>
Datum: Thu, 12 Jan
2012 12:56:39 -0500<br>
Von: Ron
Stouffer<<a
moz-do-not-send="true"
href="mailto:ronald.stouffer@noaa.gov" target="_blank">ronald.stouffer@noaa.gov</a>><br>
<mailto:<a
moz-do-not-send="true"
href="mailto:ronald.stouffer@noaa.gov" target="_blank">ronald.stouffer@noaa.gov</a>><br>
Organisation:
Geophysical Fluid
Dynamics Laboratory<br>
An: Hans Ramthun<<a
moz-do-not-send="true" href="mailto:ramthun@dkrz.de" target="_blank">ramthun@dkrz.de</a>>
<mailto:<a
moz-do-not-send="true"
href="mailto:ramthun@dkrz.de" target="_blank">ramthun@dkrz.de</a>><br>
<br>
<br>
<br>
Hi Hans,<br>
<br>
I am not sure if the
problem you point
out is my problem<br>
(something that<br>
GFDLers need to fix)
or something that
somebody involved in<br>
the CMIP<br>
data serving
software needs to
fix.<br>
<br>
If it is our (GFDL)
problem. How do we
go about fixing
it???<br>
<br>
Thanks for the
comment.<br>
-Ron<br>
<br>
On 1/12/2012 9:04
AM, Hans Ramthun
wrote:<br>
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hallo Ron,<br>
<br>
When I go to the
GFDL thredds
server<br>
(<a
moz-do-not-send="true"
href="http://esgdata.gfdl.noaa.gov/thredds/esgcet/catalog.html"
target="_blank">http://esgdata.gfdl.noaa.gov/thredds/esgcet/catalog.html</a>)<br>
I find only<br>
datasets with
version numbers
like v1,v2,...<br>
Normally I would
expect here
something like a
data:<br>
v20120112 or so.<br>
<br>
Cheers<br>
Hans<br>
<br>
<br>
Am 12.01.2012
14:08,<a
moz-do-not-send="true"
href="mailto:schriebcharlotte.pascoe@stfc.ac.uk" target="_blank">schriebcharlotte.pascoe@stfc.ac.uk</a>:<br>
</blockquote>
</blockquote>
<mailto:<a
moz-do-not-send="true"
href="mailto:charlotte.pascoe@stfc.ac.uk" target="_blank">charlotte.pascoe@stfc.ac.uk</a>:><br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Hi all,<br>
<br>
There are no
GFDL simulations
on the list of
published<br>
CIM documents<br>
on the
questionnaire
atom feed.<br>
However, there
are 26
simulations
documented in
the GFDL<br>
pages of the<br>
questionnaire.<br>
Ron, you'll have
just received an
email from the
help<br>
desk asking you<br>
to hit the
publish button
on you
simulation
documents and<br>
offering an<br>
online demo
session to help
iron out any
niggles.<br>
<br>
best,<br>
Charlotte<br>
<br>
-----Original
Message-----<br>
<a
moz-do-not-send="true"
href="mailto:From%3Ametafor-bounces@lists.enes.org" target="_blank">From:metafor-bounces@lists.enes.org</a><br>
</blockquote>
</blockquote>
</blockquote>
<mailto:<a
moz-do-not-send="true"
href="mailto:metafor-bounces@lists.enes.org" target="_blank">metafor-bounces@lists.enes.org</a>><br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">[mailto:<a moz-do-not-send="true"
href="mailto:metafor-bounces@lists.enes.org"
target="_blank">metafor-bounces@lists.enes.org</a>] On Behalf Of<br>
Bryan Lawrence<br>
Sent: 11 January
2012 13:33<br>
To: Ron Stouffer<br>
Cc: Metafor
List; sylvia
murphy<br>
Subject: Re:
[metafor]
metafor
questionnaire<br>
<br>
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">Bryan,<br>
<br>
Is the ESM2M
(GFDL) metafor
questionnaire
done and
public?<br>
<br>
-Ron<br>
</blockquote>
Hi Ron<br>
<br>
The short answer
is yes.<br>
<br>
The slightly
longer answer
is, that for
reasons I don't<br>
understand<br>
one sees a
different amount
of simulation
metadata for<br>
GFDL in the<br>
various gateways
... There are 23
simulations<br>
descriptions for
GFDL<br>
(including ESM2M
ones) at BADC,
21 at PCMDI and
10 at<br>
NCAR in their<br>
gateway 2 ...<br>
<br>
(You can see
these records by
choosing
simulation<br>
or'simulation<br>
metadata' as the
target in your
search on the
portals,<br>
rather than<br>
</blockquote>
</blockquote>
</blockquote>
data)<br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">I don't know that any of these numbers correspond to
the<br>
number in<br>
the
questionnaire
feed. It's an
interesting
quality<br>
control issue,<br>
and I think
we'll knock
something up to
compare what's<br>
public from<br>
the
questionnaire,
and what appears
in the portals.<br>
<br>
Cheers<br>
Bryan<br>
<br>
-- <br>
Bryan Lawrence<br>
University of
Reading:
Professor of
Weather and
Climate<br>
Computing.<br>
National Centre
for Atmospheric
Science:
Director of<br>
Models and Data.<br>
STFC: Director
of the Centre
for
Environmental
Data<br>
Archival.<br>
Ph:<a
moz-do-not-send="true"
href="tel:%2B44%20118%203786507" value="+441183786507" target="_blank">+44
118 3786507</a><tel:%2B44%20118%203786507>
or 1235<br>
445012;<br>
</blockquote>
</blockquote>
</blockquote>
Web:<a
moz-do-not-send="true"
href="http://home.badc.rl.ac.uk/lawrence" target="_blank">home.badc.rl.ac.uk/lawrence</a><<a
moz-do-not-send="true" href="http://home.badc.rl.ac.uk/lawrence"
target="_blank">http://home.badc.rl.ac.uk/lawrence</a>><br>
<br>
<blockquote
style="BORDER-LEFT:#ccc
1px solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">
<blockquote
style="BORDER-LEFT:#ccc
1px
solid;MARGIN:0px
0px 0px
0.8ex;PADDING-LEFT:1ex"
class="gmail_quote">_______________________________________________<br>
metafor mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:metafor@lists.enes.org" target="_blank">metafor@lists.enes.org</a><mailto:<a
moz-do-not-send="true" href="mailto:metafor@lists.enes.org"
target="_blank">metafor@lists.enes.org</a>><br>
<a
moz-do-not-send="true"
href="https://lists.enes.org/mailman/listinfo/metafor" target="_blank">https://lists.enes.org/mailman/listinfo/metafor</a><br>
</blockquote>
</blockquote>
</blockquote>
-- <br>
Estanislao Gonzalez<br>
<br>
Max-Planck-Institut
fЭr Meteorologie
(MPI-M)<br>
Deutsches
Klimarechenzentrum
(DKRZ) - German
Climate<br>
Computing Centre<br>
Room 108 -
Bundesstrasse 45a,
D-20146 Hamburg,
Germany<br>
<br>
Phone:<a
moz-do-not-send="true"
href="tel:%2B49%20%2840%29%2046%2000" value="+49404600" target="_blank">+49
(40) 46 00</a><br>
94-126<tel:%2B49%20%2840%29%2046%2000%2094-126><br>
<a
moz-do-not-send="true"
href="mailto:E-Mail%3Agonzalez@dkrz.de" target="_blank">E-Mail:gonzalez@dkrz.de</a><mailto:<a
moz-do-not-send="true" href="mailto:gonzalez@dkrz.de" target="_blank">gonzalez@dkrz.de</a>><br>
<br>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
_______________________________________________<br>
GO-ESSP-TECH mailing list<br>
<a moz-do-not-send="true"
href="mailto:GO-ESSP-TECH@ucar.edu"
target="_blank">GO-ESSP-TECH@ucar.edu</a><br>
<a moz-do-not-send="true"
href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech"
target="_blank">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a><br>
</blockquote>
</blockquote>
</blockquote>
_______________________________________________<br>
GO-ESSP-TECH mailing list<br>
<a moz-do-not-send="true"
href="mailto:GO-ESSP-TECH@ucar.edu"
target="_blank">GO-ESSP-TECH@ucar.edu</a><br>
<a moz-do-not-send="true"
href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech"
target="_blank">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a><br>
<br>
</blockquote>
<br>
<br>
</div>
</div>
-- <br>
Estanislao Gonzalez<br>
<br>
Max-Planck-Institut für Meteorologie (MPI-M)
<div><br>
Deutsches Klimarechenzentrum (DKRZ) - German
Climate Computing Centre<br>
Room 108 - Bundesstrasse 45a, D-20146 Hamburg,
Germany<br>
<br>
Phone: <a moz-do-not-send="true"
href="tel:%2B49%20%2840%29%2046%2000%2094-126"
value="+4940460094126" target="_blank">+49
(40) 46 00 94-126</a><br>
</div>
E-Mail: <a moz-do-not-send="true"
href="mailto:gonzalez@dkrz.de" target="_blank">gonzalez@dkrz.de</a><br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
******************************************<br>
<font color="#888888">Sylvia Murphy<br>
NESII/CIRES/NOAA Earth System Research Laboratory<br>
325 Broadway, Boulder CO 80305<br>
Time Zone: U.S. Mountain<br>
Web: <a moz-do-not-send="true"
href="http://www.esrl.noaa.gov/nesii/"
target="_blank">http://www.esrl.noaa.gov/nesii/</a><br>
Email: <a moz-do-not-send="true"
href="mailto:sylvia.murphy@noaa.gov"
target="_blank">sylvia.murphy@noaa.gov</a><br>
Phone: <a moz-do-not-send="true"
href="tel:303-497-7753" value="+13034977753"
target="_blank">303-497-7753</a><br>
</font><br>
<br>
</blockquote>
<br>
<br>
<pre cols="72">--
Estanislao Gonzalez
Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
Phone: <a moz-do-not-send="true" href="tel:%2B49%20%2840%29%2046%2000%2094-126" value="+4940460094126" target="_blank">+49 (40) 46 00 94-126</a>
E-Mail: <a moz-do-not-send="true" href="mailto:gonzalez@dkrz.de" target="_blank">gonzalez@dkrz.de</a> </pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
******************************************<br>
<font color="#888888">Sylvia Murphy<br>
NESII/CIRES/NOAA Earth System Research Laboratory<br>
325 Broadway, Boulder CO 80305<br>
Time Zone: U.S. Mountain<br>
Web: <a moz-do-not-send="true"
href="http://www.esrl.noaa.gov/nesii/" target="_blank">http://www.esrl.noaa.gov/nesii/</a><br>
Email: <a moz-do-not-send="true"
href="mailto:sylvia.murphy@noaa.gov" target="_blank">sylvia.murphy@noaa.gov</a><br>
Phone: 303-497-7753<br>
</font><br>
<br>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Estanislao Gonzalez
Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
Phone: +49 (40) 46 00 94-126
E-Mail: <a class="moz-txt-link-abbreviated" href="mailto:gonzalez@dkrz.de">gonzalez@dkrz.de</a> </pre>
</body>
</html>