[Go-essp-tech] Incorrect use of missing value in some MRI ocean data -- correction

Tue Dec 6 02:23:37 MST 2011

Hello Martin, Karl,

aren't there other things coming out of this:

1. who is responsible for spotting cases where land points are
represented by 0.0, and cases where they are represented by missing
value.  I think the implication is at the moment its the users.  (I
guess it could have been CMOR - though thats too late now - it could be
something in 'QC' [though I may have used not quite the right term in
saying 'QC']).

2. how are users to know the best ways of dealing with the cases?  ie to
use sftof?  (I guess the danger in giving advice in general on these
sort of things is the advise turns out to be wrong, or not quite
applicable in a particular context).

Jamie

________________________________

	From: go-essp-tech-bounces at ucar.edu
[mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of Karl Taylor
	Sent: 05 December 2011 18:59
	To: martin.juckes at stfc.ac.uk
	Cc: cmip5-users at jiscmail.ac.uk; go-essp-tech at ucar.edu;
graham.parton at stfc.ac.uk
	Subject: Re: [Go-essp-tech] Incorrect use of missing value in
some MRI ocean data -- correction

	Hi Martin,

	Right again.

	thanks,
	Karl

	On 12/5/11 10:39 AM, martin.juckes at stfc.ac.uk wrote: 

		Hello Karl,

		Seiji (from MRI) suggested people use sea-area
fraction"sftof": I think "areacello" (standard name "cell_area") refers
to the actual grid cell, which may be be partially or completely land.
"volcello" (standard name "ocean_volume") doesn't have a definition in
the current standard name table -- but from the name I would expect to
refer to the volume of ocean, so that should be OK for the height
dependent fields,

		cheers,
		Martin

		________________________________
		From: Karl Taylor [taylor13 at llnl.gov]
		Sent: 05 December 2011 18:21
		To: Juckes, Martin (STFC,RAL,RALSP)
		Cc: go-essp-tech at ucar.edu; Parton, Graham
(STFC,RAL,RALSP); cmip5-users at jiscmail.ac.uk
		Subject: Re: [Go-essp-tech] Incorrect use of missing
value in some MRI ocean data -- correction

		Hi Martin,

		You're quite right.  It states:

		"2. Unless otherwise specified, the ocean and sea-ice
output (including Oclim, Oyr, Omon, and OImon) represents a mean over
only the sea portion of each grid cell (i.e., it is interpreted as
"where ocean over ocean"), and a value of 0.0 should be reported where
the sea fraction is 0."

		I'm can't recall whether this specification was the
result of some requirement, or if it was simply a mistake.  Thinking
about it now, I think the recommendation should have been to set it to
the "missing" value, not 0.0, since over land the values are undefined.

		Given that some groups will follow the specified rule,
and others will put in the missing value for land points, users would be
advised to obtain the areacello variable (or volcello) which should be
set to 0.0 where there is no ocean.

		Any other ideas?

		Best regards,
		Karl

		On 12/5/11 9:45 AM,
martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>
<mailto:martin.juckes at stfc.ac.uk>  wrote:

		Hello Karl,

		In the following:

http://cmip-pcmdi.llnl.gov/cmip5/docs/standard_output.xls, sheet
"general", column A, row 15 and 17.

		It says "the value 0.0 should be reported where the sea
fraction is 0",

		regards,
		Martin

		________________________________
		From: Karl Taylor
[taylor13 at llnl.gov<mailto:taylor13 at llnl.gov> <mailto:taylor13 at llnl.gov>
]
		Sent: 05 December 2011 16:42
		To: Juckes, Martin (STFC,RAL,RALSP)
		Cc: go-essp-tech at ucar.edu<mailto:go-essp-tech at ucar.edu>
<mailto:go-essp-tech at ucar.edu> ; Parton, Graham (STFC,RAL,RALSP);
cmip5-users at jiscmail.ac.uk<mailto:cmip5-users at jiscmail.ac.uk>
<mailto:cmip5-users at jiscmail.ac.uk> 
		Subject: Re: [Go-essp-tech] Incorrect use of missing
value in some MRI ocean data -- correction

		Dear Martin and all,

		For 3-d ocean fields like thetao, I think the missing
value (not 0.0) should invariably be assigned to all land grid cells and
for grid cells below the bottom of the ocean.  I can't find where in the
document pointed to below it says otherwise.  I would like to correct
the document, so please tell me where it needs revision.

		thanks,
		Karl

		On 12/4/11 8:07 PM,
martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>
<mailto:martin.juckes at stfc.ac.uk> <mailto:martin.juckes at stfc.ac.uk>
<mailto:martin.juckes at stfc.ac.uk> <mailto:martin.juckes at stfc.ac.uk>
<mailto:martin.juckes at stfc.ac.uk>  wrote:

		I owe MRI an apology -- the use of zero instead of a
missing value is in fact recommended in
(http://cmip-pcmdi.llnl.gov/cmip5/docs/standard_output.xls),

		regards,
		Martin
		________________________________________
		From: Juckes, Martin (STFC,RAL,RALSP)
		Sent: 04 December 2011 08:45
		To: go-essp-tech at ucar.edu<mailto:go-essp-tech at ucar.edu>
<mailto:go-essp-tech at ucar.edu> <mailto:go-essp-tech at ucar.edu>
<mailto:go-essp-tech at ucar.edu> <mailto:go-essp-tech at ucar.edu>
<mailto:go-essp-tech at ucar.edu> ; Parton, Graham (STFC,RAL,SSTD);
cmip5-users at jiscmail.ac.uk<mailto:cmip5-users at jiscmail.ac.uk>
<mailto:cmip5-users at jiscmail.ac.uk> <mailto:cmip5-users at jiscmail.ac.uk>
<mailto:cmip5-users at jiscmail.ac.uk> <mailto:cmip5-users at jiscmail.ac.uk>
<mailto:cmip5-users at jiscmail.ac.uk> 
		Cc: Pascoe, Stephen (STFC,RAL,RALSP)
		Subject: Incorrect use of missing value in some MRI
ocean data.

		Hello,

		It has come to light that some MRI ocean data has land
grid points set to zero rather than to the declared missing value. This
is likely to confuse some applications reading and plotting the data --
I've notified Seiji Yukimoto at MRI,

		sincerely,
		Martin
		________________________________________
		From: Dennis Shea [shea at ucar.edu<mailto:shea at ucar.edu>
<mailto:shea at ucar.edu> <mailto:shea at ucar.edu> <mailto:shea at ucar.edu>
<mailto:shea at ucar.edu> <mailto:shea at ucar.edu> ]
		Sent: 03 December 2011 19:45
		To: Zhang Tao
		Cc: Juckes, Martin (STFC,RAL,RALSP);
haley at ucar.edu<mailto:haley at ucar.edu> <mailto:haley at ucar.edu>
<mailto:haley at ucar.edu> <mailto:haley at ucar.edu> <mailto:haley at ucar.edu>
<mailto:haley at ucar.edu> ; Gary Strand; Karl Taylor
		Subject: Re: [ncl-talk] How to regrid CMIP5 ocean temp.
data to regular 1X1 grid

		CMIP5 users beware!   :-(

		---

		The first rule of data processing is to 'look at the
data.'

		The problem is *not* with the cdo. Rather the source
data file
		from CMIP5 has incorrect information. I read the
surface level

		   a   =

addfile("thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc","
r")
		   ta  = a->thetao(0,0,:,:)

		   printVarSummary(ta)
		   printMinMax(ta,0)

		The variable has the following attributes

		   missing_value :      1e+20
		   _FillValue : 1e+20

		I counted the number of number of _FillValue (nmsg1) and
the
		min/max temperatures.

		(0)     Sea Water Potential Temperature: min=0
max=303.924
		(0)     nmsg1=0

		Obviously, a temperature of 0 is physically not correct.
I printed
		the array. No missing values (1e20) were encountered.
What about land?
		The actual _FillValue (missing_value) is 0.0. In an NCL
		script I reset to 0.0

		   ta at _FillValue = 0.0

		I counted the _FillValue and the min/max

		(0)     nmsg2=42671
		(0)     Sea Water Potential Temperature: min=271.293
max=303.924

		The temperature range is much more reasonable.

		So the cdo treated the original 0.0 as valid values
rather than
		_FillValue. I used the NCO operator, ncatted, to create
a copy of the
		original source file but altered the _FillValue to 0.0

		%> ncatted -O -h -a _FillValue,thetao,o,f,0

thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
		thetao_Omon_MRI_FillValue.nc

		Then ran the cdo to generate the weights

		%> cdo genbil,r360x180 thetao_Omon_MRI_FillValue.nc
		thetao_Omon_MRI_bilwgt.nc

		Then did the remap

		%> cdo remap,r360x180,thetao_Omon_MRI_bilwgt.nc
		thetao_Omon_MRI_FillValue.nc thetao_Omon.r360x180.nc

		I works!!

		See: http://www.cgd.ucar.edu/~shea/TaoZhang.png

		D

		On 12/2/11 4:12 PM, Zhang Tao wrote:

		Dennis,

		I have attached 3 plots for you to take a look. MPI.pdf
is the results
		from MPI model, I use "cdo remapbil,ll1deg_grid.nc
<input file> <output
		file> " to get it.

		I plot out the contour and shaded plots of MRI-CGCM3
which is obtained
		from your command. clearly, the following command does
not work
		well with this model. I use "cdo remapbil,ll1deg_grid.nc
<input file>
		<output file> " for this model, similar results are
obtained. There are
		lots of unrealistically points in the land.

		Tao

		%> cdo genbil,r360x180

thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
		thetao_Omon_MRI_bilwgt.nc

		&> cdo remap,r360x180,thetao_Omon_MRI_bilwgt.nc

thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
		thetao_Omon.r360x180.nc

		On 12/2/11 3:49 PM, Dennis Shea wrote:

		%> cdo genbil,r360x180

thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
		thetao_Omon_MRI_bilwgt.nc

		&> cdo remap,r360x180,thetao_Omon_MRI_bilwgt.nc

thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
		thetao_Omon.r360x180.nc

		ftp ftp.cgd.ucar.edu<ftp://ftp.cgd.ucar.edu>
<ftp://ftp.cgd.ucar.edu> <ftp://ftp.cgd.ucar.edu>
<ftp://ftp.cgd.ucar.edu> <ftp://ftp.cgd.ucar.edu>
<ftp://ftp.cgd.ucar.edu> 
		anonymous
		email
		cd pub/shea/ZhangTao
		prompt
		mget *
		quit

		=================
		10997404 Dec 2 15:46 thetao_Omon_MRI_bilwgt.nc
		1050660 Dec 2 15:47 thetao_Omon.r360x180.nc

		Hope it worked.

		I'm outta her!

		D

		On 12/02/2011 03:00 PM, Zhang Tao wrote:

		Dennis,

		Could you write down the detailed CDO command to remap
Japanese model?
		I can make a test.

		Thanks,
		Tao

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20111206/87b9a914/attachment-0001.html 