[Go-essp-tech] Incorrect use of missing value in some MRI ocean data -- correction

[Kettleborough, Jamie]

Hello Martin,

But missing value is still acceptable - yes?  (I think this is what we,
MOHC, did, and I wouldn't be surprised if others did to...)

If you accept both 0 and missing values then I guess users have to be
prepared to cope with both.

Jamie

> -----Original Message-----
> From: go-essp-tech-bounces at ucar.edu 
> [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of 
> martin.juckes at stfc.ac.uk
> Sent: 05 December 2011 04:07
> To: go-essp-tech at ucar.edu; graham.parton at stfc.ac.uk; 
> cmip5-users at jiscmail.ac.uk
> Subject: Re: [Go-essp-tech] Incorrect use of missing value in 
> some MRI ocean data -- correction
> 
> I owe MRI an apology -- the use of zero instead of a missing 
> value is in fact recommended in 
> (http://cmip-pcmdi.llnl.gov/cmip5/docs/standard_output.xls),
> 
> regards,
> Martin
> ________________________________________
> From: Juckes, Martin (STFC,RAL,RALSP)
> Sent: 04 December 2011 08:45
> To: go-essp-tech at ucar.edu; Parton, Graham (STFC,RAL,SSTD); 
> cmip5-users at jiscmail.ac.uk
> Cc: Pascoe, Stephen (STFC,RAL,RALSP)
> Subject: Incorrect use of missing value in some MRI ocean data.
> 
> Hello,
> 
> It has come to light that some MRI ocean data has land grid 
> points set to zero rather than to the declared missing value. 
> This is likely to confuse some applications reading and 
> plotting the data -- I've notified Seiji Yukimoto at MRI,
> 
> sincerely,
> Martin
> ________________________________________
> From: Dennis Shea [shea at ucar.edu]
> Sent: 03 December 2011 19:45
> To: Zhang Tao
> Cc: Juckes, Martin (STFC,RAL,RALSP); haley at ucar.edu; Gary 
> Strand; Karl Taylor
> Subject: Re: [ncl-talk] How to regrid CMIP5 ocean temp. data 
> to regular 1X1 grid
> 
> CMIP5 users beware!   :-(
> 
> ---
> 
> The first rule of data processing is to 'look at the data.'
> 
> The problem is *not* with the cdo. Rather the source data 
> file from CMIP5 has incorrect information. I read the  surface level
> 
>    a   =
> addfile("thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.
> lite.nc","r")
>    ta  = a->thetao(0,0,:,:)
> 
>    printVarSummary(ta)
>    printMinMax(ta,0)
> 
> 
> The variable has the following attributes
> 
>    missing_value :      1e+20
>    _FillValue : 1e+20
> 
> I counted the number of number of _FillValue (nmsg1) and the 
> min/max temperatures.
> 
> (0)     Sea Water Potential Temperature: min=0   max=303.924
> (0)     nmsg1=0
> 
> Obviously, a temperature of 0 is physically not correct. I 
> printed the array. No missing values (1e20) were encountered. 
> What about land?
> The actual _FillValue (missing_value) is 0.0. In an NCL 
> script I reset to 0.0
> 
>    ta at _FillValue = 0.0
> 
> I counted the _FillValue and the min/max
> 
> (0)     nmsg2=42671
> (0)     Sea Water Potential Temperature: min=271.293   max=303.924
> 
> The temperature range is much more reasonable.
> 
> So the cdo treated the original 0.0 as valid values rather 
> than _FillValue. I used the NCO operator, ncatted, to create 
> a copy of the original source file but altered the _FillValue to 0.0
> 
> %> ncatted -O -h -a _FillValue,thetao,o,f,0 
> thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
> thetao_Omon_MRI_FillValue.nc
> 
> Then ran the cdo to generate the weights
> 
> %> cdo genbil,r360x180 thetao_Omon_MRI_FillValue.nc 
> thetao_Omon_MRI_bilwgt.nc
> 
> 
> Then did the remap
> 
> %> cdo remap,r360x180,thetao_Omon_MRI_bilwgt.nc
> thetao_Omon_MRI_FillValue.nc thetao_Omon.r360x180.nc
> 
> I works!!
> 
> See: http://www.cgd.ucar.edu/~shea/TaoZhang.png
> 
> D
> 
> 
> On 12/2/11 4:12 PM, Zhang Tao wrote:
> > Dennis,
> >
> > I have attached 3 plots for you to take a look. MPI.pdf is 
> the results 
> > from MPI model, I use "cdo remapbil,ll1deg_grid.nc <input file> 
> > <output
> > file> " to get it.
> >
> > I plot out the contour and shaded plots of MRI-CGCM3 which 
> is obtained 
> > from your command. clearly, the following command does not 
> work well 
> > with this model. I use "cdo remapbil,ll1deg_grid.nc <input file> 
> > <output file> " for this model, similar results are obtained. There 
> > are lots of unrealistically points in the land.
> >
> > Tao
> >
> >
> > %> cdo genbil,r360x180
> > thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
> > thetao_Omon_MRI_bilwgt.nc
> >
> > &> cdo remap,r360x180,thetao_Omon_MRI_bilwgt.nc
> > thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
> > thetao_Omon.r360x180.nc
> >
> >
> >
> >
> > On 12/2/11 3:49 PM, Dennis Shea wrote:
> >> %> cdo genbil,r360x180
> >> thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
> >> thetao_Omon_MRI_bilwgt.nc
> >>
> >> &> cdo remap,r360x180,thetao_Omon_MRI_bilwgt.nc
> >> thetao_Omon_MRI-CGCM3_piControl_r1i1p1_230101-230102.lite.nc
> >> thetao_Omon.r360x180.nc
> >>
> >> ftp ftp.cgd.ucar.edu
> >> anonymous
> >> email
> >> cd pub/shea/ZhangTao
> >> prompt
> >> mget *
> >> quit
> >>
> >>
> >> =================
> >> 10997404 Dec 2 15:46 thetao_Omon_MRI_bilwgt.nc 1050660 Dec 2 15:47 
> >> thetao_Omon.r360x180.nc
> >>
> >> Hope it worked.
> >>
> >> I'm outta her!
> >>
> >> D
> >>
> >>
> >> On 12/02/2011 03:00 PM, Zhang Tao wrote:
> >>> Dennis,
> >>>
> >>> Could you write down the detailed CDO command to remap 
> Japanese model?
> >>> I can make a test.
> >>>
> >>> Thanks,
> >>> Tao
> --
> Scanned by iCritical.
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>