[ncl-talk] pixel data to regular grid data

[Dennis Shea]

[1] Well the way to go is ESMF regridding. See last comment.

[2] Did you look at the data?  You should.

[3]
The files are poorly constructed. The whole point of netCDF/HDF is to be
self describing. This file has issues. Consider the

  f    = addfile("SEV..., "r")
  lat2d   := f->MSG_Latitude
  printVarSummary(lat2d)Variable: lat2d
Type: float
Total Size: 55115776 bytes
           13778944 values
Number of Dimensions: 2
Dimensions and sizes:   [fakeDim0 | 3712] x [fakeDim1 | 3712]
Coordinates:
Number Of Attributes: 1
  hdf_name :    MSG_Latitude

There is *no* indication of a missing values. Yet ..

printMinMax(lat2d,0)   ===> lat2d: min=-999.999  max=81.2637

-999.999 should be flagged to indicate missing (_FillValue)

lat2d at _FillValue = -999.999

If you now count the number of _FillValue, you would find that 25% of the
points have missing coordinates. This should be accounted for in any
regridding.
dim2d = dimsizes(lat2d)
kFill    = num(ismissing(lat2d))
print("kFill="+kFill+"  TotalGridPoints="+product(dim2d))

The data for many variables are packed with scale and offset. Yet there is*
no indication on the file* of how to unpack the variables. Maybe it is
ancillary documentation. *If so, that is your responsibility to read and
account for it in the script.*

*shor*t Cloud_Top_Temperature ( fakeDim69, fakeDim70 )   <= type 'short'
units :        K
long_name :    Cloud Top Temperature from in-house  ...
actual_range : ( -32699, 32657 )
Num_Fill :     0
Num_Valid :    13778944
*valid_range* :  ( 0, 20000 )
_FillValue :   -999
scale_factor : 0.009999999776482582   <=========== 0.01
scale_factor_err :        0
add_offset :   -15000                               <===========
add_offset_err :          0

There are 4 ways to unpack

  ctt_1  = CTT at scale_factor*(CTT + CTT at add_offset)
  ctt_2  = CTT at scale_factor*(CTT - CTT at add_offset)
  ctt_3  = CTT at scale_factor*CTT  + CTT at add_offset
  ctt_4  = CTT at scale_factor*CTT   - CTT at add_offset

Print out the min/max of each. Then select the one most appropriate.
Hint: it is this one

  ctt   := CTT at scale_factor*(CTT + CTT at add_offset)   ; unpacked

      ; valid_range :  ( 0, 20000 ); type short
 vrng   = CTT at valid_range

      ; set all outside valid range to _FillValue
 CTT    = where(CTT.ge.vrng(0) .and. CTT.le.vrng(1), CTT, CTT at _FillValue)
 print("CTT: min="+min(CTT)+"  max="+max(CTT))

Look at the unpacked values. Often, satellite data have outliers. Do you
want to keep them or not??
  opt = True
  opt at PrintStat = True
  ctt_stat = stat_dispersion(ctt, opt )   ; **READ** documentation

;*********************************
; Use information from stat_dispersion to filter some outliers
;*********************************
ctt  = where(ctt.lt.-80.0 .or. ctt.gt.-5.0, ctt at _FillValue, ctt)

PLOT the raw data ....

You said 0.5x0.5 .... do you really think you can get that from the
available data?

====
Rememer the 1st rule of data processing is the ** know your data **.

On Mon, Jun 29, 2015 at 4:54 PM, Zhifeng Yang <yangzf01 at gmail.com> wrote:

> Hi
>
> I am using SEVIRI data now, and trying to interpolate the pixel data into
> regular grid data, like 0.5deg*0.5. I wrote a procedure to interpolate the
> data. Unfortunately, I need to do a lot of loops to complete the
> interpolation, which costs a great amount of time. So the process is super
> slow. I searched on the NCL website to find a similar function or procedure
> which has the similar function. However, I did not find one. So do we have
> this kind of function or procedure in NCL to interpolate the pixel data
> into regular grid data?
>
> Thank you,
> Zhifeng
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150630/6180b227/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SEVERI_SOURCE_0.png
Type: image/png
Size: 212681 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150630/6180b227/attachment.png