[ncl-talk] smops interpolation
Dennis Shea
shea at ucar.edu
Tue Sep 18 09:35:45 MDT 2018
There is an AMSR example:
*http://www.ncl.ucar.edu/Applications/HDF.shtml
<http://www.ncl.ucar.edu/Applications/HDF.shtml>*
*Dee: amsr_1.ncl*
On Fri, Sep 14, 2018 at 10:28 PM Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com>
wrote:
> Dear Dennis
> Thank you so much for your helpful replies.
> Pardon me if I ask some questions with my responsibility to solve them.
> However sometimes some problems take much time and finally I couldn't find
> any solution for them. I'll be thankful if I could have ncl-talk help to
> overcome them. Beside that, solving some problems need more experiences,
> which I still don't have enough experience with ncl and also some data
> format. I always query my data with "panoply" or "ncl_filedump" or "ncdump
> -h", but the problem is that I don't know what should I do. It is obvious
> helping in these problems are priceless and until now I've got many helps
> from ncl-talk which kicking me forward much at doing my thesis. I just
> could appreciate for these helps.
>
> However, it could be disturbing (!), but I encountered same problem with
> another data set, AMSR2.
> First I want to know whether is there any ncl script for AMSR2. I couldn't
> find anything in (https://www.ncl.ucar.edu/Applications/). If not, may I
> ask some help for AMSR2 data? I've attached three samples data of AMSR2.
> These data set don't have _FillValue, although it includes "-32767"
> (default _FillValue of short type) and "-32768" (default missing Value).
> So I've tried
> sm at _FillValue := getVarFillValue(sm)
> But what should I do about "-32768" values (missing_value)?
> Also, since that ( Geophysical_Data) in AMSR2 is 3 dimensional I've
> changed some lines as below:
> sm_new = where(lsm.eq.0 , sm_new at _FillValue , sm_new(:,:,0))
> sm_new = sm_new(::-1,:,:)
> sm_region := sm_new({latS:latN},{lonL:lonR},:)
> AMSR2_fo = linint2_points(sm_region&longitude, sm_region&latitude,
> sm_region(time | :, latitude | :, longitude | :), True , stlon, stlat, 0)
>
> Are they true?
>
> Another problem that is related to handle many h5 data, which resulted to
> following error:
> HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
> #000: H5F.c line 408 in H5Fis_hdf5(): unable open file
> major: File accessibilty
> minor: Not an HDF5 file
> #001: H5Fint.c line 532 in H5F__is_hdf5(): unable to open file
> ...
>
>
> Could I have any reference page in ncl website or any suggestion to
> overcome these problems.
>
> Sincerely
> Ehsan
> On Friday, September 14, 2018, 8:10:40 AM GMT+4:30, Dennis Shea <
> shea at ucar.edu> wrote:
>
>
> SMOPS: soil Moisture Operational Products System
> *https://www.ospo.noaa.gov/Products/land/smops/*
> <https://www.ospo.noaa.gov/Products/land/smops/>
> Values over land only
>
> [1]
> The* most important rule in data processing* is **Look at your data"*.
> Sometimes you have to look *carefully!*
> *This is a user responsibility.*
>
> netcdf NPR_SMOPS_CMAP_D20170410 {
> dimensions:
> Longitude = 1440 ;
> Latitude = 720 ;
> variables:
> *short* *Blended_SM(Latitude, Longitude) ;*
> Blended_SM:long_name = "Blended Soil Moisture" ;
> Blended_SM:units = "m^3/m^3" ;
> *Blended_SM:FillValue = -999s ; <=======*
> Blended_SM:valid_range = 0s, 10000s ;
> Blended_SM:scale_factor = 0.0001f ;
> Blended_SM:add_offset = 0.f ;
> [SNIP]
>
> The NPR_SMOPS_CMAP file *does not conform to any netCDF convention*.
> Likely an oversight. It uses *FillValue* rather than _*FillValue. *The *_
> *is important!!! NCL recognizes only *_FillValue *(Climate and Forecast
> netCDF Convention) *.* Hence, NCL's *short2flt *
> <http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>will
> not properly unpack the "*short* *Blended_SM"* variable. Likely, this is
> the source of some of the interpolation issues.
>
> How to fix this?
>
> Replace:
>
> sm = *short2flt( *
> <http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>SMOPS_fi->Blended_SM
> )
>
> with
>
> sm = SMOPS_fi->Blended_SM ; Read variable of type 'short'
> sm@*_FillValue *= sm@
> *FillValue ; add correct attribute to original variable*
> printVarSummary(sm) ; Note *_FillValue*
> attribute
> sm *:=* *short2flt*( sm ) ; Use NCL's
> overwrite syntax *:=*
>
> Add coordinates which are not on the file. Another, oversight by the file
> creator!
>
> sm!0 = "latitude"
> sm!1 = "longitude"
> sm&latitude = latitude
> sm&longitude = longitude
> printVarSummary(sm)
> printMinMax(sm, 0)
> print("---")
>
> ============
> [2] Using '*poisson_grid_fill*'
>
> [a] will 'fill-in' gaps in the satellite mosaic swaths :-)
> [b] it will also put 'soil moisture' over the oceans and lakes. :-(
>
> You can use a*n* NCL function to mask out the oceans/lakes/ice/etc
>
> *http://www.ncl.ucar.edu/Document/Functions/Shea_util/landsea_mask.shtml*
> <http://www.ncl.ucar.edu/Document/Functions/Shea_util/landsea_mask.shtml>
>
> %> ncl 20_SMOPS.poisson_fill_mask.ncl
>
> creates: SMOPS_FILL_MASK.png
> ============
>
> [3] The following uses linint2_points_Wrap to perform the station
> interpolations.
>
> %> 20_SMOPS.poisson_fill_mask_linint2.ncl
>
> On Wed, Sep 12, 2018 at 5:15 AM, Ehsan Taghizadeh <
> ehsantaghizadeh at yahoo.com> wrote:
>
> Hi,
> Again Thank you so much for your helpful comments.
> Unfortunately after a few days I still have problem with interpolating
> data on station points. In spite of using "poisson_grid_fill" and "linmsg"
> to avoid getting Fill_Value on unstructured points, these don't work; all
> values for all dates and all points are Fill_value (-0.099900) (attached
> file: 20_SMOPS_Interp.txt). Could I have any suggestion to get interpolated
> values other than "-0.099900". I've used this script for some data set and
> it helps me a lot. But for some data set, like SMOPS data, it gives me just
> Fill_Value.
> I know using "knnsearch" in matlab gives values except "-0.099900",
> however I want to interpolate with ncl.
> I've attached the script (20_SMOPS_Interp.ncl) and also station points
> (2_stations.dat) and output file (20_SMOPS_Interp.txt). I've also put
> zipped input data (SMOPS_Data.tgz) on ftp.
>
> I'll be thankful for any help.
>
> Sincerely
> Ehsan
>
>
> On Friday, September 7, 2018, 8:25:16 PM GMT+4:30, Dennis Shea <
> shea at ucar.edu> wrote:
>
>
> [1] No
> [2] Yes
>
> nlat = 720
> latitude = *fspan*(-49.875, 49.875, nlat)
> latitude at units = "*degrees_north*"
> latitude at long_name = "latitude"
> latitude!0 = "latitude"
> latitude&latitude = latitude
>
> printVarSummary(latitude)
> printMinMax(latitude,0)
>
> mlon = 1440
> longitude = *lonGlobeFo* <https://www.ncl.ucar.edu/Document/Functions/Contributed/lonGlobeFo.shtml>(mlon, "longitude", "longitude", "degrees_east") ; 0->360
> longitude = (/ longitude - 180. /) ; subtract 180 from all values
> longitude&longitude = longitude ; update coordinates printVarSummary(lon)
>
> printVarSummary(longitude)
> printMinMax(longitude,0)
>
> f = *addfile*("NPR_SMOPS_CMAP_D20170 410.nc","r")
>
>
> x = *short2flt*
> <https://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>(
> f->Blended_SM )
>
> x&latitude = latitude
> x&longitude = longitude
>
> printVarSummary(x)
>
> [3]/[4]
>
> xo = .... longitudes of land points...
>
> yo = ... latitudes ....
>
> xpts = *linint2_points*(x&longitude,x& latitude,x, False, xo,yo, 0)
>
> printVarSummary(xpts)
>
> printMinMax(xpts,0)
>
>
>
> On Tue, Sep 4, 2018 at 5:50 AM, Ehsan Taghizadeh <
> ehsantaghizadeh at yahoo.com> wrote:
>
> Thank you so much for your nice reply, as always.
> I did your comments, however I encountered some difficulties and I'll be
> thankful if I could have any help about them, too.
>
> 1. Should I change
>
> latitude = *fspan*(-49.875, 49.875, nlat)
>
> to
>
> latitude = *fspan*(-89.875, 89.875, nlat)
>
> or like longitude, use
>
> lat = *latGlobeFo*(nlat, "lat", "latitude", "degrees_north")
>
>
> 2. Should I change
>
> latitude at units = "degrees_east"
>
> to
>
> latitude at units = "degrees_north"
>
>
> 3. How should I control FillValue (-999s), because after running ncl
> script, it writes -0.1 for Nan value of the data, both in grided and
> interpolated output files.
> 4. However the most difficulty is the interpolating part, because all
> interpolated values for all station points got -0.1, means Nan. There is no
> value except -0.1 for any station points. Could anybody help me about this?
> I've attached the output of interpolated file. I'll be thankful for any
> help.
>
> Sincerely
> Ehsan
>
> On Friday, August 31, 2018, 6:14:53 PM GMT+4:30, Dennis Shea <
> shea at ucar.edu> wrote:
>
>
> Sometimes, you have to read documentation associated with the file(s)
> *Soil Moisture* Operational Products System (*SMOPS*)
>
> The file has:
>
> netcdf NPR_SMOPS_CMAP_D20170410 {
> dimensions:
>
> * Longitude = 1440 ; Latitude = 720 ;*
> variables:
> *short* Blended_SM(*Latitude, Longitude*) ;
> Blended_SM:long_name = "Blended Soil Moisture" ;
> Blended_SM:units = "m^3/m^3" ;
> Blended_SM:FillValue = -999s ;
> Blended_SM:valid_range = 0s, 10000s ;
> Blended_SM:*scale_factor *= 0.0001f ;
> Blended_SM:*add_offset* = 0.f ;
>
> [SNIP]
> :Product_Resolution = "0.25 degree" ;
> :Date_Start = "20170410" ;
> :Date_End = "20170410" ;
> [SNIP]
> ====
> It does not have Latitude[*] and Longitude[*] coordinates. However,
> **you** can create them. Then, use ESMF or (say)* linint2_Wrap *
> <https://www.ncl.ucar.edu/Document/Functions/Contributed/linint2_Wrap.shtml>or
> *linint2_points_Wrap*
> <https://www.ncl.ucar.edu/Document/Functions/Contributed/linint2_points_Wrap.shtml>
> to perform the interpolation.
>
> ====
>
> nlat = 720
> latitude = *fspan*(-49.875, 49.875, nlat)
> latitude at units = "degrees_east"
> latitude at long_name = "latitude"
> latitude!0 = "latitude"
> latitude&latitude = latitude
>
> printVarSummary(latitude)
> printMinMax(latitude,0)
>
> mlon = 1440
> longitude = *lonGlobeFo* <https://www.ncl.ucar.edu/Document/Functions/Contributed/lonGlobeFo.shtml>(mlon, "longitude", "longitude", "degrees_east") ; 0->360
> longitude = (/ longitude - 180. /) ; subtract 180 from all values
> longitude&longitude = longitude ; update coordinates printVarSummary(lon)
>
> printVarSummary(longitude)
> printMinMax(longitude,0)
>
> f = *addfile*("NPR_SMOPS_CMAP_D20170 410.nc","r")
>
>
> x = *short2flt*
> <https://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml>(
> f->Blended_SM )
>
> x&latitude = latitude
> x&longitude = longitude
>
> printVarSummary(x)
>
>
>
> On Fri, Aug 31, 2018 at 5:45 AM, Ehsan Taghizadeh <
> ehsantaghizadeh at yahoo.com> wrote:
>
> Hi,
> Until now I've asked some questions related to interpolating some dataset
> on some unstructured points (synoptic stations). Now I have a new data set
> which I'm not sure about appropriate method to interpolating. I put one of
> my data (NPR_SMOPS_CMAP_D20170410.nc) in ftp.cgd.ucar.edu. I'll be
> thankful if some one could refer me to correct script, like on (https://www.ncl.ucar.edu/
> Applications/ESMF.shtml <https://www.ncl.ucar.edu/Applications/ESMF.shtml>)
> or (https://www.ncl.ucar.edu/ Applications/regrid.shtml
> <https://www.ncl.ucar.edu/Applications/regrid.shtml>), which interpolates
> my data on some special unstructured points.
> However the printVarSummary(Blended_SM) result is as below:
>
> Variable: Blended_SM
> Type: float
> Total Size: 4147200 bytes
> 1036800 values
> Number of Dimensions: 2
> Dimensions and sizes: [Latitude | 720] x [Longitude | 1440]
> Coordinates:
> Number Of Attributes: 5
> _FillValue : 1e+20
> long_name : Blended Soil Moisture
> units : m^3/m^3
> FillValue : -999
> valid_range : ( 0, 1 )
>
>
> Sincerely
> Ehsan
>
> ______________________________ _________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
> <http://mailman.ucar.edu/mailman/listinfo/ncl-talk>
>
>
>
> ______________________________ _________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
> <http://mailman.ucar.edu/mailman/listinfo/ncl-talk>
>
>
>
> ______________________________ _________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
> <http://mailman.ucar.edu/mailman/listinfo/ncl-talk>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180918/514fcc42/attachment.html>
More information about the ncl-talk
mailing list