[ncl-talk] smops interpolation
Ehsan Taghizadeh
ehsantaghizadeh at yahoo.com
Wed Sep 19 14:31:18 MDT 2018
Hi,Thank you for your help.This script really helped me. However I should mention two points.1- I think AMSR2 longitude starts from 0 (not 0.125); so may be following line should be added while determining coordinates: lon = (/ lon - .125 /)
which results lon = [/0 0.25 0.5 0.75 1.0 ... 359 359.25 359.5 357.75/]
2- If I have more than one year for input files, I mean following line: fili = systemfunc("cd "+diri+" ; ls GW1AM2_201*h5")instead of: fili = systemfunc("cd "+diri+" ; ls GW1AM2_2017*h5")I encounter Segmentation fault (core dumped)
I've searched a little for that, and I find some issues related on previous ncl versions. Is there still this issue on ncl6.5.0
I'll be thankful for having any idea about these two points.
SincerelyEhsan
On Tuesday, September 18, 2018, 8:05:59 PM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
There is an AMSR example:
http://www.ncl.ucar.edu/Applications/HDF.shtml
Dee: amsr_1.ncl
On Fri, Sep 14, 2018 at 10:28 PM Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Dear DennisThank you so much for your helpful replies.
Pardon me if I ask some questions with my responsibility to solve them. However sometimes some problems take much time and finally I couldn't find any solution for them. I'll be thankful if I could have ncl-talk help to overcome them. Beside that, solving some problems need more experiences, which I still don't have enough experience with ncl and also some data format. I always query my data with "panoply" or "ncl_filedump" or "ncdump -h", but the problem is that I don't know what should I do. It is obvious helping in these problems are priceless and until now I've got many helps from ncl-talk which kicking me forward much at doing my thesis. I just could appreciate for these helps.
However, it could be disturbing (!), but I encountered same problem with another data set, AMSR2.
First I want to know whether is there any ncl script for AMSR2. I couldn't find anything in (https://www.ncl.ucar.edu/Applications/). If not, may I ask some help for AMSR2 data? I've attached three samples data of AMSR2. These data set don't have _FillValue, although it includes "-32767" (default _FillValue of short type) and "-32768" (default missing Value).So I've tried
sm at _FillValue := getVarFillValue(sm)
But what should I do about "-32768" values (missing_value)?Also, since that ( Geophysical_Data) in AMSR2 is 3 dimensional I've changed some lines as below: sm_new = where(lsm.eq.0 , sm_new at _FillValue , sm_new(:,:,0)) sm_new = sm_new(::-1,:,:)
sm_region := sm_new({latS:latN},{lonL:lonR},:)
AMSR2_fo = linint2_points(sm_region&longitude, sm_region&latitude, sm_region(time | :, latitude | :, longitude | :), True , stlon, stlat, 0)
Are they true?
Another problem that is related to handle many h5 data, which resulted to following error:HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0: #000: H5F.c line 408 in H5Fis_hdf5(): unable open file major: File accessibilty minor: Not an HDF5 file #001: H5Fint.c line 532 in H5F__is_hdf5(): unable to open file...
Could I have any reference page in ncl website or any suggestion to overcome these problems.
SincerelyEhsan On Friday, September 14, 2018, 8:10:40 AM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
SMOPS: soil Moisture Operational Products System
https://www.ospo.noaa.gov/Products/land/smops/
Values over land only
[1]
The most important rule in data processing is *Look at your data".
Sometimes you have to look carefully!This is a user responsibility.
netcdf NPR_SMOPS_CMAP_D20170410 {
dimensions:
Longitude = 1440 ;
Latitude = 720 ;
variables:
short Blended_SM(Latitude, Longitude) ;
Blended_SM:long_name = "Blended Soil Moisture" ;
Blended_SM:units = "m^3/m^3" ;
Blended_SM:FillValue = -999s ; <=======
Blended_SM:valid_range = 0s, 10000s ;
Blended_SM:scale_factor = 0.0001f ;
Blended_SM:add_offset = 0.f ;[SNIP]
The NPR_SMOPS_CMAP file does not conform to any netCDF convention. Likely an oversight. It uses FillValue rather than _FillValue. The _ is important!!! NCL recognizes only _FillValue (Climate and Forecast netCDF Convention) . Hence, NCL's short2flt will not properly unpack the "short Blended_SM" variable. Likely, this is the source of some of the interpolation issues.
How to fix this?
Replace:
sm = short2flt( SMOPS_fi->Blended_SM )
with
sm = SMOPS_fi->Blended_SM ; Read variable of type 'short'
sm at _FillValue = sm at FillValue ; add correct attribute to original variable
printVarSummary(sm) ; Note _FillValue attribute
sm := short2flt( sm ) ; Use NCL's overwrite syntax :=
Add coordinates which are not on the file. Another, oversight by the file creator!
sm!0 = "latitude"
sm!1 = "longitude"
sm&latitude = latitude
sm&longitude = longitude
printVarSummary(sm)
printMinMax(sm, 0)
print("---")
============[2] Using 'poisson_grid_fill'
[a] will 'fill-in' gaps in the satellite mosaic swaths :-)
[b] it will also put 'soil moisture' over the oceans and lakes. :-(
You can use an NCL function to mask out the oceans/lakes/ice/etc
http://www.ncl.ucar.edu/Document/Functions/Shea_util/landsea_mask.shtml
%> ncl 20_SMOPS.poisson_fill_mask.ncl
creates: SMOPS_FILL_MASK.png
============
[3] The following uses linint2_points_Wrap to perform the station interpolations.
%> 20_SMOPS.poisson_fill_mask_linint2.ncl
On Wed, Sep 12, 2018 at 5:15 AM, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Hi,Again Thank you so much for your helpful comments.
Unfortunately after a few days I still have problem with interpolating data on station points. In spite of using "poisson_grid_fill" and "linmsg" to avoid getting Fill_Value on unstructured points, these don't work; all values for all dates and all points are Fill_value (-0.099900) (attached file: 20_SMOPS_Interp.txt). Could I have any suggestion to get interpolated values other than "-0.099900". I've used this script for some data set and it helps me a lot. But for some data set, like SMOPS data, it gives me just Fill_Value.I know using "knnsearch" in matlab gives values except "-0.099900", however I want to interpolate with ncl.I've attached the script (20_SMOPS_Interp.ncl) and also station points (2_stations.dat) and output file (20_SMOPS_Interp.txt). I've also put zipped input data (SMOPS_Data.tgz) on ftp.
I'll be thankful for any help.
SincerelyEhsan
On Friday, September 7, 2018, 8:25:16 PM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
[1] No[2] Yes
nlat = 720
latitude = fspan(-49.875, 49.875, nlat)
latitude at units = "degrees_north"
latitude at long_name = "latitude"
latitude!0 = "latitude"
latitude&latitude = latitude
printVarSummary(latitude)
printMinMax(latitude,0)
mlon = 1440
longitude = lonGlobeFo(mlon, "longitude", "longitude", "degrees_east") ; 0->360
longitude = (/ longitude - 180. /) ; subtract 180 from all values
longitude&longitude = longitude ; update coordinates printVarSummary(lon)
printVarSummary(longitude)
printMinMax(longitude,0)
f = addfile("NPR_SMOPS_CMAP_D20170 410.nc","r")
x = short2flt( f->Blended_SM )
x&latitude = latitude
x&longitude = longitude
printVarSummary(x)
[3]/[4]
xo = .... longitudes of land points...
yo = ... latitudes ....
xpts = linint2_points(x&longitude,x& latitude,x, False, xo,yo, 0)
printVarSummary(xpts)
printMinMax(xpts,0)
On Tue, Sep 4, 2018 at 5:50 AM, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Thank you so much for your nice reply, as always.
I did your comments, however I encountered some difficulties and I'll be thankful if I could have any help about them, too.
1. Should I changelatitude = fspan(-49.875, 49.875, nlat)tolatitude = fspan(-89.875, 89.875, nlat)or like longitude, uselat = latGlobeFo(nlat, "lat", "latitude", "degrees_north")
2. Should I changelatitude at units = "degrees_east"tolatitude at units = "degrees_north"
3. How should I control FillValue (-999s), because after running ncl script, it writes -0.1 for Nan value of the data, both in grided and interpolated output files.4. However the most difficulty is the interpolating part, because all interpolated values for all station points got -0.1, means Nan. There is no value except -0.1 for any station points. Could anybody help me about this?I've attached the output of interpolated file. I'll be thankful for any help.
Sincerely
Ehsan
On Friday, August 31, 2018, 6:14:53 PM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
Sometimes, you have to read documentation associated with the file(s)
Soil Moisture Operational Products System (SMOPS)
The file has:
netcdf NPR_SMOPS_CMAP_D20170410 {
dimensions:
Longitude = 1440 ;
Latitude = 720 ;
variables:
short Blended_SM(Latitude, Longitude) ;
Blended_SM:long_name = "Blended Soil Moisture" ;
Blended_SM:units = "m^3/m^3" ;
Blended_SM:FillValue = -999s ;
Blended_SM:valid_range = 0s, 10000s ;
Blended_SM:scale_factor = 0.0001f ;
Blended_SM:add_offset = 0.f ;
[SNIP]
:Product_Resolution = "0.25 degree" ;
:Date_Start = "20170410" ;
:Date_End = "20170410" ;
[SNIP]
====
It does not have Latitude[*] and Longitude[*] coordinates. However, *you* can create them. Then, use ESMF or (say) linint2_Wrap or linint2_points_Wrap to perform the interpolation.
====
nlat = 720
latitude = fspan(-49.875, 49.875, nlat)
latitude at units = "degrees_east"
latitude at long_name = "latitude"
latitude!0 = "latitude"
latitude&latitude = latitude
printVarSummary(latitude)
printMinMax(latitude,0)
mlon = 1440
longitude = lonGlobeFo(mlon, "longitude", "longitude", "degrees_east") ; 0->360
longitude = (/ longitude - 180. /) ; subtract 180 from all values
longitude&longitude = longitude ; update coordinates printVarSummary(lon)
printVarSummary(longitude)
printMinMax(longitude,0)
f = addfile("NPR_SMOPS_CMAP_D20170 410.nc","r")
x = short2flt( f->Blended_SM )
x&latitude = latitude
x&longitude = longitude
printVarSummary(x)
On Fri, Aug 31, 2018 at 5:45 AM, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Hi,Until now I've asked some questions related to interpolating some dataset on some unstructured points (synoptic stations). Now I have a new data set which I'm not sure about appropriate method to interpolating. I put one of my data (NPR_SMOPS_CMAP_D20170410.nc) in ftp.cgd.ucar.edu. I'll be thankful if some one could refer me to correct script, like on (https://www.ncl.ucar.edu/ Applications/ESMF.shtml) or (https://www.ncl.ucar.edu/ Applications/regrid.shtml), which interpolates my data on some special unstructured points.However the printVarSummary(Blended_SM) result is as below:
Variable: Blended_SMType: floatTotal Size: 4147200 bytes 1036800 valuesNumber of Dimensions: 2Dimensions and sizes: [Latitude | 720] x [Longitude | 1440]Coordinates:Number Of Attributes: 5 _FillValue : 1e+20 long_name : Blended Soil Moisture units : m^3/m^3 FillValue : -999 valid_range : ( 0, 1 )
Sincerely
Ehsan
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180919/01580953/attachment.html>
More information about the ncl-talk
mailing list