[ncl-talk] Removing missing values from a 1D data

[Wang, Jiali]

Thanks Adam, Dave and Dennis. I didn’t see any of those in my emails, maybe I am missing something. Dennis did ask me to upload a file this morning to test.

I am going to try these suggestions shortly and will get back to you.

Thank you all,
Jiali

From: Dave Allured - NOAA Affiliate [mailto:dave.allured at noaa.gov]
Sent: Friday, November 17, 2017 1:30 PM
To: Wang, Jiali <jialiwang at anl.gov>; ncl-talk at ucar.edu
Subject: Re: [ncl-talk] Removing missing values from a 1D data

Also did you see my previous answer here?
http://mailman.ucar.edu/pipermail/ncl-talk/2017-November/010745.html

--Dave

On Fri, Nov 17, 2017 at 12:07 PM, Adam Phillips <asphilli at ucar.edu<mailto:asphilli at ucar.edu>> wrote:
Hi Jiali,
I believe Dennis answered this question a few days ago:

x[*]   with _FillValue attribute



 ;nmsg = num(ismissing(x))   ; coun number of missing



  igood = ind(.not.ismissing(x))

  xgood = x(igood)

  print(xgood)
You need to use the ind function to identify what isn't missing, and then pass those indices to your st1, lon1 and lat1 arrays.
good = ind(.not.ismissing(st1))
st2 = st1(igood)
lat2 = lat1(igood)
lon2 = lat2(igood)

Adam

On Fri, Nov 17, 2017 at 11:04 AM, Wang, Jiali <jialiwang at anl.gov<mailto:jialiwang at anl.gov>> wrote:
Dear NCL group,

I am having a similar question as this post https://www.ncl.ucar.edu/Support/talk_archives/2011/1499.html but unfortunately i didn’t get it..

I have a 1D data set (station data) as shown below. I would like to filter out the lower orders (order smaller than 2). I do this using “where”, so I get lots of missing values for those stations with lower orders. But these values take lots of space, so I want to get rid of them and output a much lighter data only for the stations with non-missing values. Below is what my data and my NCL script look like.  Could you please take a look and advice? With my script I am still having all the missing value in my output.

ncdump -c 200303091200.CHRTOUT_DOMAIN3
netcdf \200303091200 {
dimensions:
        station = 3678397 ;
        time = 1 ;
variables:
        float latitude(station) ;
                latitude:long_name = "Station latitude" ;
                latitude:units = "degrees_north" ;
        float longitude(station) ;
                longitude:long_name = "Station longitude" ;
                longitude:units = "degrees_east" ;
        float streamflow(station) ;
                streamflow:units = "meter^3 / sec" ;
                streamflow:long_name = "River Flow" ;
        int order(station) ;
                order:long_name = "Strahler Stream Order" ;
=============
a = addfile("200303101200.CHRTOUT_DOMAIN3"+".nc","r")
st = a->streamflow
lat = a->latitude
lon = a->longitude
ord = a->order ;;do filter based on this number, we only want order>2

st1 = new(dimsizes(st),typeof(st))
lat1 = new(dimsizes(lat),typeof(lat))
lon1 = new(dimsizes(lon),typeof(lon))

st at _FillValue = -9999
st1  = where(ord .gt. 2, st, st at _FillValue)
lat1 = where(ord .gt. 2, lat, st at _FillValue)
lon1 = where(ord .gt. 2, lon, st at _FillValue)

if (.not.all(ismissing(st1))) then   ;;I think this “if” statement has some problem, but I am not sure what’s the right one should be..
    system("/bin/rm -f simple.nc<http://simple.nc>")
    ncdf = addfile("simple.nc<http://simple.nc>" ,"c")
    ncdf->streamflow = st1
    ncdf->lat        = lat1
    ncdf->lon        = lon1
;;;;;;or output in txt format
      opt = True
      opt at fout = "foo.txt"
      write_matrix(st1, "10f7.2", opt)
end if

==============

Thanks!

Jiali Wang, Ph. D
Environmental Science Division
Argonne National Laboratory
Tel: 630-252-2848<tel:(630)%20252-2848>
http://www.evs.anl.gov/about-evs/staff/detail/index.cfm?/Wang/Jiali
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171117/845fffb0/attachment.html>