[ncl-talk] NCL Masking Optimization

[Mary Haley]

Hi Alan,

If the lat/lon values of your data are the same for every iteration of the
loop and/or subset of your data, then what I recommend is to use
shapefile_mask_data *once* to create a mask array of 0s and 1s, save this
mask file to a NetCDF file, and then use this to do the masking for
subsequent masking. This will go *much* faster than calling
shapefile_mask_data every time.

The "shapefiles_11.ncl" script shows how to do this:

http://www.ncl.ucar.edu/Applications/shapefiles.shtml#ex11

   - shapefile_mask_data is called with "return_mask set to True. This
   causes an array of 0s and 1s to be returned.
   - The 0s and 1s array is written to a NetCDF file
   - For subsequent masking, the mask array is read off the file once, and
   used in conjunction with the "where" function to do the masking.

If your lat/lon data is different for every iteration, then the above won't
be of much help to you.

However, just recently one of our former NCL developers sent us some
potentially faster masking code to use with shapefile data, which I haven't
had a chance to try out.  If you want to give this a try, then it would
help if I could have access to the data used with your script, so I can
test it.

--Mary


On Fri, Aug 7, 2015 at 1:31 PM, Alan Rhoades <alan.m.rhoades at gmail.com>
wrote:

> Hello,
>
> I use shapefiles and the NCL masking command for a lot of climate dataset
> analysis.  I've recently gotten to the point where I need to start masking
> 4D data for a rather large shapefile region (over several seasons of data)
> and it can take upwards of two days to complete on my local server (which
> can be an issue for troubleshooting if something goes wrong in my script).
> I was curious if you could take a look at my masking script (see attached)
> and give any pointers on how to better optimize and speed-up the masking
> process.
>
> All the best,
>
> AR
>
> --
>
> *Alan Rhoades*
> *PhD Student, Atmospheric Science Graduate Group*
> *Climate Change Water and Society (CCWAS) NSF IGERT Trainee*
> *University of California, Davis*
> *LinkedIn <https://www.linkedin.com/pub/alan-rhoades/22/5bb/52a>*
> *alan.m.rhoades at gmail.com <alan.m.rhoades at gmail.com> *
> *amrhoades at ucdavis.edu <amrhoades at ucdavis.edu>*
>
> *"It’s all really there. That’s what really gets you. But you gotta stop
> and think about it to really get the pleasure about the complexity, the
> inconceivable nature of nature."*
> *Richard Feynman*
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150810/889a1f69/attachment.html