[ncl-talk] interpolating EDGAR CO2 emission to WRF domain, affordable rgrid2rcm

Mary Haley haley at ucar.edu
Thu Apr 5 10:54:30 MDT 2018


Hi Xiaoming,

Dennis sent me your files offline and his rgrid2rcm script.

I created a version that uses ESMF_regrid, using the plotting parameters
that Dennis sent since he did such a nice job on this.

The only issue I saw with the original regridding script is that you need
to set these two options, since the source and destination grids are
regional:

  Opt at SrcRegional    = True      ; These two are important! Do not

  Opt at DstRegional    = True      ; set if you have a global lat/lon grid


Attached is an ESMF_regrid version of your script that you can try with
bilinear, neareststod, and patch.

I also changed the plotting a little so you can choose between plotting
over the native WRF map projection, or a cylindrical equidistant projection
as Dennis had used.  Finally, I paneled the original plot and regridded
plot for comparison.

Once you have the weights file *and* you have the same source and
destination grids, you can then set:

  Opt at SkipSrcGrid     = True
  Opt at SkipDstGrid     = True
  Opt at SkipWgtGen      = True

which will make subsequent regridding go much faster, because it only has
to do a sparse matrix multiply under the hood. You can only set these,
however, if are using the same source and destination grids and the same
interpolation method.

--Mary


On Thu, Apr 5, 2018 at 9:02 AM, xiaoming Hu <yuanfangcan at hotmail.com> wrote:

> Dennis
>
>
> Thanks a lot!  By reading your ncl script, I learned a lot!  Thanks for
> teaching me this affordable  rgrid2rcm.  😊
>
>  I am attaching your code below in case other people might be interested!
>
> Thanks
>
> Xiaoming
>
>   path_wrf   = "./"
>
>   diro       = "./"
>   filo       = "wrfchemi_d01_valueFromEdgar_rgrid2rcm_China.nc"
>   ptho       = diro+filo
>
> ;---Target WRF (China)  grid
>
>   fwrf       = addfile(path_wrf+"WRF.LatLon.nc","r")
>   lat2d      = fwrf->XLAT(0,:,:)
>   lon2d      = fwrf->XLONG(0,:,:)
>   printVarSummary(lat2d)
>   printMinMax(lat2d,0)    ;  min=12.5967   max=56.6092
>   printVarSummary(lon2d)
>   printMinMax(lon2d,0)    ;  min=59.5349   max=140.765
>
>   extra      = 2          ; arbitrary
>   minLat2D   = min(lat2d) - extra
>   maxLat2D   = max(lat2d) + extra
>   minLon2D   = min(lon2d) - extra
>   maxLon2D   = max(lon2d) + extra
>
> ;===============================================
> ; EDGAR grid
> ; [lat | 1800] x [lon | 3600]   ; rectilinear
> ;  lat: [-89.95..89.95]
> ;  lon: [0.05..359.95]
> ;===============================================
>
>   path_edgar = "./"
>   var_name   = "emi_co2"
>   fn_var     = "v42_CO2_2008_TOT.0.1x0.1.nc"       ; rectilinear
>   f_var      = addfile(path_edgar+fn_var,"r")
>
> ;===============================================
> ; Read *only* the EDGAR grid area surrounding China
> ;===============================================
>   var        = f_var->$var_name$({minLat2D:maxLat2D},{minLon2D:maxLon2D})
>   printVarSummary(var)
>   printMinMax(var,0)
>
> begTime = get_cpu_time()
>   var_regrid = rgrid2rcm (var&lat, var&lon, var, lat2d, lon2d, 0)
> print("rgrid2rcm: " + (get_cpu_time() - begTime) + " seconds")
>
>   var_regrid at long_name = var_name
>   copy_VarCoords(lat2d, var_regrid)
>   printVarSummary(var_regrid)
>   printMinMax(var_regrid,0)
>   print("================")
>
>   conv       = 1e9*3600/44  ; EDGAR in kg m-2 s-1  WRFchem need   mol km^-2 hr^-1
>   var_regrid = (/var_regrid*conv /)
>   var_regrid at units = "mol/km^2 hr^1"
>
>   printVarSummary(var_regrid)
>   printMinMax(var_regrid,0)
>
> ;=====================================
> ; Statistical distribution
> ; Use to set plot limits.
> ; There are outliers
> ;=====================================
>   opt = True
>   opt at PrintStat = True
>   stat_var = stat_dispersion(var_regrid, opt )
> ;=====================================
> ;; 	 [3]          LowDec=1.35338
> ;; 	 [4]          LowOct=3.07492
> ;; 	 [5]          LowSex=7.8127
> ;; 	 [6]     LowQuartile=27.2554
> ;; 	 [7]          LowTri=51.6854
> ;; 	 [8]          Median=99.2983
> ;; 	 [9]         HighTri=331.714
> ;; 	 [10]   HighQuartile=653.725
> ;; 	 [11]        HighSex=1085.59
> ;; 	 [12]        HighOct=1480.61
> ;; 	 [13]        HighDec=1816.67
> ;=====================================
> ; PLOT
> ;=====================================
>   wks = gsn_open_wks("png","wrf_EDGAR")
>
> ;---Set some basic plot options
>   res               = True
>   res at gsnMaximize   = True   ; maximize plot in frame
>   res at tiMainString  = fn_var
>
>   res at cnFillOn      = True
>   res at cnFillMode    = "RasterFill"
>   res at cnFillPalette = "WhiteBlueGreenYellowRed"
>   res at cnLinesOn     = False
>   res at cnLineLabelsOn= False
>
>   res at cnLevelSelectionMode = "ManualLevels"     ; set manual contour levels
>   res at cnMinLevelValF       = 25.0               ; set min contour level
>   res at cnMaxLevelValF       = 3000.0             ; set max contour level
>   res at cnLevelSpacingF      = 25.                ; set contour spacing
>
>   contour = gsn_csm_contour(wks,var_regrid,res)
>
>   delete([/ res at cnLevelSelectionMode, res at cnMinLevelValF \
>           , res at cnMaxLevelValF, res at cnLevelSpacingF      /] )
>
> ;---Change contour levels to better match the color map and/or data range being used
>
>   res at cnLevelSelectionMode = "ExplicitLevels"
>   res at cnLevels = (/ 10,20,40,60,80,100,150,200,250,300,350,400,500,600 \
>                   ,700,800,900,1000,1250,1500,1750,2000,2250,2500,2750,3000/)
>
>   res at mpProjection  = "CylindricalEquidistant"    ; The default
>   res at gsnAddCyclic  = False
>
> ;---Zoom in on plot
>   res at mpMinLatF     = minLat2D
>   res at mpMaxLatF     = maxLat2D
>   res at mpMinLonF     = minLon2D
>   res at mpMaxLonF     = maxLon2D
>
> ;---Additional resources desired
>   res at pmTickMarkDisplayMode = "Always"   ; nicer tickmarks
>
>  ;res at mpDataBaseVersion     = "MediumRes"       ; better and more map outlines
>  ;res at mpDataSetName         = "Earth..4"
>  ;res at mpOutlineBoundarySets = "AllBoundaries"
>  ;res at mpOutlineOn           = True
>   res at mpFillOn              = False
>
>   res at lbOrientation         = "Vertical"
>   res at tiMainOffsetYF        = -0.03           ; Move the title down
>
>   var_regrid at lat2d = lat2d
>   var_regrid at lon2d = lon2d
>   contour = gsn_csm_contour_map(wks,var_regrid,res)
>
>
>
> ------------------------------
> *From:* Dennis Shea <shea at ucar.edu>
> *Sent:* Thursday, April 5, 2018 7:16 AM
> *To:* xiaoming Hu; Xiaoming Hu
> *Subject:* Re: ESMF_regrid with Opt at InterpMethod = "neareststod" also
> slow, interpolating EDGAR CO2 emission to WRF domain
>
> Hi Xiaoming
>
> I just realized that you sent this regrid issue directly to me.
>
> Please, send all questions and responses to ncl-talk at ucar.edu
>
> When you send questions to me, you have made your-problem ... my-problem.
> There are other people who watch ncl-talk who can help with regridding &
> ESMF questions.
>
> ===
>
> Your source 'EDGAR' grid is a global rectilinear grid of size 1800 x 3600.
> The WRF/China space is a small subset.
>
> The 'regrid2rcm' is a brute-force algorithm written in fortran. It will go
> through all 6480000 point for *each* WRF grid point.
>
> You, the user, can help yourself by extracting the WRF region only from
> the EDGAR grid by using NCL's coordinate subscripting.
>
> I am very busy today. I will look at the ESMF later but it is a 'standard'
> rectilinear to curvilinear regrid.
>
> Attached a a sample use of rgrid2rcm. Still slow, 30 sec or so on a MAC.
>
> D
>
> On Wed, Apr 4, 2018 at 3:03 PM, Dennis Shea <shea at ucar.edu> wrote:
>
> His response
>
>
> ---------- Forwarded message ----------
> From: *xiaoming Hu* <yuanfangcan at hotmail.com>
> Date: Wed, Apr 4, 2018 at 12:55 PM
> Subject: ESMF_regrid with Opt at InterpMethod = "neareststod" also slow,
> interpolating EDGAR CO2 emission to WRF domain
> To: Dennis Shea <shea at ucar.edu>, Xiaoming Hu <xhu at ou.edu>
>
>
> Dear Dennis
>
>
> I put my files under: http://www.caps.ou.edu/
> micronet/CO2_and_otherGHG/WRFV3.9.1.1/YSU/wrfchem3.9.1.1_R2_
> China_Snudge_tracer16CTbc_SSTu_10mb_convOnScavOff.2016010100
> /VPRM_Preprocessor/
>
> including:
>
> http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.9.1.1
> /YSU/wrfchem3.9.1.1_R2_China_Snudge_tracer16CTbc_SSTu_10mb_c
> onvOnScavOff.2016010100/VPRM_Preprocessor/Interp_edgar42_to_
> wrfchemi_ChinafromPREP_CHEM_SRC.ncl
>
> http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.9.1.1
> /YSU/wrfchem3.9.1.1_R2_China_Snudge_tracer16CTbc_SSTu_10mb_c
> onvOnScavOff.2016010100/VPRM_Preprocessor/WRF.LatLon.nc
>
> http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.9.1.1
> /YSU/wrfchem3.9.1.1_R2_China_Snudge_tracer16CTbc_SSTu_10mb_c
> onvOnScavOff.2016010100/VPRM_Preprocessor/v42_CO2_2008_TOT.0.1x0.1.nc
>
>
> Please let me know if you trouble downloading the files or you need extra
> files.
>
>
> The interpolated CO2 emission is like:
>
>
> http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.9.1.1
> /YSU/wrfchem3.9.1.1_R2_China_Snudge_tracer16CTbc_SSTu_10mb_c
> onvOnScavOff.2016010100/wrfout_d01_E_CO2_0.png
>
>
> Thanks a lot!
>
> Xiaoming
>
>
> ------------------------------
> *From:* Dennis Shea <shea at ucar.edu>
> *Sent:* Wednesday, April 4, 2018 12:40:58 PM
> *To:* xiaoming Hu
> *Subject:* Re: [ncl-talk] rgrid2rcm too slow? ESMF_regrid with
> Opt at InterpMethod = "neareststod" also slow
>
> Hello,
>
> This is offline.
>
> [1]
> If the 'src and 'dst' files are small (less than 1GB) could u please ftp
> them?
>
> If the files are large or you want to send smaller files, then use the
> netCDF Operators to extract the needed information.
>
> Based upon:
>
>     path_edgar = "..."
>
>   fn_co2     = "v42_CO2_2008_TOT.0.1x0.1.nc"
>   f_co2      = addfile(path_edgar+fn_co2,"r")
>   emi_co2    = f_co2->emi_co2
>
>   fwrf    = addfile("wrfinput_d01.nc","r")
>   lat2d      = fwrf->XLAT(0,:,:)
>   lon2d      = fwrf->XLONG(0,:,:)
>
> %> ncks -v emi_co2  path_edgar/fn_co2  CO2.ESMF.nc
>
> %> ncks -v XLAT,XLON  wrfinput_d01.nc WRF.LatLon.nc
>
> etc
> ===
>
> ftp ftp.cgd.ucar.edu
> anonymous
> your_email
>
> cd incoming
>
> put ...
> put ...
> put ...
>
> quit
>
> ===
> Then send me the names of the files.
> I'll try to look later today.
>
> ===
>
> That said, if you are dealing with the same grids but different variables,
> then you can create a weight file and reuse.
>
> Even if it takes a "long time" to generate the weights, resusing the weights is essentially 'instantaneous'. Under the hood, it is a sparse matyrix multipl.
>
>
> http://www.ncl.ucar.edu/Document/Functions/ESMF/ESMF_regrid_gen_weights.shtm
>
> Cheers
>
> D
>
>
>
> On Wed, Apr 4, 2018 at 8:29 AM, xiaoming Hu <yuanfangcan at hotmail.com>
> wrote:
>
> I ended up using Opt at InterpMethod     = "neareststod"     for ESMF_regrid
> to make sure there is no negative values.
>
> But ESMF_regrid  with "neareststod"  is also slow.
>
> For comparison,   ESMF_regrid  with "neareststod"   took more than 1
> hour  while ESMF_regrid  with "patch" only took a few minutes
> Any comments?
>
> Thanks
> Xiaoming
>
> ------------------------------
> *From:* Dennis Shea <shea at ucar.edu>
> *Sent:* Wednesday, March 29, 2017 11:25 AM
> *To:* xiaoming Hu
> *Cc:* ncl-talk at ucar.edu; xiaoming Hu
> *Subject:* Re: [ncl-talk] rgrid2rcm too slow?
>
> Rather than:
>
> Opt at InterpMethod     = "patch"
>
> Try
>
> Opt at InterpMethod     = "bilinear"    ; or "conserve"
>
> ===
>
> I think that "patch" was developed to attain highly accurate
> derivatives.
>
>
>
> On Wed, Mar 29, 2017 at 10:07 AM, xiaoming Hu <yuanfangcan at hotmail.com>
> wrote:
>
> Dennis
>
>
> Thanks a lot!  Yes, ESMF_regrid is much quicker.
>
> It mostly worked for me except negative values generated,
>
> see my original data shown at http://www.caps.ou.edu/micr
> onet/CO2_and_otherGHG/WRFV3.8.1/YSU/wrfchem3.8.1_NARR_CONUS_
> nudge2_tracer16_Comet.2011080600/wrfout_d01_E_CO2_EDGAR_2.png
>
> and the interpolated data shown
>
> at http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.8.
> 1/YSU/wrfchem3.8.1_NARR_CONUS_nudge2_tracer16_Comet.20110806
> 00/wrfout_d01_E_CO2_0.png
>
> You may notice in the right-lower corner the contour information shows
> Min=-20.745
>
>
> Any suggestion to improve this?
>
> My interpolation script at http://www.caps.ou.edu/micr
> onet/CO2_and_otherGHG/WRFV3.8.1/YSU/wrfchem3.8.1_NARR_CONUS_
> nudge2_tracer16_Comet.2011080600/Interp_edgar42_to_wrfchemi_NEI.ncl
>
>
> Thanks a lot!
>
> Xiaoming
> ------------------------------
> *From:* Dennis Shea <shea at ucar.edu>
> *Sent:* Tuesday, March 28, 2017 11:39:23 AM
> *To:* xiaoming Hu
> *Cc:* ncl-talk at ucar.edu
> *Subject:* Re: [ncl-talk] rgrid2rcm too slow?
>
> rgrid2rcm can be slow.
>
> Use ESMF: there are exam
> http://www.ncl.ucar.edu/Applications/ESMF.shtml
>
> There are numerous examples of  going  from rectilinear ('rgrid') to
> curvilinear ('rcm')
> Examples 9, 15, 28-30, 32, 33,
>
> HTH
>
>
> On Tue, Mar 28, 2017 at 9:30 AM, xiaoming Hu <yuanfangcan at hotmail.com>
> wrote:
>
> I am doing
>
> "
> dummy1     = rgrid2rcm (elat, elon, emi_co2, lat2d, lon2d, 0)
> "
>
> where lat2d and lon2d have a dimension of 442x265
>
>
> The script appears to take forever.
>
>
> Any other ways to do a faster interpolation?
>
>
> Thanks a lot!
>
> Xiaoming
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EDGAR_to_WRF_neareststod_750.png
Type: image/png
Size: 467977 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EDGAR_to_WRF_bilinear_750.png
Type: image/png
Size: 462280 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EDGAR_to_WRF_patch_750.png
Type: image/png
Size: 464097 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: edgar42_to_wrf_ESMF.ncl
Type: application/octet-stream
Size: 7327 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0001.obj>


More information about the ncl-talk mailing list