[ncl-talk] interpolating EDGAR CO2 emission to WRF domain, affordable rgrid2rcm
Mary Haley
haley at ucar.edu
Thu Apr 5 10:54:30 MDT 2018
Hi Xiaoming,
Dennis sent me your files offline and his rgrid2rcm script.
I created a version that uses ESMF_regrid, using the plotting parameters
that Dennis sent since he did such a nice job on this.
The only issue I saw with the original regridding script is that you need
to set these two options, since the source and destination grids are
regional:
Opt at SrcRegional = True ; These two are important! Do not
Opt at DstRegional = True ; set if you have a global lat/lon grid
Attached is an ESMF_regrid version of your script that you can try with
bilinear, neareststod, and patch.
I also changed the plotting a little so you can choose between plotting
over the native WRF map projection, or a cylindrical equidistant projection
as Dennis had used. Finally, I paneled the original plot and regridded
plot for comparison.
Once you have the weights file *and* you have the same source and
destination grids, you can then set:
Opt at SkipSrcGrid = True
Opt at SkipDstGrid = True
Opt at SkipWgtGen = True
which will make subsequent regridding go much faster, because it only has
to do a sparse matrix multiply under the hood. You can only set these,
however, if are using the same source and destination grids and the same
interpolation method.
--Mary
On Thu, Apr 5, 2018 at 9:02 AM, xiaoming Hu <yuanfangcan at hotmail.com> wrote:
> Dennis
>
>
> Thanks a lot! By reading your ncl script, I learned a lot! Thanks for
> teaching me this affordable rgrid2rcm. 😊
>
> I am attaching your code below in case other people might be interested!
>
> Thanks
>
> Xiaoming
>
> path_wrf = "./"
>
> diro = "./"
> filo = "wrfchemi_d01_valueFromEdgar_rgrid2rcm_China.nc"
> ptho = diro+filo
>
> ;---Target WRF (China) grid
>
> fwrf = addfile(path_wrf+"WRF.LatLon.nc","r")
> lat2d = fwrf->XLAT(0,:,:)
> lon2d = fwrf->XLONG(0,:,:)
> printVarSummary(lat2d)
> printMinMax(lat2d,0) ; min=12.5967 max=56.6092
> printVarSummary(lon2d)
> printMinMax(lon2d,0) ; min=59.5349 max=140.765
>
> extra = 2 ; arbitrary
> minLat2D = min(lat2d) - extra
> maxLat2D = max(lat2d) + extra
> minLon2D = min(lon2d) - extra
> maxLon2D = max(lon2d) + extra
>
> ;===============================================
> ; EDGAR grid
> ; [lat | 1800] x [lon | 3600] ; rectilinear
> ; lat: [-89.95..89.95]
> ; lon: [0.05..359.95]
> ;===============================================
>
> path_edgar = "./"
> var_name = "emi_co2"
> fn_var = "v42_CO2_2008_TOT.0.1x0.1.nc" ; rectilinear
> f_var = addfile(path_edgar+fn_var,"r")
>
> ;===============================================
> ; Read *only* the EDGAR grid area surrounding China
> ;===============================================
> var = f_var->$var_name$({minLat2D:maxLat2D},{minLon2D:maxLon2D})
> printVarSummary(var)
> printMinMax(var,0)
>
> begTime = get_cpu_time()
> var_regrid = rgrid2rcm (var&lat, var&lon, var, lat2d, lon2d, 0)
> print("rgrid2rcm: " + (get_cpu_time() - begTime) + " seconds")
>
> var_regrid at long_name = var_name
> copy_VarCoords(lat2d, var_regrid)
> printVarSummary(var_regrid)
> printMinMax(var_regrid,0)
> print("================")
>
> conv = 1e9*3600/44 ; EDGAR in kg m-2 s-1 WRFchem need mol km^-2 hr^-1
> var_regrid = (/var_regrid*conv /)
> var_regrid at units = "mol/km^2 hr^1"
>
> printVarSummary(var_regrid)
> printMinMax(var_regrid,0)
>
> ;=====================================
> ; Statistical distribution
> ; Use to set plot limits.
> ; There are outliers
> ;=====================================
> opt = True
> opt at PrintStat = True
> stat_var = stat_dispersion(var_regrid, opt )
> ;=====================================
> ;; [3] LowDec=1.35338
> ;; [4] LowOct=3.07492
> ;; [5] LowSex=7.8127
> ;; [6] LowQuartile=27.2554
> ;; [7] LowTri=51.6854
> ;; [8] Median=99.2983
> ;; [9] HighTri=331.714
> ;; [10] HighQuartile=653.725
> ;; [11] HighSex=1085.59
> ;; [12] HighOct=1480.61
> ;; [13] HighDec=1816.67
> ;=====================================
> ; PLOT
> ;=====================================
> wks = gsn_open_wks("png","wrf_EDGAR")
>
> ;---Set some basic plot options
> res = True
> res at gsnMaximize = True ; maximize plot in frame
> res at tiMainString = fn_var
>
> res at cnFillOn = True
> res at cnFillMode = "RasterFill"
> res at cnFillPalette = "WhiteBlueGreenYellowRed"
> res at cnLinesOn = False
> res at cnLineLabelsOn= False
>
> res at cnLevelSelectionMode = "ManualLevels" ; set manual contour levels
> res at cnMinLevelValF = 25.0 ; set min contour level
> res at cnMaxLevelValF = 3000.0 ; set max contour level
> res at cnLevelSpacingF = 25. ; set contour spacing
>
> contour = gsn_csm_contour(wks,var_regrid,res)
>
> delete([/ res at cnLevelSelectionMode, res at cnMinLevelValF \
> , res at cnMaxLevelValF, res at cnLevelSpacingF /] )
>
> ;---Change contour levels to better match the color map and/or data range being used
>
> res at cnLevelSelectionMode = "ExplicitLevels"
> res at cnLevels = (/ 10,20,40,60,80,100,150,200,250,300,350,400,500,600 \
> ,700,800,900,1000,1250,1500,1750,2000,2250,2500,2750,3000/)
>
> res at mpProjection = "CylindricalEquidistant" ; The default
> res at gsnAddCyclic = False
>
> ;---Zoom in on plot
> res at mpMinLatF = minLat2D
> res at mpMaxLatF = maxLat2D
> res at mpMinLonF = minLon2D
> res at mpMaxLonF = maxLon2D
>
> ;---Additional resources desired
> res at pmTickMarkDisplayMode = "Always" ; nicer tickmarks
>
> ;res at mpDataBaseVersion = "MediumRes" ; better and more map outlines
> ;res at mpDataSetName = "Earth..4"
> ;res at mpOutlineBoundarySets = "AllBoundaries"
> ;res at mpOutlineOn = True
> res at mpFillOn = False
>
> res at lbOrientation = "Vertical"
> res at tiMainOffsetYF = -0.03 ; Move the title down
>
> var_regrid at lat2d = lat2d
> var_regrid at lon2d = lon2d
> contour = gsn_csm_contour_map(wks,var_regrid,res)
>
>
>
> ------------------------------
> *From:* Dennis Shea <shea at ucar.edu>
> *Sent:* Thursday, April 5, 2018 7:16 AM
> *To:* xiaoming Hu; Xiaoming Hu
> *Subject:* Re: ESMF_regrid with Opt at InterpMethod = "neareststod" also
> slow, interpolating EDGAR CO2 emission to WRF domain
>
> Hi Xiaoming
>
> I just realized that you sent this regrid issue directly to me.
>
> Please, send all questions and responses to ncl-talk at ucar.edu
>
> When you send questions to me, you have made your-problem ... my-problem.
> There are other people who watch ncl-talk who can help with regridding &
> ESMF questions.
>
> ===
>
> Your source 'EDGAR' grid is a global rectilinear grid of size 1800 x 3600.
> The WRF/China space is a small subset.
>
> The 'regrid2rcm' is a brute-force algorithm written in fortran. It will go
> through all 6480000 point for *each* WRF grid point.
>
> You, the user, can help yourself by extracting the WRF region only from
> the EDGAR grid by using NCL's coordinate subscripting.
>
> I am very busy today. I will look at the ESMF later but it is a 'standard'
> rectilinear to curvilinear regrid.
>
> Attached a a sample use of rgrid2rcm. Still slow, 30 sec or so on a MAC.
>
> D
>
> On Wed, Apr 4, 2018 at 3:03 PM, Dennis Shea <shea at ucar.edu> wrote:
>
> His response
>
>
> ---------- Forwarded message ----------
> From: *xiaoming Hu* <yuanfangcan at hotmail.com>
> Date: Wed, Apr 4, 2018 at 12:55 PM
> Subject: ESMF_regrid with Opt at InterpMethod = "neareststod" also slow,
> interpolating EDGAR CO2 emission to WRF domain
> To: Dennis Shea <shea at ucar.edu>, Xiaoming Hu <xhu at ou.edu>
>
>
> Dear Dennis
>
>
> I put my files under: http://www.caps.ou.edu/
> micronet/CO2_and_otherGHG/WRFV3.9.1.1/YSU/wrfchem3.9.1.1_R2_
> China_Snudge_tracer16CTbc_SSTu_10mb_convOnScavOff.2016010100
> /VPRM_Preprocessor/
>
> including:
>
> http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.9.1.1
> /YSU/wrfchem3.9.1.1_R2_China_Snudge_tracer16CTbc_SSTu_10mb_c
> onvOnScavOff.2016010100/VPRM_Preprocessor/Interp_edgar42_to_
> wrfchemi_ChinafromPREP_CHEM_SRC.ncl
>
> http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.9.1.1
> /YSU/wrfchem3.9.1.1_R2_China_Snudge_tracer16CTbc_SSTu_10mb_c
> onvOnScavOff.2016010100/VPRM_Preprocessor/WRF.LatLon.nc
>
> http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.9.1.1
> /YSU/wrfchem3.9.1.1_R2_China_Snudge_tracer16CTbc_SSTu_10mb_c
> onvOnScavOff.2016010100/VPRM_Preprocessor/v42_CO2_2008_TOT.0.1x0.1.nc
>
>
> Please let me know if you trouble downloading the files or you need extra
> files.
>
>
> The interpolated CO2 emission is like:
>
>
> http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.9.1.1
> /YSU/wrfchem3.9.1.1_R2_China_Snudge_tracer16CTbc_SSTu_10mb_c
> onvOnScavOff.2016010100/wrfout_d01_E_CO2_0.png
>
>
> Thanks a lot!
>
> Xiaoming
>
>
> ------------------------------
> *From:* Dennis Shea <shea at ucar.edu>
> *Sent:* Wednesday, April 4, 2018 12:40:58 PM
> *To:* xiaoming Hu
> *Subject:* Re: [ncl-talk] rgrid2rcm too slow? ESMF_regrid with
> Opt at InterpMethod = "neareststod" also slow
>
> Hello,
>
> This is offline.
>
> [1]
> If the 'src and 'dst' files are small (less than 1GB) could u please ftp
> them?
>
> If the files are large or you want to send smaller files, then use the
> netCDF Operators to extract the needed information.
>
> Based upon:
>
> path_edgar = "..."
>
> fn_co2 = "v42_CO2_2008_TOT.0.1x0.1.nc"
> f_co2 = addfile(path_edgar+fn_co2,"r")
> emi_co2 = f_co2->emi_co2
>
> fwrf = addfile("wrfinput_d01.nc","r")
> lat2d = fwrf->XLAT(0,:,:)
> lon2d = fwrf->XLONG(0,:,:)
>
> %> ncks -v emi_co2 path_edgar/fn_co2 CO2.ESMF.nc
>
> %> ncks -v XLAT,XLON wrfinput_d01.nc WRF.LatLon.nc
>
> etc
> ===
>
> ftp ftp.cgd.ucar.edu
> anonymous
> your_email
>
> cd incoming
>
> put ...
> put ...
> put ...
>
> quit
>
> ===
> Then send me the names of the files.
> I'll try to look later today.
>
> ===
>
> That said, if you are dealing with the same grids but different variables,
> then you can create a weight file and reuse.
>
> Even if it takes a "long time" to generate the weights, resusing the weights is essentially 'instantaneous'. Under the hood, it is a sparse matyrix multipl.
>
>
> http://www.ncl.ucar.edu/Document/Functions/ESMF/ESMF_regrid_gen_weights.shtm
>
> Cheers
>
> D
>
>
>
> On Wed, Apr 4, 2018 at 8:29 AM, xiaoming Hu <yuanfangcan at hotmail.com>
> wrote:
>
> I ended up using Opt at InterpMethod = "neareststod" for ESMF_regrid
> to make sure there is no negative values.
>
> But ESMF_regrid with "neareststod" is also slow.
>
> For comparison, ESMF_regrid with "neareststod" took more than 1
> hour while ESMF_regrid with "patch" only took a few minutes
> Any comments?
>
> Thanks
> Xiaoming
>
> ------------------------------
> *From:* Dennis Shea <shea at ucar.edu>
> *Sent:* Wednesday, March 29, 2017 11:25 AM
> *To:* xiaoming Hu
> *Cc:* ncl-talk at ucar.edu; xiaoming Hu
> *Subject:* Re: [ncl-talk] rgrid2rcm too slow?
>
> Rather than:
>
> Opt at InterpMethod = "patch"
>
> Try
>
> Opt at InterpMethod = "bilinear" ; or "conserve"
>
> ===
>
> I think that "patch" was developed to attain highly accurate
> derivatives.
>
>
>
> On Wed, Mar 29, 2017 at 10:07 AM, xiaoming Hu <yuanfangcan at hotmail.com>
> wrote:
>
> Dennis
>
>
> Thanks a lot! Yes, ESMF_regrid is much quicker.
>
> It mostly worked for me except negative values generated,
>
> see my original data shown at http://www.caps.ou.edu/micr
> onet/CO2_and_otherGHG/WRFV3.8.1/YSU/wrfchem3.8.1_NARR_CONUS_
> nudge2_tracer16_Comet.2011080600/wrfout_d01_E_CO2_EDGAR_2.png
>
> and the interpolated data shown
>
> at http://www.caps.ou.edu/micronet/CO2_and_otherGHG/WRFV3.8.
> 1/YSU/wrfchem3.8.1_NARR_CONUS_nudge2_tracer16_Comet.20110806
> 00/wrfout_d01_E_CO2_0.png
>
> You may notice in the right-lower corner the contour information shows
> Min=-20.745
>
>
> Any suggestion to improve this?
>
> My interpolation script at http://www.caps.ou.edu/micr
> onet/CO2_and_otherGHG/WRFV3.8.1/YSU/wrfchem3.8.1_NARR_CONUS_
> nudge2_tracer16_Comet.2011080600/Interp_edgar42_to_wrfchemi_NEI.ncl
>
>
> Thanks a lot!
>
> Xiaoming
> ------------------------------
> *From:* Dennis Shea <shea at ucar.edu>
> *Sent:* Tuesday, March 28, 2017 11:39:23 AM
> *To:* xiaoming Hu
> *Cc:* ncl-talk at ucar.edu
> *Subject:* Re: [ncl-talk] rgrid2rcm too slow?
>
> rgrid2rcm can be slow.
>
> Use ESMF: there are exam
> http://www.ncl.ucar.edu/Applications/ESMF.shtml
>
> There are numerous examples of going from rectilinear ('rgrid') to
> curvilinear ('rcm')
> Examples 9, 15, 28-30, 32, 33,
>
> HTH
>
>
> On Tue, Mar 28, 2017 at 9:30 AM, xiaoming Hu <yuanfangcan at hotmail.com>
> wrote:
>
> I am doing
>
> "
> dummy1 = rgrid2rcm (elat, elon, emi_co2, lat2d, lon2d, 0)
> "
>
> where lat2d and lon2d have a dimension of 442x265
>
>
> The script appears to take forever.
>
>
> Any other ways to do a faster interpolation?
>
>
> Thanks a lot!
>
> Xiaoming
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EDGAR_to_WRF_neareststod_750.png
Type: image/png
Size: 467977 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EDGAR_to_WRF_bilinear_750.png
Type: image/png
Size: 462280 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EDGAR_to_WRF_patch_750.png
Type: image/png
Size: 464097 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: edgar42_to_wrf_ESMF.ncl
Type: application/octet-stream
Size: 7327 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180405/42bd37f3/attachment-0001.obj>
More information about the ncl-talk
mailing list