[ncl-talk] A Proposal to incorporate our useful codes into the NCL/PyNGL (ff)

[John Clyne]

Dear Meng-Zhuo Zhang,

Thanks for reaching out to us. The NCAR NCL team is no longer adding new features to NCL/PyNGL as described in this open letter to the community:

https://geocat.ucar.edu/blog/2020/11/11/November-2020-update

NCAR’s new GeoCAT team is committed to building community Python tools for the geosciences and is actively seeking contributions. You can find out more about GeoCAT’s efforts here: https://geocat.ucar.edu/, and find information on contributing, in particular, here: https://geocat.ucar.edu/pages/contributing.html

Best,

John Clyne
National Center for Atmospheric Research
303.497.1236 (w), 303.809.1922 (c)
clyne at ucar.edu


> 
> Dear NCL/NCARG team,
> 
> 
> 
> Over past years, we devised some statistics and a diagram to evaluate climate model performance in simulating the vector field or multiple fields. These performance metrics and the diagram are useful in climate model evaluation and intercomparison. I?m writing to inquire if we can cooperate with your team to incorporate relevant function codes into the NCL/PyNGL. We briefly introduce the performance metrics and the diagram as follows:
> 
> (1)  Vector field evaluation (VFE) method
> 
> We defined three performance metrics in terms of the vector field:
> 
> Vector similarity coefficient (VSC): It measures the pattern similarity between two vector fields, which takes both the vector length and vector direction into account. In terms of the one-dimensional case, the VSC computed with anomaly field becomes the Pearson correlation coefficient.
> 
> Root mean square length of the vector field (RMSL): It measures the mean and variance of vector lengths. In the one-dimensional case, the RMSL computed with anomaly field becomes the standard deviation.
> 
> Root mean square vector difference (RMSVD): It measures the overall difference between two vector fields. In the one-dimensional case, the RMSVD computed with anomaly field becomes the commonly used root mean square difference.
> 
> The three vector field statistics (VSC, RMSL, and RMSVD) satisfy the cosine law. Thus, we used these statistics to construct a vector field evaluation (VFE) diagram, which can summarize and illustrate multiple aspects of model performance in simulating a vector field (Xu et al., 2016). The widely-used Taylor diagram can be regarded as a specific case of the VFE diagram.
> 
> (2)  Multivariable integrated evaluation method
> 
> Based on the VFE method, we further proposed a multivariable integrated evaluation (MVIE) method to support model evaluation on multiple fields (Xu et al., 2017; Zhang et al., 2021). These multiple fields consisting of either scalar or vector fields are normalized and then grouped into a whole?the multidimensional vector field for evaluation. The statistics of the MVIE measure the model performance in simulating multiple fields from the aspects of amplitude, pattern similarity, and overall difference, respectively.
> 
> 
> 
> However, using the functions provided by the NCL, we cannot yet easily calculate the performance metrics in terms of the vector field or multiple fields. Fortunately, we recently developed a Multivariable Integrated Evaluation Tool (MVIETool; Zhang et al., 2021) coded with the NCL/Python3 to facilitate the multivariable evaluation. MVIETool provides the functions to compute performance metrics regarding the vector field and multiple fields. These functions can take area weighting into account as needed. Meanwhile, they can also add variable weighting to adjust the relative importance of various variables in the performance metrics. Thus, users can conveniently evaluate model performance in terms of either vector field or multiple fields with the support of the tool. As the NCL and Python are the most popular languages in the atmospheric science community, we believe that our evaluation methods can better support model evaluation if the relevant codes can be incorporated into the NCL/
> PyNGL.
> 
> 
> 
> Your consideration of our proposal is greatly appreciated. We are looking forward to hearing from you soon.
> 
> 
> 
> 
> 
> 1.  Xu Z., Hou Z., Han Y., and Guo W., 2016: A diagram for evaluating multiple aspects of model performance in simulating vector fields, Geosci. Model Dev., http://doi.org/10.5194/gmd-2016-172
> 
> 2.     Xu Z., Han Y., and Fu C., 2017: Multivariable Integrated Evaluation of Model Performance with the Vector Field Evaluation Diagram. Geosci. Model Dev., 10, 3805?3820, https://doi.org/10.5194/gmd-10-3805-2017
> 
> 3.     Zhang M.-Z., Xu Z., Han Y., and Guo W., 2021: An improved multivariable integrated evaluation method and tool (MVIETool) v1.0 for multimodel intercomparison, Geosci. Model Dev. https://doi.org/10.5194/gmd-10-3805-2017
> 
> 
> 
> Best wishes,
> 
> Meng-Zhuo Zhang
> 
> School of Atmospheric Sciences,
> 
> Nanjing University
> 
> Nanjing, 210000
> 
> China
>