<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
Hi Jason,</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div><font face="Calibri,sans-serif">I am a PyNIO user, not a developer so I can’t speak to any intentionality, but I am more surprised that your code works with version PyNIO version 1.4.1 than that it does’t work with the beta. I don’t have the old version
anymore, but I am curious the output of type(m) with your original code? I was under the impression that the call </font><span style="color: rgb(0, 0, 0); font-family: Menlo; font-size: 11px;">f.variables[</span><span style="color: rgb(209, 47, 27); font-family: Menlo; font-size: 11px;">"mask</span><font color="#d12f1b" face="Menlo" size="2">”</font><font face="Menlo" size="2">] </font><font face="Calibri,sans-serif">had
always produced a Nio object and not a numpy array. Using the beta version I see this:</font></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<p style="margin: 0px; font-size: 11px; font-family: Menlo;"><br>
</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">m_obj = f.variables[<span style="font-variant-ligatures: no-common-ligatures; color: #d12f1b">"mask"</span>]</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;"><span style="font-variant-ligatures: no-common-ligatures; color: #bb2ca2">print</span> type(m_obj)</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">>> <class 'Nio.NioVariable'></p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">m_contents = f.variables[<span style="font-variant-ligatures: no-common-ligatures; color: #d12f1b">"mask"</span>][:]</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;"><span style="font-variant-ligatures: no-common-ligatures; color: #bb2ca2">print</span> type(m_contents)</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">>> <type 'numpy.ndarray’></p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;"><br>
</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;"><span style="font-family: Calibri, sans-serif; font-size: medium;">What does the corresponding code produce with the 1.4.1?</span></p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;"><span style="font-family: Calibri, sans-serif; font-size: medium;"><br>
</span></p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;"><span style="font-family: Calibri, sans-serif; font-size: medium;">Thanks!</span></p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;"><span style="font-family: Calibri, sans-serif; font-size: medium;">Heather</span></p>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span><<a href="mailto:pyngl-talk-bounces@ucar.edu">pyngl-talk-bounces@ucar.edu</a>> on behalf of Jason Greenlaw - NOAA Affiliate <<a href="mailto:jason.greenlaw@noaa.gov">jason.greenlaw@noaa.gov</a>><br>
<span style="font-weight:bold">Date: </span>Tuesday, December 29, 2015 at 10:36 AM<br>
<span style="font-weight:bold">To: </span>"<a href="mailto:pyngl-talk@ucar.edu">pyngl-talk@ucar.edu</a>" <<a href="mailto:pyngl-talk@ucar.edu">pyngl-talk@ucar.edu</a>><br>
<span style="font-weight:bold">Subject: </span>[pyngl-talk] PyNIO 1.5.0 Beta vs 1.4.1 - NetCDF Variable Access and Numpy<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div>Hello,</div>
<div><br>
</div>
<div>I recently installed the 1.5.0 beta versions of PyNGL and PyNIO (using 64-bit binaries for CentOS6) and attempted to run some existing code, but encountered an issue when numpy functions (e.g. numpy.ma.masked_equal()) are called with NioVariable object
arguments.</div>
<div><br>
</div>
<div>At PyNIO v1.4.1/numpy1.6.1 I was able to do the following:</div>
<div><br>
</div>
<div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div>>>> import Nio</div>
<div>
<div>>>> Nio.__version__</div>
<div>'1.4.1'</div>
</div>
<div>>>> import numpy</div>
<div>>>> numpy.__version__</div>
<div>'1.6.1'</div>
<div>>>> f = Nio.open_file("<a href="http://glofs.leofs.fields.nowcast.20151229.t12z.nc">glofs.leofs.fields.nowcast.20151229.t12z.nc</a>", "r")</div>
<div>>>> m = f.variables["mask"]</div>
<div>>>> numpy.ma.masked_equal(m, 1.0)</div>
<div>masked_array(data =</div>
<div> [[0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> ...,</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]],</div>
<div> mask =</div>
<div> [[False False False ..., False False False]</div>
<div> [False False False ..., False False False]</div>
<div> [False False False ..., False False False]</div>
<div> ...,</div>
<div> [False False False ..., False False False]</div>
<div> [False False False ..., False False False]</div>
<div> [False False False ..., False False False]],</div>
<div> fill_value = 1.0)</div>
</blockquote>
</div>
<div><br>
</div>
<div><br>
</div>
<div>However at PyNIO 1.5.0 beta/numpy 1.9.2, the numpy function call hangs, and the process begins consuming memory at an exponential rate until the call is interrupted.</div>
<div><br>
</div>
<div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div>
<div>>>> import Nio<br>
</div>
<div>>>> Nio.__version__</div>
<div>'1.5.0-beta'</div>
<div>>>> import numpy</div>
<div>>>> numpy.__version__</div>
<div>'1.9.2'</div>
</div>
<div>
<div>>>> f = Nio.open_file("<a href="http://glofs.leofs.fields.nowcast.20151229.t12z.nc">glofs.leofs.fields.nowcast.20151229.t12z.nc</a>", "r")</div>
<div>>>> m = f.variables["mask"]</div>
</div>
<div>
<div>>>> numpy.ma.masked_equal(m, 1.0)</div>
<div>^C^CTraceback (most recent call last):</div>
<div> File "<stdin>", line 1, in <module></div>
<div> File "/opt/pyngl/python/lib/python2.7/site-packages/numpy/ma/core.py", line 1982, in masked_equal</div>
<div> output = masked_where(equal(x, value), x, copy=copy)</div>
<div> File "/opt/pyngl/python/lib/python2.7/site-packages/numpy/ma/core.py", line 928, in __call__</div>
<div> (da, db) = (getdata(a, subok=False), getdata(b, subok=False))</div>
<div> File "/opt/pyngl/python/lib/python2.7/site-packages/numpy/ma/core.py", line 667, in getdata</div>
<div> data = np.array(a, copy=False, subok=subok)</div>
<div> File "/opt/pyngl/python/lib/python2.7/site-packages/PyNIO/Nio.py", line 325, in __getitem__</div>
<div> ret = get_variable(self.file, self.varname, xsel)</div>
<div> File "/opt/pyngl/python/lib/python2.7/site-packages/PyNIO/coordsel.py", line 60, in get_variable</div>
<div> ret = file.file.variables[varname][xsel]</div>
<div>KeyboardInterrupt</div>
</div>
</blockquote>
<div><br>
</div>
<div>But if I change the call to use numpy indexing, it works:</div>
<br>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div>
<div>
<div>>>> m.shape</div>
<div>(24, 81)</div>
</div>
<div>>>> numpy.ma.masked_equal(m[:,:], 1.0)</div>
<div>masked_array(data =</div>
<div> [[0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> ...,</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]</div>
<div> [0.0 0.0 0.0 ..., 0.0 0.0 0.0]],</div>
<div> mask =</div>
<div> [[False False False ..., False False False]</div>
<div> [False False False ..., False False False]</div>
<div> [False False False ..., False False False]</div>
<div> ...,</div>
<div> [False False False ..., False False False]</div>
<div> [False False False ..., False False False]</div>
<div> [False False False ..., False False False]],</div>
<div> fill_value = 1.0)</div>
</div>
</blockquote>
<br>
</div>
<div>Was this change in functionality intentional? </div>
<div><br>
</div>
<div>The NetCDF files I used are available at:</div>
<div><br>
</div>
<div> <a href="ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/nos/prod/glofs.20151229/">
ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/nos/prod/glofs.20151229/</a><br>
</div>
<div><br>
</div>
(directory will change based on date)
<div><br>
</div>
<div>Thanks,</div>
<div>Jason</div>
<div>
<div>
<div class="gmail_signature">
<div dir="ltr">
<div>--</div>
Jason Greenlaw
<div>Software Developer, ERT, Inc.<br>
<div>NOAA/NOS/OCS/CSDL
<div><a href="http://nowcoast.noaa.gov" target="_blank">http://nowcoast.noaa.gov</a></div>
<div><a href="mailto:Jason.Greenlaw@noaa.gov" target="_blank">Jason.Greenlaw@noaa.gov</a></div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>