<div dir="ltr">Dear Dave,<div>Thank you for your suggestion. I think the problem comes from where I combine so many NetCDF files together. However, I could not remove the warnings.</div><div>Best wishes,</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 19, 2019 at 11:35 PM Dave Allured - NOAA Affiliate <<a href="mailto:dave.allured@noaa.gov">dave.allured@noaa.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>That is good information from the diagnostics. All file structure details look fine to me. However I notice that you are using addfiles to open about 12780 files simultaneously. I missed this the first time. You may be running into a system limit of number of simultaneous open files.</div><div><br></div><div>This is discussed in a paragraph about half way down the documentation page for the addfiles function. From that, I suggest adding this line before your addfiles command:</div><br> setfileoption ("nc", "SuppressClose", False)</div><div dir="ltr"><br></div><div>I am not familiar with this option. It is possible that you need to place this command after addfiles, and change the first argument from "nc" to just "f" without quotes. Try it both ways if necessary.</div><div><br></div><div dir="ltr"><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 19, 2019 at 10:35 AM Setareh Rahimi <<a href="mailto:setareh.rahimi@gmail.com" target="_blank">setareh.rahimi@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><font color="#000000">Dear all,</font><div><font color="#000000">Thank you for your suggestions. However, the problem still exists. I checked the files individually and found out nothing wrong with them. I run the script for each year separately and did not get the warnings, but once run the script for all the years together the warnings appear. I attached the output from tests that Dave has suggested. </font></div><div><font face="arial, sans-serif" color="#000000">NCL version:<span style="font-size:14px;font-variant-ligatures:no-common-ligatures"> 6.6.2 </span></font></div><div><font color="#000000" face="arial, sans-serif"><span style="font-size:14px;font-variant-ligatures:no-common-ligatures">Computer system: macOS Mojave, version 10.14</span></font></div><div><font color="#000000" face="arial, sans-serif"><span style="font-size:14px;font-variant-ligatures:no-common-ligatures">Best wishes,</span></font></div><div><font color="#000000" face="Menlo"><span style="font-size:14px;font-variant-ligatures:no-common-ligatures"><br></span></font></div>
</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Sep 18, 2019 at 3:16 AM Dave Allured - NOAA Affiliate <<a href="mailto:dave.allured@noaa.gov" target="_blank">dave.allured@noaa.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Recently there are some known conditions that can cause unknown format and corrupted file messages for valid netcdf files. I recommend diagnosing individual files, not using NCL, before dismissing an entire file set as corrupted. Try this black magic first and see if NCL can then read the files:</div><div dir="ltr"><br></div> Bash: HDF5_USE_FILE_LOCKING=FALSE<br> C-shell: setenv HDF5_USE_FILE_LOCKING FALSE<div><br></div><div>If that does not work, then try these tests. The first two are guaranteed to work on all file types.</div><div><br></div><div> file <a href="http://data.nc" target="_blank">data.nc</a></div><div> od -c -N16 <a href="http://data.nc" target="_blank">data.nc</a><br></div><div><div> h5dump -BH <a href="http://data.nc" target="_blank">data.nc</a></div></div><div> ncdump -k <a href="http://data.nc" target="_blank">data.nc</a><br></div><div> ncdump -sh <a href="http://data.nc" target="_blank">data.nc</a></div><div dir="ltr"><br></div><div>If the problem has not become obvious, then post output from these tests to this mailing list. If more than 40 lines long, put all output into a text file with name ending in .txt, and send as a file attachment to your message. Please do not send any screen shots.</div><div><br></div><div>Also send your NCL version number and type of computer system.</div><div dir="ltr"><br></div><div>--Dave</div><div><br></div><div><br></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Sep 17, 2019 at 3:27 PM Dennis Shea via ncl-talk <<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>As the message states, the file is "<b>corrupted</b>". This is not an NCL issue. <br></div><div><br></div><div>[1] delete the file[s] and try reacquiring them</div><div>[2] Possibly, the source files are corrupted.</div><div>---<br></div><div>FYI: There are some Persiann examples.<br></div><div><a href="https://www.ncl.ucar.edu/Applications/HiResPrc.shtml" target="_blank"><b>https://www.ncl.ucar.edu/Applications/HiResPrc.shtml</b></a></div><div><br></div><div><br></div></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Sep 17, 2019 at 12:15 PM Setareh Rahimi <<a href="mailto:setareh.rahimi@gmail.com" target="_blank">setareh.rahimi@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear Adam, and Dennis,<div>Thanks for your help. hopefully I could combine those files together, but NCL tells there is something wrong with some files (attached image). In order to check what could be wrong, I redownload 1983 files again and got many warning for the second time. Any suggestion to remove those warning, please?</div><div>Best wishes,</div></div></blockquote></div></blockquote></div></div></div></div></div></div></blockquote></div>
</blockquote></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">S.Rahimi<br><br></div>