<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Thank you very much for your response.<div class=""><br class=""></div><div class="">I’m using NCL ver. 6.3.0 and the system info is the following:</div><div class="">Linux login02 2.6.32-696.3.2.el6.x86_64 #1 SMP Wed Jun 7 11:51:39 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux</div><div class=""><br class=""></div><div class="">The header of the target file is:</div><div class=""><br class=""></div><div class=""><div class="">netcdf tas_day_IPSL-CM5A-LR_rcp85_r1i1p1_20060101-22051231.rgrd {</div><div class="">dimensions:</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>time = 73000 ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>i = 180 ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>j = 180 ;</div><div class="">variables:</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>int time(time) ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string time:long_name = "time" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string time:units = "month" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>float i(i) ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string i:units = "none" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string i:long_name = "i index" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>float j(j) ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string j:units = "none" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string j:long_name = "j index" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>float tas(time, i, j) ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:associated_files = "baseURL: <a href="http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation" class="">http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation</a> gridspecFile: gridspec_atmos_fx_IPSL-CM5A-LR_rcp85_r0i0p0.nc areacella: areacella_fx_IPSL-CM5A-LR_rcp85_r0i0p0.nc" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:coordinates = "height" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:history = "2011-08-16T22:13:26Z altered by CMOR: Treated scalar dimension: \'height\'. 2011-08-16T22:13:26Z altered by CMOR: replaced missing value flag (9.96921e+36) with standard missing value (1e+20). 2011-08-16T22:13:45Z altered by CMOR: Inverted axis: lat." ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:cell_measures = "area: areacella" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:cell_methods = "time: mean (interval: 30 minutes)" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:original_name = "t2m" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:units = "K" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:long_name = "Near-Surface Air Temperature" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:standard_name = "air_temperature" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string tas:remap = "remapped via ESMF_regrid_with_weights: Bilinear" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>tas:missing_value = 1.e+20f ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>tas:_FillValue = 1.e+20f ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>float lat(j, i) ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string lat:long_name = "latitude" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string lat:units = "degrees_north" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>lat:_FillValue = -999.f ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>float lon(j, i) ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string lon:long_name = "longitude" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>string lon:units = "degrees_east" ;</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>lon:_FillValue = -999.f ;</div><div class="">}</div><div class=""><br class=""></div><div><blockquote type="cite" class=""><div class="">On Dec 18, 2017, at 8:05 AM, Dennis Shea <<a href="mailto:shea@ucar.edu" class="">shea@ucar.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class="">When you have a file issue, you should include some information:<br class=""><br class=""></div>(a) what what either of the fillowing show?<br class=""><br class=""></div>%> ncl_filedump <a href="http://secure-web.cisco.com/1Vrb-DzQ8Nq6PmsZlx-VCfXUfqpPMn1kXTYcQdEn-y4fG8MjIf5FObpsBYxAL2dmU2ZkqNJfPs2m48ExK2fva0vSjYwNLr6rs3gkQ6g5UEmk8YZ2M20H3syb-rPE3GNx2tTdfo_z1ua7u5k0Wb08hDwLEukpmfhGBv7kt8YDlQbBwjUBMS_HdToqmK_XiP8JoUcRtxtarP0x80S1_Z-v8v5Z8PvxcBc0moozZZf5dmaDSt4FFu4fIrM23KZHUbY46HPXuKt9NI2DWfPerKnJJLwBnFnXSQPpfgpK-FLJBenIoiKsGlrVbaphjCtP6hAq7mvHF1zP6hieqFXhU995Jask72xYsh1-1fl1VwZb4r3sJuqZFamfSy6SNO2CHfgcNy90HUGvXz8Tr1-oKAdgRxdzO1wcnFpPoAjegHXfheH2PZAQj9b07_gdKkzY6eidU/http%3A%2F%2Ftas_day_IPSL-CM5A-LR_rcp85_r1i1p1_20060101-22051231.rgrd.nc" rel="noreferrer" target="_blank" class="">tas_day_IPSL-CM5A-LR_<wbr class="">rcp85_r1i1p1_20060101-<wbr class="">22051231.rgrd.nc</a><br class=""><br class=""></div>or <br class=""><br class=""></div>%> ncdump -h <a href="http://secure-web.cisco.com/1Vrb-DzQ8Nq6PmsZlx-VCfXUfqpPMn1kXTYcQdEn-y4fG8MjIf5FObpsBYxAL2dmU2ZkqNJfPs2m48ExK2fva0vSjYwNLr6rs3gkQ6g5UEmk8YZ2M20H3syb-rPE3GNx2tTdfo_z1ua7u5k0Wb08hDwLEukpmfhGBv7kt8YDlQbBwjUBMS_HdToqmK_XiP8JoUcRtxtarP0x80S1_Z-v8v5Z8PvxcBc0moozZZf5dmaDSt4FFu4fIrM23KZHUbY46HPXuKt9NI2DWfPerKnJJLwBnFnXSQPpfgpK-FLJBenIoiKsGlrVbaphjCtP6hAq7mvHF1zP6hieqFXhU995Jask72xYsh1-1fl1VwZb4r3sJuqZFamfSy6SNO2CHfgcNy90HUGvXz8Tr1-oKAdgRxdzO1wcnFpPoAjegHXfheH2PZAQj9b07_gdKkzY6eidU/http%3A%2F%2Ftas_day_IPSL-CM5A-LR_rcp85_r1i1p1_20060101-22051231.rgrd.nc" rel="noreferrer" target="_blank" class="">tas_day_IPSL-CM5A-LR_<wbr class="">rcp85_r1i1p1_20060101-<wbr class="">22051231.rgrd.nc</a><br class=""><br class=""></div>(b) what version of NCL are you using><br class=""><br class=""></div>%> ncl -V<br class=""><br class=""></div>(c) your system info<br class=""><br class=""></div>%> uname -a<br class=""><br class="">-----------<br class=""><br class="">
fdir=“/root/dir4ncl/“<br class="">
fili="<a href="http://secure-web.cisco.com/1Vrb-DzQ8Nq6PmsZlx-VCfXUfqpPMn1kXTYcQdEn-y4fG8MjIf5FObpsBYxAL2dmU2ZkqNJfPs2m48ExK2fva0vSjYwNLr6rs3gkQ6g5UEmk8YZ2M20H3syb-rPE3GNx2tTdfo_z1ua7u5k0Wb08hDwLEukpmfhGBv7kt8YDlQbBwjUBMS_HdToqmK_XiP8JoUcRtxtarP0x80S1_Z-v8v5Z8PvxcBc0moozZZf5dmaDSt4FFu4fIrM23KZHUbY46HPXuKt9NI2DWfPerKnJJLwBnFnXSQPpfgpK-FLJBenIoiKsGlrVbaphjCtP6hAq7mvHF1zP6hieqFXhU995Jask72xYsh1-1fl1VwZb4r3sJuqZFamfSy6SNO2CHfgcNy90HUGvXz8Tr1-oKAdgRxdzO1wcnFpPoAjegHXfheH2PZAQj9b07_gdKkzY6eidU/http%3A%2F%2Ftas_day_IPSL-CM5A-LR_rcp85_r1i1p1_20060101-22051231.rgrd.nc" rel="noreferrer" target="_blank" class="">tas_day_IPSL-CM5A-LR_<wbr class="">rcp85_r1i1p1_20060101-<wbr class="">22051231.rgrd.nc</a>"<br class="">
fn=addfile(fdir+fili,"r")<br class="">
buff=fn->tas<div class=""><div class=""><div class=""><div class=""> print(“Data is stored”)</div></div></div></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Dec 18, 2017 at 3:35 AM, Guido Cioni <span dir="ltr" class=""><<a href="mailto:guidocioni@gmail.com" target="_blank" class="">guidocioni@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="">Tomoko,<div class="">9 GB is anything but "large", although the concept of "large" is highly subjective :P </div><div class=""><br class=""></div><div class="">I've successfully read SINGLE netcdf files in NCL whose size was ~500GB so that shouldn't be the problem. For some reason a netcdf file of some size, say 400 GB, which has many timesteps is read more slowly than a file with the same size but with less timesteps; that was my impression.</div><div class=""><br class=""></div><div class="">You are setting a lot of options which I think are not needed. Did you just try to read the file with this line? </div><div class=""><br class=""></div><div class=""><blockquote type="cite" class="">fn=addfile(fdir+fili,"r")<br class=""></blockquote><br class=""></div><div class="">If it still takes a lot of time it could be system-dependent. When creating the variable NCL stores it into the RAM. If the system does not have enough RAM, some virtual memory will be created on your hard drive, which can slow down everything. But honestly I don't think you're even close to saturate your system's RAM. The problem may lie somewhere else...</div><div class=""><br class=""></div><div class="">Let us know.</div><div class=""><br class=""></div><div class=""><div class=""><div class="h5"><br class=""><div class=""><blockquote type="cite" class=""><div class="">On 18. Dec 2017, at 07:17, Tomoko Koyama <<a href="mailto:Tomoko.Koyama@Colorado.EDU" target="_blank" class="">Tomoko.Koyama@Colorado.EDU</a>> wrote:</div><br class="m_6192207682118824426Apple-interchange-newline"><div class=""><div class="">I’d like to extract some data from a large netcdf file, which size is about 9GB.<br class=""><br class="">The following shows the partial script, but a submitted job was killed before “Data is stored” message appeared.<br class="">(I attempted several times with 3-hr max walltime )<br class=""><br class=""> setfileoption("nc", "FileStructure", "Advanced")<br class=""> fdir=“/root/dir4ncl/“<br class=""> fili="<a href="http://secure-web.cisco.com/1Vrb-DzQ8Nq6PmsZlx-VCfXUfqpPMn1kXTYcQdEn-y4fG8MjIf5FObpsBYxAL2dmU2ZkqNJfPs2m48ExK2fva0vSjYwNLr6rs3gkQ6g5UEmk8YZ2M20H3syb-rPE3GNx2tTdfo_z1ua7u5k0Wb08hDwLEukpmfhGBv7kt8YDlQbBwjUBMS_HdToqmK_XiP8JoUcRtxtarP0x80S1_Z-v8v5Z8PvxcBc0moozZZf5dmaDSt4FFu4fIrM23KZHUbY46HPXuKt9NI2DWfPerKnJJLwBnFnXSQPpfgpK-FLJBenIoiKsGlrVbaphjCtP6hAq7mvHF1zP6hieqFXhU995Jask72xYsh1-1fl1VwZb4r3sJuqZFamfSy6SNO2CHfgcNy90HUGvXz8Tr1-oKAdgRxdzO1wcnFpPoAjegHXfheH2PZAQj9b07_gdKkzY6eidU/http%3A%2F%2Ftas_day_IPSL-CM5A-LR_rcp85_r1i1p1_20060101-22051231.rgrd.nc" target="_blank" class="">tas_day_IPSL-CM5A-LR_<wbr class="">rcp85_r1i1p1_20060101-<wbr class="">22051231.rgrd.nc</a>"<br class=""> fn=addfile(fdir+fili,"r")<br class=""> setfileoption("nc","Format","<wbr class="">NetCDF4Classic")<br class=""> buff=fn->tas<br class=""> print(“Data is stored”)<br class=""><br class="">Does it simply take a long time to read?<br class="">Is there anyway to speed up to read a large netcdf file?<br class=""><br class=""><br class="">Thank you,<br class="">Tomoko<br class="">______________________________<wbr class="">_________________<br class="">ncl-talk mailing list<br class=""><a href="mailto:ncl-talk@ucar.edu" target="_blank" class="">ncl-talk@ucar.edu</a><br class="">List instructions, subscriber options, unsubscribe:<br class=""><a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank" class="">http://mailman.ucar.edu/<wbr class="">mailman/listinfo/ncl-talk</a><br class=""></div></div></blockquote></div><br class=""></div></div><span class="HOEnZb"><font color="#888888" class=""><div class="">
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><br class="m_6192207682118824426Apple-interchange-newline">Guido Cioni</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><a href="http://secure-web.cisco.com/1VTkTCOmq-V7se95ulfH2_V2onf2c5MHHSTkyctv2YOqoL1PVEAl_MqESQrE3TCnUXt5-Plwk6X6Ijz7gW23j1IO1btnfLdwl5uueV2Byy2e8E0u3WyoeKH12w2VLjTkl-D2S3X_BVWJH-e346H9PjUFHsGPt8LKQoSWmxYYoaBF17tgPyud58ZrGEy1Qz0AIEN_hEEVgf3aZxxopkxDkeph0fLWlG48ghQ4jtD4XBzTLqXCYE6HY0OpDrszaiyfBqlA8nNl2lBEzsl0vnu9k5hLcDideVoHSjwf4q4FmI7Eyzo3SPdcG_z7wCvLKzMRx2gLbgEOIgS35uovnkmWg4MteYUkJvSQjsVXdhrfOe7LnBXeHA6rs2JZYtZCdLzsqT7rE_dzKlyPUK2cav6PhfYhz7MwFpKBJmj0joQ3s4RlnsHFtLB-nxRtZq9PTt9JL/http%3A%2F%2Fguidocioni.altervista" target="_blank" class="">http://guidocioni.altervista</a>.<wbr class="">org</div>
</div>
<br class=""></font></span></div></div><br class="">______________________________<wbr class="">_________________<br class="">
ncl-talk mailing list<br class="">
<a href="mailto:ncl-talk@ucar.edu" class="">ncl-talk@ucar.edu</a><br class="">
List instructions, subscriber options, unsubscribe:<br class="">
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank" class="">http://mailman.ucar.edu/<wbr class="">mailman/listinfo/ncl-talk</a><br class="">
<br class=""></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""></div></body></html>