[ncl-talk] Unable to read files with addfiles

Giorgio Graffino g.graffino at tim.it
Wed Jun 21 11:42:20 MDT 2023


Hi Dennis,
 
Thanks for your reply. I just realized that I skipped one line in the 
list of commands, which is the one actually triggering the error. Please 
see the list of commands below, plus the one you asked me to run.
 
dir = "/home/users/ggraffin/CMIP5/"
 
mslp_list = systemfunc("ls 
"+dir+"historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm011*mslp_commongrid.nc")
 
print(mslp_list)                                                         
                                   
Variable: mslp_list
Type: string
Total Size: 80 bytes
           10 values
Number of Dimensions: 1
Dimensions and sizes:    [10]
Coordinates: 
(0)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc
(1)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0111_mslp_commongrid.nc
(2)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0112_mslp_commongrid.nc
(3)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0113_mslp_commongrid.nc
(4)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0114_mslp_commongrid.nc
(5)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0115_mslp_commongrid.nc
(6)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0116_mslp_commongrid.nc
(7)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0117_mslp_commongrid.nc
(8)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0118_mslp_commongrid.nc
(9)    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0119_mslp_commongrid.nc
 
mslp_file = addfiles(mslp_list,"r")
 
print(mslp_file)                                                         
                                   
Variable: mslp_file
Type: list
Total Size: 4 bytes
           1 values
Number of Dimensions: 1
Dimensions and sizes:    [1]
Coordinates: 
Type: list <concat | fifo>
Total items: 10
List Item 0:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 1:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0111_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 2:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0112_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 3:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0113_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 4:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0114_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 5:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0115_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 6:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0116_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 7:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0117_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 8:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0118_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
List Item 9:    NclFileVarClass
Variable: unnamed
Type: file
File 
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0119_mslp_commongrid.nc
Number of global attributes:     7
Number of dimensions:     4
Number of variables:     5
ListSetType(mslp_file,"join")
mslp = mslp_file[:]->slp(:,20*12:155*12+12-1,{-10:10},{135:270})
 
fatal:["NclFile.c":2100]:Subscript out of range, error in subscript #0
fatal:Memory allocation failure:[errno=12]
fatal:["Execute.c":8635]:Execute: Error occurred at or near line 36
 
Cheers,
Giorgio
 
------ Messaggio Originale ------
Da: shea at ucar.edu
A: g.graffino at tim.it Cc: ncl-talk at ucar.edu
Inviato: martedì 20 giugno 2023 19:40
Oggetto: Re: [ncl-talk] Unable to read files with addfiles
 
I can not see an obvious issue.
 
Given and the "join" option 
 
 
     time = 156  // unlimited
     bnds = 2
     lon = 180
     lat = 120

 
and, say, 10 ensembles, the slp variable should be

 
   slp(ncl_join , time, lat , lon )     => slp(10,156,120,180) 
 
================================================
 
What do the following 'print' statements show?  
 

mslp_list = systemfunc("ls 
/home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm011*mslp_commongrid.nc 
<http://mslp_commongrid.nc> ")
print(mslp_list)    ; <========   
 
mslp_file = addfiles(mslp_list,"r")
print(mslp_file)    ; <========
ListSetType(mslp_file,"join")
 

 
On Tue, Jun 20, 2023 at 11:37 AM Giorgio Graffino via ncl-talk 
<ncl-talk at mailman.ucar.edu <mailto:ncl-talk at mailman.ucar.edu> > wrote:
 
Hello NCL people,
 
I'm trying to read files from the MPI-ESM1.1 large ensemble with 
addfiles. I'm always getting the same error, regardless of the sample 
size, so it shouldn't be an actual memory issue.
 
mslp_list = systemfunc("ls 
/home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm011*mslp_commongrid.nc 
<http://mslp_commongrid.nc> ")
mslp_file = addfiles(mslp_list,"r")
ListSetType(mslp_file,"join")
fatal:["NclFile.c":2100]:Subscript out of range, error in subscript #0
fatal:Memory allocation failure:[errno=12]
 
I can't even use functions like getfiledimsizes or getfileatts, because 
I get the following error. 
dsizes = getfileatts(mslp_file(0)) 
fatal:Argument type mismatch on argument (0) of (getfileatts) can not 
coerce 
 
Both mslp_file and mslp_list have 10 elements as they should.
 
printVarSummary(mslp_file)          
Variable: mslp_file
Type: list <join | fifo>
Total items: 10
printVarSummary(mslp_list)
Variable: mslp_list
Type: string
Total Size: 80 bytes
           10 values
Number of Dimensions: 1
Dimensions and sizes:    [10]
Coordinates: 
 
I'm sure it's a file-dependent issue, because addfiles works fine on 
subset of files from other sources. I'm asking for suggestions about 
things I can try so I can understand what's going on. Here is the file 
dump of one of the files.
 
Copyright (C) 1995-2019 - All Rights Reserved
University Corporation for Atmospheric Research
NCAR Command Language Version 6.6.2
The use of this software is governed by a License Agreement.
See http://www.ncl.ucar.edu/ <http://www.ncl.ucar.edu/>  for more 
details.
Variable: f
Type: file
filename:    MPI-ESM1.1_historical_lkm0110_mslp_commongrid
path:    /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc 
<http://MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc>
  file global attributes:
     CDI : Climate Data Interface version 1.9.5 
(http://mpimet.mpg.de/cdi <http://mpimet.mpg.de/cdi> )
     Conventions : CF-1.4
     history : Tue Jun 20 17:34:23 2023: ncap2 -O -s time=float(time) 
/home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc 
<http://MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc> 
/home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc 
<http://MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc>
Tue Apr 18 17:40:36 2023: cdo remapbil,CMIP5/CMIP5_grid.txt 
/home/users/qx920595/baobab/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_year.nc 
<http://MPI-ESM1.1_historical_lkm0110_mslp_year.nc> 
/home/users/qx920595/baobab/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc 
<http://MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc>
Tue Apr 18 17:40:30 2023: ncks -v slp 
/storage/basic/baobab/jonathan/thorsten/MPI-ESM1.1_historical_lkm0110_yearmean.nc 
<http://MPI-ESM1.1_historical_lkm0110_yearmean.nc> 
/home/users/qx920595/baobab/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_year.nc 
<http://MPI-ESM1.1_historical_lkm0110_mslp_year.nc>
Sat Sep 24 17:07:31 2016: cdo -f nc -t echam6 yearmean 
/scratch/m/m300057/historical_ensemble/lkm0110_BOT 
/scratch/m/m300057/historical_ensemble/MPI-ESM1.1_historical_lkm0110_yearmean.nc 
<http://MPI-ESM1.1_historical_lkm0110_yearmean.nc>
     source : ECHAM6
     institution : Max-Planck-Institute for Meteorology
     NCO : netCDF Operators version 5.0.7 (Homepage = http://nco.sf.net 
<http://nco.sf.net> , Code = http://github.com/nco/nco 
<http://github.com/nco/nco> )
     CDO : Climate Data Operators version 1.9.5 
(http://mpimet.mpg.de/cdo <http://mpimet.mpg.de/cdo> )
  dimensions:
     time = 156  // unlimited
     bnds = 2
     lon = 180
     lat = 120
  variables:
     float time ( time )
        axis :    T
        bounds :    time_bnds
        calendar :    proleptic_gregorian
        standard_name :    time
        units :    day as %Y%m%d.%f
     double time_bnds ( time, bnds )
     double lon ( lon )
        standard_name :    longitude
        long_name :    longitude
        units :    degrees_east
        axis :    X
     double lat ( lat )
        standard_name :    latitude
        long_name :    latitude
        units :    degrees_north
        axis :    Y
     float slp ( time, lat, lon )
        long_name :    mean sea level pressure
        units :    Pa
        code :    151
        table :    128
 
Thanks a lot.
 
Giorgio
_______________________________________________
ncl-talk mailing list
ncl-talk at mailman.ucar.edu <mailto:ncl-talk at mailman.ucar.edu>
List instructions, subscriber options, unsubscribe:
https://mailman.ucar.edu/mailman/listinfo/ncl-talk 
<https://mailman.ucar.edu/mailman/listinfo/ncl-talk>
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20230621/08f26287/attachment-0001.htm>


More information about the ncl-talk mailing list