[ncl-talk] Unable to read files with addfiles
Giorgio Graffino
g.graffino at tim.it
Wed Jun 21 11:42:20 MDT 2023
Hi Dennis,
Thanks for your reply. I just realized that I skipped one line in the
list of commands, which is the one actually triggering the error. Please
see the list of commands below, plus the one you asked me to run.
dir = "/home/users/ggraffin/CMIP5/"
mslp_list = systemfunc("ls
"+dir+"historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm011*mslp_commongrid.nc")
print(mslp_list)
Variable: mslp_list
Type: string
Total Size: 80 bytes
10 values
Number of Dimensions: 1
Dimensions and sizes: [10]
Coordinates:
(0) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc
(1) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0111_mslp_commongrid.nc
(2) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0112_mslp_commongrid.nc
(3) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0113_mslp_commongrid.nc
(4) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0114_mslp_commongrid.nc
(5) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0115_mslp_commongrid.nc
(6) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0116_mslp_commongrid.nc
(7) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0117_mslp_commongrid.nc
(8) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0118_mslp_commongrid.nc
(9) /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0119_mslp_commongrid.nc
mslp_file = addfiles(mslp_list,"r")
print(mslp_file)
Variable: mslp_file
Type: list
Total Size: 4 bytes
1 values
Number of Dimensions: 1
Dimensions and sizes: [1]
Coordinates:
Type: list <concat | fifo>
Total items: 10
List Item 0: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 1: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0111_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 2: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0112_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 3: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0113_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 4: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0114_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 5: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0115_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 6: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0116_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 7: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0117_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 8: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0118_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
List Item 9: NclFileVarClass
Variable: unnamed
Type: file
File
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0119_mslp_commongrid.nc
Number of global attributes: 7
Number of dimensions: 4
Number of variables: 5
ListSetType(mslp_file,"join")
mslp = mslp_file[:]->slp(:,20*12:155*12+12-1,{-10:10},{135:270})
fatal:["NclFile.c":2100]:Subscript out of range, error in subscript #0
fatal:Memory allocation failure:[errno=12]
fatal:["Execute.c":8635]:Execute: Error occurred at or near line 36
Cheers,
Giorgio
------ Messaggio Originale ------
Da: shea at ucar.edu
A: g.graffino at tim.it Cc: ncl-talk at ucar.edu
Inviato: martedì 20 giugno 2023 19:40
Oggetto: Re: [ncl-talk] Unable to read files with addfiles
I can not see an obvious issue.
Given and the "join" option
time = 156 // unlimited
bnds = 2
lon = 180
lat = 120
and, say, 10 ensembles, the slp variable should be
slp(ncl_join , time, lat , lon ) => slp(10,156,120,180)
================================================
What do the following 'print' statements show?
mslp_list = systemfunc("ls
/home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm011*mslp_commongrid.nc
<http://mslp_commongrid.nc> ")
print(mslp_list) ; <========
mslp_file = addfiles(mslp_list,"r")
print(mslp_file) ; <========
ListSetType(mslp_file,"join")
On Tue, Jun 20, 2023 at 11:37 AM Giorgio Graffino via ncl-talk
<ncl-talk at mailman.ucar.edu <mailto:ncl-talk at mailman.ucar.edu> > wrote:
Hello NCL people,
I'm trying to read files from the MPI-ESM1.1 large ensemble with
addfiles. I'm always getting the same error, regardless of the sample
size, so it shouldn't be an actual memory issue.
mslp_list = systemfunc("ls
/home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm011*mslp_commongrid.nc
<http://mslp_commongrid.nc> ")
mslp_file = addfiles(mslp_list,"r")
ListSetType(mslp_file,"join")
fatal:["NclFile.c":2100]:Subscript out of range, error in subscript #0
fatal:Memory allocation failure:[errno=12]
I can't even use functions like getfiledimsizes or getfileatts, because
I get the following error.
dsizes = getfileatts(mslp_file(0))
fatal:Argument type mismatch on argument (0) of (getfileatts) can not
coerce
Both mslp_file and mslp_list have 10 elements as they should.
printVarSummary(mslp_file)
Variable: mslp_file
Type: list <join | fifo>
Total items: 10
printVarSummary(mslp_list)
Variable: mslp_list
Type: string
Total Size: 80 bytes
10 values
Number of Dimensions: 1
Dimensions and sizes: [10]
Coordinates:
I'm sure it's a file-dependent issue, because addfiles works fine on
subset of files from other sources. I'm asking for suggestions about
things I can try so I can understand what's going on. Here is the file
dump of one of the files.
Copyright (C) 1995-2019 - All Rights Reserved
University Corporation for Atmospheric Research
NCAR Command Language Version 6.6.2
The use of this software is governed by a License Agreement.
See http://www.ncl.ucar.edu/ <http://www.ncl.ucar.edu/> for more
details.
Variable: f
Type: file
filename: MPI-ESM1.1_historical_lkm0110_mslp_commongrid
path: /home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc
<http://MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc>
file global attributes:
CDI : Climate Data Interface version 1.9.5
(http://mpimet.mpg.de/cdi <http://mpimet.mpg.de/cdi> )
Conventions : CF-1.4
history : Tue Jun 20 17:34:23 2023: ncap2 -O -s time=float(time)
/home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc
<http://MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc>
/home/users/ggraffin/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc
<http://MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc>
Tue Apr 18 17:40:36 2023: cdo remapbil,CMIP5/CMIP5_grid.txt
/home/users/qx920595/baobab/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_year.nc
<http://MPI-ESM1.1_historical_lkm0110_mslp_year.nc>
/home/users/qx920595/baobab/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc
<http://MPI-ESM1.1_historical_lkm0110_mslp_commongrid.nc>
Tue Apr 18 17:40:30 2023: ncks -v slp
/storage/basic/baobab/jonathan/thorsten/MPI-ESM1.1_historical_lkm0110_yearmean.nc
<http://MPI-ESM1.1_historical_lkm0110_yearmean.nc>
/home/users/qx920595/baobab/CMIP5/historical/MPI-ESM1.1/MPI-ESM1.1_historical_lkm0110_mslp_year.nc
<http://MPI-ESM1.1_historical_lkm0110_mslp_year.nc>
Sat Sep 24 17:07:31 2016: cdo -f nc -t echam6 yearmean
/scratch/m/m300057/historical_ensemble/lkm0110_BOT
/scratch/m/m300057/historical_ensemble/MPI-ESM1.1_historical_lkm0110_yearmean.nc
<http://MPI-ESM1.1_historical_lkm0110_yearmean.nc>
source : ECHAM6
institution : Max-Planck-Institute for Meteorology
NCO : netCDF Operators version 5.0.7 (Homepage = http://nco.sf.net
<http://nco.sf.net> , Code = http://github.com/nco/nco
<http://github.com/nco/nco> )
CDO : Climate Data Operators version 1.9.5
(http://mpimet.mpg.de/cdo <http://mpimet.mpg.de/cdo> )
dimensions:
time = 156 // unlimited
bnds = 2
lon = 180
lat = 120
variables:
float time ( time )
axis : T
bounds : time_bnds
calendar : proleptic_gregorian
standard_name : time
units : day as %Y%m%d.%f
double time_bnds ( time, bnds )
double lon ( lon )
standard_name : longitude
long_name : longitude
units : degrees_east
axis : X
double lat ( lat )
standard_name : latitude
long_name : latitude
units : degrees_north
axis : Y
float slp ( time, lat, lon )
long_name : mean sea level pressure
units : Pa
code : 151
table : 128
Thanks a lot.
Giorgio
_______________________________________________
ncl-talk mailing list
ncl-talk at mailman.ucar.edu <mailto:ncl-talk at mailman.ucar.edu>
List instructions, subscriber options, unsubscribe:
https://mailman.ucar.edu/mailman/listinfo/ncl-talk
<https://mailman.ucar.edu/mailman/listinfo/ncl-talk>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20230621/08f26287/attachment-0001.htm>
More information about the ncl-talk
mailing list