[ncl-talk] Query regarding to the multiple files
Mary Haley
haley at ucar.edu
Tue Mar 6 10:47:12 MST 2018
It looks like you have a space after the last "/" in your diri string:
diri = "/Volumes/TIWARI/Arctic/Climatology/ "
------------------------------------------^
It should not be there!
diri = "/Volumes/TIWARI/Arctic/Climatology/"
--Mary
On Mon, Mar 5, 2018 at 8:57 PM, Shani Tiwari <pshanitiwari at gmail.com> wrote:
> Dear Dennis sir
> I modified he directory. Since in the present
> directory, there are only 17 .nc. I tried from both fils = systemfunc
> ("ls "+diri+"*.nc") and fils = systemfunc ("ls "+diri+"
> g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green*.nc") but both give the
> same result.
> Once I print the fils using *print (fils), the out is totally strange
> which is *
>
> Variable: fils
>
> Type: string
>
> Total Size: 328 bytes
>
> 41 values
>
> Number of Dimensions: 1
>
> Dimensions and sizes: [41]
>
> Coordinates:
>
> (0) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-
> 20001231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (1) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20010101-
> 20011231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (2) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20020101-
> 20021231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (3) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20030101-
> 20031231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (4) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20040101-
> 20041231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (5) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20050101-
> 20051231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (6) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20060101-
> 20061231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (7) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20070101-
> 20071231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (8) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20080101-
> 20081231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (9) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20090101-
> 20091231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (10) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20100101-
> 20101231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (11) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20110101-
> 20111231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (12) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20120101-
> 20121231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (13) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20130101-
> 20131231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (14) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20140101-
> 20141231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (15) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20150101-
> 20151231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (16) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20160101-
> 20161231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (17)
>
> (18) /Volumes/TIWARI/Arctic/Climatology/MISR/:
>
> (19) AOD anomaly from MISR.png
>
> (20) AOD climatology from MISR.png
>
> (21) Aerosol Climatology during JJA.000001.png
>
> (22) g4.ints.MIL3MAE_4_Optical_depth_average_Green.20000101-
> 20161231.JJA.180W_60N_180E_90N.csv
>
> (23) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-
> 20001231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (24) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20010101-
> 20011231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (25) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20020101-
> 20021231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (26) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20030101-
> 20031231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (27) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20040101-
> 20041231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (28) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20050101-
> 20051231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (29) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20060101-
> 20061231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (30) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20070101-
> 20071231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (31) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20080101-
> 20081231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (32) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20090101-
> 20091231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (33) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20100101-
> 20101231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (34) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20110101-
> 20111231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (35) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20120101-
> 20121231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (36) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20130101-
> 20131231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (37) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20140101-
> 20141231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (38) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20150101-
> 20151231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (39) g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20160101-
> 20161231.SEASON_JJA.180W_60N_180E_90N.nc
>
> (40) misr.ncl
>
> * Kindly let me know where is the fault in this script.*
>
>
> Regards
>
>
>
>
> ------------------------------------------------------------
> Shani Tiwari, Ph.D.
> Graduate School of Environmental Studies,
> Nagoya University, Nagoya, Japan
> Mob: +91 9451952997 <+91%2094519%2052997>
> Skype ID: shani.tiwari2
> Research Gate: Dr. Shani Tiwari
> <https://www.researchgate.net/profile/Shani_Tiwari2>
> Google Scholar Web-page: Dr. Shani Tiwari
> <https://scholar.google.co.jp/citations?user=IjIjMTMAAAAJ&hl=en>
> -------------------------------------------------------------
>
> On Tue, Mar 6, 2018 at 12:34 PM, Dennis Shea <shea at ucar.edu> wrote:
>
>>
>> (0) # of files = 41
>>
>>
>> I think you must be more specific in your 'systemfunc' usage.
>>
>>
>>
>> diri = "/Volumes/TIWARI/Arctic/Climatology/ "
>> fils = systemfunc ("ls "+diri+"*.nc")
>>
>> The above is getting the name of *ALL* the netCDF files in the
>> directory.
>>
>> Maybe:
>>
>> fils = systemfunc ("ls "+diri+"g4.timeAvg.MIL3MAE_4_O
>> ptical_depth_average_Green*.nc")
>>
>>
>>
>>
>>
>> On Mon, Mar 5, 2018 at 8:26 PM, Shani Tiwari <pshanitiwari at gmail.com>
>> wrote:
>>
>>> Dear Mary
>>> There are total 17 files in the folder and the output of
>>> the printVarSummaries as :
>>>
>>> *Script :*
>>>
>>> begin
>>>
>>> diri = "/Volumes/TIWARI/Arctic/Climatology/ "
>>> fils = systemfunc ("ls "+diri+"*.nc")
>>> f1 = addfiles (fils, "r") ;
>>>
>>> print("# of files = " + dimsizes(fils))
>>> print(f1[0])
>>>
>>> ListSetType (f1, "join")
>>> aerosol = f1[:]->MIL3MAE_4_Optical_depth_average_Green
>>> AOD = mask(aerosol,(aerosol.lt.0 .or. aerosol.gt.1),False)
>>>
>>>
>>> Mean = dim_avg_n_Wrap(AOD, 0)
>>> ; copy_VarCoords(Normal_AOD(0,:,:), Mean)
>>> Mean!0="lat"
>>> Mean!1="lon"
>>> Mean&lat = aerosol&lat ; assign coordinate values and
>>> Mean&lon = aerosol&lon ; units attributes
>>>
>>> printVarSummary (aerosol)
>>>
>>> end
>>>
>>> *And the output is :*
>>>
>>> (0) # of files = 41
>>>
>>>
>>> Variable: unnamed
>>>
>>> Type: file
>>>
>>> filename: g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-20
>>> 001231.SEASON_JJA.180W_60N_180E_90N
>>>
>>> path: g4.timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-20
>>> 001231.SEASON_JJA.180W_60N_180E_90N.nc
>>>
>>> file global attributes:
>>>
>>> nco_openmp_thread_number : 1
>>>
>>> Conventions : CF-1.4
>>>
>>> NCO : "4.5.3"
>>>
>>> history : Mon Mar 5 08:09:56 2018: ncks -O -x -v time_bnds
>>> timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-20001
>>> 231.SEASON_JJA.180W_60N_180E_90N.nc timeAvg.MIL3MAE_4_Optical_dept
>>> h_average_Green.20000101-20001231.SEASON_JJA.180W_60N_180E_90N.nc
>>>
>>> Mon Mar 5 08:09:56 2018: ncatted -a valid_range,,d,, -O -o
>>> timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-20001
>>> 231.SEASON_JJA.180W_60N_180E_90N.nc timeAvg.MIL3MAE_4_Optical_dept
>>> h_average_Green.20000101-20001231.SEASON_JJA.180W_60N_180E_90N.nc
>>>
>>> Mon Mar 5 08:09:56 2018: ncatted -O -a title,global,o,c,MIL3MAE_4_Optical_depth_average_Green
>>> Averaged over 2000-01-01 to 2000-12-31 timeAvg.MIL3MAE_4_Optical_dept
>>> h_average_Green.20000101-20001231.SEASON_JJA.180W_60N_180E_90N.nc
>>>
>>> Mon Mar 5 08:09:55 2018: ncks -x -v time -o
>>> timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-20001
>>> 231.SEASON_JJA.180W_60N_180E_90N.nc -O timeAvg.MIL3MAE_4_Optical_dept
>>> h_average_Green.20000101-20001231.SEASON_JJA.180W_60N_180E_90N.nc
>>>
>>> Mon Mar 5 08:09:55 2018: ncwa -o timeAvg.MIL3MAE_4_Optical_dept
>>> h_average_Green.20000101-20001231.SEASON_JJA.180W_60N_180E_90N.nc -a
>>> time -O timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-20001
>>> 231.SEASON_JJA.180W_60N_180E_90N.nc
>>>
>>> Mon Mar 5 08:09:55 2018: ncra -D 2 -H -O -o
>>> timeAvg.MIL3MAE_4_Optical_depth_average_Green.20000101-20001
>>> 231.SEASON_JJA.180W_60N_180E_90N.nc -d lat,60.000000,90.000000 -d
>>> lon,-180.000000,180.000000
>>>
>>> start_time : 2000-06-01T00:00:00Z
>>>
>>> end_time : 2000-08-31T23:59:59Z
>>>
>>> userstartdate : 2000-01-01T00:00:00Z
>>>
>>> userenddate : 2000-12-31T23:59:59Z
>>>
>>> title : JJA (2000-Jun - 2000-Aug) Aerosol Optical Depth 555 nm
>>> monthly 0.5 deg. [MISR MIL3MAE v4] over 2000-Jun - 2000-Aug, Region 180W,
>>> 60N, 180E, 90N
>>>
>>> plot_hint_title : JJA (2000-Jun - 2000-Aug)
>>>
>>> plot_hint_subtitle : Aerosol Optical Depth 555 nm monthly 0.5 deg.
>>> [MISR MIL3MAE v4] over 2000-Jun - 2000-Aug, Region 180W, 60N, 180E, 90N
>>>
>>> plot_hint_caption : - Seasons with missing months are discarded.
>>>
>>> dimensions:
>>>
>>> lat = 60
>>>
>>> lon = 720
>>>
>>> latv = 2
>>>
>>> lonv = 2
>>>
>>> variables:
>>>
>>> float MIL3MAE_4_Optical_depth_average_Green ( lat, lon )
>>>
>>> long_name : Aerosol Optical Depth 555 nm
>>>
>>> _FillValue : -9999
>>>
>>> cell_methods : OpticalDepth, Band, ParticleType: mean time:
>>> mean
>>>
>>> standard_name : optical_depth_average
>>>
>>> quantity_type : Total Aerosol Optical Depth
>>>
>>> product_short_name : MIL3MAE
>>>
>>> product_version : 4
>>>
>>> units : 1
>>>
>>> latitude_resolution : 0.5
>>>
>>> longitude_resolution : 0.5
>>>
>>> coordinates : lat lon
>>>
>>>
>>> double lat ( lat )
>>>
>>> units : degrees_north
>>>
>>> standard_name : latitude
>>>
>>> bounds : lat_bnds
>>>
>>>
>>> double lat_bnds ( lat, latv )
>>>
>>> units : degrees_north
>>>
>>>
>>> double lon ( lon )
>>>
>>> units : degrees_east
>>>
>>> standard_name : longitude
>>>
>>> bounds : lon_bnds
>>>
>>>
>>> double lon_bnds ( lon, lonv )
>>>
>>> units : degrees_east
>>>
>>>
>>> warning:A valid instance of variable MIL3MAE_4_Optical_depth_average_Green
>>> was not found in one or more elements of the file list
>>>
>>>
>>> Variable: aerosol
>>>
>>> Type: float
>>>
>>> Total Size: 5875200 bytes
>>>
>>> 1468800 values
>>>
>>> Number of Dimensions: 3
>>>
>>> *Dimensions and sizes: [ncl_join | 34] x [lat | 60] x [lon | 720]*
>>>
>>> Coordinates:
>>>
>>> lat: [60.25..89.75]
>>>
>>> lon: [-179.75..179.75]
>>>
>>> Number Of Attributes: 11
>>>
>>> long_name : Aerosol Optical Depth 555 nm
>>>
>>> _FillValue : -9999
>>>
>>> cell_methods : OpticalDepth, Band, ParticleType: mean time: mean
>>>
>>> standard_name : optical_depth_average
>>>
>>> quantity_type : Total Aerosol Optical Depth
>>>
>>> product_short_name : MIL3MAE
>>>
>>> product_version : 4
>>>
>>> units : 1
>>>
>>> latitude_resolution : 0.5
>>>
>>> longitude_resolution : 0.5
>>>
>>> coordinates : lat lon
>>>
>>> warning:dim_rmvmean_n: 444 rightmost sections of the input array
>>> contained all missing values
>>>
>>>
>>>
>>> Since , I added only 17 files, then the number of ncl_join should be 17
>>> instead of 34.
>>>
>>>
>>> I hope that now will understand my problem.
>>>
>>>
>>>
>>> Regards
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------
>>> Shani Tiwari, Ph.D.
>>> Graduate School of Environmental Studies,
>>> Nagoya University, Nagoya, Japan
>>> Mob: +91 9451952997 <+91%2094519%2052997>
>>> Skype ID: shani.tiwari2
>>> Research Gate: Dr. Shani Tiwari
>>> <https://www.researchgate.net/profile/Shani_Tiwari2>
>>> Google Scholar Web-page: Dr. Shani Tiwari
>>> <https://scholar.google.co.jp/citations?user=IjIjMTMAAAAJ&hl=en>
>>> -------------------------------------------------------------
>>>
>>> On Tue, Mar 6, 2018 at 5:15 AM, Mary Haley <haley at ucar.edu> wrote:
>>>
>>>> Dear Shani,
>>>>
>>>> It would help if you provide more information about how many files you
>>>> actually have, and what "aerosol" looks like.
>>>>
>>>> Can you please include the output from your "printVarSummary(aerosol)"
>>>> call, and also add the following two print statements to your script after
>>>> the addfiles call, and provide me with the output from that as well:
>>>>
>>>> print("# of files = " + dimsizes(fils))
>>>> print(f1[0])
>>>>
>>>> Thanks,
>>>>
>>>> --Mary
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Mar 5, 2018 at 2:35 AM, Shani Tiwari <pshanitiwari at gmail.com>
>>>> wrote:
>>>>
>>>>> Dear NCL users
>>>>> I added multiple files using "systemfunc"
>>>>> and "ListSetType ". But the printVarSummary gave "ncl_join" size *exactly
>>>>> double* to the number of files present in to the directory.
>>>>>
>>>>> begin
>>>>>
>>>>> diri = "/Volumes/TIWARI/Arctic/Climatology/ " ; input
>>>>> directory
>>>>> fils = systemfunc ("ls "+diri+"*.nc") ; file paths
>>>>> f1 = addfiles (fils, "r")
>>>>> ListSetType (f1, "join")
>>>>> aerosol = f1[:]->MIL3MAE_4_Optical_depth_average_Green
>>>>> AOD = mask(aerosol,(aerosol.lt.0 .or. aerosol.gt.1),False)
>>>>>
>>>>>
>>>>> Mean = dim_avg_n_Wrap(AOD, 0)
>>>>> ; copy_VarCoords(Normal_AOD(0,:,:), Mean)
>>>>> Mean!0="lat"
>>>>> Mean!1="lon"
>>>>> Mean&lat = aerosol&lat ; assign coordinate values and
>>>>> Mean&lon = aerosol&lon ; units attributes
>>>>>
>>>>> printVarSummary (aerosol)
>>>>>
>>>>> end
>>>>>
>>>>> Kindly suggest me, how to overcome to this?
>>>>>
>>>>>
>>>>> Regards
>>>>>
>>>>>
>>>>> ------------------------------------------------------------
>>>>> Shani Tiwari, Ph.D.
>>>>> Graduate School of Environmental Studies,
>>>>> Nagoya University, Nagoya, Japan
>>>>> Mob: +91 9451952997 <+91%2094519%2052997>
>>>>> Skype ID: shani.tiwari2
>>>>> Research Gate: Dr. Shani Tiwari
>>>>> <https://www.researchgate.net/profile/Shani_Tiwari2>
>>>>> Google Scholar Web-page: Dr. Shani Tiwari
>>>>> <https://scholar.google.co.jp/citations?user=IjIjMTMAAAAJ&hl=en>
>>>>> -------------------------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> ncl-talk mailing list
>>>>> ncl-talk at ucar.edu
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180306/28972b1e/attachment.html>
More information about the ncl-talk
mailing list