[ncl-talk] Problems with averaging over a do loop

Walter Kolczynski walter.kolczynski at noaa.gov
Fri Jun 5 12:27:12 MDT 2015 

In that example, NCL would compute the average as 5. If you actually 
want to make all the missing values zero, it would be better to use the 
where function to make it clear that is what you are doing:

a = addfiles(fili,"r")
ListSetType(a, "join")  ; create new record dimension when reading files
AODVIS = a[:]->VAR_147_GDS0_EATM
AODVIS = where( ismissing(AODVIS), 0.0, AODVIS )    ; replace all 
missing values with 0.0
AOD = dim_avg_n_Wrap( AODVIS, 0 )                   ; average along 0th 
dimension and keep metadata

However, missing values are generally supposed to represent unknown 
data, not 0.0, so make sure this is appropriate for your application (I 
don't know anything about FIM-chem data).

- Walter

On 05-Jun-15 14:11, BLIUJUS, STEVEN D CTR USAF AFMC AFLCMC/HBAW-OL wrote:
> For the FillValue though, here is my question. Let's say point 2,2 (whatever lat,lon that is) is -9999 in one file but 5.0 in the other, what value will it be given when using the average? I don't want it to show 5, it should show 2.5.
>
> -----Original Message-----
> From: ncl-talk-bounces at ucar.edu [mailto:ncl-talk-bounces at ucar.edu] On Behalf Of Walter Kolczynski
> Sent: Friday, June 05, 2015 1:08 PM
> To: ncl-talk at ucar.edu
> Subject: Re: [ncl-talk] Problems with averaging over a do loop
>
> One more I just saw:
>
> 5) You are assigning AODVIS at _FillValue, and then immediately deleting it. (a) This is pointless, (b) You should only need to assign the fill value if the value assigned when the grib is read is incorrect for some reason (rare), (c) zero is generally not used as a fill value (remember that anything equal to the fill value will be treated as missing and ignored in most NCL computations).
>
> - Walter
>
>
> On 05-Jun-15 13:56, Walter Kolczynski wrote:
>
>
> 	Steven,
> 	
> 	Several things:
> 	
> 	1) You didn't create the AODSum array with two dimensions, you created an array with one dimension of length 2 (check your printVarSummary). To create a multi-dimensional array with new you need to pass in an array of the dimension sizes.
> 	2) NCL has functions to average over a specified dimension. You should read all of your files in at once with a[:]->VAR_147_GDS0_EATM and then use dim_avg_n_Wrap to average. This will both be cleaner (no need for an AODSum at all) and eliminate the do loop (which are slow in interpreted languages such as NCL).
> 	3) Since you want to create a new record dimension when reading in the variables, you should be using "join" mode instead of the default "cat" for your file list by calling ListSetType(a, "join").
> 	4) Because you are reading in LAT and LON across all files, you are creating an array that has multiple copies of the lat/lon arrays, which is likely not what you want. Since lat/lon do not change, you should just read them from the first file in the array (a[0]), plus...
> 	4a) You do not need to assign LAT and LON as attributes of AODVIS. They are already coordinates for the appropriate dimensions of AODVIS.
> 	
> 	- Walter
> 	
> 	
> 	On 05-Jun-15 12:15, BLIUJUS, STEVEN D CTR USAF AFMC AFLCMC/HBAW-OL wrote:
> 	
>
> 		I am trying to take an average of a value in several files. The values are
> 		an array. In this case the Variable is AOD which is based on Lat/Lon. I can
> 		manually sum them and create and average, but I am trying to create a do
> 		loop such that if I have several files (40) I can average that value. All
> 		files have the exact same lat/lon, just the AOD value is different. I have
> 		attached two scripts. The first one is gribtest.ncl. This script is where I
> 		manually add the two files. The other attached is AOD.ncl. This is where I
> 		am trying to create a do loop. I am trying to create an empty array before
> 		the do loop called AODSum. However, when the loop executes I dies when
> 		trying to add with the message:
> 		
> 		fatal:Plus: Number of dimensions do not match, can't continue
> 		fatal:["Execute.c":8128]:Execute: Error occurred at or near line 37 in file
> 		AOD.ncl
> 		
> 		(0)     Error: scalar_field: If the input data is 1-dimensional, you must
> 		set sfXArray and sfYArray to 1-dimensional arrays of the same length.
> 		warning:create: Bad HLU id passed to create, ignoring it
> 		
> 		I created the array with two dimensions so I am not sure why it is failing.
> 		The two files are much too large to send via email, so if those need to be
> 		viewed I'd have to send them another way. Here is all the variable summaries
> 		though:
> 		
> 		Variable: LAT
> 		Type: float
> 		Total Size: 2888 bytes
> 		            722 values
> 		Number of Dimensions: 1
> 		Dimensions and sizes:   [g0_lat_1 | 722]
> 		Coordinates:
> 		            g0_lat_1: [-90..90]
> 		Number Of Attributes: 9
> 		  long_name :   latitude
> 		  GridType :    Cylindrical Equidistant Projection Grid
> 		  units :       degrees_north
> 		  Dj :  0.5
> 		  Di :  0.5006954
> 		  Lo2 : 360
> 		  La2 : 90
> 		  Lo1 :  0
> 		  La1 : -90
> 		
> 		Variable: LON
> 		Type: float
> 		Total Size: 5760 bytes
> 		            1440 values
> 		Number of Dimensions: 1
> 		Dimensions and sizes:   [g0_lon_2 | 1440]
> 		Coordinates:
> 		            g0_lon_2: [ 0..360]
> 		Number Of Attributes: 9
> 		  long_name :   longitude
> 		  GridType :    Cylindrical Equidistant Projection Grid
> 		  units :       degrees_east
> 		  Dj :  0.5
> 		  Di :  0.5006954
> 		  Lo2 : 360
> 		  La2 : 90
> 		  Lo1 :  0
> 		  La1 : -90
> 		
> 		Variable: AODSum
> 		Type: float
> 		Total Size: 8 bytes
> 		            2 values
> 		Number of Dimensions: 1
> 		Dimensions and sizes:   [2]
> 		Coordinates:
> 		Number Of Attributes: 1
> 		  _FillValue :  9.96921e+36
> 		
> 		Variable: AODVIS
> 		Type: float
> 		Total Size: 1039680 bytes
> 		            259920 values
> 		Number of Dimensions: 2
> 		Dimensions and sizes:   [g0_lat_1 | 361] x [g0_lon_2 | 720]
> 		Coordinates:
> 		            g0_lat_1: [-90..90]
> 		            g0_lon_2: [ 0..360]
> 		Number Of Attributes: 15
> 		  LON : <ARRAY of 1440 elements>
> 		  LAT : <ARRAY of 722 elements>
> 		  sub_center :  Ocean Prediction Center
> 		  center :      U.S. Air Force - Global Weather Center
> 		  long_name :   Unknown Variable Name
> 		  units :       unknown
> 		  _FillValue :  1e+20
> 		  level_indicator :     200
> 		  gds_grid_type :       0
> 		  parameter_table_version :     129
> 		  parameter_number :    147
> 		  level :       0
> 		  forecast_time :       0
> 		  forecast_time_units : hours
> 		  initial_time :        07/20/2010 (00:00)
> 		
> 		Steven Bliujus, Contractor
> 		SEMS/16WS WXE
> 		557th Weather Wing
> 		DNS: 232-7151
> 		
>
> 		
> 		
> 		_______________________________________________
> 		ncl-talk mailing list
> 		ncl-talk at ucar.edu
> 		List instructions, subscriber options, unsubscribe:
> 		http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
> 	--
> 	Walter Kolczynski, Jr.
> 	Global Ensemble Team
> 	NOAA/NWS/NCEP/EMC (via I.M. Systems Group)
> 	(301) 683-3781
>
>

-- 
Walter Kolczynski, Jr.
Global Ensemble Team
NOAA/NWS/NCEP/EMC (via I.M. Systems Group)
(301) 683-3781

More information about the ncl-talk mailing list