[ncl-talk] GOES16 netCDF files
Tim Risner
trisner at awis.com
Fri Jan 19 15:16:16 MST 2018
I’m attempting to use the same code that is previously mentioned to map regions that are outside the CONUS, which requires the “Full Disk” set of files, versus the “CONUS” files that are working.
Variables and files are the same, of course the area is bigger, therefore the variables are much larger.
I deleted all variables in the code once they were no longer needed and have been mulling the code over for a few days now, but can’t seem to find a solution to clean the memory sufficiently.
One solution would be to only use the portion of the file I need (i.e. only the western Atlantic ocean), but I can’t seem to only strip that section of the file/variable.
> On Jan 19, 2018, at 16:04, Rick Brownrigg <brownrig at ucar.edu> wrote:
>
> Hi Tim,
>
> Those are big variables, but not that big. Are you trying to read in large numbers of them?
>
> Rick
>
>
>> On Fri, Jan 19, 2018 at 1:54 PM, Tim Risner <trisner at awis.com> wrote:
>> Works great Mary, and you can easily pull the sat wavelength you need from it.
>> One issue I did run into was if you use the “Full Disk” files rather then the “CONUS” files used in the example, you get
>> what I expect to be a memory issue?
>>
>> Error to screen
>> *******************
>> “fatal: NclMalloc Failed:[errno=12] Segmentation fault” error.
>>
>>
>> info
>> ******
>> Variable: phi
>> Type: double
>> Total Size: 235358208 bytes
>> 29419776 values
>> Number of Dimensions: 2
>> Dimensions and sizes: [5424] x [5424]
>> Coordinates:
>> Number Of Attributes: 1
>> _FillValue : -999
>>
>> Variable: lambda
>> Type: double
>> Total Size: 235358208 bytes
>> 29419776 values
>> Number of Dimensions: 2
>> Dimensions and sizes: [5424] x [5424]
>> Coordinates:
>> Number Of Attributes: 1
>> _FillValue : -999
>>
>> Variable: data
>> Type: float
>> Total Size: 117679104 bytes
>> 29419776 values
>> Number of Dimensions: 2
>> Dimensions and sizes: [y | 5424] x [x | 5424]
>> Coordinates:
>> y: [0..5423]
>> x: [0..5423]
>> Number Of Attributes: 16
>> lon2d : <ARRAY of 29419776 elements>
>> lat2d : <ARRAY of 29419776 elements>
>> long_name : ABI Cloud and Moisture Imagery reflectance factor
>> standard_name : toa_lambertian_equivalent_albedo_multiplied_by_cosine_solar_zenith_angle
>> _Unsigned : true
>> sensor_band_bit_depth : 12
>> downsampling_method : average
>> units : 1
>> resolution : y: 0.000056 rad x: 0.000056 rad
>> coordinates : band_id_C02 band_wavelength_C02 t y x
>> grid_mapping : goes_imager_projection
>> cell_methods : t: point area: sum (interval: 0.000014 rad)
>> ancillary_variables : DQF_C02
>> _FillValue_original : -1
>> _FillValue : -1
>> valid_range : ( 0, 1.299999 )
>>
>>
>>
>>
>>
>>> On Jan 11, 2018, at 6:07 PM, Mary Haley <haley at ucar.edu> wrote:
>>>
>>> Hi Tim,
>>>
>>> My apologies for getting back to you so late. We got side-tracked with another harder GOES16 issue (regridding a large GOES16 variable) and wanted to finish that up to make sure we were doing things correctly.
>>>
>>> Rick and Bill (members of the NCL development team) spent some time looking into documentation for GOES16 and figured out how to calculate the lat / lon values, given the Y / X coordinates and map projection information on the file. No sooner than this was done that the satellite was repositioned, and we had to tweak the calculations a little.
>>>
>>> I've attached the NCL scripts that Rick created, and the plot it generates. This script should work on the new GOES16 data as well.
>>>
>>> You need to run:
>>>
>>> ncl plot_goes16.ncl
>>>
>>> The CMI_16 variable was arbitrarily chosen, but we did chose 16 over the other numbers because it was more interesting.
>>>
>>> I understand that there are different channels, and a better way to plot this data might be to overlay the different channels with different opacities set. I need to look into how another user did this, and I'll post back here if I ever get around to it.
>>>
>>> --Mary
>>>
>>>
>>>
>>>
>>>> On Wed, Dec 13, 2017 at 1:02 PM, Tim Risner <trisner at awis.com> wrote:
>>>> thanks Rick, others.
>>>> version is 6.4.0
>>>> running on Linux Amazon EC2 instance. x86_64 x86_64 x86_64 GNU/Linux
>>>> testing GOES16 file attached.
>>>> I obtained such file from the Amazon AWS bucket system that is now publicly accessible by the gov't/Amazon.
>>>> Information on file lists it as a GOES16 netcdf file.
>>>>
>>>> I assumed I could simply replace the file in the example for GOES 11 example here: http://www.ncl.ucar.edu/Applications/Scripts/goes_1.ncl
>>>> and all would work if the files and datasets were exchangeable.
>>>> Assumptions are likely my problem.
>>>>
>>>> Error output:
>>>>
>>>> fatal:["Execute.c":132]:variable (data) is not in file (/var/NCL/INPUT/GOES16.nc)
>>>> fatal:["Execute.c":6381]:variable (data) is not in file (f)
>>>> fatal:["Execute.c":8640]:Execute: Error occurred at or near line 4 in file /var/NCL/CODE/testgoes.nc
>>>>
>>>>
>>>>
>>>>
>>>> Tim Risner
>>>> Operational Meteorologist
>>>> (888) 798-9955 ext 1006
>>>>
>>>>
>>>>
>>>>> On Wed, Dec 13, 2017 at 9:02 AM, Rick Brownrigg <brownrig at ucar.edu> wrote:
>>>>> Hi Tim,
>>>>>
>>>>> Do you have a dataset you could share? I download some data from this site:
>>>>>
>>>>> https://www.goes-r.gov
>>>>>
>>>>> but not knowing what I'm really looking for, I could not ascertain that it was specifically GOES16. NCL was able to read it.
>>>>>
>>>>> I should also ask what version of NCL are you working with?
>>>>>
>>>>> If your dataset is too large to email, you can post it to our FTP site:
>>>>>
>>>>> http://ncl.ucar.edu/ftp_files.shtml
>>>>>
>>>>> Rick
>>>>>
>>>>>> On Wed, Dec 13, 2017 at 7:35 AM, Tim Risner <trisner at awis.com> wrote:
>>>>>> Hello All,
>>>>>> I am curious as to what the GOES16 netCDF plan is for NCL.
>>>>>> I've attempted to ingest the new GOES16 netCDF into generic GOES code found on the examples pages, but am getting a variables not found error when attempting to produce an image. Variable not found is "data". I assume this is due to the differences between the GOES16 netCDF files and the older GOES products? or is it just operator error and GOES16 netcdf files should work?
>>>>>>
>>>>>> Thanks!!!
>>>>>>
>>>>>>
>>>>>> Tim Risner
>>>>>> Operational Meteorologist
>>>>>> (888) 798-9955 ext 1006
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> ncl-talk mailing list
>>>>>> ncl-talk at ucar.edu
>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> ncl-talk at ucar.edu
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>>
>>> <goes16.png><plot_goes16.ncl><goesYXtoLatLon.ncl>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180119/2c790818/attachment.html>
More information about the ncl-talk
mailing list