[ncl-talk] GOES16 netCDF files

Rick Brownrigg brownrig at ucar.edu
Fri Jan 19 15:52:33 MST 2018 

Hi Tim,

Sorry to bug you, I've had a heck of time finding datasets for this
satellite (presumably because its still in calibration/testing mode) -- I
can find all manner of information and preprocessed imagery, but actual
netcdf files -- very frustrating. Can I ask where you are getting your data
from?

Rick

On Fri, Jan 19, 2018 at 3:16 PM, Tim Risner <trisner at awis.com> wrote:

> I’m attempting to use the same code that is previously mentioned to map
> regions that are outside the CONUS, which requires the “Full Disk” set of
> files, versus the “CONUS” files that are working.
> Variables and files are the same, of course the area is bigger, therefore
> the variables are much larger.
>
> I deleted all variables in the code once they were no longer needed and
> have been mulling the code over for a few days now, but can’t seem to find
> a solution to clean the memory sufficiently.
>
> One solution would be to only use the portion of the file I need (i.e.
> only the western Atlantic ocean), but I can’t seem to only strip that
> section of the file/variable.
>
>
>
> On Jan 19, 2018, at 16:04, Rick Brownrigg <brownrig at ucar.edu> wrote:
>
> Hi Tim,
>
> Those are big variables, but not that big.  Are you trying to read in
> large numbers of them?
>
> Rick
>
>
> On Fri, Jan 19, 2018 at 1:54 PM, Tim Risner <trisner at awis.com> wrote:
>
>> Works great Mary, and you can easily pull the sat wavelength you need
>> from it.
>> One issue I did run into was if you use the “Full Disk” files rather then
>> the “CONUS” files used in the example, you get
>> what I expect to be a memory issue?
>>
>> Error to screen
>> *******************
>> “fatal: NclMalloc Failed:[errno=12] Segmentation fault” error.
>>
>>
>> info
>> ******
>> Variable: phi
>> Type: double
>> Total Size: 235358208 bytes
>>             29419776 values
>> Number of Dimensions: 2
>> Dimensions and sizes: [5424] x [5424]
>> Coordinates:
>> Number Of Attributes: 1
>>   _FillValue : -999
>>
>> Variable: lambda
>> Type: double
>> Total Size: 235358208 bytes
>>             29419776 values
>> Number of Dimensions: 2
>> Dimensions and sizes: [5424] x [5424]
>> Coordinates:
>> Number Of Attributes: 1
>>   _FillValue : -999
>>
>> Variable: data
>> Type: float
>> Total Size: 117679104 bytes
>>             29419776 values
>> Number of Dimensions: 2
>> Dimensions and sizes: [y | 5424] x [x | 5424]
>> Coordinates:
>>             y: [0..5423]
>>             x: [0..5423]
>> Number Of Attributes: 16
>>   lon2d : <ARRAY of 29419776 elements>
>>   lat2d : <ARRAY of 29419776 elements>
>>   long_name : ABI Cloud and Moisture Imagery reflectance factor
>>   standard_name : toa_lambertian_equivalent_albe
>> do_multiplied_by_cosine_solar_zenith_angle
>>   _Unsigned : true
>>   sensor_band_bit_depth : 12
>>   downsampling_method : average
>>   units : 1
>>   resolution : y: 0.000056 rad x: 0.000056 rad
>>   coordinates : band_id_C02 band_wavelength_C02 t y x
>>   grid_mapping : goes_imager_projection
>>   cell_methods : t: point area: sum (interval: 0.000014 rad)
>>   ancillary_variables : DQF_C02
>>   _FillValue_original : -1
>>   _FillValue : -1
>>   valid_range : (  0, 1.299999 )
>>
>>
>>
>>
>>
>> On Jan 11, 2018, at 6:07 PM, Mary Haley <haley at ucar.edu> wrote:
>>
>> Hi Tim,
>>
>> My apologies for getting back to you so late. We got side-tracked with
>> another harder GOES16 issue (regridding a large GOES16 variable) and wanted
>> to finish that up to make sure we were doing things correctly.
>>
>> Rick and Bill (members of the NCL development team) spent some time
>> looking into documentation for GOES16 and figured out how to calculate the
>> lat / lon values, given the Y / X coordinates and map projection
>> information on the file. No sooner than this was done that the satellite
>> was repositioned, and we had to tweak the calculations a little.
>>
>> I've attached the NCL scripts that Rick created, and the plot it
>> generates. This script should work on the new GOES16 data as well.
>>
>> You need to run:
>>
>>  ncl plot_goes16.ncl
>>
>> The CMI_16 variable was arbitrarily chosen, but we did chose 16 over the
>> other numbers because it was more interesting.
>>
>> I understand that there are different channels, and a better way to plot
>> this data might be to overlay the different channels with different
>> opacities set.  I need to look into how another user did this, and I'll
>> post back here if I ever get around to it.
>>
>> --Mary
>>
>>
>>
>>
>> On Wed, Dec 13, 2017 at 1:02 PM, Tim Risner <trisner at awis.com> wrote:
>>
>>> thanks Rick, others.
>>> version is 6.4.0
>>> running on Linux Amazon EC2 instance. x86_64 x86_64 x86_64 GNU/Linux
>>> testing GOES16 file attached.
>>> I obtained such file from the Amazon AWS bucket system that is now
>>> publicly accessible by the gov't/Amazon.
>>> Information on file lists it as a GOES16 netcdf file.
>>>
>>> I assumed I could simply replace the file in the example for GOES 11
>>> example here: http://www.ncl.ucar.edu/Applications/Scripts/goes_1.ncl
>>> and all would work if the files and datasets were exchangeable.
>>> Assumptions are likely my problem.
>>>
>>> Error output:
>>>
>>> fatal:["Execute.c":132]:variable (data) is not in file
>>> (/var/NCL/INPUT/GOES16.nc)
>>> fatal:["Execute.c":6381]:variable (data) is not in file (f)
>>> fatal:["Execute.c":8640]:Execute: Error occurred at or near line 4 in
>>> file /var/NCL/CODE/testgoes.nc
>>>
>>>
>>>
>>>
>>> *Tim Risner*
>>> *Operational Meteorologist*
>>> *(888) 798-9955 ext 1006 <(888)%20798-9955>*
>>>
>>> <http://www.awis.com/>[image:
>>> https://www.facebook.com/AWISWeatherServices/]
>>> <https://www.facebook.com/AWISWeatherServices/>
>>> <https://twitter.com/awisweather>
>>>
>>> On Wed, Dec 13, 2017 at 9:02 AM, Rick Brownrigg <brownrig at ucar.edu>
>>> wrote:
>>>
>>>> Hi Tim,
>>>>
>>>> Do you have a dataset you could share?  I download some data from this
>>>> site:
>>>>
>>>> https://www.goes-r.gov
>>>>
>>>> but not knowing what I'm really looking for, I could not ascertain that
>>>> it was specifically GOES16. NCL was able to read it.
>>>>
>>>> I should also ask what version of NCL are you working with?
>>>>
>>>> If your dataset is too large to email, you can post it to our FTP site:
>>>>
>>>> http://ncl.ucar.edu/ftp_files.shtml
>>>>
>>>> Rick
>>>>
>>>> On Wed, Dec 13, 2017 at 7:35 AM, Tim Risner <trisner at awis.com> wrote:
>>>>
>>>>> Hello All,
>>>>> I am curious as to what the GOES16 netCDF plan is for NCL.
>>>>> I've attempted to ingest the new GOES16 netCDF into generic GOES code
>>>>> found on the examples pages, but am getting a variables not found error
>>>>> when attempting to produce an image. Variable not found is "data". I assume
>>>>> this is due to the differences between the GOES16 netCDF files and the
>>>>> older GOES products? or is it just operator error and GOES16 netcdf files
>>>>> should work?
>>>>>
>>>>> Thanks!!!
>>>>>
>>>>>
>>>>> *Tim Risner*
>>>>> *Operational Meteorologist*
>>>>> *(888) 798-9955 ext 1006 <(888)%20798-9955>*
>>>>>
>>>>> <http://www.awis.com/>[image:
>>>>> https://www.facebook.com/AWISWeatherServices/]
>>>>> <https://www.facebook.com/AWISWeatherServices/>
>>>>> <https://twitter.com/awisweather>
>>>>>
>>>>> _______________________________________________
>>>>> ncl-talk mailing list
>>>>> ncl-talk at ucar.edu
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>> <goes16.png><plot_goes16.ncl><goesYXtoLatLon.ncl>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180119/3cd20168/attachment.html>

More information about the ncl-talk mailing list