[ncl-talk] Convert unpacked netCDF file to packed one

[Setareh Rahimi]

Dear Dennis,
Many thanks for your help. Once I follow the
http://www.ncl.ucar.edu/Applications/method_1.shtml method to save the new
nc file, I face some errors, in fact I am too new in writhing nc files.
That method just give me :

netcdf new-hgt.2019 {

dimensions:

time = UNLIMITED ; // (0 currently)


// global attributes:

:creation_date = "Tue Jan 12 16:33:58 +0330 2021" ;

:Conventions = "CF-1.0 " ;

:source_file = "NCEP/DOE AMIP-II Reanalysis (Reanalysis-2) Model" ;

:title = "4x daily NCEP/DOE Reanalysis 2" ;


How can I have the correct nc file?


Best wishes

On Mon, Jan 11, 2021 at 11:35 PM Dennis Shea <shea at ucar.edu> wrote:

> I was just typing a response that contained the same information as Dave's.
>
> Likely, you have no control "the model I am running only accept packed nc
> file with scale factor."
> As noted by DaveA, the file you are using is netCDF-4
> [ Perhaps too much info but netCDF-4's compression is described *here ]*
> <https://www.unidata.ucar.edu/blogs/developer/entry/netcdf_compression>
> ===============
>
> %> *ncdump -k* air.2019.nc
> netCDF-4 classic model
>
> In fact, I believe *all* the netCDF files created by "NCEP-DOE AMIP-II
> Reanalysis" (and others) will be netCDF-4.
>
> Maybe you could have the model developers alter the code.
> Really, they should have checked if the 'scale_factor' and/or 'add_offset'
> attributes are present.
> If not, use the variable directly.
> ====================
>
>             f = addfile("air.2019.nc","r")
>             x = f->air
>             printVarSummary(x)                 ; full uncompressed values
>                                                              ; *** LOOK AT
> THE SIZE****
>                                                              ; It is MUCH
> LARGER than the ENTIRE netCDF file
>
>       xShort   = *pack_values*(xFloat, "short", False)
>       *printVarSummary* <http://www.ncl.ucar.edu/Document/Functions/Built-in/printVarSummary.shtml>(xShort)
>
>       xShort   = pack_values(x, "short", False)
>       printVarSummary(xShort)  ; *** LOOK AT THE SIZE****
>                                ; HALF the size of 'x'
>                                ; but much larger than the compressed value!
>
>  The use (say) the following to write 'xShort' a new (much larger) file that the original
>
> *http://www.ncl.ucar.edu/Applications/method_1.shtml* <http://www.ncl.ucar.edu/Applications/method_1.shtml>
>
> Then use that new file for the model.
>
> Likely,* EVERY *variable will have to be done like this for the "model you are using
>
>
> *===*
>
>
>
> *It is much easier for the model code to be altered.*
>
>
> *Good Luck*
>
>
> On Mon, Jan 11, 2021 at 12:46 PM Dave Allured - NOAA Affiliate via
> ncl-talk <ncl-talk at mailman.ucar.edu> wrote:
>
>> I see.  I suggest the best solution would be a simple upgrade to the
>> model code, so that it could read both packed and unpacked air temp files.
>> This will enable the model to use future data updates without extra
>> processing.
>>
>> In fortran, this would be an if statement to check whether there is a
>> scale_factor attribute.  If not, then skip reading scale_factor and
>> add_offset, also skip the unpacking statement or loop that converts packed
>> data values to unpacked.  Just read in the unpacked values directly.  You
>> would need two read statements, the original one for the packed data type,
>> and a new one for the unpacked data type (fortran real).
>>
>> If something prevents you from changing the model code, then just convert
>> the air temp file to packed using the NCL *pack_values* function.  See
>> the general documentation for how to read and write a netcdf file.
>>
>>
>> On Mon, Jan 11, 2021 at 12:00 PM Setareh Rahimi <setareh.rahimi at gmail.com>
>> wrote:
>>
>>> Dear Dave,
>>> Thank you for your reply. Yes I am sure, because the model I am running
>>> only accept packed nc file with scale factor.
>>> Please have a look at attached file showing the error I face while
>>> running the model with mentioned nc file.
>>> Best regard
>>>
>>> On Mon, Jan 11, 2021 at 22:20 Dave Allured - NOAA Affiliate <
>>> dave.allured at noaa.gov> wrote:
>>>
>>>> That file is already compressed to 31% of its natural array size, using
>>>> netcdf-4 internal compression.  Are you sure you want to do that?  If yes,
>>>> then use the NCL function *pack_values*.
>>>>
>>>>
>>>> On Mon, Jan 11, 2021 at 11:07 AM Setareh Rahimi via ncl-talk <
>>>> ncl-talk at mailman.ucar.edu> wrote:
>>>>
>>>>> Dear all,
>>>>> I need to convert an unpacked netCDF file to a packed one (
>>>>> ftp://ftp2.psl.noaa.gov/Datasets/ncep.reanalysis2/pressure/air.2019.nc),
>>>>> I wonder if this possible using NCL?
>>>>> Many thanks in advance,
>>>>> Best wishes,
>>>>> --
>>>>> S.Rahimi
>>>>>
>>>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at mailman.ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>

-- 
S.Rahimi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20210112/87a28ada/attachment.html>