[ncl-talk] new variables from rows

Dennis Shea shea at ucar.edu
Mon Jul 24 08:44:18 MDT 2017


Ascii/text data sets can be a bit of a nuisance to handle.  Good
bookkeeping is important. For example, what index should be used to start
or end a calculation.  Sometimes, as in the file you attached, there are
characters that you may want to eliminate. In your file, it is the double
quote [ " ].

I've attached a script that, hopefully, addresses a comment by Damjan
concerning his original script: "it is not elegant." Obviously, you have to
know something about NCL's function libraries. As we know, one 'good' thing
about NCL is that it has a lot of functions;  the 'bad' thing is that NCL
has lots of functions. :-)  or :-(

Here are some for handling text/ascii.

See:
   http://www.ncl.ucar.edu/Document/Functions/string.shtml

http://www.ncl.ucar.edu/Document/Functions/Contributed/readAsciiTable.shtml
   http://www.ncl.ucar.edu/Document/Functions/Built-in/asciiread.shtml
   http://www.ncl.ucar.edu/Applications/list_io.shtml
        + Read ascii; Read CSV
        + Output netCDF

---
Some good programming practices

  [0] Write clear code.
       Nothing fancy/slick that obscures what you are doing. Optimize only
when necessary.

  [1] Code development
       [a]  Use 'print' / 'printVarSummary' / 'printMinMax' to look at
variables
       [b]  Test code segments.

  [2] Document code

  [3] Indent code blocks for readability

  [4] Isolate specific and/or repetitive  tasks in functions or procedures

       [a] functions when 'something' is to be returned and, subsequently,
used.
            See 'create_time' in attached script
       [b] procedures when performing specific tasks and nothing is to be
returned. For example:
            [i]  Generating a plot. In particular, one with many resource
settings.
            [ii] Creating a file or writing a variable to an existing file
(See 'create_var_nc)

  [5] Suggestion: create your own library/libraries to avoid rewriting code.
       Further, these could serve as the basis for newer
functions/procedures.

---

One nuisance in Damjan's data file is the double quotes [ " ]. As used by
both Rashad's script and the attached script, the following NCL function
can be used:
   http://www.ncl.ucar.edu/Document/Functions/Built-in/str_get_dq.shtml

FYI: there is a separate function for single quote [ ' ]
   http://www.ncl.ucar.edu/Document/Functions/Built-in/str_get_sq.shtml

===
The attached script has numerous 'print' statements to track what is
happening. *Please look at the code and the corresponding output.*

%> ncl TOA5.DamjanJelic.ncl

%> ncdump -h TOA5_STUP_10s_5.nc >! ncdump.TOA5

Good Luck


On Mon, Jul 24, 2017 at 5:31 AM, Rashed Mahmood <rashidcomsis at gmail.com>
wrote:

> Hi Damjan,
> Sure you can contact me directly to discuss similar problems... However,
> to increase your chances of a best possible answer it is always good to
> discuss questions on NCL forum.
>
> Cheers,
> Rashed
>
> On Mon, Jul 24, 2017 at 2:00 PM, Damjan Jelic <dnjelic at gmail.com> wrote:
>
>> Dear Rashed,
>>
>> thank you very much for the script! it solves my problem for sure and i
>> learn from it very much since it shows several function didnt know exist.
>>
>> i will feel free to contact you directly for similar problems if thats ok
>> with you.
>>
>> Sincerely!
>> from Croatia,
>> Damjan Jelic
>>
>> On Sun, Jul 23, 2017 at 12:51 PM, Rashed Mahmood <rashidcomsis at gmail.com>
>> wrote:
>>
>>> Hi Damjan,
>>> The attached script would read variable names, units and data from the
>>> file, and write in a netcdf file. It still needs to specify number of
>>> header lines, and the line numbers for variable names and units.
>>>
>>> Since the variable and unit names in the text files has double quotes, I
>>> had to add a "tric" using two do loops to get rid of double quotes from
>>> individual strings (lines:22-36, Perhaps there is some proper way for it
>>> which I am not aware of).
>>>
>>> The script seems to work fine, I think!. Attached are input,output and
>>> the script files.
>>>
>>> hope that helps,
>>> Rashed
>>>
>>> Date: Fri, 21 Jul 2017 16:50:07 +0200
>>> From: Damjan Jeli? <damjan.jelic at fmf.uni-lj.si>
>>> Subject: [ncl-talk] new variables from rows
>>> To: ncl-talk at ucar.edu
>>> Message-ID: <81b23a5fb192458c543a235e55871440 at fmf.uni-lj.si>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Dear NCL support,
>>>
>>> i have txt files (attached) containing 35 different variables which im
>>> trying to save in netcdf format. current problem is that i need to
>>> specify every variable on its own and it is not elegant (attached) and
>>> later on it will be problem since columns will be rearranged in random
>>> order from time to time.
>>> Can it be done somehow like this:
>>> .data=asciiread(...
>>> ....
>>> variable_5 = (str_get_field(data(1::),5,","))
>>> print(variable_5(0)); which is variable name of column 5
>>> print(variable_5(0))=variable_5(3:end)
>>>
>>> or in written form:
>>> i load data, ignore first row,
>>> read data by columns
>>> use first row as new variable name
>>> use second row tu set units
>>> skip 3. row and use all the rest as data.
>>>
>>> Kind regards,
>>> Damjan
>>>
>>>
>>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170724/057ad605/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TOA5.DamjanJelic.ncl
Type: application/octet-stream
Size: 7411 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170724/057ad605/attachment.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TOA5_STUP_10s_5.dat
Type: application/octet-stream
Size: 9515 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170724/057ad605/attachment-0001.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ncdump.TOA5
Type: application/octet-stream
Size: 2767 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170724/057ad605/attachment-0002.obj 


More information about the ncl-talk mailing list