[ncl-talk] Floating point issue

Wed Feb 6 12:14:20 MST 2019

Graziela,

This is a normal issue with floating point numbers.  Decimal fractions can
not be represented exactly in binary floating point, except for a small
subset that are exact sums of negative powers of 2.  For example, 42.75 can
be represented exactly, but 42.6 can not.  When reading decimal strings
from text formats like CSV, NCL and other languages will generally select
the nearest available floating point value.  Therefore, when working with
assumed decimal numbers, you get small roundoff errors that need special
treatment.  Please see this article:

https://en.wikipedia.org/wiki/Floating-point_arithmetic#Representable_numbers,_conversion_and_rounding

NCL's default print function does not show all significant floating point
digits.  To properly debug this, use sprintf to show all significant
digits.  Single precision float requires at least 9 significant decimal
digits to display all possible unique internal values.  Double precision
requires at least 17 decimal digits to show all possible values uniquely.

    print (sprintf ("%27.17g", (/ a, b, c /) ))

The normal way to handle inexact comparisons is to set a small threshold
based on the expected numeric range and precision, then use threshold
comparisons rather than equality.  Threshold comparisons may be absolute or
relative, and should be customized for each application.  Try this:

    delta = 0.001
    test1 = -50.2
    b = ind ( abs (xlon1(0:29) - test1) .lt. delta )

Finally, single precision float is okay for one or two fractional digits,
but is often not precise enough for geographic coordinates in many
applications.  I recommend you upgrade to type double for all processing of
lat/lon coordinates.  When reading CSV, use todouble rather than tofloat.
Also decrease the threshold value "delta"  in threshold comparisons.

--Dave

On Wed, Feb 6, 2019 at 6:31 AM Graziela Luzia <grazi.luzia at gmail.com> wrote:
>
> Hello ncl-users.
>
> I'm converting a csv file in netcdf and apparently my script, although
not very efficient (I'm a begginer) was working fine. But for some points
it dindn't write correctly. I've checked everything and I supose the
problem is converting string to float, but I can't figure out how can I
solve this problem.
>
> I don't understand why it shows me a value like 42.6 but it is like it
was 42.600002. It causes me trouble when I compare it with another variable
equal to 42.6, it returns a false result. Also, it only happen to some
values, like 42.6 (but not with 50.2) as you can see below.
>
> I've tried to use decimalPlaces, but it results the same.
> I'd apreciate any information. Thanks in advance.
>
> uname-a: Linux Tower2 4.15.0-45-generic #48~16.04.1-Ubuntu SMP Tue Jan 29
18:03:48 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
>
> Graziela.
>
> ---
> lines = asciiread("resultado.csv",-1,"string")
> data = lines(1:) ; exclude line 0 header
>
> ;xlon = decimalPlaces(tofloat(str_get_field(data,1,",")),1,True)
> xlon=tofloat(str_get_field(data,1,","))
>
> xlon1 = new(600,"float",-999)
>    do i=0,599
>       xlon1(i)=xlon(i*8)+0.1
>    end do
>
> print(xlon1(0:29))
>
> a=xlon1(23)-xlon1(24)
> print(a)
>
> b=ind(xlon1(0:29).eq.-50.2)
> c=ind(xlon1(0:29).eq.-42.6)
> print(b)
> print(c)
>
> ----
> labren at Tower2:~/Documentos/Produto3$ ncl teste.ncl
>  Copyright (C) 1995-2015 - All Rights Reserved
>  University Corporation for Atmospheric Research
>  NCAR Command Language Version 6.3.0
>  The use of this software is governed by a License Agreement.
>  See http://www.ncl.ucar.edu/ for more details.
>
> Variable: xlon1 (subsection)
> Type: float
> Total Size: 120 bytes
>             30 values
> Number of Dimensions: 1
> Dimensions and sizes: [30]
> Coordinates:
> Number Of Attributes: 1
>   _FillValue : -999
> (0) -50.2
> (1) -50.2
> (2) -50.2
> (3) -50.2
> (4) -50.2
> (5) -50.2
> (6) -50.2
> (7) -50.2
> (8) -50.2
> (9) -50.2
> (10) -50.2
> (11) -50.2
> (12) -50.2
> (13) -50.2
> (14) -50.2
> (15) -50.2
> (16) -50.2
> (17) -50.2
> (18) -50.2
> (19) -50.2
> (20) -50.2
> (21) -50.2
> (22) -50.2
> (23) -50.2
> (24) -42.6
> (25) -42.6
> (26) -42.6
> (27) -42.6
> (28) -42.6
> (29) -42.6
>
> Variable: a
> Type: float
> Total Size: 4 bytes
>             1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 1
>   _FillValue : -999
> (0) -7.599998
>
>
> Variable: b
> Type: integer
> Total Size: 96 bytes
>             24 values
> Number of Dimensions: 1
> Dimensions and sizes: [24]
> Coordinates:
> (0) 0
> (1) 1
> (2) 2
> (3) 3
> (4) 4
> (5) 5
> (6) 6
> (7) 7
> (8) 8
> (9) 9
> (10) 10
> (11) 11
> (12) 12
> (13) 13
> (14) 14
> (15) 15
> (16) 16
> (17) 17
> (18) 18
> (19) 19
> (20) 20
> (21) 21
> (22) 22
> (23) 23
>
> Variable: c
> Type: integer
> Total Size: 4 bytes
>             1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
> Number Of Attributes: 1
>   _FillValue : -2147483647
> (0) -2147483647
>
> My csv file contain values like this:
>
> xlon,ylat,ano,mes,direcao,v1,v2,v3,v4
> -50.3,-30.4,2016,1,45,0,0.0020206556,0.0017961383
> -50.3,-30.4,2016,1,90,0.0002245173,0.0004490346,0.0049393803,0.0246969017
> -50.3,-30.4,2016,1,135,0.0011225864,0.0029187247,0.0114503817,0.0177368657
> -50.3,-30.4,2016,1,180,0,0.0017961383,0.0038167939,0.0141445891 (...)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190206/8e2ba88b/attachment.html>