[ncl-talk] string to float conversion

[Rashed Mahmood]

Hi all,

I am wondering if there is an option to stop the function "tofloat" from
rounding. Here is a simple example that explains the problem:

abc  =
(/"130511001","130511002","130511003","130511004","130511005","130511006"/)
cde1  = sprintf("%10f",tofloat(abc))
cde2  = toint(cde1)
cde3  = toint(abc)

print(abc+" = "+cde1+" = "+cde2+" = "+cde3)

This results:

(0)     130511001 = 130511000.000000 = 130511000 = 130511001
(1)     130511002 = 130511000.000000 = 130511000 = 130511002
(2)     130511003 = 130511000.000000 = 130511000 = 130511003
(3)     130511004 = 130511008.000000 = 130511008 = 130511004
(4)     130511005 = 130511008.000000 = 130511008 = 130511005
(5)     130511006 = 130511008.000000 = 130511008 = 130511006


This loss of precision could be acceptable if the values were some data
numbers, however, if these are unique station IDs then this is not correct.

I know the solution is simply use "toint" or "todouble" to avoid this
situation. Just curious if a rounding option is possible in "tofloat".

Cheers,
Rashed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180427/68147d6a/attachment.html>