[ncl-talk] unexpected results from very simple code

[danghy at gmail.com]

Thanks, Rick!  I'll read the document.



danghy at gmail.com
 
From: Rick Brownrigg
Date: 2015-07-21 11:19
To: danghy at gmail.com
CC: Arne Melsom; ncl-talk
Subject: Re: [ncl-talk] unexpected results from very simple code
Hi Hongyan,

As Arne indicated, with 32-bit floating point, you can't expect more than 7 significant digits (14 for double), and in your case, you are asking for precision 2 orders of magnitude beyond that. 

For the truly curious, all the gory reality about computing and precision/accuracy is detailed in this doc:

http://www.cse.msu.edu/~cse320/Documents/FloatingPoint.pdf

HTH...
Rick

On Mon, Jul 20, 2015 at 7:26 PM, danghy at gmail.com <danghy at gmail.com> wrote:
Hi Arne,

Thanks a lot for the information!  I will probably use "integer" to solve the problem.  But I was really curious why the floating point number returns "210200048", not "210200049"... the perfect case would be that it returns "210200051".

Thanks again.

Hongyan


danghy at gmail.com
 
From: Arne Melsom
Date: 2015-07-20 17:27
To: danghy
CC: ncl-talk
Subject: Re: [ncl-talk] unexpected results from very simple code
Hi Danghy,
 
I think that you can't expect (4 byte) floating point numbers to deliver the accuracy (no. digits) that you request. In your case, I suggest using double precision or integer representation. Here's a part of your example using the original representation and the two alternatives:
 
begin
 
a=210200051.
print("output / a :")
print(tostring(a))
print(sprintf("%9.0f",a))
 
ad=210200051.d
print("output / ad:")
print(tostring(ad))
print(sprintf("%9.0f",ad))
 
ai=210200051
print("output / ai:")
print(tostring(ai))
print(sprinti("%9i",ai))
 
end
 
 
...which gives the following output:
 
(0) output / a :
(0) 210200048.000000
(0) 210200048
(0) output / ad:
(0) 210200051.000000
(0) 210200051
(0) output / ai:
(0) 210200051
(0) 210200051
 
Hope this helps!
Arne M.
 
 
 
----- Original Message -----
> Dear all,
> 
> I met a problem when reading "station ID" - I've simplified the code as
> follows:
> 
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
> begin
> 
> degrees = (/210200051, 121, 36, 43,38,58,26/)
> 
> print(sprintf("%9.0f",degrees(0)))
> print(tostring(degrees(0)))
> 
> a=210200051.
> print(tostring(a))
> print(sprintf("%9.0f",a))
> 
> end
> 
> The NCL output on screen is:
> 
> ncl test.ncl
> Copyright (C) 1995-2015 - All Rights Reserved
> University Corporation for Atmospheric Research
> NCAR Command Language Version 6.3.0
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
> (0) 210200048
> (0) 210200051
> (0) 210200048.000000
> (0) 210200048
> 
> Only the second method "tostring" gives the correct station ID. Could you
> please give me a clue why is this happening?
> 
> Appreciate!
> 
> 
> danghy at gmail.com
> 
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> 

_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150721/f9414416/attachment.html