All Forums Database
Hillol 2 posts Joined 01/14
17 Jul 2014
sel cast(10**9 as int) giving wrong result

Hi ,
sel cast(10**8 as int) is giving result as 99,999,999 .
while it should be 100,000,000.00. also it is well below the Int limit of 2,147,483,647.
for any cast(10**n as int) it is giving similar result,that is deducting 1 from the actual.
Could anyone explain why this is happening ,is there any implicit conversion going on here.
Thanks...
 

dnoeth 4628 posts Joined 11/04
17 Jul 2014

The exponentiation operator ** results in a FLOAT, which might be the reason.
In TD14 you could use POWER(10,8) instead which returns a NUMBER (and a correct result ofter the CAST).
Before you could CAST to a DECIMAL(38,0) instead.

Dieter

You must sign in to leave a comment.