All Forums Database
rob_watson 3 posts Joined 04/16
08 Sep 2016
TO_NUMBER returns wrong value

Hi,
I'm trying to convert a series of hex values (stored as strings) to numbers ( BIGINT) using the TO_NUMBER function . A number of the results giivng the incorrect number
SELECT TO_NUMBER( '03ea02653f6938ba','XXXXXXXXXXXXXXXX' ) returns 282040560542890190
According to my scientific calculator the value is 282040560542890170
SELECT TO_NUMBER( '03ea02653f693e63', 'XXXXXXXXXXXXXXXX' ) returns 282040560542891600. Yet the value appears to be 282040560542891619
We're running Teradata Version 14.0.7.15.
Teradata Studio V15.10.11
Any ideas how to get the correct value ?
Thanks in advance
Rob

Tags:
You must sign in to leave a comment.