# MaxCoderz

for your 1 bit pleasure!

 All times are UTC

 Page 1 of 1 [ 3 posts ]
 Print view Previous topic | Next topic
Author Message
 Post subject: nice bug (fPart(64) > 0)Posted: Sun 20 Jan, 2008 10:03 am
 Calc King

Joined: Sat 05 Aug, 2006 7:22 am
Posts: 1513
fPart(log(2^64)/log(2))
I would think that is 0 (zero) because log(2^64)/log(2) = 64
but no, apparently it's a different kind of 64, which looks exactly like 64 on the screen but is actually a bit bigger.

the result is 1E-12
this does not happen with fPart(64

not really 'breaking news' I know, but this bug caught me when I wanted to know what numbers n and m (from N) for which n^m (where the ^ means power, not xor) comes closest to 2^64-1 for n != 2^x.
I'm not exactly sure why I wanted to know this, but I was doing some stuff with the Diffie-Hellman exchange so it might have something to do with that...

Top

 Post subject: Posted: Sun 20 Jan, 2008 10:10 am
 Maxcoderz Staff

Joined: Wed 15 Dec, 2004 6:06 am
Posts: 579
Location: Chicago!
Yay, rounding errors...

_________________
You know your hexadecimal output routine is broken when it displays the character 'G'.

Top

 Post subject: Re: nice bug (fPart(64) > 0)Posted: Thu 29 Apr, 2010 2:37 pm
 New Member

Joined: Wed 28 Apr, 2010 2:34 pm
Posts: 1
Also on the 83 and 84 (all models that I've encountered thus far) i^x for any x>=7 yields a wrong answer.
and On the Nspire all logs that should resolve to an integer greater then 2 is are not actually the integer its slightly off so like log(8,2)-3=-1E-13 that became a real problem when I didn't know this and tried to index an array on log(expr(something in the form of a*x^n)/a,2) it took me a week to figure out that i had to add round the result to 0 decimal places to make it work.

Top

 Display posts from previous: All posts1 day7 days2 weeks1 month3 months6 months1 year Sort by AuthorPost timeSubject AscendingDescending
 Page 1 of 1 [ 3 posts ]

 All times are UTC

#### Who is online

Users browsing this forum: No registered users and 9 guests

 You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum

Search for:
 Jump to:  Select a forum ------------------ General    News and Information    General TI Discussion    Off Topic    Announce Your Projects    Pixel Art Projects    Staff Side Projects & Featured Projects    Aether 3D    BBC BASIC    Latenite, Brass and EarlyMorning    Metroid    Nostromo    Smash Bros    Super Mario - and the Elemental Crystal    Tankies    xLIB    solidFRAME Products    Staff Side Products & Featured Products    Desolate    Grayscale Dev Kit    The Verdante Forest    Trapped    Discontinued Projects Programming    Programming Help    Program Ideas    Programming Competition
Powered by phpBB ® Forum Software © phpBB Group | DVGFX2 by: Matt