The Net Age

Thursday, November 11, 2004

MS Calculator Error

The problem was found by $0cr4t3$:

MS Calculator has a problem in calculating negative numbers.
When in Scientific mode and trying the transformation of "-1" from Decimal to HEX calc.exe produces an unwanted result FFFFFFFFFFFFFFFF

What is this hex subtraction:
0xA3 - 0xF5 =
e.g.
http://www.google.com/search?hl=en&lr=&q=0xA3-0xF5&btnG=Search

Answer: 0xA3 - 0xF5 = -0x52
The answer has been given by Google.com

More on Google.com calculating abilities:

http://www.googleguide.com/calculator.html
http://www.google.com/help/features.html#calculator

___________________________

Another problem found by man0war:

Try the following sequence of events while in HEX mode
A(HEX) -> +/- -> +/- -> = -> +/-

That means push A then +/- then +/- again then
= and then +/- again and you have FFFFFFFFFFFFFFF6

___________________________

After searching("calculator hex") at support.microsoft.com only the following link that was of relevance was found.

Solution to an old problem, still very effective:
"Microsofts solution to another problem when calculating large numbers:
To prevent this problem from occurring, do not use Calculator to convert large decimal numbers to hexadecimal. "

http://support.microsoft.com/default.aspx?scid=kb;en-us;187514
___________________________

0 Comments:

Post a Comment

<< Home