Mike Orth – Booya!

The “Number” data type in Flex is used for floating-point numbers (i.e. decimals like 42.11). It is usually the default type when using numerical values, even though Flash Player handles int and uint more effeciently. The Number class uses the IEEE 754 standard which gives you values up to 53 bits (highest of which is approx. 1.79e+308).

When doing calculations involving Numbers, calculations are done using binary addition. This can really throw you off course in a few rare instances if you’re not careful! Take the following example:

b = .3, right? Right.. only, not. The value comes out to 0.30000000000000004 instead. Ok, not too far off, but imagine making several calculations on the result and you can start to sway off course. The following example is a watered-down version of what I encountered lately. I knew about this type of discrepancy, but it took me a couple of days to figure out where things were going wrong because I didn’t assume that something was failing at such a base level; you typically expect things like subtraction to work correctly.

Whoa. Things start getting funky (btw, in case you’re not familiar with the notation, that second to last one is 5 with 17 decimal places, “.00000000000000005″). Why the discrepancy? Again, it has to do with converting the numbers to binary and evalutating them. You can read more on the IEEE 754 standard here. This is typically not something you will have to deal with, but when you do encounter it, you can spin your wheels for a while.

Try it for yourself here.
Demo application that adds/subtracts and includes some default cases.