|
1 .. _tut-fp-issues: |
|
2 |
|
3 ************************************************** |
|
4 Floating Point Arithmetic: Issues and Limitations |
|
5 ************************************************** |
|
6 |
|
7 .. sectionauthor:: Tim Peters <tim_one@users.sourceforge.net> |
|
8 |
|
9 |
|
10 Floating-point numbers are represented in computer hardware as base 2 (binary) |
|
11 fractions. For example, the decimal fraction :: |
|
12 |
|
13 0.125 |
|
14 |
|
15 has value 1/10 + 2/100 + 5/1000, and in the same way the binary fraction :: |
|
16 |
|
17 0.001 |
|
18 |
|
19 has value 0/2 + 0/4 + 1/8. These two fractions have identical values, the only |
|
20 real difference being that the first is written in base 10 fractional notation, |
|
21 and the second in base 2. |
|
22 |
|
23 Unfortunately, most decimal fractions cannot be represented exactly as binary |
|
24 fractions. A consequence is that, in general, the decimal floating-point |
|
25 numbers you enter are only approximated by the binary floating-point numbers |
|
26 actually stored in the machine. |
|
27 |
|
28 The problem is easier to understand at first in base 10. Consider the fraction |
|
29 1/3. You can approximate that as a base 10 fraction:: |
|
30 |
|
31 0.3 |
|
32 |
|
33 or, better, :: |
|
34 |
|
35 0.33 |
|
36 |
|
37 or, better, :: |
|
38 |
|
39 0.333 |
|
40 |
|
41 and so on. No matter how many digits you're willing to write down, the result |
|
42 will never be exactly 1/3, but will be an increasingly better approximation of |
|
43 1/3. |
|
44 |
|
45 In the same way, no matter how many base 2 digits you're willing to use, the |
|
46 decimal value 0.1 cannot be represented exactly as a base 2 fraction. In base |
|
47 2, 1/10 is the infinitely repeating fraction :: |
|
48 |
|
49 0.0001100110011001100110011001100110011001100110011... |
|
50 |
|
51 Stop at any finite number of bits, and you get an approximation. This is why |
|
52 you see things like:: |
|
53 |
|
54 >>> 0.1 |
|
55 0.10000000000000001 |
|
56 |
|
57 On most machines today, that is what you'll see if you enter 0.1 at a Python |
|
58 prompt. You may not, though, because the number of bits used by the hardware to |
|
59 store floating-point values can vary across machines, and Python only prints a |
|
60 decimal approximation to the true decimal value of the binary approximation |
|
61 stored by the machine. On most machines, if Python were to print the true |
|
62 decimal value of the binary approximation stored for 0.1, it would have to |
|
63 display :: |
|
64 |
|
65 >>> 0.1 |
|
66 0.1000000000000000055511151231257827021181583404541015625 |
|
67 |
|
68 instead! The Python prompt uses the builtin :func:`repr` function to obtain a |
|
69 string version of everything it displays. For floats, ``repr(float)`` rounds |
|
70 the true decimal value to 17 significant digits, giving :: |
|
71 |
|
72 0.10000000000000001 |
|
73 |
|
74 ``repr(float)`` produces 17 significant digits because it turns out that's |
|
75 enough (on most machines) so that ``eval(repr(x)) == x`` exactly for all finite |
|
76 floats *x*, but rounding to 16 digits is not enough to make that true. |
|
77 |
|
78 Note that this is in the very nature of binary floating-point: this is not a bug |
|
79 in Python, and it is not a bug in your code either. You'll see the same kind of |
|
80 thing in all languages that support your hardware's floating-point arithmetic |
|
81 (although some languages may not *display* the difference by default, or in all |
|
82 output modes). |
|
83 |
|
84 Python's builtin :func:`str` function produces only 12 significant digits, and |
|
85 you may wish to use that instead. It's unusual for ``eval(str(x))`` to |
|
86 reproduce *x*, but the output may be more pleasant to look at:: |
|
87 |
|
88 >>> print str(0.1) |
|
89 0.1 |
|
90 |
|
91 It's important to realize that this is, in a real sense, an illusion: the value |
|
92 in the machine is not exactly 1/10, you're simply rounding the *display* of the |
|
93 true machine value. |
|
94 |
|
95 Other surprises follow from this one. For example, after seeing :: |
|
96 |
|
97 >>> 0.1 |
|
98 0.10000000000000001 |
|
99 |
|
100 you may be tempted to use the :func:`round` function to chop it back to the |
|
101 single digit you expect. But that makes no difference:: |
|
102 |
|
103 >>> round(0.1, 1) |
|
104 0.10000000000000001 |
|
105 |
|
106 The problem is that the binary floating-point value stored for "0.1" was already |
|
107 the best possible binary approximation to 1/10, so trying to round it again |
|
108 can't make it better: it was already as good as it gets. |
|
109 |
|
110 Another consequence is that since 0.1 is not exactly 1/10, summing ten values of |
|
111 0.1 may not yield exactly 1.0, either:: |
|
112 |
|
113 >>> sum = 0.0 |
|
114 >>> for i in range(10): |
|
115 ... sum += 0.1 |
|
116 ... |
|
117 >>> sum |
|
118 0.99999999999999989 |
|
119 |
|
120 Binary floating-point arithmetic holds many surprises like this. The problem |
|
121 with "0.1" is explained in precise detail below, in the "Representation Error" |
|
122 section. See `The Perils of Floating Point <http://www.lahey.com/float.htm>`_ |
|
123 for a more complete account of other common surprises. |
|
124 |
|
125 As that says near the end, "there are no easy answers." Still, don't be unduly |
|
126 wary of floating-point! The errors in Python float operations are inherited |
|
127 from the floating-point hardware, and on most machines are on the order of no |
|
128 more than 1 part in 2\*\*53 per operation. That's more than adequate for most |
|
129 tasks, but you do need to keep in mind that it's not decimal arithmetic, and |
|
130 that every float operation can suffer a new rounding error. |
|
131 |
|
132 While pathological cases do exist, for most casual use of floating-point |
|
133 arithmetic you'll see the result you expect in the end if you simply round the |
|
134 display of your final results to the number of decimal digits you expect. |
|
135 :func:`str` usually suffices, and for finer control see the :meth:`str.format` |
|
136 method's format specifiers in :ref:`formatstrings`. |
|
137 |
|
138 |
|
139 .. _tut-fp-error: |
|
140 |
|
141 Representation Error |
|
142 ==================== |
|
143 |
|
144 This section explains the "0.1" example in detail, and shows how you can perform |
|
145 an exact analysis of cases like this yourself. Basic familiarity with binary |
|
146 floating-point representation is assumed. |
|
147 |
|
148 :dfn:`Representation error` refers to the fact that some (most, actually) |
|
149 decimal fractions cannot be represented exactly as binary (base 2) fractions. |
|
150 This is the chief reason why Python (or Perl, C, C++, Java, Fortran, and many |
|
151 others) often won't display the exact decimal number you expect:: |
|
152 |
|
153 >>> 0.1 |
|
154 0.10000000000000001 |
|
155 |
|
156 Why is that? 1/10 is not exactly representable as a binary fraction. Almost all |
|
157 machines today (November 2000) use IEEE-754 floating point arithmetic, and |
|
158 almost all platforms map Python floats to IEEE-754 "double precision". 754 |
|
159 doubles contain 53 bits of precision, so on input the computer strives to |
|
160 convert 0.1 to the closest fraction it can of the form *J*/2\*\**N* where *J* is |
|
161 an integer containing exactly 53 bits. Rewriting :: |
|
162 |
|
163 1 / 10 ~= J / (2**N) |
|
164 |
|
165 as :: |
|
166 |
|
167 J ~= 2**N / 10 |
|
168 |
|
169 and recalling that *J* has exactly 53 bits (is ``>= 2**52`` but ``< 2**53``), |
|
170 the best value for *N* is 56:: |
|
171 |
|
172 >>> 2**52 |
|
173 4503599627370496L |
|
174 >>> 2**53 |
|
175 9007199254740992L |
|
176 >>> 2**56/10 |
|
177 7205759403792793L |
|
178 |
|
179 That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The |
|
180 best possible value for *J* is then that quotient rounded:: |
|
181 |
|
182 >>> q, r = divmod(2**56, 10) |
|
183 >>> r |
|
184 6L |
|
185 |
|
186 Since the remainder is more than half of 10, the best approximation is obtained |
|
187 by rounding up:: |
|
188 |
|
189 >>> q+1 |
|
190 7205759403792794L |
|
191 |
|
192 Therefore the best possible approximation to 1/10 in 754 double precision is |
|
193 that over 2\*\*56, or :: |
|
194 |
|
195 7205759403792794 / 72057594037927936 |
|
196 |
|
197 Note that since we rounded up, this is actually a little bit larger than 1/10; |
|
198 if we had not rounded up, the quotient would have been a little bit smaller than |
|
199 1/10. But in no case can it be *exactly* 1/10! |
|
200 |
|
201 So the computer never "sees" 1/10: what it sees is the exact fraction given |
|
202 above, the best 754 double approximation it can get:: |
|
203 |
|
204 >>> .1 * 2**56 |
|
205 7205759403792794.0 |
|
206 |
|
207 If we multiply that fraction by 10\*\*30, we can see the (truncated) value of |
|
208 its 30 most significant decimal digits:: |
|
209 |
|
210 >>> 7205759403792794 * 10**30 / 2**56 |
|
211 100000000000000005551115123125L |
|
212 |
|
213 meaning that the exact number stored in the computer is approximately equal to |
|
214 the decimal value 0.100000000000000005551115123125. Rounding that to 17 |
|
215 significant digits gives the 0.10000000000000001 that Python displays (well, |
|
216 will display on any 754-conforming platform that does best-possible input and |
|
217 output conversions in its C library --- yours may not!). |
|
218 |
|
219 |