The big issue here is what you're going to use your numbers for. If you're going to do a lot of fast floating point operations for something like graphics or neural networks, these errors are fine. Speed is more important than exact accuracy.
If you're handling money, or numbers representing some other real, important concern where accuracy matters, most likely any number you intend to show to the user as a number, floats are not what you need.
Back when I started using Groovy, I was very pleased to discover that Groovy's default decimal number literal was translated to a BigDecimal rather than a float. For any sort of website, 9 times out of 10, that's what you need.
I'd really appreciate it if Javascript had a native decimal number type like that.
Decimal numbers are not conceptually any more or less exact than binary numbers. For example, you can't represent 1/3 exactly in decimal, just like you can't represent 1/5 exactly in binary.
When handling money, we care about faithfully reproducing the human-centric quirks of decimal numbers, not "being more accurate". There's no reason in principle to regard a system that can't represent 1/3 as being fundamentally more accurate because it happens to be able to represent 1/5.
Money are really best dealt with as integers, any time you'd use a non-integer number, use some fixed multiple that makes it an integer, then divide by the excess factor at the end of the calculation. For instance computing 2.15% yearly interest on a bank account might be done as follows:
DaysInYear = 366
InterestRate = 215
DayBalanceSum = 0
for each Day in Year
DayBalanceSum += Day.Balance
InterestRaw = DayBalanceSum * InterestRate
InterestRaw += DaysInYear * 5000
Interest = InterestRaw / (DaysInYear * 10000)
Balance += Interest
Balance should always be expressed in the smallest fraction of currency that we conventionally round to, like 1 yen or 1/100 dollar. Adding in half of the divisor before dividing effectively turns floor division into correctly rounded division.
The real lesson is, no matter what base (radix) you use, floating point math is inexact.
The value of floating point is that it can represent extremely huge or extremely infinitesimal values.
If you're working with currency / money, floating point is the wrong thing to use. For the entire history of human civilization, currency has always been an integer type, possibly with a fixed decimal point. Money has always been integers for as long as commerce has existed, and long before computers.
If you're building games, or AI, or navigating to Pluto, then floating point is the tool to use.
> Decimal numbers are not conceptually any more or less exact than binary numbers.
True but irrelevant. The problem isn't with the math fundamentals, it's the programmers.
The issue is if you get your integer handling wrong it usually stands out. Maybe that's because integers truncate rather than round, maybe it's because the program has to handle all those fractions of cents manually rather than letting the hardware do it so he has to think about it.
In any case integer code that works in unit tests usually continues to work, but floating point code passing all unit tests will be broken on some floating point implementations and not others. The reason is pretty obvious: floating point is inexact, but the implementations contain a ton of optimisations to hide that inexactness so it rarely raises it's ugly head.
When it does it's in the worst possible way. In a past day job I build cash registers and accounting systems. If you use floating point where exact results are required I can guarantee you your future self will be haunted by a never ending stream of phone calls from auditors telling you code that has worked solidly in thousands of installations over a decade can not add up. And god help you if you ever made the mistake of writing "if a == b" because you forgot a and b are floating point. Compiler writers should do us all a favour and not define == and != for floating point.
Back when I was doing this no complier implemented anything beyond 32 bit integer arithmetic, in fact there was no open source either. So you had to write a multi precision library and all expression evaluation had to be done using function calls. Despite floating point giving you hardware 56 bit arithmetic (which was enough), you were still better off using those clunky integers.
As others have said here: if you need exact results (and, yes currency is the most common use case), for the love of god do it using integers.
> If you're going to do a lot of fast floating point operations for something like graphics or neural networks, these errors are fine. Speed is more important than exact accuracy.
Um... that really depends. If you have an algorithm that is numerically unstable, these errors will quickly lead to a completely wrong result. Using a different type is not going to fix that, of course, and you need to fix the algorithm.
From your description, I fail to understand how does it depend. You're saying that the algorithm is wrong, and changing the type doesn't help. If the type is not the issue, what difference does it make?
In the world of money, it is rare to have to work past 3 decimal places. Bond traders operate on 32nds, so that might present some difficulties, but they really just want rounding at the hundreds.
Now, when you’re talking about central bank accruals (or similar sized deposits) that’s a bit different. In these cases, you have a very specific accrual multiple, multiplied by a balance in the multiple hundreds of billions or trillions. In these cases, precision with regards to the interest accrual calculation is quite significant, as rounding can short the payor/payee by several millions of dollars.
Hence the reason bond traders have historically traded in fractions of 32.
A sample bond trade:
‘Twenty sticks at a buck two and five eights bid’
‘Offer At 103 full’
‘Don’t break my balls with this, I got last round at delmonicos last night’
‘Offer 103 firm, what are we doing’
‘102-7 for 50 sticks’
‘Should have called me earlier and pulled the trigger, 50 sticks offer 103-2’
‘Fuck you, I’m your daughter’s godfather’
‘In that case, 40 sticks, 103-7 offer’
‘Fuck you, 10 sticks, 102-7, and you buy me a steak, and my daughter a new dress’
‘5 sticks at 104, 45 at 102-3 off tape, and you pick up bar tab and green fees’
‘Done’
‘You own it’
That’s kinda how bonds are traded.
Ref:
Stick: million
Bond pricing: dollar price + number divided by 32
Delmonicos: money bonfire with meals served
I'm curious about the "off tape" part. Presumably this means not on a ticker or not made public somehow - how are these transactions publicized and/or hidden?
Hear, hear! It would be great if javascript had any integral type that we could build decimals, rationals, arbitrarily-large integers and so on off. It’s technically doable with doubles if you really know what you’re doing, but it would be so much easier with an integral type.
How is a true integer easier than just pretending a double is an integer? In both cases, you have to be aware of the range of values they can hold to prevent overflow (integers) or rounding (doubles), and you have to be careful not to perform operations that aren't valid for integers to avoid truncation (integers) or non-zero decimal places (doubles).
'Decimal' is a red herring. The number base doesn't matter. (And what are you going to do when you need currency coversions, anyways?)
Floats are a digital approximation of real numbers, because computers were originally designed for solving math problems - trigonometry and calculus, that is.
For money you want rational numbers, not reals. Unfortunately, computers never got a native rational number type, so you'll have to roll your own.
Historically, it's correct-but-too-vague to say computers were for "solving math problems". Historic computer problems should be divided into two types: business problems and scientific/engineering problems. Business problems include things like tabulation and accounting. Programmable digital computers go back at least as far as UNIVAC I, in 1951 (using programmable digital computers for science doesn't go back THAT MUCH farther).
Prior to the IBM/360 (1964), mainframes sold for business purposes generally had no support for floating point arithmetic. They used fixed-point arithmetic. At the hardware level I think this is just integer math (I think?), but at a compiler level you can have different data types which are seen to be fractions with fixed accuracy. I believe I've read that COBOL had this feature since I-don't-know-how-far-back.
This sort of software fixed-point is still standard in SQL and many other places. Some languages, and many application-specific frameworks, have pre-existing fixed-point support. So it's also not accurate to say that you necessarily need to roll your own, though certainly in some contexts you'll need to.
And for money, you very much do not want arbitrary rational numbers. The important thing with money is that results are predictable and not fudgable. The problem with .1 + .2 != .3 is not that anyone cares about 4E-17 dollars, it's that they freak out when the math isn't predictable. Using rationals might be more predictable than using floats, but fixed-point is better still. And that's fixed-point base-10, because it's what your customers use when they check your work.
The type of any numeric literal is any type of the `Num` class. That means that they can be floating point, fractional, or integers "for free" depending on where you use them in your programs.
`0.75 + pi` is of type `Floating a => a`, but `0.75 + 1%4` is of type `Rational`.
Hm... what happens if you've got a neural network trained to make decisions in the financial domain?
Is there a way to exploit the difference between numeric precision underlying the neural network and the precision used to represent the financial transactions?
Neural networks are by their very nature a bit vague, random and unpredictable. Their output is not suitable as a direct, real monetary value you can rely on. At best, they predict trends, approximations or classifications.
I'd agree for saner defaults, especially in web development. I can understand that if you want to have strictly one number type it may make sense to opt for floating point to eke out the performance when you do need it, but I'd rather see high-precision as the default (as most expect that you'd be able to write an accurate calculator app in JavaScript without much work) and opt-in to the benefit of floating point operations.
I remember in college when we learned about this and I had the thought, "Why don't we just store the numerator and denominator?", and threw together a little C++ class complete with (then novel, to me) operator-overloads, which implemented the concept. I felt very proud of myself. Then years later I learned that it's a thing people actually use: https://en.wikipedia.org/wiki/Rational_data_type
An other compromise in to use fixed point which is effectively a rational with a fixed denominator. Extremely popular on machines which can handle integer arithmetics but not floating point (since you can trivially do fixed-point arithmetics using integer operations, you just need to be very careful when you handle overflows). If you look at the code of old school games (including classics like Doom if memory serves) the game engine used fixed-point to work on commodity hardware without FPU.
There's also BCD (binary coded decimal) that can solve some problems by avoiding the decimal-to-binary conversions if you're mainly dealing with decimal values. For instance 0.2 can't usually be represented in binary but of course it poses no problem in BCD.
Beware that BCD, and decimal in general, accumulates roundoff error at a much higher rate than binary, if you do any inexact operations.
It is more common these days to use base-1000, instead, when you need exact decimal representations. You can fit three base-1000 "digits" in a 32-bit word, with two bits left over for sign plus any other flag you find useful. (One such use could be to make a zero in the second place indicate that the rest of the word is actually binary; then regular arithmetic works on such words.) Calculations in base-1000 are quite a lot faster than BCD.
Almost always when people think they need decimal, binary -- even binary floating-point, if the numbers are small enough -- is much, much better. Just be sure to represent everything as an integer number of the smallest unit, say pennies; and scale (*100, /100) on I/O.
Fixed/floating is an interesting tradeoff for many real-time strategy games too where changes in game state are a synchronized simulation. Fixed point math in software can give more reliable and cross-platform math operations, but with a performance cost (eg: Homeworld: Deserts of Kharak). Using the CPU's floating-point hardware is faster, but you often have to ensure the correct CPU registers are set before doing calculations and those registers can be changed by other software such as a DirectX driver or the operating system (eg: Age of Empires II, Rise of Nations. etc).
> There's also BCD (binary coded decimal) that can solve some problems by avoiding the decimal-to-binary conversions if you're mainly dealing with decimal values. For instance 0.2 can't usually be represented in binary but of course it poses no problem in BCD.
BCD is/was super common in measurement equipment for internal calculations for this reason, and also because it is trivial to format for display (LED/LCD/VFDs) or text output (bus system, printer/plotter).
It's actually in use in many places, for things like handling currency and money, and for when you get funny corner cases involving rounding such numbers and pooling the change.
Whenever I see someone handling currency in floats, something inside me wither and die a small death.
> Whenever I see someone handling currency in floats, something inside me wither and die a small death.
Meh. When used correctly in the right circumstances it is acceptable to use floats.
Here's an example. Suppose you are pricing bonds, annuities, or derivatives. All the intermediate calculations make essential use of floating point operation. The Black–Scholes model for example requires the logarithm, the exponential, the square root, and the CDF of the normal distribution. None of that is doable without floats.
Even for simpler examples it is sometimes okay to use floats. If you only ever need to store an exact number of cents, you can totally store the number of cents in a double. Integer operations are exact using IEEE-754 double operations when they are smaller than 2^53-1 or so. There's usually no benefit of doing so, but hey it's possible.
Currency handling is almost never done with rationals (numerator and denominator) and is frequently (and correctly so!) done with fixed or floating point decimal types.
It's not always terrible. I've seen doubles appropriately used in cases where performance was paramount, and floating point error was either not relevant or less important.
That said, yeah, when working with money in situations where money matters, some sort of decimal or rational datatype should be the rule, not the exception.
Well, this is one of those things where context matters.
In trading, it's super common to use floating point arithmetic for decision logic since it's very fast and straightforward to write. The actual trade execution, however, almost always relies on integer arithmetic because then money is actually being used (and hence must be tracked properly).
It's not therefore inherently incorrect to do currency conversions with floats in some situations provided that the actual transaction execution relies on fixed precision or decimal arithmetic.
When I was in college the professor of my software engineering class explicitly warned us to never use floating point numbers for money. He went on at length of the dangers of floating points for dealing with money and warned us that people can get really upset if they feel like they've been screwed out of money.
He had decades of experience in the software development industry and I got the feeling that he'd seen the effect of this issue personally.
I haven't worked in fintech but I've read that money is often represented (at least in storage) as plain integers, since for example US currency only ever goes to two decimal places. But I guess once you start operating on it you run into potential truncation unless you use rationals.
> It's actually in use in many places, for things like handling currency and money
Hm, are you sure? I don't believe "rational" types which encode numbers as a numerator and denominator are typically used for currency/money.
If they were, would the denominator always be 100 or 1000? I guess you could use a rational type that way, although it'd be a small subset of what rational data types are intended for. But I guess it'd be "safe"? Not totally sure actually, one question would be if rounding works how you want when you do things like apply an interest percentage to a monetary amount. (I am not very familiar with rational data types, and am not sure how rounding works -- or even if they do rounding at all, or just keep increasing the magnitude of the denominator for exact precision, which is probably _not_ what you'd want with currency, for reasons not only of performance but of desired semantics).
You are correct an IEEE-754 floating point type is inappropriate for currency. I believe for currency you would generally use a fixed-point type (rather than floating point type), or non-IEEE "arbitrary precision floating point" type like ruby's BigDecimal (ruby also offers a Rational type. https://ruby-doc.org/core-2.5.0/Rational.html . This is a different thing than the arbitrary-precision BigDecimal. I have never used Rational or seen it used. It is not generally used for money/currency.) Or maybe even a binary-coded decimal value? (Not sure if that's the same thing as "arbitrary-precision floating point" of ruby's BigDecimal or not).
There are several possible correct and appropriate data encodings/types for currency, that will have the desired precision and calculation semantics... I am not sure if rational data type is one of them, and I don't believe it is common (and it would probably be much less performant than the options taht are common). Postgres, for instance, does not have a "rational" type built in, although there appears to be a third-party extension for it. Yet postgres is obviously frequently used to store currency values! I believe many other popular rdbms have no rational data type support at all.
I'm not actually sure what domains rational data types are good for. Probably not really anything scientific measurement based either (the IEEE-754 floating point types ARE usually good for that, that is their use case!) The wikipedia page sort of hand-wavily says "algebraic computation", which I'm not enough about math to really know what that means. I have never myself used rational data types, I don't think! Although I was aware of them; they are neat.
Currency in banking is handled with bigints. Not rationals, just bigints of the smallest unit (i.e., 1 cent). This forces you to order operations so that divisions are done last or not at all.
I actually ran into a bug recently while implementing my first raytracer, where the point calculated from the sphere-intersect test would just occasionally end up inside the sphere due to floating point imprecision, so the diffuse sample rays would have their origins completely in the dark, leading to randomly black pixels. Solved it by bumping every intersection out by 0.01 in the direction of its normal.
And then of course there have been several other "x.abs() < 0.01" cases for various purposes. So I could definitely see that being an interesting experiment.
But rationals are more expensive to compute with (compared to floating-point; this is another example of the trade-off between performance and accuracy.)
it's also a range-storage trade-off. if you use two fixed width integers to represent a rational, the minimum and maximum values are the same as that of the integer type. floating point gives a far wider range for the same number of bits.
IEEE floats are pretty complicated, but today’s CPUs have dedicated support for those and not for rationals, so we use them where we probably shouldn’t.
Yeah, I assumed as much. I wasn't really thinking about that at the time, but knowing now that it exists in the wild, the only conceivable reason for it not to be used everywhere would be some kind of performance penalty.
When I was reading about this I thought why don't the print functions just by default round to the nearest 10 decimal places or similar so 0.30000000000000004 prints as 0.3 unless you specify you don't want that. And I wrote a function in javascript to round like that though it was surprisingly tricky and messy to do so.
While it's true that floating point has its limitations, this stuff about not using it for money seems overblown to me. I've worked in finance for many years, and it really doesn't matter that much. There are de minimis clauses in contracts that basically say "forget about the fractions of a cent". Of course it might still trip up your position checking code, but that's easily fixed with a tiny tolerance.
It’s not painless. E.g. dividing $100.00 by 12 month in integer cents requires 11 $8.33 and one $8.37 (or better 4x(2x8.33+8.34), depending on definition of ‘better’). You can forget this $0.04, but it will jump around in reports until you get rid of it – it requires someone’s attention anyway, no matter how small it is. Otoh, in unrounded floating point that will lead to a mismatch between (integer) payments and calculations. In rounded fp it’s the same problem, except when you’re trying very hard for error bits to accumulate (like cross-multiplying dataset sums with no intermediate rounding, which is nonsense in financial calc and where regular fixpoint integers will overflow anyway).
What I’m trying to show here is that both integers and floating point are not suitable for doing ‘simple’ financial math. But we get used to this Bresenhamming in integers and do not perceive it as solving an error correction problem.
That's one of the worst domain name ever. When the topic comes along, I always remember about "that single-serving website with a domain name that looks like a number" and then take a surprisingly long time searching for it.
I have written a test framework and I am quite familiar with these problems, and comparing floating point numbers is a PITA. I had users complaining that 0.3 is not 0.3.
The code managing these comparisons turned out to be more complex than expected. The idea is that values are represented as ranges, so, for example, the IEEE-754 "0.3" is represented as ]0.299~, 0.300~[ which makes it equal to a true 0.3, because 0.3 is within that range.
It may be the worst domain name ever, but the site only exists because I thought that using "0" as a subdomain was a neat trick, and worked back from there to figure out what to do with it.
FWIW - the only way I can ever find my own website is by searching for it in my github repositories. So I definitely agree, it's not a terribly memorable domain.
It's the first result for "floating point site" on Google. Sure the domain itself is impossible to remember, but you don't have to remember the actual number, just what it stands for.
Remember filter bubble. My first result is not your first result. (although in this case it happens to be, but we both probably search a lot on programming)
> That's one of the worst domain name ever. When the topic comes along, I always remember about "that single-serving website with a domain name that looks like a number" and then take a surprisingly long time searching for it.
That's why we need regular expressions support in every search box, browser history, bookmarks and Google included.
(Well, it's a big document. I searched for the string "addition", which occurs just 41 times.)
I failed, but I believe I can show that the standard requires addition to be commutative in all cases:
1. "Clause 5 of this standard specifies the result of a single arithmetic operation." (§10.1)
2. "All conforming implementations of this standard shall provide the operations listed in this clause for all
supported arithmetic formats, except as stated below. Unless otherwise specified, each of the computational
operations specified by this standard that returns a numeric result shall be performed as if it first produced
an intermediate result correct to infinite precision and with unbounded range, and then rounded that
intermediate result, if necessary, to fit in the destination’s format" (§5.1)
Obviously, addition of real numbers is commutative, so the intermediate result produced for addition(a,b) must be equal to that produced for addition(b,a). I hope, but cannot guarantee, that the rounding applied to that intermediate result would not then depend on the order of operands provided to the addition operator.
3. "The operation addition(x, y) computes x+y. The preferred exponent is min(Q(x), Q(y))." (§5.4.1). This is the entire definition of addition, as far as I could find. (It's also defined, just above this statement, as being a general-computational operation. According to §5.1, a general-computational operation is one which produces floating-point or integer results, rounds all results according to §4, and might signal floating-point exceptions according to §7.)
4. The standard encourages programming language implementations to treat IEEE 754 addition as commutative (§10.4):
> A language implementation preserves the literal meaning of the source code by, for example:
> - Applying the properties of real numbers to floating-point expressions only when they preserve numerical results and flags raised:
> -- Applying the commutative law only to operations, such as addition and multiplication, for which neither the numerical values of the results, nor the representations of the results, depend on the order of the operands.
> -- Applying the associative or distributive laws only when they preserve numerical results and flags raised.
> -- Applying the identity laws (0 + x and 1 × x) only when they preserve numerical results and flags raised.
This looks like a guarantee that, in IEEE 754 addition, "the representation of the result" (i.e. the sign/exponent/significand triple, or a special infinite or NaN value - §3.2) does not "depend on the order of the operands". §3.2 specifically allows an implementation to map multiple bitstrings ("encodings") to a single "representation", so it's possible that the bit pattern of the result of an addition may differ depending on the order of the addends.
5. "Except for the quantize operation, the value of a floating-point result (and hence its cohort) is determined
by the operation and the operands’ values; it is never dependent on the representation or encoding of an
operand."
"The selection of a particular representation for a floating-point result is dependent on the operands’
representations, as described below, but is not affected by their encoding." (both from §5.2)
HOWEVER...
6. §6, dealing with infinite and NaN values, implicitly contemplates that there might be a distinction between addition(a,b) and addition(b,a):
> Operations on infinite operands are usually exact and therefore signal no exceptions, including, among
others,
> - addition(∞, x), addition(x, ∞), subtraction(∞, x), or subtraction(x, ∞), for finite x (§6.1)
Yep. The TL;DR of a numerical analysis class I took is that if you're going to sum a list of floats, sort it by increasing numeric value first so that the tiny values aren't rounded to zero every time.
Readit News
If you're handling money, or numbers representing some other real, important concern where accuracy matters, most likely any number you intend to show to the user as a number, floats are not what you need.
Back when I started using Groovy, I was very pleased to discover that Groovy's default decimal number literal was translated to a BigDecimal rather than a float. For any sort of website, 9 times out of 10, that's what you need.
I'd really appreciate it if Javascript had a native decimal number type like that.
When handling money, we care about faithfully reproducing the human-centric quirks of decimal numbers, not "being more accurate". There's no reason in principle to regard a system that can't represent 1/3 as being fundamentally more accurate because it happens to be able to represent 1/5.
The value of floating point is that it can represent extremely huge or extremely infinitesimal values.
If you're working with currency / money, floating point is the wrong thing to use. For the entire history of human civilization, currency has always been an integer type, possibly with a fixed decimal point. Money has always been integers for as long as commerce has existed, and long before computers.
If you're building games, or AI, or navigating to Pluto, then floating point is the tool to use.
True but irrelevant. The problem isn't with the math fundamentals, it's the programmers.
The issue is if you get your integer handling wrong it usually stands out. Maybe that's because integers truncate rather than round, maybe it's because the program has to handle all those fractions of cents manually rather than letting the hardware do it so he has to think about it.
In any case integer code that works in unit tests usually continues to work, but floating point code passing all unit tests will be broken on some floating point implementations and not others. The reason is pretty obvious: floating point is inexact, but the implementations contain a ton of optimisations to hide that inexactness so it rarely raises it's ugly head.
When it does it's in the worst possible way. In a past day job I build cash registers and accounting systems. If you use floating point where exact results are required I can guarantee you your future self will be haunted by a never ending stream of phone calls from auditors telling you code that has worked solidly in thousands of installations over a decade can not add up. And god help you if you ever made the mistake of writing "if a == b" because you forgot a and b are floating point. Compiler writers should do us all a favour and not define == and != for floating point.
Back when I was doing this no complier implemented anything beyond 32 bit integer arithmetic, in fact there was no open source either. So you had to write a multi precision library and all expression evaluation had to be done using function calls. Despite floating point giving you hardware 56 bit arithmetic (which was enough), you were still better off using those clunky integers.
As others have said here: if you need exact results (and, yes currency is the most common use case), for the love of god do it using integers.
Um... that really depends. If you have an algorithm that is numerically unstable, these errors will quickly lead to a completely wrong result. Using a different type is not going to fix that, of course, and you need to fix the algorithm.
Now, when you’re talking about central bank accruals (or similar sized deposits) that’s a bit different. In these cases, you have a very specific accrual multiple, multiplied by a balance in the multiple hundreds of billions or trillions. In these cases, precision with regards to the interest accrual calculation is quite significant, as rounding can short the payor/payee by several millions of dollars.
Hence the reason bond traders have historically traded in fractions of 32.
A sample bond trade:
‘Twenty sticks at a buck two and five eights bid’ ‘Offer At 103 full’ ‘Don’t break my balls with this, I got last round at delmonicos last night’ ‘Offer 103 firm, what are we doing’ ‘102-7 for 50 sticks’ ‘Should have called me earlier and pulled the trigger, 50 sticks offer 103-2’ ‘Fuck you, I’m your daughter’s godfather’ ‘In that case, 40 sticks, 103-7 offer’ ‘Fuck you, 10 sticks, 102-7, and you buy me a steak, and my daughter a new dress’ ‘5 sticks at 104, 45 at 102-3 off tape, and you pick up bar tab and green fees’ ‘Done’ ‘You own it’
That’s kinda how bonds are traded.
Ref: Stick: million Bond pricing: dollar price + number divided by 32 Delmonicos: money bonfire with meals served
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
Floats are a digital approximation of real numbers, because computers were originally designed for solving math problems - trigonometry and calculus, that is.
For money you want rational numbers, not reals. Unfortunately, computers never got a native rational number type, so you'll have to roll your own.
Prior to the IBM/360 (1964), mainframes sold for business purposes generally had no support for floating point arithmetic. They used fixed-point arithmetic. At the hardware level I think this is just integer math (I think?), but at a compiler level you can have different data types which are seen to be fractions with fixed accuracy. I believe I've read that COBOL had this feature since I-don't-know-how-far-back.
This sort of software fixed-point is still standard in SQL and many other places. Some languages, and many application-specific frameworks, have pre-existing fixed-point support. So it's also not accurate to say that you necessarily need to roll your own, though certainly in some contexts you'll need to.
And for money, you very much do not want arbitrary rational numbers. The important thing with money is that results are predictable and not fudgable. The problem with .1 + .2 != .3 is not that anyone cares about 4E-17 dollars, it's that they freak out when the math isn't predictable. Using rationals might be more predictable than using floats, but fixed-point is better still. And that's fixed-point base-10, because it's what your customers use when they check your work.
The type of any numeric literal is any type of the `Num` class. That means that they can be floating point, fractional, or integers "for free" depending on where you use them in your programs.
`0.75 + pi` is of type `Floating a => a`, but `0.75 + 1%4` is of type `Rational`.
Is there a way to exploit the difference between numeric precision underlying the neural network and the precision used to represent the financial transactions?
Was proposed in the late 90's Mike Cowlishaw but the rest of the standards committee would have none of it.
Proposal: https://github.com/littledan/proposal-bigdecimal
Slides: https://docs.google.com/presentation/d/1qceGOynkiypIgvv0Ju8u...
Give it =0.1+0.2-0.3 and it will see what you are trying to do and return 0.
Give it anything slightly more complicated such as =(0.1+0.2-0.3) and this won't trip, in this example displaying 5.55112E-17 or similar.
Deleted Comment
https://people.eecs.berkeley.edu/~wkahan/Mind1ess.pdf
(and plenty of other rants...:
https://people.eecs.berkeley.edu/~wkahan/ )
There's also BCD (binary coded decimal) that can solve some problems by avoiding the decimal-to-binary conversions if you're mainly dealing with decimal values. For instance 0.2 can't usually be represented in binary but of course it poses no problem in BCD.
It is more common these days to use base-1000, instead, when you need exact decimal representations. You can fit three base-1000 "digits" in a 32-bit word, with two bits left over for sign plus any other flag you find useful. (One such use could be to make a zero in the second place indicate that the rest of the word is actually binary; then regular arithmetic works on such words.) Calculations in base-1000 are quite a lot faster than BCD.
Almost always when people think they need decimal, binary -- even binary floating-point, if the numbers are small enough -- is much, much better. Just be sure to represent everything as an integer number of the smallest unit, say pennies; and scale (*100, /100) on I/O.
BCD is/was super common in measurement equipment for internal calculations for this reason, and also because it is trivial to format for display (LED/LCD/VFDs) or text output (bus system, printer/plotter).
Whenever I see someone handling currency in floats, something inside me wither and die a small death.
Meh. When used correctly in the right circumstances it is acceptable to use floats.
Here's an example. Suppose you are pricing bonds, annuities, or derivatives. All the intermediate calculations make essential use of floating point operation. The Black–Scholes model for example requires the logarithm, the exponential, the square root, and the CDF of the normal distribution. None of that is doable without floats.
Even for simpler examples it is sometimes okay to use floats. If you only ever need to store an exact number of cents, you can totally store the number of cents in a double. Integer operations are exact using IEEE-754 double operations when they are smaller than 2^53-1 or so. There's usually no benefit of doing so, but hey it's possible.
That said, yeah, when working with money in situations where money matters, some sort of decimal or rational datatype should be the rule, not the exception.
In trading, it's super common to use floating point arithmetic for decision logic since it's very fast and straightforward to write. The actual trade execution, however, almost always relies on integer arithmetic because then money is actually being used (and hence must be tracked properly).
It's not therefore inherently incorrect to do currency conversions with floats in some situations provided that the actual transaction execution relies on fixed precision or decimal arithmetic.
He had decades of experience in the software development industry and I got the feeling that he'd seen the effect of this issue personally.
I still remember that warning well.
Hm, are you sure? I don't believe "rational" types which encode numbers as a numerator and denominator are typically used for currency/money.
If they were, would the denominator always be 100 or 1000? I guess you could use a rational type that way, although it'd be a small subset of what rational data types are intended for. But I guess it'd be "safe"? Not totally sure actually, one question would be if rounding works how you want when you do things like apply an interest percentage to a monetary amount. (I am not very familiar with rational data types, and am not sure how rounding works -- or even if they do rounding at all, or just keep increasing the magnitude of the denominator for exact precision, which is probably _not_ what you'd want with currency, for reasons not only of performance but of desired semantics).
You are correct an IEEE-754 floating point type is inappropriate for currency. I believe for currency you would generally use a fixed-point type (rather than floating point type), or non-IEEE "arbitrary precision floating point" type like ruby's BigDecimal (ruby also offers a Rational type. https://ruby-doc.org/core-2.5.0/Rational.html . This is a different thing than the arbitrary-precision BigDecimal. I have never used Rational or seen it used. It is not generally used for money/currency.) Or maybe even a binary-coded decimal value? (Not sure if that's the same thing as "arbitrary-precision floating point" of ruby's BigDecimal or not).
There are several possible correct and appropriate data encodings/types for currency, that will have the desired precision and calculation semantics... I am not sure if rational data type is one of them, and I don't believe it is common (and it would probably be much less performant than the options taht are common). Postgres, for instance, does not have a "rational" type built in, although there appears to be a third-party extension for it. Yet postgres is obviously frequently used to store currency values! I believe many other popular rdbms have no rational data type support at all.
I'm not actually sure what domains rational data types are good for. Probably not really anything scientific measurement based either (the IEEE-754 floating point types ARE usually good for that, that is their use case!) The wikipedia page sort of hand-wavily says "algebraic computation", which I'm not enough about math to really know what that means. I have never myself used rational data types, I don't think! Although I was aware of them; they are neat.
Which specific places have you seen it used in?
And then of course there have been several other "x.abs() < 0.01" cases for various purposes. So I could definitely see that being an interesting experiment.
https://en.m.wikipedia.org/wiki/Arbitrary-precision_arithmet...
and gets harder when you want exact irrationals too https://www.google.com/search?q=exact+real+arithmetic
But maybe that just results in all the floating point weirdness again, just not for small rationals.
Let's say we need to do a comparison. Set
and Which is greater?Or even more feindish, Set
and which is greater?By what algorithm would you do the computation, and could you guarantee me the same compute time as comparing 2/3 and 4/5?
Deleted Comment
(Shameless Common Lisp plug: http://clhs.lisp.se/Body/t_ration.htm)
Dead Comment
2015.000000000000: https://news.ycombinator.com/item?id=10558871
I assume that's the reason they made the mantissa linear, even though having the whole thing logarithmic makes more sense.
What I’m trying to show here is that both integers and floating point are not suitable for doing ‘simple’ financial math. But we get used to this Bresenhamming in integers and do not perceive it as solving an error correction problem.
I have written a test framework and I am quite familiar with these problems, and comparing floating point numbers is a PITA. I had users complaining that 0.3 is not 0.3.
The code managing these comparisons turned out to be more complex than expected. The idea is that values are represented as ranges, so, for example, the IEEE-754 "0.3" is represented as ]0.299~, 0.300~[ which makes it equal to a true 0.3, because 0.3 is within that range.
Maybe the creator's theory is that people will search for 0.30000000000000004 when they run into it after running their code.
FWIW - the only way I can ever find my own website is by searching for it in my github repositories. So I definitely agree, it's not a terribly memorable domain.
That's why we need regular expressions support in every search box, browser history, bookmarks and Google included.
"simple" discussion: https://floating-point-gui.de/errors/comparison/
more advanced: https://randomascii.wordpress.com/2012/02/25/comparing-float...
Deleted Comment
Also the "field" of floating point numbers is not commutative†, (can run on JS console:)
x=0;for (let i=0; i<10000; i++) { x+=0.0000000000000000001; }; x+=1
--> 1.000000000000001
x=1;for (let i=0; i<10000; i++) { x+=0.0000000000000000001; };
--> 1
Although most of the time a+b===b+a can be relied on. And for most of the stuff we do on the web it's fine!††
† edit: Please s/commutative/associative/, thanks for the comments below.
†† edit: that's wrong! Replace with (a+b)+c === a+(b+c)
What is failing is associativity, i.e. (a+b)+c==a+(b+c)
For example
(.0000000000000001 + .0000000000000001 ) + 1.0
--> 1.0000000000000002
.0000000000000001 + (.0000000000000001 + 1.0)
--> 1.0
In your example, you are mixing both properties,
(.0000000000000001 + .0000000000000001) + 1.0
--> 1.0000000000000002
(1.0 + .0000000000000001) + .0000000000000001
--> 1.0
but the difference is caused by the lack of associativity, not by the lack of commutativity.
[1] Perhaps you must exclude -0.0. I think it is commutative even with -0.0, but I'm never 100% sure.
(Well, it's a big document. I searched for the string "addition", which occurs just 41 times.)
I failed, but I believe I can show that the standard requires addition to be commutative in all cases:
1. "Clause 5 of this standard specifies the result of a single arithmetic operation." (§10.1)
2. "All conforming implementations of this standard shall provide the operations listed in this clause for all supported arithmetic formats, except as stated below. Unless otherwise specified, each of the computational operations specified by this standard that returns a numeric result shall be performed as if it first produced an intermediate result correct to infinite precision and with unbounded range, and then rounded that intermediate result, if necessary, to fit in the destination’s format" (§5.1)
Obviously, addition of real numbers is commutative, so the intermediate result produced for addition(a,b) must be equal to that produced for addition(b,a). I hope, but cannot guarantee, that the rounding applied to that intermediate result would not then depend on the order of operands provided to the addition operator.
3. "The operation addition(x, y) computes x+y. The preferred exponent is min(Q(x), Q(y))." (§5.4.1). This is the entire definition of addition, as far as I could find. (It's also defined, just above this statement, as being a general-computational operation. According to §5.1, a general-computational operation is one which produces floating-point or integer results, rounds all results according to §4, and might signal floating-point exceptions according to §7.)
4. The standard encourages programming language implementations to treat IEEE 754 addition as commutative (§10.4):
> A language implementation preserves the literal meaning of the source code by, for example:
> - Applying the properties of real numbers to floating-point expressions only when they preserve numerical results and flags raised:
> -- Applying the commutative law only to operations, such as addition and multiplication, for which neither the numerical values of the results, nor the representations of the results, depend on the order of the operands.
> -- Applying the associative or distributive laws only when they preserve numerical results and flags raised.
> -- Applying the identity laws (0 + x and 1 × x) only when they preserve numerical results and flags raised.
This looks like a guarantee that, in IEEE 754 addition, "the representation of the result" (i.e. the sign/exponent/significand triple, or a special infinite or NaN value - §3.2) does not "depend on the order of the operands". §3.2 specifically allows an implementation to map multiple bitstrings ("encodings") to a single "representation", so it's possible that the bit pattern of the result of an addition may differ depending on the order of the addends.
5. "Except for the quantize operation, the value of a floating-point result (and hence its cohort) is determined by the operation and the operands’ values; it is never dependent on the representation or encoding of an operand."
"The selection of a particular representation for a floating-point result is dependent on the operands’ representations, as described below, but is not affected by their encoding." (both from §5.2)
HOWEVER...
6. §6, dealing with infinite and NaN values, implicitly contemplates that there might be a distinction between addition(a,b) and addition(b,a):
> Operations on infinite operands are usually exact and therefore signal no exceptions, including, among others,
> - addition(∞, x), addition(x, ∞), subtraction(∞, x), or subtraction(x, ∞), for finite x (§6.1)
OK.
You've identified a problem, but it isn't that addition is noncommutative.1.0 + 1e-16 == 1e-16 + 1.0 == 1.0 as well as 1.0 + 1e-15 == 1e-15 + 1.0 == 1.000000000000001
however (1.0 + (1e-16 + 1e-16)) == 1.0 + 2e-16 == 1.0000000000000002, whereas ((1.0 + 1e-16) + 1e-16) == 1.0 + 1e-16 == 1.0
https://en.wikipedia.org/wiki/Kahan_summation_algorithm