I am converting some Excel Calculations to Powershell:
=ROUNDDOWN((250000.00-0)*5.00%/12,6)
This gives a value of 1041.666666
I have converted it to Powershell:
[Math]::Round((250000.00 - 0) * (5.00/100) / 12,6)
But this gives a value of 1041.666667
The problem is that the last number matters and the fact that powershell doesnt round the 7 down to 6. This causes problems down the line as the final result for the Excel is 1461.59 but for powershell it is 1461.42
So, what's the best way to get a correct round-down to a certain number of decimal places in Powershell?
Math.Round rounds to the nearest number, not down. You'd have to roll your own version which uses Floor, e.g.:
function Get-RoundedDown($d, $digits) {
$scale = [Math]::Pow(10, $digits)
[Math]::Truncate($d * $scale) / $scale
}
Get-RoundedDown (250000 * .05 / 12) 6
Note, though, that in general there is no guarantee that the number you see as a result of that actually only has six digits after the decimal point since what we're doing here is not friendly towards binary floating-point numbers. So it may well be that you receive a number like 1441.666600000003 as a result and there's little you can do about that except switching to decimal.
Related
I have some raw data exported from my ERP. Most of the data has two decimals, but there are a few lines contain numbers with three or more decimal places... When summing up all data with powershell I have a 0,01€ discrepancy between my ERP PDF report and sum of exported data. I tracked the discrepancy to a single number - 2.205. Powershell rounds this number to 2.2 not 2.21 as expected. If I try any other number - 2.215, 2.225 they all get rounded to 2.22, 2.23..
Why is 2.205 not treated the same???
.NET's default mid-point rounding strategy is ToEven, meaning that .5 values are rounded to the closest even integer.
Important: The following example use [decimal] numbers - as implied by number-literal suffix d - not [double]s , so as to avoid problems that stem from [double] values, which internally use a binary representation that typically does not have an exact decimal equivalent, so that a number that may appear to have a 5 in the relevant decimal place may actually be just above or below that exact decimal value,[1] which means that mid-point rounding may not apply - see Rounding and precision.
In the case at hand, rounding -2.205 to 2 decimal places means that 0 is the closest even integer, and [Math]::Round(-2.205d, 2) therefore yields -2.20.[2]
It sounds like you want AwayFromZero as the mid-point rounding strategy, which rounds to the next higher integer in absolute terms:
PS> [Math]::Round(-2.205d, 2, 'AwayFromZero')
-2.21
[1] To see if a given fractional value has an exact decimal representation, you can use something like the following:
$num = [double] -2.205; '{0:F28}' -f $num
Unless the input number is echoed with trailing zeros only, the given value has no exact decimal representation; [double] value -2.205 does not, but 0.5 does, for instance.
[2] Note that [Math]::Round(2.225d, 2) therefore yields 2.22, not 2.23, as you state in your question.
If I try any other number - 2.215, 2.225 they all get rounded to 2.22, 2.23
Are you sure about that 2.23 result? Here's my reslts when I run it:
PS H:\> [math]::Round(2.205,2)
2.2
PS H:\> [math]::Round(2.215,2)
2.22
PS H:\> [math]::Round(2.225,2)
2.22
PS H:\> [math]::Round(2.235,2)
2.24
PS H:\> [math]::Round(2.245,2)
2.24
Taken from Scripting guy:
Notice that here the decimal, .25, rounded down to .2. This is because the number (2) is even. In the following example, the .35 rounds up because the number (3) is odd.
To fix it, define Midpoint Routing per Microsoft's documentation:
[math]::Round(2.205,2,1)
I am writing a PowerShell script that converts a number with a decimal point into an integer.
$val = 1024.24
How to convert this value to integer? (I want it to be 1024)
Use floor rounding, which rounds to the lower whole number
[Math]::Floor($val)
Edit: if just discarding decimal part is not what you are looking for you can use [Math]::Round($val) which will round the number like normal math rounding or you can use [Math]::Ceiling($val) which will round up (in your case it will round to 1025) and it is probably not what you need but it is good to know that you have these options as well.
I am working on a calculator in Swift, but I have a small problem:
when multiplying two numbers, I don't have the same results as a regular calculator.
For example:
In Swift :
0.333328247070312 * 16 = 5.33325195312499
In a regular calculator:
0.333328247070312 * 16 = 5.333251953
What should I do to get the same results as a regular calculator in Swift?
Your "regular calculator" seems wrong or weird in result printing, so, you should not rely on it. I've rechecked your calculation in Python3 which is known to calculate all in binary64 double and print the most exact decimal form:
>>> 0.333328247070312 * 16
5.333251953124992
it's even more detailed (by one digit) than Swift output. Your output also can't be verified as binary32 calculation, because the latter has ~7 correct decimal digits, and usually isn't printed with more digits. What this calculator is? I'd suppose some Pascal-based tool due to its custom 6-byte float.
Try to ask your calculator to print the most detailed form. If it fails, throw it away and use a most exact tool to verify, or, if your task is really to get the same result, figure out more details about its processing.
These two long numbers are the same except for the last digit.
test = [];
test(1) = 33777100285870080;
test(2) = 33777100285870082;
but the last digit is lost when the numbers are put in the array:
unique(test)
ans = 3.3777e+16
How can I prevent this? The numbers are ID codes and losing the last digit is screwing everything up.
Matlab uses 64-bit floating point representation by default for numbers. Those have a base-10 16-digit precision (more or less) and your numbers seem to exceed that.
Use something like uint64 to store your numbers:
> test = [uint64(33777100285870080); uint64(33777100285870082)];
> disp(test(1));
33777100285870080
> disp(test(2));
33777100285870082
This is really a rounding error, not a display error. To get the correct strings for output purposes, use int2str, because, again, num2str uses a 64-bit floating point representation, and that has rounding errors in this case.
To add more explanation to #rubenvb's solution, your values are greater than flintmax for IEEE 754 double precision floating-point, i.e, greater than 2^53. After this point not all integers can be exactly represented as doubles. See also this related question.
I use mod() to compare if a number's 0.01 digit is 2 or not.
if mod(5.02*100, 10) == 2
...
end
The result is mod(5.02*100, 10) = 2 returns 0;
However, if I use mod(1.02*100, 10) = 2 or mod(20.02*100, 10) = 2, it returns 1.
The result of mod(5.02*100, 10) - 2 is
ans =
-5.6843e-14
Could it be possible that this is a bug for matlab?
The version I used is R2013a. version 8.1.0
This is not a bug in MATLAB. It is a limitation of floating point arithmetic and conversion between binary and decimal numbers. Even a simple decimal number such as 0.1 has cannot be exactly represented as a binary floating point number with finite precision.
Computer floating point arithmetic is typically not exact. Although we are used to dealing with numbers in decimal format (base10), computers store and process numbers in binary format (base2). The IEEE standard for double precision floating point representation (see http://en.wikipedia.org/wiki/Double-precision_floating-point_format, what MATLAB uses) specifies the use of 64 bits to represent a binary number. 1 bit is used for the sign, 52 bits are used for the mantissa (the actual digits of the number), and 11 bits are used for the exponent and its sign (which specifies where the decimal place goes).
When you enter a number into MATLAB, it is immediately converted to binary representation for all manipulations and arithmetic and then converted back to decimal for display and output.
Here's what happens in your example:
Convert to binary (keeping only up to 52 digits):
5.02 => 1.01000001010001111010111000010100011110101110000101e2
100 => 1.1001e6
10 => 1.01e3
2 => 1.0e1
Perform multiplication:
1.01000001010001111010111000010100011110101110000101 e2
x 1.1001 e6
--------------------------------------------------------------
0.000101000001010001111010111000010100011110101110000101
0.101000001010001111010111000010100011110101110000101
+ 1.01000001010001111010111000010100011110101110000101
-------------------------------------------------------------
1.111101011111111111111111111111111111111111111111111101e8
Cutting off at 52 digits gives 1.111101011111111111111111111111111111111111111111111e8
Note that this is not the same as 1.11110110e8 which would be 502.
Perform modulo operation: (there may actually be additional error here depending on what algorithm is used within the mod() function)
mod( 1.111101011111111111111111111111111111111111111111111e8, 1.01e3) = 1.111111111111111111111111111111111111111111100000000e0
The error is exactly -2-44 which is -5.6843x10-14. The conversion between decimal and binary and the rounding due to finite precision have caused a small error. In some cases, you get lucky and rounding errors cancel out and you might still get the 'right' answer which is why you got what you expect for mod(1.02*100, 10), but In general, you cannot rely on this.
To use mod() correctly to test the particular digit of a number, use round() to round it to the nearest whole number and compensate for floating point error.
mod(round(5.02*100), 10) == 2
What you're encountering is a floating point error or artifact, like the commenters say. This is not a Matlab bug; it's just how floating point values work. You'd get the same results in C or Java. Floating point values are "approximate" types, so exact equality comparisons using == without some rounding or tolerance are prone to error.
>> isequal(1.02*100, 102)
ans =
1
>> isequal(5.02*100, 502)
ans =
0
It's not the case that 5.02 is the only number this happens for; several around 0 are affected. Here's an example that picks out several of them.
x = 1.02:1000.02;
ix = mod(x .* 100, 10) ~= 2;
disp(x(ix))
To understand the details of what's going on here (and in many other situations you'll encounter working with floats), have a read through the Wikipedia entry for "floating point", or my favorite article on it, "What Every Computer Scientist Should Know About Floating-Point Arithmetic". (That title is hyperbole; this article goes deep and I don't understand half of it. But it's a great resource.) This stuff is particularly relevant to Matlab because Matlab does everything in floating point by default.