I've searched for this but i didn't find anything, i hope this is not a doubled question.
I'm doing a formula in TSQL like this:
#Temp = SQRT((((#Base1 - 1) * (#StDev1 * #StDev1))
+ ((#AvgBase - 1) * (#AvgStDev * #AvgStDev)))
* ((1 / #Base1) + (1 / #AvgBase))
/ (#Base1 + #AvgBase - 2))
But it always returns me a 0.
#Base1 and #AvgBase are int, the rest of parameters are float, but i've also tried with decimal(15,15).
I tried also changing the self multiplication with the function POWER() but the only problem i can't solve is this part: (1 / #Base1) + (1 / #AvgBase), because #Base1 and #AvgBase are so big, and the result of the calc is 0,0001... and some more numbers. How can i force the engine to not round the result to 0? Thanks
EDIT: I solved it changing the #AvgBase and the #Base1 to float type. I guess that the result 1/#param, with #param -> int gives you the rounded result and when you go for casting it or whatever, you are working on a rounded result anyway.
have you tried to create a #INVBase1 = (1/#Base1) ? will this also be rounded to 0? what happens when you play around with the data format of this new variable?
alternatively have you tried
/ ((#Base1) + (#AvgBase))
instead of
* ((1 / #Base1) + (1 / #AvgBase))
Related
Very strange issue I am noticing... This link says the numeric data type should be able to precisely handle 16383 digits after the decimal: https://www.postgresql.org/docs/10/datatype-numeric.html
So can someone plz explain to me why this function returns 9499.99999999999905:
(((60000::numeric / 50500 * 50500) - 50500) * (50500::numeric / 50500))::numeric
The correct answer is 9500.
When I use this function I get the right answer:
(((60000::numeric(20,11) / 50500 * 50500) - 50500) * (50500::numeric(20,11) / 50500))::numeric(20,11)
and this gives wrong answer:
(((60000::numeric(25,16) / 50500 * 50500) - 50500) * (50500::numeric(25,16) / 50500))::numeric(25,16) = 9499.9999999999990500
The odd thing is this same issue is happening on this website: https://web2.0calc.com/
if you paste the formula:
((60000.0000000 / 50500 * 50500) - 50500) * (50500.0000000 / 50500) = 9500.
But if I instead add an extra 0 to each of those:
((60000.00000000 / 50500 * 50500) - 50500) * (50500.00000000 / 50500) = 9499.99999999999999999999999999999999999999999999999999999999999905.
Even weirder, for both postgres and this website, if I break down the formula into two executions like this:
((60000.00000000 / 50500 * 50500) - 50500) = 9500
(50500.00000000 / 50500) = 1
9500 * 1 = 9500.
What the heck is going on here?
That you get the right answer with numeric(20,11) and the wrong one with numeric(25,16) is a coincidence: both will have rounding errors, because they calculate only to a limited precision, but in the first case the rounded result happens to be the correct one.
The same is the case for 60000.0000000 and 60000.00000000: they are interpreted as numeric values with different scale.
SELECT scale(60000.0000000), scale(60000.00000000);
scale | scale
-------+-------
7 | 8
(1 row
The only thing that is not obvious is why you get such a bad scale when you cast to numeric without any scale or precision.
The scale of 60000::numeric is 0, and 50500 is also converted to a numeric with scale 0. Now if you divide two numeric values, the resulting scale is calculated by the function select_div_scale, and a comment there clarifies the matter:
/*
* The result scale of a division isn't specified in any SQL standard. For
* PostgreSQL we select a result scale that will give at least
* NUMERIC_MIN_SIG_DIGITS significant digits, so that numeric gives a
* result no less accurate than float8; but use a scale not less than
* either input's display scale.
*/
NUMERIC_MIN_SIG_DIGITS has the value 16. The problem that this heuristic solves is that the scale of the division cannot be determined by looking at the scale of the arguments. So PostgreSQL chooses a value of 16, unless one of the arguments has a bigger scale. This avoids ending up with extremely large result values, unless someone explicitly asks for it.
This is a example equation which I want to be solved:
let equation = (5-2) * (10-5) / (4-2) * (10-5)
print (equation)
//35
The result which is printed is 35. But the right result would be 1,5. Whats wrong?
your expression is incorrect I hope you want the result 1.5
put '(' correctly * and / Precedence to execution are same but () is greater than * and /
let equation = ((5-2) * (10-5)) / ((4-2) * (10-5))
print (equation)
if you put the multiplication in another '()' then you will get result one perhaps the right part is integer so its auto conver to integer type
let equation = Double ( (5 - 2) * (10 - 5)) / Double ((4 - 2) * ( 10 - 5 ))
print (equation)
this code will print 1.5
Just look out operators Precedence in programming language
This should work:
let numerator: Double = (5-2) * (10-5)
let denumerator: Double = (4-2) * (10-5)
Fist you calculate the numerator and denumerator. And finally the result:
print(result)
let result: Double = numerator/denumerator
//1.5
As #araf has answered you should look out for the operator precedence in programming language.
Which follow a simple rule of the BODMAS evaluated in following order:
Brackets
Orders
Division and Multiplication (left to right)
Addition and Subtraction (left to right)
In your scenario:
let equation = (5-2) * (10-5) / (4-2) * (10-5)
the output is as follows:
3*5/2*4 = 15/2*5 = 7*5 = 35
#L.Stephan has suggested a better approach of calculating numerator and denumerator separately and then perform the division part.
To know more you can check this link:
https://en.wikipedia.org/wiki/Order_of_operations
This question already has answers here:
Division not working properly in Swift
(3 answers)
Closed 6 years ago.
print(String(Float(2 * (10 / 9))))
Why does this code print "2.0"?
Using a calculator, "2 * (10 / 9)" would equal 2.222222.....
You are calculating with integer numbers and cast the (integer) result to Float.
Do your calculation with floating point types (Double) instead:
print(String(Float(2.0 * (10.0 / 9.0))))
No need to cast though:
print(2.0 * (10.0 / 9.0))
2.0 * (10.0 / 9.0) would give your the expected result.
In your case, Swift does the calculations based on Integers first (result = 2), then converts this to a float (result = 2.0) and this into a String (result = "2.0")
To get the correct result, it should read:
print(String(Float(2.0 * (10.0 / 9.0))))
You then could leave out the two type conversations:
print(2.0 * (10.0 / 9.0))
I have following query where I am trying to do an arithmetic operation on columns. I have used the correct casting based on the postgres documentation but (http://www.postgresql.org/docs/8.3/static/datatype-numeric.html) but someone my bigint columns still cut of the results and avoid decimal points on the result sets.
select * from (
select (degree_one + degree_two) as degree_easy,
degree_three as degree_hard,
(((degree_one + degree_two)/(degree_one + degree_two + degree_three))::decimal) as easy_percent, ((degree_three/(degree_one + degree_two + degree_three))::decimal) as hard_percent from recommendation_degree
) as dc
where dc.degree_easy >= 4 and dc.degree_hard >= 4
What am I doing wrong here? In addition to decimal, I have tried float, real but both of them gives the same result.
You can try casting the dividend and divisor in the divisions to force the calculation to be done on decimal values, otherwise I think you'll end up doing integer division (which you then cast to decimal) and the result will be incorrect.
Try this:
select * from (
select (degree_one + degree_two) as degree_easy,
degree_three as degree_hard,
(degree_one::decimal + degree_two::decimal)/(degree_one::decimal + degree_two::decimal + degree_three::decimal) as easy_percent,
degree_three::decimal/(degree_one::decimal + degree_two::decimal + degree_three::decimal) as hard_percent
from recommendation_degree
) as dc
where dc.degree_easy >= 4 and dc.degree_hard >= 4
It might not be necessary to cast both dividend and divisor, it should work when only casting the divisor I think.
Sample SQL Fiddle
I have been trying this for a while but thus far haven't had any luck.
What is the easiest way to retrieve a random number between two very precise numbers on iOS?
For example, I want a random number between 41.37783830549337 and 41.377730629131634, how would I accomplish this?
Thank you so much in advance!
Edit: I tried this:
double min = 41.37783830549337;
double max = 41.377730629131634;
double test = ((double)rand() / RAND_MAX) * (max - min) + min;
NSLog(#"Min:%lf, max:%lf, result:%lf",min,max,test);
But the results weren't quite as precise as I was hoping, and ended up like this::
Min:41.377838, max:41.377731, result:41.377838
You can normalise the output of rand to any range you want:
((double)rand() / RAND_MAX) * (max - min) + min
[Note: This is pure C, I'm assuming it works equivalently in Obj-C.]
[Note 2: Replace double with the data-type of your choice as appropriate.]
[Note 3: Replace rand with the random-number source of your choice as appropriate.]