wrong result with floating numbers - tsql

I have this table with 1 records. Im trying to compute something call Puntaje, to get the Puntaje Result I have to follow the following formula:
Puntaje = (Infracciones * 10) / Horas
Horas = Segundos / 60 / 60
I wrote the following script, but I have some doubt and problem.
1) Is there another way to assign the values to #variables or another way to compute the sum?
2) Why the Puntaje result is 0.00, have to be: 0.854
Im using MS SQL Server 2012
Can someone help me to resolve this? Thank you in advance.
/* content of table: #Customer_Drivers
DriverId Segundos KM QtyExcesos QtyFreAce QtyDesc Puntaje IDC
6172 717243 1782 17 0 0 0 0
*/
DECLARE #Customer_Drivers TABLE (
DriverId INT,
Segundos INT,
KM INT,
QtyExcesos INT,
QtyFreAce INT,
QtyDesc INT,
Puntaje INT,
IDC INT
);
SET NOCOUNT ON;
INSERT INTO #Customer_Drivers (DriverId, Segundos, KM, QtyExcesos, QtyFreAce, QtyDesc, Puntaje, IDC)
VALUES (6172, 717243, 1782, 17, 0, 0, 0, 0);
SET NOCOUNT OFF;
DECLARE #DriverId INT = 6172;
DECLARE #Horas INT;
DECLARE #QtyExcesos INT ;
DECLARE #QtyFreAce INT ;
DECLARE #QtyDesc INT ;
DECLARE #Infracciones INT;
DECLARE #Puntaje Decimal(18,2);
SET #Horas = (SELECT Segundos FROM #Customer_Drivers WHERE DriverId = #DriverId) / 60 / 60;
SET #QtyExcesos = (SELECT QtyExcesos FROM #Customer_Drivers WHERE DriverId = #DriverId);
SET #QtyFreAce = (SELECT QtyFreAce FROM #Customer_Drivers WHERE DriverId = #DriverId);
SET #QtyDesc = (SELECT QtyDesc FROM #Customer_Drivers WHERE DriverId = #DriverId);
SET #Infracciones = (#QtyExcesos + #QtyFreAce + #QtyDesc);
SET #Puntaje = ( #Infracciones * 10) /#Horas;
PRINT #Horas
PRINT #QtyExcesos
PRINT #QtyFreAce
PRINT #QtyDesc
PRINT #Puntaje
/* OUTPUT
199 -- #Horas
17 -- #QtyExcesos
0 -- #FreAce
0 -- #QtyDesc
0.00 -- #Puntaje must be = 0.854
*/

Even though #Puntaje is declared as Decimal(18,2), that doesn't mean your calculation will be treated as a decimal. The problem is that ( #Infracciones * 10) / #Horas is using all integers so this expression will result in the integer value 0. Then this integer 0 is converted to a decimal and stored in #Puntaje.
To fix this, you need to convert part of the expression to a decimal first so that the result will be a decimal:
SET #Puntaje = ( CAST(#Infracciones AS Decimal(18,2)) * 10) / #Horas

You are using integers in your calculation, so the result will be rounded off (or truncated) to the nearest integer. Use decimal values, or use 'cast' :
#Puntaje = (cast(#Infracciones as decimal(18,2)) * 10.0) / cast(#Horas as decimal(18,2))
Check my syntax - just typed this on without trying it

1) You can use SELECT #Horas = Segundos/3600, #QtyExcesos = QtyExcesos ... FROM [RS_Reports].[dbo].[Customer_Drivers] WHERE DriverId = #DriverId. This should work providing that there is one line of results.
2) Already answered by others, you have to divide by decimal to get a decimal, i.e. you'll have to convert #Horas to Decimal

From what I've learned. #variable should be some parameters light input parameter and output parameter.... Try to execute your Stored Procedure and see what you got in SQL Server Management Studio.
There should be a return value.

Related

SQL Server - Decimal values not precisely calculated

For the below calculation, the expected result is -0.0000671. But the result of the code is -0.00006800000.
declare
#V_A decimal (38,11) = 99.99329,
#V_B decimal (38,11) = 100,
#V_RESULT decimal (38,11);
print '#V_A = '+cast(#V_A as varchar) --Printed as "99.99329000000"
print '#V_B = '+cast(#V_B as varchar) --Printed as "100.00000000000"
SET #V_RESULT = (#V_A / #V_B) - 1;
print '#V_RESULT = '+cast(#V_RESULT as varchar) --Printed as "-0.00006800000"
Interestingly, the below code returns exact result as expected.
select (99.99329 / 100) - 1 --Returns "-0.000067100".
Could you please help me to get the same output in the above T-SQL code? I don't want to use float as it will increase decimal places. Thanks in advance.
Environment: Azure SQL DB, DTU based.
Casting to float during calculation resolves the issue.
#V_A decimal (38,11) = 99.99329,
#V_B decimal (38,11) = 100,
#V_RESULT decimal (38,11);
print '#V_A = '+cast(#V_A as varchar) --Printed as "99.99329000000"
print '#V_B = '+cast(#V_B as varchar) --Printed as 100.00000000000
SET #V_RESULT = (cast(#V_A as float) / cast(#V_B as float)) - 1; --Printed as -0.00006710000
print '#V_RESULT = '+cast(#V_RESULT as varchar)

Multiply two host variables in embedded SQL for PostgreSQL

I have problem multiplying two host variables in embedded SQL for PostgreSQL.
The SQL-query is big but I have cut out the part that doesn't work.
Declaration:
EXEC SQL BEGIN DECLARE SECTION;
int var1;
int var2;
EXEC SQL END DECLARE SECTION;
Code:
CASE WHEN product_id = :var1 THEN :var1 * :var2
ELSE 0 END
The compilation works but I get the following error message at execution:
Errcode: -400
Errmsg: operator is not unique: unknown * unknown on line 1394
If I change the code to
CASE WHEN product_id = :var1 THEN 1 * :var2
ELSE 0 END
or
CASE WHEN product_id = :var1 THEN product_id * :var2
ELSE 0 END
or
CASE WHEN product_id = :var1 THEN :var1 * (1 * :var2)
ELSE 0 END
it works.
Is it possible to multiply two host variables? If not, is there any workaround? The last code example above works but I would like a solution that is not as ugly.
Try casting to integer:
CASE WHEN product_id = :var1 THEN :var1::integer * :var2::integer
ELSE 0 END
The error
operator is not unique...
can mostly be fixed by casting to the expected type

Removing decimal places

I have a set of decimal values, but I need to remove the values after the decimal if they are zero.
17.00
23.50
100.00
512.79
become
17
23.50
100
512.79
Currently, I convert to a string and replace out the trailing .00 - Is there a better method?
REPLACE(CAST(amount as varchar(15)), '.00', '')
This sounds like it is purely a data presentation problem. As such you should let the receiving application or reporting software take care of the formatting.
You could try converting the .00s to datatype int. That would truncate the decimals. However, as all the values appear in one column they will have to have the same type. Making everything an int would ruin your rows with actual decimal places.
As a SQL solution to a presentation problem, I think what you have is OK.
I would advice you to compare the raw decimal value with itself floored. Example code:
declare
#Val1 decimal(9,2) = 17.00,
#Val2 decimal(9,2) = 23.50;
select
case when FLOOR ( #Val1 ) = #Val1
then cast( cast(#Val1 as int) as varchar)
else cast(#Val1 as varchar) end,
case when FLOOR ( #Val2 ) = #Val2
then cast( cast(#Val2 as int) as varchar)
else cast(#Val2 as varchar) end
-------------
17 | 23.50

Assign range to variable using between

I have the following decitiontree:
declare #placeholder varchar(20)
If #Number1 = 1
AND #Number2 = 0
BEGIN SET #placeholder = 'NULL'
END
ELSE IF #Number1 = 1
AND #Number2 > 0
BEGIN SET #placeholder = Between (#Number2*10) AND (#Number2*10+9)
END
ELSE
BEGIN
SET #placeholder = #Othervariable
END
I need the Variable for the query:
SELECT * FROM Table
WHERE #Placeholder is null or ID = #placeholder.
But the 'Between' part is not working. Can anyone help me with it?
This won't work in SQL server - there is no variable type that holds something like lambda expression, say Between (#Number2*10) AND (#Number2*10+9).
One way is to store this in string (say, nvarchar(max)) and execute using exec() or sp_executesql().
The other (usually more optimized) way is to form the main expression to include sub-expressions along with exclusion criteria.
Here is an example:
DECLARE #Number1 int = 1 -- input variable 1
DECLARE #Number2 int = 1 -- input variable 2
DECLARE #excl1 bit = 0 -- exclusion criteria 1 (ec1)
DECLARE #excl2 bit = 0 -- exclusion criteria 2 (ec2)
-- fill excl. crit.
SET #excl1 = CASE WHEN #Number1 = 1 AND #Number2 = 0 THEN 1 ELSE 0 END
SET #excl2 = CASE WHEN #Number1 = 1 AND #Number2 > 0 THEN 1 ELSE 0 END
-- just output to see what's happening
PRINT #Number1
PRINT #Number2
PRINT #excl1
PRINT #excl2
SELECT *
FROM Table
WHERE
-- if ec1 is active, we apply sub-expression 1
(#excl1 = 0 OR
(#excl1 = 1 AND ID IS NULL))
AND
-- if ec2 is active, we apply sub-expression 2
(#excl2 = 0 OR
(#excl2 = 1 AND ID BETWEEN #Number2 * 10 AND #Number2 * 10 + 9))

tsql how to validate a number's scale

I need to validate the number of digits to the right of the decimal (the scale)
0, is a valid number in any of the places (tenths, hundredths, thousandths, etc.).
Any tips or tricks?... w/o an extensive regex library, and no built in function, I would prefer a function that accepts the number, the number of places the scale should equal, and then return a bit.
Following up with Maess's suggestion I came up with this:
CREATE FUNCTION [dbo].[GetScale]
(
#tsValue varchar(250)
, #tiScale int
)
RETURNS int
AS
BEGIN
DECLARE
#tiResult int
, #tiValueScale int
SET #tiResult = 0
SELECT #tiValueScale = LEN( SUBSTRING ( #tsValue, PATINDEX('%.%', #tsValue) + 1, LEN(#tsValue) ) )
IF (#tiValueScale = #tiScale)
SET #tiResult = 1
RETURN #tiResult
END
GO
Seems to work as desired. Thanks for the help.
Just as a followup... i ran into an issue where a number didnt have a decimal (which returns the patindex to 0) and the number was the same size as the scale, it would return a false positive... so i add an additional select from the patindex to determine if it does exist or not... it now looks like this:
- =============================================
ALTER FUNCTION [dbo].[GetScale]
(
#tsValue varchar(250)
, #tiScale int
)
RETURNS int
AS
BEGIN
DECLARE
#tiResult int
, #tiValueScale int
, #tiDecimalExists int
SET #tiResult = 0
SET #tiDecimalExists = 0
SELECT #tiDecimalExists = PATINDEX('%.%', #tsValue)
IF (#tiDecimalExists != 0)
BEGIN
SELECT #tiValueScale = LEN( SUBSTRING ( #tsValue, #tiDecimalExists + 1, LEN(#tsValue) ) )
IF (#tiValueScale = #tiScale)
SET #tiResult = 1
END
RETURN #tiResult
END
I tried Anthony's solution with some success, but there is some undesirable side effects when the first whole number is 9.
For example...
select 0.11 as fraction, Math.NumberOfDecimalPlaces(0.11) dp union
select 9.1, Math.NumberOfDecimalPlaces(9.1) union
select 9.01, Math.NumberOfDecimalPlaces(9.01) union
select 9.0, Math.NumberOfDecimalPlaces(9.0) union
select 99.0, Math.NumberOfDecimalPlaces(99.0) union
select 10999.0, Math.NumberOfDecimalPlaces(10999.0) union
select 8.0, Math.NumberOfDecimalPlaces(8.0) union
select 0, Math.NumberOfDecimalPlaces(0)
Produces...
0.00 0
0.11 2
8.00 0
9.00 -1
9.01 2
9.10 1
99.00 -2
10999.00 -3
Which shows some incorrect calculations when 9 is the first whole number.
I've made a small improvement to Anthony's original function.
CREATE FUNCTION [Math].[NumberOfDecimalPlaces]
(
#fraction decimal(38,19)
)
RETURNS INT
AS
BEGIN
RETURN FLOOR(LOG10(REVERSE(ABS(#fraction % 1) +1))) +1
END
This simply strips of the whole number part of the fraction. Which when implemented produces...
0.00 0
0.11 2
8.00 0
9.00 0
9.01 2
9.10 1
99.00 0
10999.00 0
The correct result
CREATE FUNCTION dbo.DecimalPlaces(#n decimal(38,19))
RETURNS int
AS
BEGIN
RETURN FLOOR(LOG10(REVERSE(ABS(#n % 1) + 1))) + 1
END
Edit (Feb 5 '15):
Thanks sqlconsumer. I have included your fix for nines before the decimal point.