Problem: I am trying to create a wide view (~5000 columns), which works across data sources fine JDV. However, when I try to create the view with a join on 2+ table from data source, the optimizer pushes down the join to the source. The current source cannot handle more then 1600 columns.
Example: When trying to join Member_DX1 and Member_DX2 at client, JDV pushes the enter code herecombined join to postgres as one getting the too max column error.
/* TABLE 1 */
CREATE VIEW Member_DX1 (
MEMB_BID Integer
, DX130402000000 Integer
, DX180608000000 Integer
, DX20401070000 Integer
.... /* 1000 more */
as
SELECT dx.memb_bid
, case dx.EPI_1_DX4 when 130402000000 then 1 else 0 END as DX130402000000
, case dx.EPI_1_DX4 when 180608000000 then 1 else 0 END as DX180608000000
, case dx.EPI_1_DX4 when 20401070000 then 1 else 0 END as DX20401070000
...
FROM BDR.ENH_EPI_DETAIL dx
/* TABLE 2 */
CREATE VIEW Member_DX2 (
MEMB_BID Integer
, DX200102010000 Integer
, DX90125000000 Integer
, DX160603070000 Integer
... /* 1000 more ...
SELECT dx.memb_bid /* FOREIGN TABLE */
, case dx.EPI_1_DX4 when 200102010000 then 1 else 0 END as DX200102010000
, case dx.EPI_1_DX4 when 90125000000 then 1 else 0 END as DX90125000000
, case dx.EPI_1_DX4 when 160603070000 then 1 else 0 END as DX160603070000
...`enter code here`
FROM BDR.ENH_EPI_DETAIL dx
then my query in (e.g. dBeaver) looks like this:
SELECT * from Member_DX1 dx1
join Member_DX2 dx2
on dx1.MEMB_BID = dx2.MEMB_BID
The current source cannot handle more then 1600 columns.
Can you capture that as an issue for Teiid? Then we can take appropriate compensating action automatically.
then my query in (e.g. dBeaver) looks like this:
If you see this issue affecting all of your user queries, then you can turn join support off at the translator level via translator overrides - SupportsInnerJoin, SupportsOuterJoins, etc.. If there is a pk/fk relationship and you can modify the metadata, you can add an extension property allow-join as false to prevent the pushdown - see Join Compensation http://teiid.github.io/teiid-documents/master/content/reference/Federated_Optimizations.html
Related
I developed the following function:
create function kv_fn_ValuationPerItem_AW (#dDate date, #active bit)
returns table
as
return
(
select
Code ItemCode
, Description_1 ItemDescription
, ItemGroup
, Qty_On_Hand CurrentQtyOnHand
, AveUCst CurrentAvgCost
, Qty_On_Hand*AveUCst CurrentValue
from _bvSTTransactionsFull t
inner join StkItem s on t.AccountLink = s.StockLink
where ServiceItem = 0
and ItemActive = #active
and TxDate <= #dDate
group by Code, Description_1, ItemGroup, Qty_On_Hand, AveUCst
)
The function requires two parameters:
Date
Is the item Active - 1 = Active & 0 = Inactive
If I use the function as stipulated above, by specifying 1 for the Active Parameter, then the results will only be for Active Items.
If I specify 0, then it'll return all inactive Items.
How do I alter this function to cater for Active Items or both Active & Inactive?
i.e. if the parameter is 1, the where clause should read as ItemActive = #active, but when it's 0, the where clause should read as ItemActive in (1,0), How do I change the function to work like this?
I tried a case, but my syntax is not correct...
It's as simple as adding an or to your where cluase:
...
and (ItemActive = 1 OR #active = 0)
...
BTW, you might want to do it like this instead:
and (ItemActive = #active OR #active IS NULL)
which means that when you pass in 1 as #active you'll get only the active items, when you pass in 0 you'll get only the inactive members, but when you pass in null you'll get all records, regardless of the value in the ItemActive column.
Thanks Shnugo & Zohar for your answers,
Please amend your answers, then I'll mark yours as the answer.
The solution to my problem was to alter the Function as following:
create function kv_fn_ValuationPerItem_AW (#dDate date, #active bit)
returns table
as
return
(
select
Code ItemCode
, Description_1 ItemDescription
, ItemGroup
, Qty_On_Hand CurrentQtyOnHand
, AveUCst CurrentAvgCost
, Qty_On_Hand*AveUCst CurrentValue
from _bvSTTransactionsFull t
inner join StkItem s on t.AccountLink = s.StockLink
where ServiceItem = 0
and ItemActive in (1,#active)
and TxDate <= #dDate
group by Code, Description_1, ItemGroup, Qty_On_Hand, AveUCst
)
I think you are looking for this:
DECLARE #mockup TABLE(ID INT IDENTITY,SomeValue VARCHAR(100),Active BIT);
INSERT INTO #mockup VALUES('Row 1 is active',1)
,('Row 2 is active',1)
,('Row 3 is inactive',0)
,('Row 4 is inactive',0);
DECLARE #OnlyActive BIT=0; --set this to 1 to see active rows only
SELECT *
FROM #mockup m
WHERE (#OnlyActive=0 OR m.Active=1);
The idea is: If the parameter is set to 0 this expression is always true, if not, the column Active must be set to 1.
Hint: I used paranthesis, which was not needed in this simple case. But in your more complex WHERE clause they will be needed...
Hint2: I named the parameter OnlyActive, which expresses a bit better what you are looking for. You might turn the parameter to ShowAll with an invers logic too...
I have a table like this:
amount type app owe
1 a 10 10
2 a 8 -2
3 a 20 12
4 i 30 10
5 a 40 10
owe is:
(type == 'a')?app - sum(owe) where amount < (amount for current row):max(app-sum(owe)where amount<(amount for current row),0)
So I'd need a window function on the column that the window function is on. There are these partition on rows between rows unlimited preceding and prior row, but it has to be on a different column, not the column I'm summing. Is there a way to reference the same column the window function is on
I tried an alias
case
when type = a
then app - sum(owe)over(ROWS BETWEEN UNBOUNDED PRECEDING AND 1 preceding) as owe
else
greatest(0,app - sum(owe)over(ROWS BETWEEN UNBOUNDED PRECEDING AND 1 preceding))
end as owe
But since owe doesn't exist when I made it, I get:
owe doesn't exist.
Is there some other way?
You cannot do that with window functions. Your only chance using SQL is a recursive CTE:
WITH RECURSIVE tab_owe AS (
SELECT amount, type, app,
CASE WHEN type = 'a'
THEN app
ELSE GREATEST(app, 0)
END AS owe
FROM tab
ORDER BY amount LIMIT 1
UNION ALL
SELECT t.amount, t.type, t.app,
CASE WHEN t.type = 'a'
THEN t.app - sum(tab_owe.owe)
ELSE GREATEST(t.app - sum(tab_owe.owe), 0)
END AS owe
FROM (SELECT amount, type, app
FROM tab
WHERE amount > (SELECT max(amount) FROM tab_owe)
ORDER BY amount
LIMIT 1) AS t
CROSS JOIN tab_owe
GROUP BY t.amount, t.type, t.app
)
SELECT amount, type, app, owe
FROM tab_owe;
(untested)
This would be much easier to write in procedural code, sou consider using a table function.
This is what I came up with. Of course, I'm not a real programmer, so I'm sure there's a smarter way:
insert into mort (amount, "type", app)
values
(1,'a',10),
(2,'a',8),
(3,'a',20),
(4,'i',30),
(5,'a',40)
CREATE OR REPLACE FUNCTION mort_v ()
RETURNS TABLE (
zamount int,
ztype text,
zapp int,
zowe double precision
) AS $$
DECLARE
var_r record;
charlie double precision;
sam double precision;
BEGIN
charlie = 0;
FOR var_r IN(SELECT
amount,
"type",
app
FROM mort order by 1)
LOOP
zamount = var_r.amount;
ztype = var_r.type;
zapp = var_r.app;
sam = var_r.app - charlie;
if ztype = 'a' then
zowe = sam;
else
zowe = greatest(sam, 0);
end if;
charlie = charlie + zowe;
RETURN NEXT;
END LOOP;
END; $$
LANGUAGE 'plpgsql';
select * from mort_v()
So with my limited skills you'll notice I had to add a 'z' in front of the columns that are already in the table so I can spit it out again. If your table has 30 columns you'd normally have to do this 30 times. But, I asked a real engineer and he mentioned that if you just spit out the primary key with the calculated column, you can just join it back to the original table. That's smarter than what I have. If there's an even better solution, that would be great. This does serve as a nice reference to how to do something like a cursor in postgre and how to make variables without a '#' in front like in mssqlserver.
Master Table
Code UserName
1 UserOne
2 UserTwo
3 UserThree
Details Table
Code UserCode ParamName ParamValue
1 1 NameOne ValueOne
1 1 NameTwo ValueTwo
1 1 NameThree ValueThree
and so on
The above is my Master and Details table. I wanna write a query which will convert the rows of details table into columns. The desired output is given below:
Code UserCode NameOne NameTwo NameThree and so on
1 1 ValueOne ValueTwo ValueThree and so on
How can I achieve this? Any suggestion will be great in advance.
This is a common problem, without know anyting about BDMS to use, i suggest two low-level solution:
adding column by JOIN
adding column by subselect
Adding column by subselect consist in adding a subselect to take each dato do you need to traspose into column.
Adding column by JOIN consist into add a left join to data (whit right data cut), ad expose field you need into columns.
These solution are each static and valid only if you have a fixed number of column to traspsose. A way to let that dinamy could be to intruduce a store procedure.
I hope the below query will help you achieve what you want
SELECT DISTINCT
(SELECT ParamValue FROM tblDetails WHERE ParamName ='TestOne' AND UserCode = tb.UserCode) AS TestOne,
(SELECT ParamValue FROM tblDetails WHERE ParamName ='TestTwo' AND UserCode = tb.UserCode) AS TestTwo,
(SELECT ParamValue FROM tblDetails WHERE ParamName ='TestThree' AND UserCode = tb.UserCode) AS TestThree
FROM tblDetails tb
or you can use PIVOT if the ParamName can have many values which cannot be guaranteed in advance.
A few general / common strategies...
You can use a PIVOT query...
You can use a CASE (TSQL) or DECODE (PLSQL) type of statement...
SELECT
...
CASE Parmname WHEN 'NameOne' THEN [ValueOne] ELSE '' END as NameOne,
CASE Parmname WHEN 'NameTwo' THEN [ValueTwo] ELSE '' END as NameTwo
...
You can use DERIVED TABLES...
SELECT
...
N1.Parmname,
N2.Parmname,
...
FROM
...
LEFT JOIN (SELECT * FROM tbl_Detail WHERE Parmname = 'NameOne') N1
ON...
LEFT JOIN (SELECT * FROM tbl_Detail WHERE Parmname = 'NameTwo') N2
...
...etcetera
Here's a simple query we do for ad hoc requests from our Marketing department on the leads we received in the last 90 days.
SELECT ID
,FIRST_NAME
,LAST_NAME
,ADDRESS_1
,ADDRESS_2
,CITY
,STATE
,ZIP
,HOME_PHONE
,MOBILE_PHONE
,EMAIL_ADDRESS
,ROW_ADDED_DTM
FROM WEB_LEADS
WHERE ROW_ADDED_DTM BETWEEN #START AND #END
They are asking for more derived columns to be added that show the number of previous occurences of ADDRESS_1 where the EMAIL_ADDRESS matches. But they want is for different date ranges.
So the derived columns would look like this:
,COUNT_ADDRESS_1_LAST_1_DAYS,
,COUNT_ADDRESS_1_LAST_7_DAYS
,COUNT_ADDRESS_1_LAST_14_DAYS
etc.
I've manually filled these derived columns using update statements when there was just a few. The above query is really just a sample of a much larger query with many more columns. The actual request has blossomed into 6 date ranges for 13 columns. I'm asking if there's a better way then using 78 additional update statements.
I think you will have a hard time writing a query that includes all of these 78 metrics per e-mail address without actually creating a query that hard-codes the different choices. However you can generate such a pivot query with dynamic SQL, which will save you some keystrokes and will adjust dynamically as you add more columns to the table.
The result you want to end up with will look something like this (but of course you won't want to type it):
;WITH y AS
(
SELECT
EMAIL_ADDRESS,
/* aggregation portion */
[ADDRESS_1] = COUNT(DISTINCT [ADDRESS_1]),
[ADDRESS_2] = COUNT(DISTINCT [ADDRESS_2]),
... other columns
/* end agg portion */
FROM dbo.WEB_LEADS AS wl
WHERE ROW_ADDED_DTM >= /* one of 6 past dates */
GROUP BY wl.EMAIL_ADDRESS
)
SELECT EMAIL_ADDRESS,
/* pivot portion */
COUNT_ADDRESS_1_LAST_1_DAYS = *count address 1 from 1 day ago*,
COUNT_ADDRESS_1_LAST_7_DAYS = *count address 1 from 7 days ago*,
... other date ranges ...
COUNT_ADDRESS_2_LAST_1_DAYS = *count address 2 from 1 day ago*,
COUNT_ADDRESS_2_LAST_7_DAYS = *count address 2 from 7 days ago*,
... other date ranges ...
... repeat for 11 more columns ...
/* end pivot portion */
FROM y
GROUP BY EMAIL_ADDRESS
ORDER BY EMAIL_ADDRESS;
This is a little involved, and it should all be run as one script, but I'm going to break it up into chunks to intersperse comments on how the above portions are populated without typing them. (And before long #Bluefeet will probably come along with a much better PIVOT alternative.) I'll enclose my interspersed comments in /* */ so that you can still copy the bulk of this answer into Management Studio and run it with the comments intact.
Code/comments to copy follows:
/*
First, let's build a table of dates that can be used both to derive labels for pivoting and to assist with aggregation. I've added the three ranges you've mentioned and guessed at a fourth, but hopefully it is clear how to add more:
*/
DECLARE #d DATE = SYSDATETIME();
CREATE TABLE #L(label NVARCHAR(15), d DATE);
INSERT #L(label, d) VALUES
(N'LAST_1_DAYS', DATEADD(DAY, -1, #d)),
(N'LAST_7_DAYS', DATEADD(DAY, -8, #d)),
(N'LAST_14_DAYS', DATEADD(DAY, -15, #d)),
(N'LAST_MONTH', DATEADD(MONTH, -1, #d));
/*
Next, let's build the portions of the query that are repeated per column name. First, the aggregation portion is just in the format col = COUNT(DISTINCT col). We're going to go to the catalog views to dynamically derive the list of column names (except ID, EMAIL_ADDRESS and ROW_ADDED_DTM) and stuff them into a #temp table for re-use.
*/
SELECT name INTO #N FROM sys.columns
WHERE [object_id] = OBJECT_ID(N'dbo.WEB_LEADS')
AND name NOT IN (N'ID', N'EMAIL_ADDRESS', N'ROW_ADDED_DTM');
DECLARE #agg NVARCHAR(MAX) = N'', #piv NVARCHAR(MAX) = N'';
SELECT #agg += ',
' + QUOTENAME(name) + ' = COUNT(DISTINCT '
+ QUOTENAME(name) + ')' FROM #N;
PRINT #agg;
/*
Next we'll build the "pivot" portion (even though I am angling for the poor man's pivot - a bunch of CASE expressions). For each column name we need a conditional against each range, so we can accomplish this by cross joining the list of column names against our labels table. (And we'll use this exact technique again in the query later to make the /* one of past 6 dates */ portion work.
*/
SELECT #piv += ',
COUNT_' + n.name + '_' + l.label
+ ' = MAX(CASE WHEN label = N''' + l.label
+ ''' THEN ' + QUOTENAME(n.name) + ' END)'
FROM #N as n CROSS JOIN #L AS l;
PRINT #piv;
/*
Now, with those two portions populated as we'd like them, we can build a dynamic SQL statement that fills out the rest:
*/
DECLARE #sql NVARCHAR(MAX) = N';WITH y AS
(
SELECT
EMAIL_ADDRESS, l.label' + #agg + '
FROM dbo.WEB_LEADS AS wl
CROSS JOIN #L AS l
WHERE wl.ROW_ADDED_DTM >= l.d
GROUP BY wl.EMAIL_ADDRESS, l.label
)
SELECT EMAIL_ADDRESS' + #piv + '
FROM y
GROUP BY EMAIL_ADDRESS
ORDER BY EMAIL_ADDRESS;';
PRINT #sql;
EXEC sp_executesql #sql;
GO
DROP TABLE #N, #L;
/*
Now again, this is a pretty complex piece of code, and perhaps it can be made easier with PIVOT. But I think even #Bluefeet will write a version of PIVOT that uses dynamic SQL because there is just way too much to hard-code here IMHO.
*/
Can anyone tell me how to translate the following T-SQL statement:
SELECT fileld1 = CASE
WHEN T.option1 THEN -1
ELSE
CASE WHEN T.option2 THEN 0
ELSE 1
END
END
FROM Table1 AS T
The point is I need to validate two different options from the table for a single field in the select statement..
I have tried to do somthing with an IF statement in pl/sql, but it just doesnt work for me:
SELECT IF T.option1 THEN -1
ELSE IF T.option2 THEN 0
ELSE 1
END
FROM Table1 AS T
I am not actually sure how to write IF statement inside the SELECT statement..
And also, I need to do it INSIDE the select statement because I am constructing a view.
Use:
SELECT CASE
WHEN T.option1 = ? THEN -1
WHEN T.option2 = ? THEN 0
ELSE 1
END AS field1
FROM Table1 AS T
I can't get your original TSQL to work - I get:
Msg 4145, Level 15, State 1, Line 4
An expression of non-boolean type specified in a context where a condition is expected, near 'THEN'.
...because there's no value evaluation. If you're checking if the columns are null, you'll need to use:
SELECT CASE
WHEN T.option1 IS NULL THEN -1
WHEN T.option2 IS NULL THEN 0
ELSE 1
END AS field1
FROM Table1 AS T
...or if you need when they are not null:
SELECT CASE
WHEN T.option1 IS NOT NULL THEN -1
WHEN T.option2 IS NOT NULL THEN 0
ELSE 1
END AS field1
FROM Table1 AS T
CASE expressions shortcircuit - if the first WHEN matches, it returns the value & exits handling for that row - so the options afterwards aren't considered.
If I remember correctly, PL/SQL also supports the case. You just would have to move the column alias from "field1=" before the expression to "AS filed1" after the expression.