TSQL CTE: How to avoid circular traversal? - tsql

I have written a very simple CTE expression that retrieves a list of all groups of which a user is a member.
The rules goes like this, a user can be in multiple groups, and groups can be nested so that a group can be a member of another group, and furthermore, groups can be mutual member of another, so Group A is a member of Group B and Group B is also a member of Group A.
My CTE goes like this and obviously it yields infinite recursion:
;WITH GetMembershipInfo(entityId) AS( -- entity can be a user or group
SELECT k.ID as entityId FROM entities k WHERE k.id = #userId
UNION ALL
SELECT k.id FROM entities k
JOIN Xrelationships kc on kc.entityId = k.entityId
JOIN GetMembershipInfo m on m.entityId = kc.ChildID
)
I can't find an easy solution to back-track those groups that I have already recorded.
I was thinking of using an additional varchar parameter in the CTE to record a list of all groups that I have visited, but using varchar is just too crude, isn't it?
Is there a better way?

You need to accumulate a sentinel string within your recursion. In the following example I have a circular relationship from A,B,C,D, and then back to A, and I avoid a loop with the sentinel string:
DECLARE #MyTable TABLE(Parent CHAR(1), Child CHAR(1));
INSERT #MyTable VALUES('A', 'B');
INSERT #MyTable VALUES('B', 'C');
INSERT #MyTable VALUES('C', 'D');
INSERT #MyTable VALUES('D', 'A');
; WITH CTE (Parent, Child, Sentinel) AS (
SELECT Parent, Child, Sentinel = CAST(Parent AS VARCHAR(MAX))
FROM #MyTable
WHERE Parent = 'A'
UNION ALL
SELECT CTE.Child, t.Child, Sentinel + '|' + CTE.Child
FROM CTE
JOIN #MyTable t ON t.Parent = CTE.Child
WHERE CHARINDEX(CTE.Child,Sentinel)=0
)
SELECT * FROM CTE;
Result:
Parent Child Sentinel
------ ----- --------
A B A
B C A|B
C D A|B|C
D A A|B|C|D

Instead of a sentinel string, use a sentinel table variable. Function will catch circular reference no matter how many hops the circle is, no issues with maximum length of nvarchar(max), easily modified for different data types or even multipart keys, and you can assign the function to a check constraint.
CREATE FUNCTION [dbo].[AccountsCircular] (#AccountID UNIQUEIDENTIFIER)
RETURNS BIT
AS
BEGIN
DECLARE #NextAccountID UNIQUEIDENTIFIER = NULL;
DECLARE #Sentinel TABLE
(
ID UNIQUEIDENTIFIER
)
INSERT INTO #Sentinel
( [ID] )
VALUES ( #AccountID )
SET #NextAccountID = #AccountID;
WHILE #NextAccountID IS NOT NULL
BEGIN
SELECT #NextAccountID = [ParentAccountID]
FROM [dbo].[Accounts]
WHERE [AccountID] = #NextAccountID;
IF EXISTS(SELECT 1 FROM #Sentinel WHERE ID = #NextAccountID)
RETURN 1;
INSERT INTO #Sentinel
( [ID] )
VALUES ( #NextAccountID )
END
RETURN 0;
END

Related

Get all instances of primary keys of a table

This is a simple example of what I need, for any given table, I need to get all the instances of the primary keys, this is a little example, but I need a generic way to do it.
create table foo
(
a numeric
,b text
,c numeric
constraint pk_foo primary key (a,b)
)
insert into foo(a,b,c) values (1,'a',1),(2,'b',2),(3,'c',3);
select <the magical thing>
result
a|b
1 |1|a|
2 |2|b|
3 |3|c|
.. ...
I need to control if the instances of the primary keys are changed by the user, but I don't want to repeat code in too many tables! I need a generic way to do it, I will put <the magical thing>
in a function to put it on a trigger before update and blah blah blah...
In PostgreSQL you must always provide a resulting type for a query. However, you can obtain the code of the query you need, and then execute the query from the client:
create or replace function get_key_only_sql(regclass) returns string as $$
select 'select '|| (
select string_agg(quote_ident(att.attname), ', ' order by col)
from pg_index i
join lateral unnest(indkey) col on (true)
join pg_attribute att on (att.attrelid = i.indrelid and att.attnum = col)
where i.indrelid = $1 and i.indisprimary
group by i.indexrelid
limit 1) || ' from '||$1::text
end;
$$ language sql;
Here's some client pseudocode using the function above:
sql = pgexecscalar("select get_key_only_sql('mytable'::regclass)");
rs = pgopen(sql);

How to work around the "Recursive CTE member can refer itself only in FROM clause" requirement?

I'm trying to run a graph search to find all nodes accessible from a starting point, like so:
with recursive
nodes_traversed as (
select START_NODE ID
from START_POSITION
union all
select ed.DST_NODE
from EDGES ed
join nodes_traversed NT
on (NT.ID = ed.START_NODE)
and (ed.DST_NODE not in (select ID from nodes_traversed))
)
select distinct * from nodes_traversed
Unfortunately, when I try to run that, I get an error:
Recursive CTE member (nodes_traversed) can refer itself only in FROM clause.
That "not in select" clause is important to the recursive expression, though, as it provides the ending point. (Without it, you get infinite recursion.) Using generation counting, like in the accepted answer to this question, would not help, since this is a highly cyclic graph.
Is there any way to work around this without having to create a stored proc that does it iteratively?
Here is my solution that use global temporary table, I have limited recursion by level and nodes from temporary table.
I am not sure how it will work on large set of data.
create procedure get_nodes (
START_NODE integer)
returns (
NODE_ID integer)
as
declare variable C1 integer;
declare variable C2 integer;
begin
/**
create global temporary table id_list(
id integer
);
create index id_list_idx1 ON id_list (id);
*/
delete from id_list;
while ( 1 = 1 ) do
begin
select count(distinct id) from id_list into :c1;
insert into id_list
select id from
(
with recursive nodes_traversed as (
select :START_NODE AS ID , 0 as Lv
from RDB$DATABASE
union all
select ed.DST_NODE , Lv+1
from edges ed
join nodes_traversed NT
on
(NT.ID = ed.START_NODE)
and nt.Lv < 5 -- Max recursion level
and nt.id not in (select id from id_list)
)
select distinct id from nodes_traversed);
select count(distinct id) from id_list into :c2;
if (c1 = c2) then break;
end
for select distinct id from id_list into :node_id do
begin
suspend ;
end
end

T-SQL Loop in a stored proc

how do I loop through a comma separated variable using tsql in a stored proc
So for instance my list would look like this
"1,2,3,4,5,6,7,8,9,10"
and I would loop thought this list and made some necessary table
insert based on this list
You could do it a couple ways, but if this would be a list of ID's it could be done like this as well. It would change your list format a bit.
UPDATE table
SET column = value
WHERE ID in ('1','2','3','4','5','6','7','8','9','10')
You could do a loop as well
DECLARE #List CHAR(100)
DECLARE #ListItem int
DECLARE #Pos int
SET #List = '1,2,3,4,5,6,7,8,9,10'
WHILE LEN(#List) > 0
BEGIN
--Pull Item Frim List
SET #Pos = CHARINDEX(',', #List)
IF #Pos = 0
BEGIN
SET #ListItem = #List
END
ELSE
BEGIN
SET #ListItem = SUBSTRING(#List, 1, #Pos - 1)
END
UPDATE table
SET column = value
WHERE ID = #ListItem
--Remove Item Frim List
IF #Pos = 0
BEGIN
SET #List = ''
END
ELSE
BEGIN
SET #List = SUBSTRING(#List, #Pos + 1, LEN(#List) - #Pos)
END
END
I'd try to avoid looping and insert the rows directly from your comma list.
Use a table values parameter (new in SQl Server 2008). Set it up by creating the actual table parameter type:
CREATE TYPE IntTableType AS TABLE (ID INTEGER PRIMARY KEY)
Your procedure would then be:
Create Procedure up_TEST
#Ids IntTableType READONLY
AS
SELECT *
FROM ATable a
WHERE a.Id IN (SELECT ID FROM #Ids)
RETURN 0
GO
if you can't use table value parameters, see: "Arrays and Lists in SQL Server 2005 and Beyond, When Table Value Parameters Do Not Cut it" by Erland Sommarskog, then there are many ways to split string in SQL Server. This article covers the PROs and CONs of just about every method. in general, you need to create a split function. This is how a split function can be used to insert rows:
INSERT INTO YourTableA (colA)
SELECT
b.col1
FROM dbo.yourSplitFunction(#Parameter) b
I prefer the number table approach to split a string in TSQL but there are numerous ways to split strings in SQL Server, see the previous link, which explains the PROs and CONs of each.
For the Numbers Table method to work, you need to do this one time table setup, which will create a table Numbers that contains rows from 1 to 10,000:
SELECT TOP 10000 IDENTITY(int,1,1) AS Number
INTO Numbers
FROM sys.objects s1
CROSS JOIN sys.objects s2
ALTER TABLE Numbers ADD CONSTRAINT PK_Numbers PRIMARY KEY CLUSTERED (Number)
Once the Numbers table is set up, create this split function:
CREATE FUNCTION [dbo].[FN_ListToTable]
(
#SplitOn char(1) --REQUIRED, the character to split the #List string on
,#List varchar(8000)--REQUIRED, the list to split apart
)
RETURNS TABLE
AS
RETURN
(
----------------
--SINGLE QUERY-- --this will not return empty rows
----------------
SELECT
ListValue
FROM (SELECT
LTRIM(RTRIM(SUBSTRING(List2, number+1, CHARINDEX(#SplitOn, List2, number+1)-number - 1))) AS ListValue
FROM (
SELECT #SplitOn + #List + #SplitOn AS List2
) AS dt
INNER JOIN Numbers n ON n.Number < LEN(dt.List2)
WHERE SUBSTRING(List2, number, 1) = #SplitOn
) dt2
WHERE ListValue IS NOT NULL AND ListValue!=''
);
GO
You can now easily split a CSV string into a table and join on it:
Create Procedure up_TEST
#Ids VARCHAR(MAX)
AS
SELECT * FROM ATable a
WHERE a.Id IN (SELECT ListValue FROM dbo.FN_ListToTable(',',#Ids))
GO
or insert rows from it:
Create Procedure up_TEST
#Ids VARCHAR(MAX)
,#OtherValue varchar(5)
AS
INSERT INTO YourTableA
(colA, colB, colC)
SELECT
ListValue, #OtherValue, GETDATE()
FROM dbo.FN_ListToTable(',',#Ids)
GO
Using CTE (Common Table Expression) is the most elegant solution I think check this question on stackoverflow,
T-SQL: Opposite to string concatenation - how to split string into multiple records

tsql - using internal stored procedure as parameter is where clause

I'm trying to build a stored procedure that makes use of another stored procedure. Taking its result and using it as part of its where clause, from some reason I receive an error:
Invalid object name 'dbo.GetSuitableCategories'.
Here is a copy of the code:
select distinct top 6 * from
(
SELECT TOP 100 *
FROM [dbo].[products] products
where products.categoryId in
(select top 10 categories.categoryid from
[dbo].[GetSuitableCategories]
(
-- #Age
-- ,#Sex
-- ,#Event
1,
1,
1
) categories
ORDER BY NEWID()
)
--and products.Price <=#priceRange
ORDER BY NEWID()
)as d
union
select * from
(
select TOP 1 * FROM [dbo].[products] competingproducts
where competingproducts.categoryId =-2
--and competingproducts.Price <=#priceRange
ORDER BY NEWID()
) as d
and here is [dbo].[GetSuitableCategories] :
if (#gender =0)
begin
select * from categoryTable categories
where categories.gender =3
end
else
begin
select * from categoryTable categories
where categories.gender = #gender
or categories.gender =3
end
I would use an inline table valued user defined function. Or simply code it inline is no re-use is required
CREATE dbo.GetSuitableCategories
(
--parameters
)
RETURNS TABLE
AS
RETURN (
select * from categoryTable categories
where categories.gender IN (3, #gender)
)
Some points though:
I assume categoryTable has no gender = 0
Do you have 3 genders in your categoryTable? :-)
Why do pass in 3 parameters but only use 1? See below please
Does #sex map to #gender?
If you have extra processing on the 3 parameters, then you'll need a multi statement table valued functions but beware these can be slow
You can't use the results of a stored procedure directly in a select statement
You'll either have to output the results into a temp table, or make the sproc into a table valued function to do what you doing.
I think this is valid, but I'm doing this from memory
create table #tmp (blah, blah)
Insert into #tmp
exec dbo.sprocName

T-SQL: Pivot but for semicolon-separated values instead of columns

I've got semicolon-separated values in a column Values in my table:
Values
1;2;3;4;5
I would like to transform it in a procedure to have there values as rows:
Values
1
2
3
4
5
How could I do it in T-SQL?
Solution 1(using xml):
declare #str varchar(20)
declare #xml as xml
set #str= '1;2;3;4;5'
SET #xml = cast(('<x>'+replace(#str,';' ,'</x><x>')+'</x>') as xml)
SELECT col.value('.', 'varchar(10)') as value FROM #xml.nodes('x') as tbl(col)
Solution 2(using recursive cte)
declare #str as varchar(100)
declare #delimiter as char(1)
set #delimiter = ';'
set #str = '1;2;3;4;5' -- original data
set #str = #delimiter + #str + #delimiter
;with num_cte as
(
select 1 as rn
union all
select rn +1 as rn
from num_cte
where rn <= len(#str)
)
, get_delimiter_pos_cte as
(
select
ROW_NUMBER() OVER (ORDER BY rn) as rowid,
rn as delimiterpos
from num_cte
cross apply( select substring(#str,rn,1) AS chars) splittedchars
where chars = #delimiter
)
select substring(#str,a.delimiterpos+1 ,c2.delimiterpos - a.delimiterpos - 1) as Countries
from get_delimiter_pos_cte a
inner join get_delimiter_pos_cte c2 on c2.rowid = a.rowid+1
option(maxrecursion 0)
The thing that struck me as possibly leaving room for an additional answer, or additional improvement was that most of the answers/links given were how to split values like this for a single scalar value as opposed to how to apply that kind of splitting logic for a column of values in a table.
I include both a numbers table solution and an XML solution. The XML solution was inspired by the earlier post priyanka.sarkar. I think that a numbers table solution, using an actual numbers table instead of the CTE as in the below solution is probably the fastest, but the XML approach deserves to be developed upon because it's really nice looking.
So, here goes my attempt.
CREATE PROCEDURE PARSE_DELIMITED_VALUES
AS
WITH FIRST_NUMBERS (N) AS (
SELECT 1 UNION ALL SELECT 1
), SECOND_NUMBERS (N) AS (
SELECT E1.N
FROM FIRST_NUMBERS E1
CROSS JOIN FIRST_NUMBERS E2
), THIRD_NUMBERS (N) AS (
SELECT E1.N
FROM SECOND_NUMBERS E1
CROSS JOIN SECOND_NUMBERS E2
), FOURTH_NUMBERS (N) AS (
SELECT E1.N
FROM THIRD_NUMBERS E1
CROSS JOIN THIRD_NUMBERS E2
), FIFTH_NUMBERS (N) AS (
SELECT E1.N
FROM FOURTH_NUMBERS E1
CROSS JOIN FOURTH_NUMBERS E2
), NUMBERS (N) AS (
SELECT N
FROM NUMBERS
WHERE N <= 8000 /*adjust these as needed to come up with a max number equal to the max character length allowed in the Values column*/
/*or better yet, if you can, just remove this first...numbers... header stuff so long as you create a temp or permanent table that contains the same numbers to work with*/
)
SELECT SUBSTRING(
MYTABLE.Values,
CASE
WHEN NUMBERS.NUMBER = 1 THEN 1
ELSE NUMBERS.NUMBER + 1
END,
CASE CHARINDEX(';', MYTABLE.Values, NUMBERS.NUMBER + 1)
WHEN 0 THEN LEN('^' + MYTABLE.Values + '^') - 2 + 1
ELSE CHARINDEX(';', MYTABLE.Values, NUMBERS.NUMBER + 1)
END
- CASE
WHEN NUMBERS.NUMBER = 1 THEN 1
ELSE NUMBERS.NUMBER + 1
END
) AS PARSED_VALUE
FROM MYTABLE
INNER JOIN NUMBERS
ON NUMBERS.NUMBER <= LEN('^' + MYTABLE.Values + '^') - 2
AND (
NUMBERS.NUMBER = 1
OR SUBSTRING(MYTABLE.Values, NUMBERS.NUMBER, 1) = ';'
)
GO
-- if your values column can contain NULL values I would change the join at the end as follows:
--from INNER JOIN NUMBERS
--to LEFT OUTER JOIN NUMBERS
The above would probably be most performant if the WITH NUMBERS ... CTEs were replaced by a temporary or permanent table containing the same numeric values.
On the other hand the CTE does the job and keeps it more in one place.
CREATE PROCEDURE PARSE_DELIMITED_VALUES
AS
SELECT E.x.value('.', 'VARCHAR(MAX)') AS PARSED_VALUE
FROM (
SELECT CAST('<x>' + REPLACE(Values, ';', '</x><x>') + '</x>' AS XML) my_x
FROM MYTABLE
) TT
CROSS APPLY my_x.nodes('/x') AS E(x)
GO
-- if your values column can contain NULL values I would change the join at the end as follows:
from `CROSS APPLY`
to `OUTER APPLY`
It's not the most elegant approach, but this might be worth a try. It creates a Sql Command as a string, and at the end executes it.
DECLARE #Values VARCHAR(8000)
-- Flatten all values lists into one string
SET #Values = REPLACE(REPLACE((SELECT [Value] FROM [dbo.MyTable] FOR XML PATH('')), '<Value>', ''), '</Value>', ';')
SET #Values = SUBSTRING(#Values, 0, LEN(#Values))
DECLARE #SeparatorIndex INT
SET #SeparatorIndex = (SELECT TOP 1 PATINDEX('%[;]%', #Values))
DECLARE #InsertClause VARCHAR(50)
SET #InsertClause = 'INSERT INTO [dbo.MyTable] VALUES ('
DECLARE #SQL VARCHAR(500)
SET #SQL = #InsertClause + SUBSTRING(#Values, 0, #SeparatorIndex) + '); '
SET #Values = RIGHT(#Values, LEN(#Values) - (#SeparatorIndex - 1))
SET #SQL = REPLACE(#SQL + (SELECT (REPLACE(#Values, ';', '); ' + #InsertClause))) + ')', '; )', '')
EXEC (#SQL)
The command ends up (in Sql Server 2005) as:
INSERT INTO [dbo.MyTable] VALUES (1); INSERT INTO [dbo.MyTable] VALUES (2); INSERT INTO [dbo.MyTable] VALUES (3); INSERT INTO [dbo.MyTable] VALUES (4); INSERT INTO [dbo.MyTable] VALUES (5) ...'
Do you actually mean, "rows," as in, "tuples," (so you can insert the data into another table, one element per row) or do you mean you want the data displayed vertically?
I'd think a string Replace (look up T-SQL's String Functions) would do the trick, no? Depending on the output target, you'd replace ; with CRLF or . You could even use Replace to create dynamic SQL Insert statements that could be executed by the SP to do row inserts (if that was your intent).
For presentation purposes, this is bad practice.
If it is purely for presentation and you are permitted, I'd output everything as XML then XSLT it any way you want. Honestly, I don't remember the last time I operated directly on a recordset. I always output to XML.