Microsoft Report Builder with Postgresql and multiple CTEs - postgresql

I have a problem creating DataSets with CTE in the current Microsoft Report Designer.
My DataSource is a PostgreSQL Database and I use the current Version of PGNP to connect to.
The connection works fine, I can create DataSets based on Queries and get my data.
BUT if I try to create more complex queries, like for example with multiple CTE, I won't get any fields.
Exampe:
SELECT 'Test' as test
Will return one column (test) with one row (Test)
WITH Test as (SELECT 'Test' as test)
SELECT Test
, 'Success' as result
FROM Test
Will return two columns (test, result) with one row (Test, Success)
Till now, everything is ok.
But if I want to use more then one CTE, I have to face the following problems:
WITH Test as (SELECT 'Test' as test),
TestB as (SELECT 'Test B' as test)
SELECT ta.test, tb.test, 'Success' as result
FROM Test ta, TestB tb
Will fail with the following error message:
"Undefined table Test"
;WITH Test as (SELECT 'Test' as test),
TestB as (SELECT 'Test B' as test)
SELECT ta.test, tb.test, 'Success' as result
FROM Test ta, TestB tb
Will return no columns and no rows.
Any idea why this happens?
Edit A:
Here a (shorternd) Version of my actual query:
WITH calls AS (
SELECT acv.id
, acv.created
, acv.state
, acv.statistics_category
, acv.duration
, aqv.id as queue_id
, aqv.name as queue_name
, acv.target
, acv.target_name
, acv.queue_time
, acv.ringing_duration
, acv.hold_time
FROM acd_call_view acv
LEFT JOIN acd_queue_view aqv ON (aqv.id = acv.fk_acdqueue_id)
ORDER BY acv.id, acv.created
),
calls_answered as (
SELECT *
FROM calls
WHERE statistics_category = 'answered'
)
SELECT * FROM calls_answered
Result: "Unknown table calls"
Edit B:
Sorry, had two leftovers from the shortening remaining in the query
Edit C:
Tested Query in pgAdmin III: Working
Tested Query in JasperSoft Studio: Working
Tested Query in MS Report Builder with PGNP: "Undefined table calls"
Edit D:
As soon as I remove the second CTE, I get results
WITH calls AS (
SELECT acv.id
, acv.created
, acv.state
, acv.statistics_category
, acv.duration
, aqv.id as queue_id
, aqv.name as queue_name
, acv.target
, acv.target_name
, acv.queue_time
, acv.ringing_duration
, acv.hold_time
FROM acd_call_view acv
LEFT JOIN acd_queue_view aqv ON (aqv.id = acv.fk_acdqueue_id)
ORDER BY acv.id, acv.created
)
SELECT * FROM calls
Works fine, so I get results, the PGNP Ole DB Driver seems to work fine, the connection is up, everything is okay.
As soon, as I add the second CTE, I get the error

Works here (just plain psql terminal) , so the error must be in your framework/client application:
CREATE TABLE acd_call_view
( id INTEGER NOT NULL
, created timestamp
, state integer
, statistics_category text
, duration INTEGER
, target INTEGER
, target_name INTEGER
, queue_time INTEGER
, ringing_duration INTEGER
, hold_time INTEGER
, fk_acdqueue_id INTEGER
);
CREATE TABLE acd_queue_view
( id INTEGER NOT NULL
, name text
);
WITH calls AS (
SELECT acv.id
, acv.created
, acv.state
, acv.statistics_category
, acv.duration
, aqv.id as queue_id
, aqv.name as queue_name
, acv.target
, acv.target_name
, acv.queue_time
, acv.ringing_duration
, acv.hold_time
FROM acd_call_view acv
LEFT JOIN acd_queue_view aqv ON aqv.id = acv.fk_acdqueue_id
ORDER BY acv.id, acv.created
)
, calls_answered as (
SELECT *
FROM calls
WHERE statistics_category = 'answered'
)
SELECT * FROM calls_answered
;

Okay, here the solution:
Seems like the PGNP OleDB Driver cannot handle multiple CTEs.
Installed the ODBC Driver and configured a new DateSource via ODBC.
Now I get my results and am happy :-)
Nevertherless, thank you joop for your time.

I contacted PGNP support, and they responded quickly that the bugs in the CTEs handling were fixed in build 1.4.0.3425.

Related

How to use CASE clause (DB2) to display values from a different table?

I'm working in a bank so I had to adjust the column names and information in the query to fit the external web, so if there're any weird mistakes know it is somewhat fine.
I'm trying to use the CASE clause to display data from a different table, I know this is a workaround but due to certain circumstances I'm obligated to use it, plus it is becoming interesting to figure out if there's an actual solution.
The error I'm receiving for the following query is:
"ERROR [21000] [IBM][CLI Driver][DB2] SQL0811N The result of a scalar
fullselect, SELECT INTO statement, or VALUES INTO statement is more
than one row."
select bank_num, branch_num, account_num, client_id,
CASE
WHEN exists(
select *
from bank.services BS
where ACCS.client_id= BS.sifrur_lakoach
)
THEN (select username from bank.services BS where BS.client_id = ACCS.client_id)
ELSE 'NONE'
END username_new
from bank.accounts accs
where bank_num = 431 and branch_num = 170
EDIT:
AFAIK we're using DB2 v9.7:
DSN11015 - DB21085I Instance "DB2" uses "64" bits and DB2 code release "SQL09075" with
level identifier "08060107".
Informational tokens are "DB2 v9.7.500.702", "s111017", "IP23287", and Fix Pack "5".
Use listagg function to include all results.
select bank_num, branch_num, account_num, client_id,
CASE
WHEN exists(
select *
from bank.services BS
where ACCS.client_id= BS.sifrur_lakoach
)
THEN (select LISTAGG(username, ', ') from bank.services BS
where BS.client_id = ACCS.client_id)
ELSE 'NONE'
END username_new
from bank.accounts accs
where bank_num = 431 and branch_num = 170

Repeated use of parameter for multiple UDF's in FROM throws an invalid column name error

When using multiple table-valued functions in a query like beneath, SSMS throws an error. Also, the [Date] parameter of [PRECALCPAGES_asof] is underlined in red.
I am trying to understand why this fails. I think this might be related to the way the SQL Server engine works. Have looked into documentation on MSDN but unfortunately I do not know what to look for. Why is this caused and is there a way around it?
Query
SELECT
[Date]
, COUNT(*)
FROM
[Warehouse].[dbo].[DimDate]
CROSS APPLY
[PROJECTS_asof]([Date])
INNER JOIN
[PRECALCPAGES_asof]([Date]) ON [PRECALCPAGES_asof].[PROJECTID] = [PROJECTS_asof].[PROJECTID]
GROUP BY
[Date]
Error
Msg 207, Level 16, State 1, Line 9
Invalid column name 'Date'.
Functions
CREATE FUNCTION [ProfitManager].[PROJECTS_asof]
(
#date DATETIME
)
RETURNS TABLE AS
RETURN
(
SELECT
[PROJECTID]
, [PROJECT]
, ...
FROM
Profitmanager.[PROJECTS_HISTORY]
WHERE
[RowStartDate] <= #date
AND
[RowEndDate] > #date
)
GO
CREATE FUNCTION [ProfitManager].[PRECALCPAGES_asof]
(
#date DATETIME
)
RETURNS TABLE AS
RETURN
(
SELECT
[PAGEID]
, [PAGENAME]
, ...
FROM
Profitmanager.[PRECALCPAGES_HISTORY]
WHERE
[RowStartDate] <= #date
AND
[RowEndDate] > #date
)
GO
I think you can't use fields from tables as parameters to a function in a join. You should use cross apply.
SELECT
[Date]
, COUNT(*)
FROM
[Warehouse].[dbo].[DimDate]
CROSS APPLY
[PROJECTS_asof]([Date])
CROSS APPLY
[PRECALCPAGES_asof]([Date])
WHERE
[PRECALCPAGES_asof].[PROJECTID] = [PROJECTS_asof].[PROJECTID]
GROUP BY
[Date]

Is there a JDFTVAL equivalent for SQL?

For Iseries/IBMi DB2.
I am joining multiple files/tables together.
I have written the code in both DDS and SQL.
The DDS Logical File is working exactly as expected, but I can not use it for embedded sql in rpgle as it then defaults to the SQE engine resulting in horrendous performance.
The SQL view, on the other hand had NULLs until I used IFNULL( MBRDESCR, ''). But now MBRDECSR is a VARCHAR. Which is unacceptable.
So how do I create a SQL join without NULLs and VARCHARs?
Requested Sample Code:
DDS:
JDFTVAL
R TRANSR JFILE(TRANSPF MBRPF)
J JOIN(1 2)
JFLD(MBRID MBRID)
*
TRANSID JREF(1)
MBRID JREF(1)
MBRNAME JREF(2)
MBRSURNME JREF(2)
*
K TRANSID
K MBRID
SQL:
CREATE VIEW TRANSV01 AS (
SELECT TRANSID ,
MBRID ,
CAST(IFNULL(MBRNAME , '') as Char(20)) ,
CAST(IFNULL(MBRSURNME, '') as Char(25))
FROM TRANSPF
--Member Name
LEFT OUTER JOIN MBRPF on MBRID = MBRID
) RCDFMT TRANSR;
Please note the following:
Example above is simplified
Not every MBRID in the TRANSPF has a corresponding entry in the MBRPF (ie. no referential constraint). Thus when MBRPF is joined to the TRANSPF, there will be NULL values in MBRNAME, MBRSURNME. Unless JDFTVAL or IFNULL() is used.
I prefer not to have a VARCHAR, because of performance and extname() in rpgle.
I prefer not to have NULL values, I do not want the pgm to have to handle them.
Assuming it's the 'Allows the null value' that you find undesirable, use a UNION. The first SELECT chooses all the rows that match, which will set the NOT NULL property for you. The second SELECT chooses all the rows that don't have a match - you provide filler fields for those.
CREATE VIEW TRANSV01 AS (
SELECT TRANSID ,
MBRID ,
MBRNAME ,
MBRSURNME
FROM TRANSPF
--Member Name
JOIN MBRPF on MBRID = MBRID
UNION
SELECT TRANSID ,
MBRID ,
CAST('') as Char(20)) ,
CAST('') as Char(25))
FROM TRANSPF
--Member Name
EXCEPTION JOIN MBRPF on MBRID = MBRID
) RCDFMT TRANSR;

How to keep the following TSQL query from running my server at 100%?

The following queries run in an sproc targeting the ItemData table in SQL Server 2008R2:
SELECT TOP(500) ItemListID, GeoCity, GeoState, GeoDisplay, Title, Link, Description, CleanDescription, OptimizedDescription, PubDateParsed, ImageBytes, DateAdded FROM ( SELECT TOP(500) ItemListID, GeoCity, GeoState, GeoDisplay, Title, Link, Description, CleanDescription, OptimizedDescription, PubDateParsed, ImageBytes, DateAdded, ROW_NUMBER()
OVER( ORDER BY ItemListID DESC )
AS RowNumber
FROM ItemData
WHERE CONTAINS(Title, #FTSSearchTerm ) -- ' + #OriginalSearchTerm + '"')
AND ( WebsiteID=1 AND
(#GeoCity = '-1' OR GeoCity = #GeoCity) AND
(#GeoState = '-1' OR GeoState = #GeoState) )
) ItemData WHERE RowNumber >= ( #PageNum - 1) * #PageSize + 1 AND RowNumber <= #PageNum * #PageSize ORDER BY ItemListID DESC
SELECT #NumberOfResultsReturned = ##ROWCOUNT
SELECT #ActualNumberOfResults = COUNT(*) FROM ItemData WHERE CONTAINS(Title, #FTSSearchTerm ) -- ' + #OriginalSearchTerm + '"') AND ( WebsiteID=1 AND (#GeoCity = '-1' OR GeoCity = #GeoCity) AND (#GeoState = '-1' OR GeoState = #GeoState) )
Depending on the data the query uses either CONTAINS or FREETEXT.
With load this query runs very slow and peeks the server at 100%.
I have set the following indexes:
What do I need to do so these queries stop running so hot?
Thanks.
-- UPDATE --
The table has one clustered index which only consists of ItemListID, and FTS on Title and Description.
I have added a non-clustered index (incorrectly named in the Identity name) as follows:
Without actually looking into the execution plan, it seems like you need a non-clustered index on Title, GeoCity, GeoState, and WebsiteID with the following include columns: ItemListID, GeoDisplay, Link, Description, CleanDescription, OptimizedDescription, PubDateParsed, ImageBytes, DateAdded
This will allow the execution plan to use the one non-clustered index that contains all of the information you are looking for in this query. Without it, it will use one of the indexes you showed and still have to go to the table to get the data you need.
This won't totally fix your problem though, depending on how much data is in your table, doing the Contains on Title to do searching will always be expensive. It would be best if you could leverage full text searching to do the searching portion.
Hopefully this helps!

Update column to be different aggregate values

I am creating a script that for "merging" and deleting duplicate rows from a table. The table contains address information, and uses an integer field for storing information about the email as bit flags (column name lngValue). For example, lngValue & 1 == 1 means its the primary address.
There are instances of the same email being entered twice, but sometimes with different lngValues. To resolve this, I need to take the lngValue from all duplicates and assign them to one surviving record and delete the rest.
My biggest headache so far as been with the "merging" of the records. What I want to do is bitwise or all lngValues of duplicate records together. Here is what I have so far, which only finds the value of all lngValues bitwise or'ed together.
Warning: messy code ahead
declare #duplicates table
(
lngInternetPK int,
lngContactFK int,
lngValue int
)
insert into #duplicates (lngInternetPK, lngContactFK, lngValue)
(
select tblminternet.lngInternetPK, tblminternet.lngContactFK, tblminternet.lngValue from tblminternet inner join
(select strAddress, lngcontactfk, count(*) as count from tblminternet where lngValue & 256 <> 256 group by strAddress, lngcontactfk) secondemail
On tblminternet.strAddress = secondemail.strAddress and
tblminternet.lngcontactfk = secondemail.lngcontactfk
where count > 1 and tblminternet.strAddress is not null and tblminternet.lngValue & 256 <> 256 --order by lngContactFK, strAddress
)
update #duplicates set lngValue = t.val
from
(select (sum(dupes.lngValue) & 65535) as val from
(select here.lngInternetPK, here.lngContactFK, here.lngValue from tblminternet here inner join
(select strAddress, lngcontactfk, count(*) as count from tblminternet where lngValue & 256 <> 256 group by strAddress, lngcontactfk) secondemail
On here.strAddress = secondemail.strAddress and
here.lngcontactfk = secondemail.lngcontactfk
where count > 1 and here.strAddress is not null and here.lngValue & 256 <> 256) dupes, tblminternet this
where this.lngContactFK = dupes.lngContactFK
) t
where lngInternetPK in (select lngInternetPK from #duplicates)
Edit:
As requested here is some sample data:
Table Name: tblminternet
Column Names:
lngInternetPK
lngContactFK
lngValue
strAddress
Example row 1:
lngInternetPK: 1
lngContactFK: 1
lngValue: 33
strAddress: "me#myaddress.com"
Example row 2:
lngInternetPK: 2
lngContactFK: 1
lngValue: 40
strAddress: "me#myaddress.com"
If these two were merged here is the desired result:
lngInternetPK: 1
lngContactFK: 1
lngValue: 41
strAddress: "me#myaddress.com"
Other necessary rules:
Each contact can have multiple emails, but each email row must be distinct ( each email can only appear as one row).
SQL Server lacks native bitwise aggregates, that's why we need to emulate them.
The main idea here is to generate a set of bits from 0 to 15, for each bit apply the bitmask to the value and select MAX (which will give us an OR for a given bit), then select the SUM (which will merge the bit masks).
The we just update the first lngInternetPK for any given (lngContactFK, strValue) with the new value of lngValue, and delete all duplicates.
;WITH bits AS
(
SELECT 0 AS b
UNION ALL
SELECT b + 1
FROM bits
WHERE b < 15
),
v AS
(
SELECT i.*,
(
SELECT SUM(value)
FROM (
SELECT MAX(lngValue & POWER(2, b)) AS value
FROM tblmInternet ii
CROSS JOIN
bits
WHERE ii.lngContactFK = i.lngContactFK
AND ii.strAddress = i.strAddress
GROUP BY
b
) q
) AS lngNewValue
FROM (
SELECT ii.*, ROW_NUMBER() OVER (PARTITION BY lngContactFK, strAddress ORDER BY lngInternetPK) AS rn
FROM tblmInternet ii
) i
WHERE rn = 1
)
UPDATE v
SET lngValue = lngNewValue;
;WITH v AS
(
SELECT ii.*, ROW_NUMBER() OVER (PARTITION BY lngContactFK, strAddress ORDER BY lngInternetPK) AS rn
FROM tblmInternet ii
)
DELETE v
WHERE rn > 1
See this article in my blog for more detailed explanations:
SQL Server: aggregate bitwise OR
I believe the following query gets you what you want. This routine assumes a max of two duplicate addresses per contact. If there's more than one dup per contact, the query will have to be modified. I hope this helps.
Declare #tblminternet
Table
( lngInternetPK int,
lngContactFK int,
lngValue int,
strAddress varchar(255)
)
Insert Into #tblminternet
select 1, 1, 33, 'me#myaddress.com'
union
select 2, 1, 40, 'me#myaddress.com'
union
select 3, 2, 33, 'me#myaddress2.com'
union
select 4, 2, 40, 'me#myaddress2.com'
union
select 5, 3, 2, 'me#myaddress3.com'
--Select * from #tblminternet
Select Distinct
A.lngContactFK ,
A.lngValue | B.lngValue as 'Bitwise OR',
A.strAddress
From #tblminternet A, #tblminternet B
Where A.lngContactFK = B.lngContactFK
And A.strAddress = B.strAddress
And A.lngInternetPK != B.lngInternetPK
You can create SQL Server Aggregate functions in .NET that you can then implement in SQL server inline. I think this requires a minimum of SQL server 2005 and Visual Studio 2010. I did one using Visual Studio 2013 Community Edition (free even for commercial use) for use with .NET 2 and SQL Server 2005.
See the MSDN article: https://msdn.microsoft.com/en-us/library/91e6taax(v=vs.90).aspx
First you'll need to enable the CLR feature in SQL server: https://msdn.microsoft.com/en-us/library/ms131048.aspx
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'clr enabled', 1;
GO
RECONFIGURE;
GO
Create a SQL Server -> SQL Server Database Project
Right-click on the new project and select Properties
Configure the targeted SQL Server version under Project Settings
Configure the targeted CLR language under SQL CLR (such as VB)
Right-click on the new project and select Add -> New Item...
When the dialog pops up, select SQL Server -> SQL CLR VB -> SQL CLR VB Aggregate
Now you can write your bitwise code in VB:
Imports System
Imports System.Data
Imports System.Data.SqlClient
Imports System.Data.SqlTypes
Imports Microsoft.SqlServer.Server
<Serializable()> _
<Microsoft.SqlServer.Server.SqlUserDefinedAggregate(Format.Native)> _
Public Structure AggregateBitwiseOR
Private CurrentAggregate As SqlTypes.SqlInt32
Public Sub Init()
CurrentAggregate = 0
End Sub
Public Sub Accumulate(ByVal value As SqlTypes.SqlInt32)
'Perform Bitwise OR against aggregate memory
CurrentAggregate = CurrentAggregate OR value
End Sub
Public Sub Merge(ByVal value as AggregateBitwiseOR)
Accumulate(value.Terminate())
End Sub
Public Function Terminate() As SqlInt32
Return CurrentAggregate
End Function
End Structure
Now deploy it: https://msdn.microsoft.com/en-us/library/dahcx0ww(v=vs.90).aspx
Build the project using the menu bar: Build -> Build ProjectName (if the build fails with error 04018 then download a new version of the data tools # http://msdn.microsoft.com/en-US/data/hh297027 or by going to the menu bar: Tools -> Extensions And Updates, then under updates select update for Microsoft SQL Server Update For Database Tooling)
Copy your compiled DLL to C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn and to C:\
Register the DLL:
CREATE ASSEMBLY [CLRTools] FROM ‘c:CLRTools.dll’ WITH PERMISSION_SET = SAFE
Create the aggregate in SQL:
CREATE AGGREGATE [dbo].[AggregateBitwiseOR](#value INT)
RETURNS INT
EXTERNAL NAME [CLRTools].[CLRTools.AggregateBitwiseOR];
If you get the error "Incorrect syntax near 'EXTERNAL'" then change the database compatibility level using following commands:
For SQL Server 2005: EXEC sp_dbcmptlevel 'DatabaseName', 90
For SQL Server 2008: EXEC sp_dbcmptlevel 'DatabaseName', 100
Test your code:
SELECT dbo.AggregateBitwiseOR(Foo) AS Foo FROM Bar
I found this article helpful: http://www.codeproject.com/Articles/37377/SQL-Server-CLR-Functions