I am trying to normalize my tables to make the db more efficient.
To do this I have removed several columns from a table that I was updating several columns on.
Here is the original query when all the columns were in the table:
UPDATE myActDataBaselDataTable
set [Correct Arrears 2]=(case when [Maturity Date]='' then 0 else datediff(d,convert(datetime,#DataDate, 102),convert(datetime,[Maturity Date],102)) end)
from myActDataBaselDataTable
Now I have removed [Maturity Date] from the table myActDataBaselDataTable and it's necessary to retrieve that column from the base reference table ACTData, where it is called Mat.
In my table myActDataBaselDataTable the Account number field is a concatenation of 3 fields in ACTData, thus
myActDataBaselDataTable.[Account No]=ac.[Unit] + ' ' + ac.[ACNo] + ' ' + ac.[Suffix]
(where ac is the alias for ACTData)
So, having looked at the answers given elsewhere on SO (such as 1604091: update-a-table-using-join-in-sql-server), I tried to modify this particular update statement as below, but I cannot get it right:
UPDATE myActDataBaselDataTable
set dt.[Correct Arrears 2]=(
case when ac.[Mat]=''
then 0
else datediff(d,convert(datetime,'2014-04-30', 102),convert(datetime,ac.[Mat],102))
end)
from ACTData ac
inner join myActDataBaselDataTable dt
ON dt.[Account No]=ac.[Unit] + ' ' + ac.[ACNo] + ' ' + ac.[Suffix]
I either get an Incorrect syntax near 'From' error, or The multi-part identifier "dt.Correct Arrears 2" could not be bound.
I'd be grateful for any guidance on how to get this right, or suugestiopns about how to do it better.
thanks
EDIT:
BTW, when I run the below as a SELECT it returns data with no errors:
select case when [ac].[Mat]=''
then 0
else datediff(d,convert(datetime,'2014-04-30', 102),convert(datetime,[ac].[Mat],102))
end
from ACTData ac
inner join myActDataBaselDataTable dt
ON dt.[Account No]=ac.[Unit] + ' ' + ac.[ACNo] + ' ' + ac.[Suffix]
In a join update, update the alias
update dt
What is confusing is that in later versions of SQL you don't need to use the alias in the update line
Related
Answered / Solved.
Long story short, I need a stored procedure that would get the data from a few different views and put it into a .CSV file. Easy enough, but me being me, I decided to write something that could get the data from any table one could potentially desire. I decided to go with 2 procedures in the end:
Loop through a table with all the parameters (catalog, schema, table name, export path/file name etc. etc.) and feed it to 2nd stored procedure (in theory it should make it easier to manage in future, if/when different data needs to be exported). This one is fairly straightforward and doesn't cause any issues.
Pick up the column names (which was surprisingly easy - below in case it helps anyone)
select #SQL = 'insert into Temp_Export_Headers ' +
'select COLUMN_NAME ' +
'from [' + #loc_Source_Database + '].information_schema.columns ' +
'where table_name = ''' + #loc_Source_Table + ''''
and
select #Headers = coalesce(#Headers + ',', '') + convert(varchar, Column_Name)
from Temp_Export_Headers
After that, I want to dump all the data from "actual" table into temp one, which in itself is easy enough, but that's where things start to go downhill for me.
select #SQL =
'drop table if exists TempData ' +
'select * ' +
'into TempData ' +
'from [' + #loc_Source_Database + '].' + #loc_Source_Schema + '.' + #loc_Source_Table + ' with (nolock) '
Select * is just temporary, will probably replace it with a variable later on, for now it can live in this state on dev.
Now I want to loop through TempData and insert things I want (everything at the moment, will add some finesse and where clauses in near future) and put it into yet another temp table that holds all the stuff for actual CSV export.
Is there any way to add a self incrementing column to my TempData without having to look for and get rid of the original PK / Identity? (Different tables will have different values / names for those, making it a bit of a nightmare for someone with my knowledge / experience to loop through in a sensible manner, so I'd just like a simple column starting with 1 and ending with whatever last row number is)
#ShubhamPandey 's answer was exactly what I was after, code below is a product of my tired mind on the verge of madness (It does, however, work)
select #SQL =
'alter table TempData ' +
'add Uni_Count int'
select #SQL2 =
'declare #UniCount int ' +
'select #UniCount = 0 ' +
'update tempdata with (rowlock) ' +
'set #UniCount = Uni_Count = #UniCount + 1'
Both versions execute quicker than select * into without any other manipulation. Something I cannot yet comprehend.
Is there a better / more sensible way of doing this? (My reasoning with the loop - there will potentially be a lot of data for some of the tables / views, with most of them executed daily, plan was to export everything on Sat/Sun when system isn't that busy, and have daily "updates" going from last highest unique id to current.)
Looping was a horrible idea. To illustrate just how bad it was:
Looping through 10k rows meant execution time of 1m 21s.
Not looping through 500k rows resulted in execution time of 56s.
Since you are doing a table creation while insertion, you can always go forward with a statement like:
select #SQL =
'drop table if exists TempData ' +
'select ROW_NUMBER() OVER (<some column name>) AS [Id], * ' +
'into TempData ' +
'from [' + #loc_Source_Database + '].' + #loc_Source_Schema + '.' + #loc_Source_Table + ' with (nolock) '
This would create an auto-incrementing index for you in the TempData table
Here's a simple query we do for ad hoc requests from our Marketing department on the leads we received in the last 90 days.
SELECT ID
,FIRST_NAME
,LAST_NAME
,ADDRESS_1
,ADDRESS_2
,CITY
,STATE
,ZIP
,HOME_PHONE
,MOBILE_PHONE
,EMAIL_ADDRESS
,ROW_ADDED_DTM
FROM WEB_LEADS
WHERE ROW_ADDED_DTM BETWEEN #START AND #END
They are asking for more derived columns to be added that show the number of previous occurences of ADDRESS_1 where the EMAIL_ADDRESS matches. But they want is for different date ranges.
So the derived columns would look like this:
,COUNT_ADDRESS_1_LAST_1_DAYS,
,COUNT_ADDRESS_1_LAST_7_DAYS
,COUNT_ADDRESS_1_LAST_14_DAYS
etc.
I've manually filled these derived columns using update statements when there was just a few. The above query is really just a sample of a much larger query with many more columns. The actual request has blossomed into 6 date ranges for 13 columns. I'm asking if there's a better way then using 78 additional update statements.
I think you will have a hard time writing a query that includes all of these 78 metrics per e-mail address without actually creating a query that hard-codes the different choices. However you can generate such a pivot query with dynamic SQL, which will save you some keystrokes and will adjust dynamically as you add more columns to the table.
The result you want to end up with will look something like this (but of course you won't want to type it):
;WITH y AS
(
SELECT
EMAIL_ADDRESS,
/* aggregation portion */
[ADDRESS_1] = COUNT(DISTINCT [ADDRESS_1]),
[ADDRESS_2] = COUNT(DISTINCT [ADDRESS_2]),
... other columns
/* end agg portion */
FROM dbo.WEB_LEADS AS wl
WHERE ROW_ADDED_DTM >= /* one of 6 past dates */
GROUP BY wl.EMAIL_ADDRESS
)
SELECT EMAIL_ADDRESS,
/* pivot portion */
COUNT_ADDRESS_1_LAST_1_DAYS = *count address 1 from 1 day ago*,
COUNT_ADDRESS_1_LAST_7_DAYS = *count address 1 from 7 days ago*,
... other date ranges ...
COUNT_ADDRESS_2_LAST_1_DAYS = *count address 2 from 1 day ago*,
COUNT_ADDRESS_2_LAST_7_DAYS = *count address 2 from 7 days ago*,
... other date ranges ...
... repeat for 11 more columns ...
/* end pivot portion */
FROM y
GROUP BY EMAIL_ADDRESS
ORDER BY EMAIL_ADDRESS;
This is a little involved, and it should all be run as one script, but I'm going to break it up into chunks to intersperse comments on how the above portions are populated without typing them. (And before long #Bluefeet will probably come along with a much better PIVOT alternative.) I'll enclose my interspersed comments in /* */ so that you can still copy the bulk of this answer into Management Studio and run it with the comments intact.
Code/comments to copy follows:
/*
First, let's build a table of dates that can be used both to derive labels for pivoting and to assist with aggregation. I've added the three ranges you've mentioned and guessed at a fourth, but hopefully it is clear how to add more:
*/
DECLARE #d DATE = SYSDATETIME();
CREATE TABLE #L(label NVARCHAR(15), d DATE);
INSERT #L(label, d) VALUES
(N'LAST_1_DAYS', DATEADD(DAY, -1, #d)),
(N'LAST_7_DAYS', DATEADD(DAY, -8, #d)),
(N'LAST_14_DAYS', DATEADD(DAY, -15, #d)),
(N'LAST_MONTH', DATEADD(MONTH, -1, #d));
/*
Next, let's build the portions of the query that are repeated per column name. First, the aggregation portion is just in the format col = COUNT(DISTINCT col). We're going to go to the catalog views to dynamically derive the list of column names (except ID, EMAIL_ADDRESS and ROW_ADDED_DTM) and stuff them into a #temp table for re-use.
*/
SELECT name INTO #N FROM sys.columns
WHERE [object_id] = OBJECT_ID(N'dbo.WEB_LEADS')
AND name NOT IN (N'ID', N'EMAIL_ADDRESS', N'ROW_ADDED_DTM');
DECLARE #agg NVARCHAR(MAX) = N'', #piv NVARCHAR(MAX) = N'';
SELECT #agg += ',
' + QUOTENAME(name) + ' = COUNT(DISTINCT '
+ QUOTENAME(name) + ')' FROM #N;
PRINT #agg;
/*
Next we'll build the "pivot" portion (even though I am angling for the poor man's pivot - a bunch of CASE expressions). For each column name we need a conditional against each range, so we can accomplish this by cross joining the list of column names against our labels table. (And we'll use this exact technique again in the query later to make the /* one of past 6 dates */ portion work.
*/
SELECT #piv += ',
COUNT_' + n.name + '_' + l.label
+ ' = MAX(CASE WHEN label = N''' + l.label
+ ''' THEN ' + QUOTENAME(n.name) + ' END)'
FROM #N as n CROSS JOIN #L AS l;
PRINT #piv;
/*
Now, with those two portions populated as we'd like them, we can build a dynamic SQL statement that fills out the rest:
*/
DECLARE #sql NVARCHAR(MAX) = N';WITH y AS
(
SELECT
EMAIL_ADDRESS, l.label' + #agg + '
FROM dbo.WEB_LEADS AS wl
CROSS JOIN #L AS l
WHERE wl.ROW_ADDED_DTM >= l.d
GROUP BY wl.EMAIL_ADDRESS, l.label
)
SELECT EMAIL_ADDRESS' + #piv + '
FROM y
GROUP BY EMAIL_ADDRESS
ORDER BY EMAIL_ADDRESS;';
PRINT #sql;
EXEC sp_executesql #sql;
GO
DROP TABLE #N, #L;
/*
Now again, this is a pretty complex piece of code, and perhaps it can be made easier with PIVOT. But I think even #Bluefeet will write a version of PIVOT that uses dynamic SQL because there is just way too much to hard-code here IMHO.
*/
I am summer intern new to T-SQL and I have to run an sql select statement on various databases. What I would like to do is use 'if exists' to keep an error from occuring because some of the databases on the list to have this statement executed on no longer exist. However, I cannot figure out how to apply it to my statement. Any help would be greatly appreciated. Below is the statment me and another intern wrote:
select distinct mg.MatterName, mg.ClientNumber, mg.MatterNumber,grp.groupName as SecurityGroup
from (select distinct mat.matterName, mat.clientNumber, mat.matterNumber, usr.GroupID
from <db_name>.dbo.matter mat
inner join <db_name>.dbo.usrAccount usr
on usr.NTlogin=mat.matterCreateBy) as mg
inner join <db_name>.dbo.usrGroup grp
on mg.groupID=grp.groupID
order by matterName
the < db_name> is where the passed in parameter that is the name of the database, would go.
You could use sp_MSforeachdb to enumerate all of the databases on the instance.
This would be similar to:
exec sp_MSforeachdb 'select distinct mg.MatterName, mg.ClientNumber, mg.MatterNumber,grp.groupName as SecurityGroup from (select distinct mat.matterName, mat.clientNumber, mat.matterNumber, usr.GroupID from ?.dbo.matter mat inner join ?.dbo.usrAccount usr on usr.NTlogin=mat.matterCreateBy) as mg inner join ?.dbo.usrGroup grp on mg.groupID=grp.groupID order by matterName'
Alternatively, you could use dynamic sql to manufacture a script:
select 'use ' + name + ';' + char(13) + 'select distinct mg.MatterName, mg.ClientNumber, mg.MatterNumber,grp.groupName as SecurityGroup' +CHAR(13) + 'from (select distinct mat.matterName, mat.clientNumber, mat.matterNumber, usr.GroupID' + char(13) + 'from dbo.matter mat' + char(13) + 'inner join dbo.usrAccount usr on usr.NTlogin=mat.matterCreateBy) as mg' + char(13) + 'inner join dbo.usrGroup grp on mg.groupID=grp.groupID' + CHAR(13) + 'order by matterName;'
from master.sys.databases where database_id>4
If you redirect your output to "Results to Text" in SSMS then run the script, you will see a script written that you can then put into a query editor to execute.
I got it to work. I think this is a bit hackey but what I did was catch the exception thrown and just change a label on the page to reflect that the database doesnt exist.
DataAccess dal = new DataAccess();
dal.SelectedConnectionString = "WebServer08";
String exNetName = Request.QueryString["name"];
if (exNetName != null && !exNetName.Equals(""))
{
try
{
gvMatters.DataSource = dal.GetMatters(exNetName);
gvMatters.DataBind();
}
catch (Exception ex)
{
noDB.Text = "This database doesn't exist.";
gvMatters.Visible = false;
}
}
And i just left the SQL statement the way it was rather than try to screw around with it
I wish to write a Query for SAP B1 (t-sql) that will list all Income and Expenses Items by total and month by month.
I have successfully written a Query using PIVOT, but I do not want the column headings to be hardcoded like: Jan-11, Feb-11, Mar-11 ... Dec-11.
Rather I want the column headings to be parametrically generated, so that if I input:
--------------------------------------
Query - Selection Criteria
--------------------------------------
Posting Date greater or equal 01.09.10
Posting Date smaller or equal 31.08.11
[OK] [Cancel]
the Query will generate the following columns:
Sep-10, Oct-10, Nov-10, ..... Aug-11
I guess DYNAMIC PIVOT can do the trick.
So, I modified one SQL obtained from another forum to suit my purpose, but it does not work. The error message I get is Incorrect Syntax near 20100901.
Could anybody help me locate my error?
Note: In SAP B1, '[%1]' is an input variable
Here's my query:
/*Section 1*/
DECLARE #listCol VARCHAR(2000)
DECLARE #query VARCHAR(4000)
-------------------------------------
/*Section 2*/
SELECT #listCol =
STUFF(
( SELECT DISTINCT '],[' + CONVERT(VARCHAR, MONTH(T0.RefDate), 102)
FROM JDT1
FOR XML PATH(''))
, 1, 2, '') + ']'
------------------------------------
/*Section 3*/
SET #query = '
SELECT * FROM
(
SELECT
T0.Account,
T1.GroupMask,
T1.AcctName,
MONTH(T0.RefDate) as [Month],
(T0.Debit - T0.Credit) as [Amount]
FROM dbo.JDT1 T0
JOIN dbo.OACT T1 ON T0.Account = T1.AcctCode
WHERE
T1.GroupMask IN (4,5,6,7) AND
T0.[Refdate] >= '[%1]' AND
T0.[Refdate] <= '[%2]'
) S
PIVOT
(
Sum(Amount)
FOR [Month] IN ('+#listCol+')
) AS pvt
'
--------------------------------------------
/*Section 4*/
EXECUTE (#query)
I don't know SAP, but a couple of things spring to mind:
It looks like you want #listCol to contain a collection of numbers within square brackets, for example [07],[08],[09].... However, your code appears not to put a [ at the start of this string.
Try replacing the lines
T0.[Refdate] >= '[%1]' AND
T0.[Refdate] <= '[%2]'
with
T0.[Refdate] >= ''[%1]'' AND
T0.[Refdate] <= ''[%2]''
(I also added a space before the AND in the first of these two lines while I was editing your question.)
I am trying to total by account, the expenses & income per account. There are multiples of both the income & the expenses per account. I am struggling with this as am still learning SQL and thought that someone else likely has already addressed this? I sure would appreciate the help!
I know that this SQL server code is not correct but it at least gives a bit clearer picture of what I am attempting to do.
IF(SELECT(OBJECT_ID('TEMPDB..#Total'))) IS NOT NULL DROP TABLE #Total
declare #Expenses decimal(13,2),
#income decimal(13,2)
set #expenses = sum(EXP_CHILD_CARE_AMOUNT)
+ sum(EXP_FOOD_AMOUNT)
+ sum(EXP_LIFE_INSURANCE_AMOUNT)
+ sum(EXP_TRANSPORTATION_AMOUNT)
+ sum(EXP_TUITION_AMOUNT)
+ sum(EXP_USER_2_AMOUNT)
+ sum(EXP_USER_3_AMOUNT)
+ sum(EXP_UTILITIES_AMOUNT)
set #income = (sum(NET_PAY_AMOUNT)
+ sum(OTHER_INCOME_AMOUNT)
SELECT F.LOAN_NUMBER, #Income, #Expenses
INTO #Total
FROM OPENQUERY(SvrLink, '
SELECT F.Account, #Income, #Expenses
FROM finances F
inner join account a on(a.Account = f.Account)
where a.balance > 0
FETCH ONLY WITH UR ')
...ok, assuming that some fields are grouped into the same table (if not you'll just have to write the joins), you just need 1 query. (I wish all languages were as concise...)
#AccountId is your desired account id.
SELECT l.LOAN_NUMBER, l.AccountId,
(SELECT sum(EXP_CHILD_CARE_AMOUNT) + sum(EXP_FOOD_AMOUNT) +
sum(EXP_LIFE_INSURANCE_AMOUNT) + sum(EXP_TRANSPORTATION_AMOUNT) +
sum(EXP_TUITION_AMOUNT) + sum(EXP_USER_2_AMOUNT) +
sum(EXP_USER_3_AMOUNT) + sum(EXP_UTILITIES_AMOUNT)
as ExpenseTotal FROM Expenses_Guessing_The_Table_Name
WHERE AccountId = #AccountId) as ExpenseTotal,
(SELECT sum(NET_PAY_AMOUNT) + sum(OTHER_INCOME_AMOUNT) as IncomeTotal
FROM Income_Guessing_The_Table_Name
WHERE AccountId = #AccountId) as IncomeTotal
FROM Loans l
WHERE l.AccountId = #AccountId AND l.Balance > 0