I need to remove redundant GO statements from a large SQL file before it gets passed through Invoke-sqlcmd for deployment.
Multiple GO statements together causes "There are no batches in the input script" and using -OutputSqlErrors $false masks all other errors.
Get-Unique deletes all duplicate data - which is not desirable. I would only like to delete the duplicate GO statements
Current Script:
Exec (#SQLScript)
Print #SQLScript
End
GO
GO
if obj is not null
drop procedure
go
CREATE PROC
#al varchar(16),
#rule varchar(128)
END CATCH
GO
GO
If Exists (Select * From Table)
Go
Set #Start = DateAdd(m, 1, #Start)
End
GO
GO
I would like to get a script like this:
Exec (#SQLScript)
Print #SQLScript
End
GO
if obj is not null
drop procedure
go
CREATE PROC
#al varchar(16),
#rule varchar(128)
END CATCH
GO
If Exists (Select * From Table)
Go
Set #Start = DateAdd(m, 1, #Start)
End
GO
If you load the script into a variable, you can use regular expressions to match and replace multiple "GO" statements. For ex:
$ReplacedText = $OriginalScript -replace '(GO(\n)*){2,}',"GO`n"
The Regular expression matches "GO" that may or may not be followed by a new line, 2 or more times. and replace it with a single "GO" followed by a new line.
Related
In SSIS 2008 I have a variable called #[User::EANcode] It contains a string with a product eancode like '1234567891123'. The value is derived from a filename like'1234567891123.jpg' via a foreach loop.
However, sometimes the filenames contain an extra '_1', '_2' etc. at the end like '1234567891123_1.jpg' resulting in a value '1234567891123_1' in the EANcode variable.
This happens when there is more than one image for the same EANcode (product). The _N addition is always a number and it is always at the end of the name/string.
What is the expression to find/cath the '_1' (or_2 or_N etc) so you can store it in another variable called #[User::Addition]?
If there is no addition, the variable stays empty which is fine.
The reason I need to get this _N addition into a separate variable is that I later on need it to rename the filename but paste the addition back at the end.
Thanks!
I think you're looking for CHARINDEX() in conjunction with SUBSTRING(). With that, you can split off that _# to another variable like this (copy/pasta and execute to see. Play with the #temp1 variable to see the limitations of the code):
declare #temp1 varchar(20), #temp2 varchar(20)
set #temp1 = '1234567891123_12'
IF CHARINDEX('_', #temp1) > 1
set #temp2 = SUBSTRING(#temp1,CHARINDEX('_', #temp1),LEN(#temp1)-CHARINDEX('_',#temp1)+1)
select #temp1, #temp2
Hope it helps!
Fairly new to DB2 sql, so forgive my ignorance :)
I have a trigger with a condition inside. I want to then insert some params depending on the condition.. Here it is:
I've looked at DB2 documentation for triggers and also for if statements, and at least to my eyes it appears to comply with it, however i get a -104 error (Illegal symbol token) on the insert line.
The insert works fine provided i use values not from 'N'.
OK, it works if i have nothing in the if then statement.. but only if i have nothing!
Thanks
My guess would be that DB2 is confused by your statement terminators. You use a semicolon to terminate the CREATE TRIGGER statement, but at the same time you use the same terminator inside the CREATE TRIGGER statement, after the INSERT.
In whatever client you use to execute your code, redefine the statement terminator to something else and place that terminator at the end of CREATE TRIGGER, after END ID. For example, if you use the command line processor, save this to a file trig.sql:
CREATE OR REPLACE TRIGGER AUTO_INSERT_DEPO_NOMINEE
AFTER INSERT ON CA_ENTITLEMENT
REFERENCING NEW AS N
FOR EACH ROW
IF NOT EXISTS (SELECT D.DEPO_CD FROM DEPO D WHERE D.DEPO_CD = N.DEPO_CD)
THEN
INSERT INTO DEPO (DEPO_CD, DEPO_NME, BRANCH_CD, AUTO_GENERATED)
VALUES(N.DEPO_CD, NULL, N.BRANCH_CD, 'Y');
END IF#
then run it, specifying "#" as the statement terminator:
db2 -td# -f mytrig.sql
Or: What is not a T-SQL statement?
Except to resolve ambiguity, T-SQL syntax does not require a semicolon to terminate a statement. Despite this, Itzik Ben-Gan recommends using a semicolon to terminate a T-SQL statement because it makes code cleaner, more readable, easier to maintain, and more portable.
I don't know a precise definition of what a valid T-SQL statement is, so I might be confused here. But as far as I know, a BEGIN...END block is a T-SQL statement, so should be terminated by a semicolon. For example:
IF OBJECT_ID('tempdb.dbo.#TempTable') IS NOT NULL
BEGIN
DROP TABLE #TempTable;
END;
The code example in Microsoft's BEGIN...END documentation supports this conjecture:
USE AdventureWorks2008R2;
GO
BEGIN TRANSACTION;
GO
IF ##TRANCOUNT = 0
BEGIN
SELECT FirstName, MiddleName
FROM Person.Person WHERE LastName = 'Adams';
ROLLBACK TRANSACTION;
PRINT N'Rolling back the transaction two times would cause an error.';
END;
ROLLBACK TRANSACTION;
PRINT N'Rolled back the transaction.';
GO
/*
Rolled back the tranaction.
*/
Itzik Ben-Gan contradicts this in the code example of Excercise 1-1 of T-SQL Fundamentals:
SET NOCOUNT ON;
USE TSQLFundamentals2008;
IF OBJECT_ID('dbo.Nums', 'U') IS NOT NULL DROP TABLE dbo.Nums;
CREATE TABLE dbo.Nums(n INT NOT NULL PRIMARY KEY);
DECLARE #i AS INT = 1;
BEGIN TRAN
WHILE #i <= 100000
BEGIN
INSERT INTO dbo.Nums VALUES(#i);
SET #i = #i + 1;
END
COMMIT TRAN
SET NOCOUNT OFF;
Microsoft's Transact-SQL Syntax Conventions document states that the semicolon "will be required in a future version" of T-SQL.
Commenting on Microsoft's intention to require the semicolon in a future version of T-SQL, Itzik notes some exceptions that aren't supposed to be terminated:
So far it was a requirement to use a semicolon only in specific cases. Now it looks like the plan is to make it a required terminator for all* T-SQL statements in some future version of SQL Server.
(*) Naturally there are cases that aren’t supposed to be terminated with a semicolon; those include (but are not limited to):
BEGIN
BEGIN TRAN
IF
ELSE
WHILE
BEGIN TRY
END TRY
BEGIN CATCH
Itzik seems to be consistent with himself, but Microsoft itself does not follow his recommendations. Compare Microsoft's BEGIN TRANSACTION; and Itzik's BEGIN TRAN in the previous examples.
In the code I maintain, I have seen even the BEGIN keyword terminated by semicolon:
IF #HasWidget = 0x1
BEGIN;
SELECT WidgetID
FROM tbWidgets;
END;
I believe a T-SQL parser may consider the semicolon following the BEGIN keyword to terminate an empty statement rather than terminate the BEGIN keyword itself; I don't believe that BEGIN itself is a valid T-SQL statement.
This conjecture is supported by the fact that SQL Server 2008 successfully parses and executes the following query:
SELECT 0;;
It's so confusing because there is no widely available specification of the T-SQL language, like the Java Language Specification for Java, so nowhere is there a formal definition of a T-SQL statement.
Am I wrong? Does such a specification exist for T-SQL, and is it publicly available?
Otherwise, should just I believe what Itzik says?
T-SQL syntax does not require a semicolon to terminate a statement.
Actually, this is deprecated1. I can't remember for sure, but I think you can still get away with not using them in the upcoming SQL Server 2012, but some version after that will likely require a semi-colon for every statement. Using a semi-colon is also technically required by the ansi standard. The point is that now is the time to get in the habit of using one for every statement.
As a practical matter, I don't expect them to follow through with this directly. Rather, I expect SQL Server Management Studio and other development tools to first start issuing warnings instead of errors, perhaps for several versions. This will help developers find and fix all the old non-compliant code. But that doesn't lessen the message: semi-colons are coming, and soon.
For a simple heuristic on when not to use a semi-colon, think of the code as if it were a procedural language that used curly brackets for blocks, like C/C++. Statements that would be paired with an opening (not closing) curly bracket if written in the procedure language should not get a semi-colon.
1It's almost all the way at the bottom of the page
Summary, based on the OP's original, quoted list.
Yes semi-colon:
BEGIN TRAN;
No semi-colon:
BEGIN
IF
ELSE
WHILE
BEGIN TRY
END TRY
BEGIN CATCH
Also, use them after END and END CATCH.
Details:
BEGIN TRAN is a statement and should be terminated with a semi-colon.
Microsoft's documentation notes the optional semi-colon:
BEGIN { TRAN | TRANSACTION }
[ { transaction_name | #tran_name_variable }
[ WITH MARK [ 'description' ] ]
]
[ ; ]
Microsoft's example has semi-colons:
BEGIN TRAN T1;
UPDATE table1 ...;
BEGIN TRAN M2 WITH MARK;
UPDATE table2 ...;
SELECT * from table1;
COMMIT TRAN M2;
UPDATE table3 ...;
COMMIT TRAN T1;
Both of the above are from:
https://msdn.microsoft.com/en-us/library/ms188929(v=sql.90).aspx
They match the current documentation:
https://msdn.microsoft.com/en-us/library/ms188929(v=sql.120).aspx
As for BEGIN...END, the Microsoft documentation does not provide clear guidance.
The definition has no semi-colon:
BEGIN
{
sql_statement | statement_block
}
END
However, their example shows a semi-colon after END:
IF ##TRANCOUNT = 0
BEGIN
SELECT FirstName, MiddleName
FROM Person.Person WHERE LastName = 'Adams';
ROLLBACK TRANSACTION;
PRINT N'Rolling back the transaction two times would cause an error.';
END;
https://msdn.microsoft.com/en-us/library/ms190487.aspx
That trailing semi-colon is not consistent with Microsoft's own documentation for IF control of flow language construct:
IF Boolean_expression
{ sql_statement | statement_block }
[ ELSE
{ sql_statement | statement_block } ]
Neither that definition nor their code example shows any semi-colon:
DECLARE #compareprice money, #cost money
EXECUTE Production.uspGetList '%Bikes%', 700,
#compareprice OUT,
#cost OUTPUT
IF #cost <= #compareprice
BEGIN
PRINT 'These products can be purchased for less than
$'+RTRIM(CAST(#compareprice AS varchar(20)))+'.'
END
ELSE
PRINT 'The prices for all products in this category exceed
$'+ RTRIM(CAST(#compareprice AS varchar(20)))+'.'
https://msdn.microsoft.com/en-us/library/ms182717(v=sql.110).aspx
However, their ELSE documentation, while also not showing any semi-colon in the definition, does show one in the example, after the final END.
Definition:
IF Boolean_expression { sql_statement | statement_block }
[ ELSE { sql_statement | statement_block } ]
Example:
IF 1 = 1 PRINT 'Boolean_expression is true.'
ELSE PRINT 'Boolean_expression is false.' ;
https://msdn.microsoft.com/en-us/library/ms182587(v=sql.110).aspx
The ANSI standard doesn't resolve the ambiguity because these are non-standard extensions:
Control-of-flow statements are not covered by the ANSI SQL standard
because these are proprietary SQL extensions. The SQL Server Books
Online is sketchy on the subject and many of the examples (as of this
writing) are inconsistent and do not always include statement
terminators. Furthermore, control-of-flow statement blocks are
confusing due to the many variations, nesting, and optional BEGIN/END
specifications.
http://www.dbdelta.com/always-use-semicolon-statement-terminators/
However, the behavior of the server sheds some light. The following is not a syntax error in SQL Server 2005:
DECLARE #foo int;
IF #foo IS NULL
BEGIN
WITH Blah AS
(
SELECT
'a' AS a
)
SELECT
a
FROM Blah;
END
So the BEGIN itself does not require a semi-colon. However, the following does produce a syntax error in SQL Server 2005:
DECLARE #foo int;
IF #foo IS NULL
BEGIN
WITH Blah AS
(
SELECT
'a' AS a
)
SELECT
a
FROM Blah;
END
WITH Blah2 AS
(
SELECT
'a' AS a
)
SELECT
a
FROM Blah2;
The above results in this error:
Msg 319, Level 15, State 1, Line 13 Incorrect syntax near the keyword
'with'. If this statement is a common table expression or an
xmlnamespaces clause, the previous statement must be terminated with a
semicolon.
It also throws that error in SQL Server 2008 R2.
It gets even more confusing. Microsoft's documentation for TRY...CATCH shows an optional semi-colon after the END CATCH, and their examples are consistent with that.
BEGIN TRY
{ sql_statement | statement_block }
END TRY
BEGIN CATCH
[ { sql_statement | statement_block } ]
END CATCH
[ ; ]
However, if you have a CTE immediately after a BEGIN TRY, without a semi-colon, it will throw an error.
BEGIN TRY
WITH Blah AS
(
SELECT
'a' AS a
)
SELECT
a
FROM Blah;
END TRY
BEGIN CATCH
END CATCH
In SQL Server 2008 R2, the above batch throws this error:
Msg 319, Level 15, State 1, Line 2 Incorrect syntax near the keyword
'with'. If this statement is a common table expression, an
xmlnamespaces clause or a change tracking context clause, the previous
statement must be terminated with a semicolon.
The error implies that BEGIN TRY is a statement (which it isn't), and that a semi-colon "fixes" the issue (which it does). That's right, this works:
BEGIN TRY;
WITH Blah AS
(
SELECT
'a' AS a
)
SELECT
a
FROM Blah;
END TRY
BEGIN CATCH
END CATCH
However, Microsoft says that's not good practice:
Posted by Microsoft on 12/29/2009 at 12:11 PM I am resolving the
corresonding SQL11 bug as "by design". Here is the explanation:
The semicolon between END TRY and BEGIN CATCH should not be allowed,
because they are actually not different statements, but parts of the
same TRY-CATCH statement. We only allow semicolons when they separate
two statements in a sequence.
A word of explanation why then we allow semicolons after BEGIN TRY and
BEGIN CATCH. These keywords serve as opening "parentheses" that start
an embedded statement sequence. Semicolons after BEGIN TRY/BEGIN CATCH
get parsed as part of that embedded sequence, with the first statement
in the sequence being empty. While we allow this syntax, I would not
recommend it as a good coding practice because it creates a wrong
impression of BEGIN TRY/BEGIN CATCH being independent, standalone
statements.
The recommended way to handle that situation is with an extra BEGIN...END for clarity:
BEGIN TRY
BEGIN
WITH Blah AS
(
SELECT
'a' AS a
)
SELECT
a
FROM Blah;
END
END TRY
BEGIN CATCH
END CATCH
However, that END before the END TRY should probably have a semi-colon. After all, this will throw an error:
BEGIN TRY
BEGIN
WITH Blah AS
(
SELECT
'a' AS a
)
SELECT
a
FROM Blah;
END
WITH Blah2 AS
(
SELECT
'b' AS b
)
SELECT
b
FROM Blah2;
END TRY
BEGIN CATCH
END CATCH
Maybe always preceding a CTE WITH a semi-colon isn't so silly.
The only situation in which I frequently using a semicolon is when using Common Table Expressions via the WITH keyword - and only then because the WITH keyword must be preceded by a semicolon otherwise it returns an error. In those cases, I write
;WITH [exp]...
i.e. I precede the WITH with a semicolon, rather than terminate the previous statement.
Semicolon usage in SQL seems to be very rare; I occasionally see it after a stored procedure or function declaration by that is the exception rather than the rule. Of all the developers I've worked with I don't believe any have really used the semicolon in the way that you described.
Statements like
BEGIN;
SELECT WidgetID
FROM tbWidgets;
END;
are hard to understand - if BEGIN; is considered a statement independent of its corresponding END;, why is SELECT WidgetID not a valid statement independent of its corresponding FROM?
Was trying to datestamp the output filename - but keep getting errors -- along the lines of:
Select * from Orders
output to 'c:'+ select (CONVERT(varchar(10), GETDATE(), 120)) + 'orders.csv'
Any help appreciated...
I had similar problem on Sybase 9 - I had to write each procedure / function body inside spearate file named as that procedure. So I had to dynamically create file name using name of each procedure (or function). Below is my solution (works in isql):
begin
declare folder varchar (40);
declare fileName varchar(60);
-- folder parameter is our destination path - the 4 backslashes are important here
set folder = 'c:\\\\test\\\\sql\\\\';
-- here we are iterating over all procedures / functions
for p as curs dynamic scroll cursor for
Select distinct sysobjects.name
from sysobjects inner join syscomments
on sysobjects.id = syscomments.id where sysobjects.type = 'P'
do
-- each procedure must be inside separate file named like that procedure
set fileName = folder + name + '.sql';
-- finally, there are answer to original question:
-- we are exporting data into file, whose name is defined dynamically
unload select proc_defn from SYS.SYSPROCEDURE where proc_name = name
to fileName escapes off;
end for;
end
output to is a dbisql command, so it's interpreted on the client. This means that you can't use expressions for the filename, since they are executed on the server. However you can use the unload select statement (which does run on the server) with the into client file clause to do what you want.
See docs on the unload select statement here.
Disclaimer: I work for Sybase in SQL Anywhere engineering.
I have a stored procedure called "sp_BulkInsert" that inserts one .csv file into my database, where you specify the full path of the file when you execute it. I am trying to create another stored procedure called "sp_ResultsDump" where you specify the folder path, which then searches the folder for all .csv files, creates a table with file names, then loops through the rows of that table while executing "sp_BulkInsert" for each .csv file in the folder (the names of which are recorded in the previous table).
Here is the code:
--Step 0: Create Stored Procedure
CREATE PROCEDURE sp_ResultsDump
#PathFolder VARCHAR(2000)
AS
--Step 1: Create table of file names
IF EXISTS (SELECT 1
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE='BASE TABLE'
AND TABLE_NAME='Files')
DROP TABLE Files
CREATE TABLE Files(FileID INT IDENTITY NOT NULL, FileName VARCHAR(max))
DECLARE #PathExec VARCHAR(1000)
SET #PathExec = "dir '"+#PathFolder+"'.csv /B"
INSERT INTO Files(FileName) EXEC master..xp_cmdshell #PathExec
DELETE Files WHERE FileName IS NULL
--Step 2: Get # of files, declare and initialize iterator
DECLARE #RowCount INT, #I INT
SET #RowCount = (SELECT COUNT(FileName) FROM Files)
SET #I = 1
--Step 3: Loop through the rows of a table and execute sp_ResultsDump for each file
WHILE (#I <= #RowCount)
BEGIN
DECLARE #FileName VARCHAR(1000)
SELECT #FileName = FileName FROM Files WHERE FileID = #I
SELECT #FileName = #PathFolder+#FileName
EXEC sp_BulkInsert #FileName
SET #I = #I + 1
END
I have confirmed that Steps 1-3 work when I specify the folder (without creating a stored procedure or a dynamic #variable), however storing the #PathFolder seems to be the problem. For example, I want to grab all .csv files from C:\, and each #FileName through the loop will loop through the file names contained in table Files, column FileName.
What I want to do is to be able to execute the following code so that I can get all .csv files in a specified folder and successfully bulk insert them into my database:
EXEC sp_ResultsDump 'c:\'
The reason for this is because the folder path may change later, and I want the user to be able to specify it.
I believe that "SELECT #FileName = #PathFolder+#FileName" is incorrect, and I tried all sorts of combinations of quotation marks and +'s. Steps 1 and 3 both seem to have problems with #PathFolder.
I guess I just need help with my while loop, because I think if my while loop is correct, this should be good.
Any suggestions? Simple syntax error somewhere? Thanks in advance.
I think your problem is with the following SET command
SET #PathExec = "dir '"+#PathFolder+"'.csv /B"
It appears to be mixing the double-quotes and single-quotes. Try changing it to this
SET #PathExec = 'dir "' + #PathFolder + '.csv" /B'
SET #PathExec = 'dir "' + #PathFolder + '.csv" /B'
It appears to be missing *. Try changing it to this
SET #PathExec = 'dir "' + #PathFolder + '*.csv" /B'
Its working fine for me. Otherwise use this:
EXEC master..xp_cmdshell ‘DIR C:\inbox\*.csv /b’