When I export a SQL script from SQL developer (right click on DB entity like VIEW then "Quick DDL->Save to file" in context menu), SQL developer exports its SQL code with some prefixed comment like:
--------------------------------------------------------
-- File created - srijeda-rujna-21-2022
--------------------------------------------------------
--------------------------------------------------------
-- DDL for View THE_NAME_OF_MY_VIEW
--------------------------------------------------------
How can I change this (where is the template that SQL developer uses for this export)?
It's not anywhere you can get access to it, to change. So if you don't like it, generate the DDL yourself.
Thankfully, SQL Developer makes this easy with the 'DDL' command.
Simply do this in a SQL worksheet and run as a Script (F5)
ddl emp_details_view -- sub your view name, obviously
And the output, from our HR schema:
CREATE OR REPLACE FORCE EDITIONABLE VIEW "HR"."EMP_DETAILS_VIEW" ("EMPLOYEE_ID", "JOB_ID", "MANAGER_ID", "DEPARTMENT_ID", "LOCATION_ID", "COUNTRY_ID", "FIRST_NAME", "LAST_NAME", "SALARY", "COMMISSION_PCT", "DEPARTMENT_NAME", "JOB_TITLE", "CITY", "STATE_PROVINCE", "COUNTRY_NAME", "REGION_NAME") DEFAULT COLLATION "USING_NLS_COMP" AS
SELECT
e.employee_id,
e.job_id,
e.manager_id,
e.department_id,
d.location_id,
l.country_id,
e.first_name,
e.last_name,
e.salary,
e.commission_pct,
d.department_name,
j.job_title,
l.city,
l.state_province,
c.country_name,
r.region_name
FROM
employees e,
departments d,
jobs j,
locations l,
countries c,
regions r
WHERE e.department_id = d.department_id
AND d.location_id = l.location_id
AND l.country_id = c.country_id
AND c.region_id = r.region_id
AND j.job_id = e.job_id
WITH READ ONLY;
And if you want that in a file, just add this directly before the DDL
cd c:\path\to\file
spool view.sql
Related
I just noticed that I could alter my stored procedure code with a misspelled user defined function in it.
I noticed that at 1st time I execute the SP.
Is there any way to get a compile error when an SP include an invalid user-defined function name in it?
At compile time? No.
You can, however, use some of SQL's dependency objects (if using MS SQL) to find problems just after deployment, or as part of your beta testing. Aaron Bertran has a pretty nice article rounding up the options, depending upon the version of SQL Server.
Here is an example using SQL Server 2008 sys object called sql_expression_dependencies
CREATE FUNCTION dbo.scalarTest
(
#input1 INT,
#input2 INT
)
RETURNS INT
AS
BEGIN
-- Declare the return variable here
DECLARE #ResultVar int
-- Add the T-SQL statements to compute the return value here
SELECT #ResultVar = #input1 * #input2
-- Return the result of the function
RETURN #ResultVar
END
GO
--Fn Works!
SELECT dbo.ScalarTest(2,2)
GO
CREATE PROCEDURE dbo.procTest
AS
BEGIN
SELECT TOP 1 dbo.scalarTest(3, 3) as procResult
FROM sys.objects
END
GO
--Sproc Works!
EXEC dbo.procTest
GO
--Remove a dependency needed by our sproc
DROP FUNCTION dbo.scalarTest
GO
--Does anything have a broken dependency? YES
SELECT OBJECT_NAME(referencing_id) AS referencing_entity_name,
referenced_entity_name, *
FROM sys.sql_expression_dependencies
WHERE referenced_id IS NULL --dependency is missing
GO
--Does it work? No
EXEC dbo.procTest
GO
I am considering using RCP to run a generic datastage job, but the initial SQL changes each time it's called. Is there a process in which I can use a User Activity Variable to inject SQL from a text file or something so I can use the same datastage?
I know this Routine can read a file to look up parameters:
Routine = ‘ReadFile’
vFileName = Arg1
vArray = ”
vCounter = 0
OPENSEQ vFileName to vFileHandle
Else Call DSLogFatal(“Error opening file list: “:vFileName,Routine)
Loop
While READSEQ vLine FROM vFileHandle
vCounter = vCounter + 1
vArray = Fields(vLine,’,’,1)
vArray = Fields(vLine,’,’,2)
vArray = Fields(vLine,’,’,3)
Repeat
CLOSESEQ vFileHandle
Ans = vArray
Return Ans
But does that mean I just store the SQL in one Single line, even if it's long?
Thanks.
Why not just have the SQL within the routine itself and propagate parameters?
I have multiple queries within a single routine that does just that (one for source and one for AfterSQL statement)
This is an example and apologies I'm answering this on my mobile!
InputCol=Trim(pTableName)
If InputCol='Table1' then column='Day'
If InputCol='Table2' then column='Quarter, Day'
SQLCode = ' Select Year, Month, '
SQLCode := column:", Time, "
SQLCode := " to_date(current_timestamp, 'YYYY-MM-DD HH24:MI:SS'), "
SQLCode := \ "This is example text as output" \
SQLCode := "From DATE_TABLE"
crt SQLCode
I've used the multiple encapsulations in the example above, when passing out to a parameter make sure you check the ', " have either been escaped or are displaying correctly
Again, apologies for the quality but I hope it gives you some ideas!
You can give this a try
As you mentioned ,maintain the SQL in a file ( again , if the SQL keeps changing , you need to build a logic to automate populating the new SQL)
In the Datastage Sequencer , use a Execute Command Activity to open the SQL file
eg : cat /home/bk/query.sql
In the job activity which calls your generic job . you should map the command output of your EC activity to a job parameter
so if EC activity name is exec_query , then the job parameter will be
exec_query.$CommandOuput
When you run the sequence , your query will flow from
SQL file --> EC activity-->Parameter in Job activity-->DB stage( query parameterised)
Has you thinked to invoke a shellscript who connect to database and execute the SQL script from the sequential job? You could use sqlplus to connect in the shellscript and read the file with the SQL and use it. To execute the shellscript from the sequential job use a ExecCommand Stage (sh, ./, ...), it depends from the interpreter.
Other way to solve this, depends of the modification degree of your SQL; you could invoke a routine base who handle the parameters and invokes your parallel job.
The principal problem that I think you could have, is the limit of the long of the variable where you could store the parameter.
Tell me what option you choose and I could help you more.
I'm developing a database using the Red Gate SQL Developer tools. SQL Test, the SSMS add-in that runs tSQLt tests, lacks a way to rename test classes.
I have a test called [BackendLayerCustomerAdministrationTests].[test uspMaintainCustomerPermissions throws error when PermissionValue is missing or empty].
The name is so long it breaks Deployment Manager.
2013-12-05 18:48:40 +00:00 ERROR The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
There are other unwieldly test names in this class, so I want to start by shortening the class name.
A more succinct class name would be CustomerTests.
sp_rename is no help here.
EXECUTE sys.sp_rename
#objname = N'BackendLayerCustomerAdministrationTests',
#newname = N'CustomerTests';
Msg 15225, Level 11, State 1, Procedure sp_rename, Line 374
No item by the name of 'BackendLayerCustomerAdministrationTests' could be found in the current database 'ApiServices', given that #itemtype was input as '(null)'.
How do I change it?
tSQLt test classes are schemas with a special extended property.
Cade Roux's great solution for renaming schemas is to create a new schema, transfer all the objects, then drop the old schema.
If we did that here we'd lose the extended property.
Let's adapt it for the tSQLt framework.
How to rename a tSQLt test class
Create a new test class.
EXECUTE tSQLt.NewTestClass
#ClassName = 'CustomerTests';
You should see the old class and the new class together in the tSQLt.TestClasses view.
SELECT *
FROM tSQLt.TestClasses;
Name SchemaId
----------------------------------------- ----------
SQLCop 7
BackendLayerCustomerAdministrationTests 10
CustomerTests 14
Cade used Chris Shaffer's select variable concatenation trick to build a list of transfer statements, and print the result.
DECLARE #sql NVARCHAR(MAX) = N'';
SELECT #sql = #sql +
N'ALTER SCHEMA CustomerTests
TRANSFER BackendLayerCustomerAdministrationTests.' + QUOTENAME(name) + N';' +
CHAR(13) + CHAR(10)
FROM sys.objects
WHERE SCHEMA_NAME([schema_id]) = N'BackendLayerCustomerAdministrationTests';
PRINT #sql;
Ugly, but effective.
Copy the output and execute as a new query.
ALTER SCHEMA CustomerTests
TRANSFER BackendLayerCustomerAdministrationTests.[test uspMaintainCustomer validate merged data];
ALTER SCHEMA CustomerTests
TRANSFER BackendLayerCustomerAdministrationTests.[test uspMaintainCustomerPermissions throws error when PermissionValue is missing or empty];
I've shown only two tests here, but it should work for all of them.
Now drop the old test class.
EXECUTE tSQLt.DropClass
#ClassName = N'BackendLayerCustomerAdministrationTests';
The old class should be gone from view.
SELECT *
FROM tSQLt.TestClasses;
Name SchemaId
----------------------------------------- ----------
SQLCop 7
CustomerTests 14
Run all your tests again to check that it worked.
EXECUTE tSQLt.RunAll;
+----------------------+
|Test Execution Summary|
+----------------------+
|No|Test Case Name |Result |
+--+----------------------------------------------------------------------------+-------+
|1|[CustomerTests].[test uspMaintainCustomer throws error on missing APIKey] |Success|
|2|[CustomerTests].[test uspMaintainCustomerPermissions validate merged data] |Success|
|3|[SQLCop].[test Decimal Size Problem] |Success|
|4|[SQLCop].[test Procedures Named SP_] |Success|
|5|[SQLCop].[test Procedures using dynamic SQL without sp_executesql] |Success|
|6|[SQLCop].[test Procedures with ##Identity] |Success|
|7|[SQLCop].[test Procedures With SET ROWCOUNT] |Success|
-------------------------------------------------------------------------------
Test Case Summary: 7 test case(s) executed, 7 succeeded, 0 failed, 0 errored.
-------------------------------------------------------------------------------
Success!
Sorry to come into this so late! I'm a developer who's working on SQL Test.
We've just added the ability to rename test classes to the latest version of SQL Test.
http://www.red-gate.com/products/sql-development/sql-test/
It's now as simple as right clicking on the context menu for a test class, or pressing F2:
Please bear in mind that this option will not appear for old versions of tSQLt. To upgrade, right click on the database to uninstall the framework, then do Add database... to re-add it (the right-most button in the window):
Alternatively, you could just call a new procedure in tSQLt called tSQLt.RenameClass, which is what SQL Test calls behind the scenes.
Please let us know if you have any issues with this!
David
What is your workflow like? If you have all your tests for that test class in one script with exec tSQLt.NewTestClass 'BackendLayerCustomerAdministrationTests' then you can just find and replace the testclass name and you are done.
e.g.
EXEC tSQLt.DropClass 'BackendLayerCustomerAdministrationTests'
GO
EXEC tSQLt.NewTestClass 'CustomerTests'
GO
CREATE PROC [CustomerTests].[test_Insert_AddsACustomer]
AS
etc, etc
This will work because the EXEC tSQLt.NewTestClass 'CustomerTests' will drop all objects in the testclass and they will be recreated as the rest of the script runs.
Simplest is probably:
EXEC tSQLt.RenameClass 'old test class name', 'new test class name';
See the tSQLt docs for RenameClass
It seems Red-gate have added that ability to SQL Test since this question was posted, but the raw SQL code is somehow leaner and cleaner (whether or not you use the excellent SQL Test)
It looks like Psycopg has a custom command for executing a COPY:
psycopg2 COPY using cursor.copy_from() freezes with large inputs
Is there a way to access this functionality from with SQLAlchemy?
accepted answer is correct but if you want more than just the EoghanM's comment to go on the following worked for me in COPYing a table out to CSV...
from sqlalchemy import sessionmaker, create_engine
eng = create_engine("postgresql://user:pwd#host:5432/db")
ses = sessionmaker(bind=engine)
dbcopy_f = open('/tmp/some_table_copy.csv','wb')
copy_sql = 'COPY some_table TO STDOUT WITH CSV HEADER'
fake_conn = eng.raw_connection()
fake_cur = fake_conn.cursor()
fake_cur.copy_expert(copy_sql, dbcopy_f)
The sessionmaker isn't necessary but if you're in the habit of creating the engine and the session at the same time to use raw_connection you'll need separate them (unless there is some way to access the engine through the session object that I don't know). The sql string provided to copy_expert is also not the only way to it, there is a basic copy_to function that you can use with subset of the parameters that you could past to a normal COPY TO query. Overall performance of the command seems fast for me, copying out a table of ~20000 rows.
http://initd.org/psycopg/docs/cursor.html#cursor.copy_to
http://docs.sqlalchemy.org/en/latest/core/connections.html#sqlalchemy.engine.Engine.raw_connection
If your engine is configured with a psycopg2 connection string (which is the default, so either "postgresql://..." or "postgresql+psycopg2://..."), you can create a psycopg2 cursor from an SQL Alchemy session using
cursor = session.connection().connection.cursor()
which you can use to execute
cursor.copy_from(...)
The cursor will be active in the same transaction as your session currently is. If a commit or rollback happens, any further use of the cursor with throw a psycopg2.InterfaceError, you would have to create a new one.
You can use:
def to_sql(engine, df, table, if_exists='fail', sep='\t', encoding='utf8'):
# Create Table
df[:0].to_sql(table, engine, if_exists=if_exists)
# Prepare data
output = cStringIO.StringIO()
df.to_csv(output, sep=sep, header=False, encoding=encoding)
output.seek(0)
# Insert data
connection = engine.raw_connection()
cursor = connection.cursor()
cursor.copy_from(output, table, sep=sep, null='')
connection.commit()
cursor.close()
I insert 200000 lines in 5 seconds instead of 4 minutes
It doesn't look like it.
You may have to just use psycopg2 to expose this functionality and forego the ORM capabilities. I guess I don't really see the benefit of ORM in such an operation anyway since it's a straight bulk insert and dealing with individual objects a la an ORM would not really make a whole lot of sense.
If you're starting from SQLAlchemy, you need to first get to the connection engine (also known by the property name bind on some SQLAlchemy objects):
engine = create_engine('postgresql+psycopg2://myuser:password#localhost/mydb')
# or
engine = session.engine
# or any other way you know to get to the engine
From the engine you can isolate a psycopg2 connection:
# get a psycopg2 connection
connection = engine.connect().connection
# get a cursor on that connection
cursor = connection.cursor()
Here are some templates for the COPY statement to use with cursor.copy_expert(), a more complete and flexible option than copy_from() or copy_to() as it is indicated here: https://www.psycopg.org/docs/cursor.html#cursor.copy_expert.
# to dump to a file
dump_to = """
COPY mytable
TO STDOUT
WITH (
FORMAT CSV,
DELIMITER ',',
HEADER
);
"""
# to copy from a file:
copy_from = """
COPY mytable
FROM STDIN
WITH (
FORMAT CSV,
DELIMITER ',',
HEADER
);
"""
Check out what the options above mean and others that may be of interest to your specific situation https://www.postgresql.org/docs/current/static/sql-copy.html.
IMPORTANT NOTE: The link to the documentation of cursor.copy_expert() indicates to use STDOUT to write out to a file and STDIN to copy from a file. But if you look at the syntax on the PostgreSQL manual, you'll notice that you can also specify the file to write to or from directly in the COPY statement. Don't do that, you're likely just wasting your time if you're not running as root (who runs Python as root during development?) Just do what's indicated in the psycopg2's docs and specify STDIN or STDOUT in your statement with cursor.copy_expert(), it should be fine.
# running the copy statement
with open('/path/to/your/data/file.csv') as f:
cursor.copy_expert(copy_from, file=f)
# don't forget to commit the changes.
connection.commit()
You don't need to drop down to psycopg2, use raw_connection nor a cursor.
Just execute the sql as usual, you can even use bind parameters with text():
engine.execute(text('''copy some_table from :csv
delimiter ',' csv'''
).execution_options(autocommit=True),
csv='/tmp/a.csv')
You can drop the execution_options(autocommit=True) if this PR will be accepted
I am writing a PL/SQL stored procedure using Toad in Eclipse. I have it working just fine and the procedure is below:
CREATE OR REPLACE PROCEDURE crl_sync
IS
unit_separator CONSTANT char:= CHR(31);
record_separator CONSTANT char:= CHR(30);
CURSOR c_bc is
SELECT m.barcode, s.id, s.tracking_tags
FROM model m, sample s
WHERE m.id = s.id;
r_bc c_bc%ROWTYPE;
BEGIN
DBMS_OUTPUT.ENABLE(1000000);
DBMS_OUTPUT.PUT_LINE('--CRL_SYNC--');
OPEN c_bc;
LOOP
FETCH c_bc into r_bc;
EXIT WHEN c_bc%NOTFOUND;
DBMS_OUTPUT.PUT_LINE('barcode = '||r_bc.barcode);
END LOOP;
CLOSE c_bc;
END;
/
It compiles and executes just fine; note the DBMS_OUTPUT.ENABLE line and the put_line at the start to rule out a problem with the query but still nothing in the toad window in Eclipse. Can someone please provide a pointer as to what I'm doing wrong here?
Thanks.
[To show this question as answered]
Try the recommendation on this page:
There are two editors for Oracle, called "SQL Worksheet" and "Stored
Procedure Editor", you can open them with icons in the Connections
View. DBMS Output view works with SQL WORKSHEET. So, go to the DBMS
output view and enable it with the "bulb" icon. Now, open the SQL
Worksheet.