Is ELT possible in Datastage by using join,aggregate components ( without running directly a SQL statement from a database stage or by calling a stored procedure?)
Example:
In oracle db, i have a sql as shown below.
In IBM datastage, is it possible to run this sql with etl components( without running directly a SQL statement from a database stage or by calling a stored procedure)
insert into trg1
select
cust_expense.cust_id, cust.cust_name,
cust_expense.month, sum(cust_expense.expense)
from cust_expense, cust
group by
cust_expense.cust_id, cust.cust_name,
cust_expense.month
Related
PostgreSQL offers a way to query a remote database through dblink.
Similarly (sort-of), Exasol provides a way to connect to a remote Postgres database via the following syntax:
CREATE CONNECTION JDBC_PG
TO 'jdbc:postgresql://...'
IDENTIFIED BY '...';
SELECT * FROM (
IMPORT FROM JDBC AT JDBC_PG
STATEMENT 'SELECT * FROM MY_POSTGRES_TABLE;'
)
-- one can even write direct joins such as
SELECT
t.COLUMN,
r.other_column
FROM MY_EXASOL_TABLE t
LEFT JOIN (
IMPORT FROM JDBC AT JDBC_PG
STATEMENT 'SELECT key, other_column FROM MY_POSTGRES_TABLE'
) r ON r.key = t.KEY
This is very convenient to import data from PostgreSQL directly into Exasol without having to use a temporary file (csv, pg_dump...).
Is it possible to achieve the same thing from Snowflake (querying a remote PostgreSQL database from Snowflake with a direct live connection)? I couldn't find any mention of it in the documentation.
Have you looked into using external functions? It's not exactly what you're looking for (Snowflake doesn't have that capability yet) but it can be used as a workaround in some use cases. For instance, you could create a Python function on AWS Lambda that queries PostgreSQL for small amounts of data (due to Lambda limits) or have it trigger a PostgreSQL process that dumps to S3 to trigger Snowpipe for the bulk import use case.
We have a select query as below . Query to fetch the data is running more than 5 hours.
select ColumnA,
ColumnB,
ColumnC,
ColumnD,
ColumnE
from Table
where CodeN <> 'Z'
Is there any way we can collect stats or any other way to improve performance .. ?
And in DB2 do we have any table where we can check whether collect stats are collected on the below table..
The RUNSTATS command collects table & indexes statistics. Note, that this is a Db2 command and not an SQL statement, so you may either run it with Db2 Command Line Processor (CLP) or using relational interface with a special Stored Procedure, which is able to run such commands:
RUNSTATS command using the ADMIN_CMD procedure.
Statistics is stored in the SYSSTAT schema views. Refer to Road map to the catalog views - Table 2. Road map to the updatable catalog views.
How many rows exist in table?
and not equal operator '<>' not indexable predicates
Currently I am using H2 for integration test and DB2 for DEV and UAT.
To get the next value from a sequence, we are using Values NEXTVAL FOR SCHEMANAME.SEQNAME SQL for Db2 and which is working as expected.
However the above SQL not working in the H2 Database and we are using Select NEXTVAL ('SCHEMANAME','SEQNAME') in H2. This is not working in Db2
However I need to use same SQL query or both DB2 and H2 to get NEXTVAL.
Can we use one SQL to get NEXTVAL in both H2 and DB2?
The SQL Standard has a NEXT VALUE FOR sequenceGeneratorName expression. This expression is supported by both databases:
https://www.ibm.com/support/knowledgecenter/en/SSEPEK_12.0.0/sqlref/src/tpc/db2z_sequencereference.html
https://h2database.com/html/grammar.html#sequence_value_expression
You can test it with
VALUES NEXT VALUE FOR SCHEMANAME.SEQNAME;
Note that you need a recent version of H2 (1.4.199, for example), because old versions of H2 don't support table value constructor (VALUES …).
I need to submit a series of queries to an Oracle server over ODBC from an MS SQL server and store the results as a table on the MS SQL server.
It has to be a pass through because the query requires a server side function defined on the Oracle server.
I can't save the table on the Oracle server and then access it via ODBC because of licensing restrictions from the vendor of the db running on Oracle.
Here's the code that returns the correct results, but I don't know how to save them:
DECLARE #BibID AS bigint
DECLARE BibList CURSOR FOR
SELECT BIB_ID FROM tblActiveSerialsThatHave740s
OPEN BibList
FETCH NEXT FROM BibList INTO #BibID
WHILE ##FETCH_STATUS=0
BEGIN
EXECUTE
('SELECT
AMDB.BIB_DATA.BIB_ID As BIB_ID,
AMDB.GetAllBibTag(AMDB.BIB_DATA.BIB_ID, ''740'', ''2'') As F740_All
FROM
AMDB.BIB_DATA
WHERE
AMDB.BIB_DATA.BIB_ID = ' + #BibID + '
GROUP BY BIB_ID '
)
AT REPORT
FETCH NEXT FROM BibList INTO #BibID
END
DEALLOCATE BibList
You need to use INSERT INTO to capture the results of an EXECUTE.
Because you are executing a passthrough query, the Distributed Transaction Coordinator is going to come into play and you might need to ensure that a distributed transaction is not created (it's unlikely to be necessary to have a distributed transaction in your case) or that the Distributed Transaction Coordinator service is running:
http://blogs.msdn.com/b/sqlprogrammability/archive/2008/08/22/how-to-create-an-autonomous-transaction-in-sql-server-2008.aspx
http://technet.microsoft.com/en-us/library/ms178532.aspx
http://www.sqlservercentral.com/Forums/Topic861249-392-1.aspx
I want to execute multiple statements from my data access layer using C# dan IBM's DB2 data provider.
(environment: DB2/AS400 os version: V5R4)
eg in TSQL:
declare varA integer;
select varA= count(*) from tableA;
select * from tableB where col1 <= varA
with SQL server ; I can concatenate those 3 statements into a string
and assign the text to DBCommand.CommandText.
How to execute multiple statements(compound statement) against DB2 database via DBCommand (using IBM DB2 data provider)
I tried using begin and end block but still failed
BEGIN
statement1;
statement2;
statement3;
END
Thank you
I do not think it's possible.
I had already tried something similar some time ago, and the only solution I found is to dynamically create a stored procedure, calling it, and finally delete it.