I have very simple SSIS package which brings 8 columns from PostgreSQL into SQL server. There are only 10 rows feeding through. Now requirement changed by adding another
- join
- with extra column to bring back.
- Also one where clause condition to be removed.
When changed and preview using OLE DB source editor it throws error 'Value does not fall within the expected range'
script runs successfully in PostgreSQL within 40 sec(though it should not take that long).
In 'OLE DB source editor:
When adding Join - it previews(successfully)
when adding Join and bringing column - it previews (successfully)
when adding Join, bringing column and removing condition ( condition let it bring 149 records instead of 10 ) - (throws error)
I guess it is very basic error and someone with experience will be able to spot it quickly. Will be very thankful for kind and prompt response..
I have checked datatype for new column source and destination and it matches. character (255)
I have run script using PGAdmin 4 and it runs successfully in PostgreSQL.
.
I am using a Java based program and I am writing a simple select query inside that program to retrieve data from the PostgreSQL database. The data come with the header which is an error for the rest of my codes.
How do I get rid of all column headings in an SQL query? I just want to
print out the raw data without any headings.
I am using Building Controls Virtual Test Bed (BCVTB) to connect my database to EnergyPlus. This BCVTB has a database actor that you can write a query in it and receive data and send it to your other simulation program. I decided to use PostgreSQL. however when I write Select * From mydb, it brings data with the column names (header). I just want raw data without header. what should I do?
PostgreSQL does not send table headings, not like a CSV file. The protocol (as used via JDBC) sends the rows. The driver does request a description of the rows that includes column names, but it is not part of the result set rows like the "header first" convention for CSV.
Whatever is happening must be a consequence of the BCVTB tools you are using, and I suggest pursuing it on that side of things.
I want to execute a SQL file with multiple queries inside it using Talend.
To my understanding of Talend there is no straight forward way of doing it.
My SQL File contains multiple Select Statement related to Various tables.
Ex: Select age,name from Table_AB; Select count(Salary), emp_id from Emp_table;
I have referred http://bekwam.blogspot.in/2011/02/running-file-of-sql-statements-with.html ; The queries get executed but i want to output the results of the tOracleRow to an output file or log file and send the file to an Email recipient.
What i Have Done till now is: Talend Job to execute a SQL file with multiple queries
Can Anyone please help me on how to Output the results of an tOracleRow to an output file and mail the results to an Email recipient. I am able to do it for a single table with a schema , but confused on how to do it for various Tables and output it
I have a CSV file and a database table as inputs. I need to keep all the records that are in the database table but exclude those that are in the CSV file. It all gets written to a CSV file in the end.
All my attempts have resulted in hundreds of duplicates so I'm rather challenged here.
This is a quick-and-dirty trick, but it should fit your needs.
Connect your database and your CVS input to a TMap- Use your DB as main connection and CSV as Lookup. Something like:
tMySqlInput---->Main--->tMap---out---->tFileDelimitedOutput
^
|
|
Lookup
|
|
tFileDelimitedInput
Inside the tMap, map the join columns as usual. Click on the wrench just above the lookup mapping area and set the join type as Inner join. Be sure that "Match Model" is not set to All, to avoid cross joins. Then, create the "out" output connection. Finally, click the wrench of this connection and set "Catch lookup inner join reject" to true
On "out" (so, the output file) you will have only the unmatched rows from the DB. Depending on your data, you may need to add a tAggregateRow just before tFileDelimitedOutput to eventually clean duplicates.
I am currently experimenting with Tableau Extract API to generate some TDE from the tables I have in a PostgreSQL database. I was able to write a code to generate the TDE from single table, but I would like to do this for multiple joined tables. To be more specific, if I have two tables that are inner joined by some field, how would I generate the TDE for this?
I can see that if I am working with small number of tables, I could use a SQL query with JOIN clauses to create a one gigantic table, and generate the TDE from that table.
>> SELECT * FROM table_1 INNER JOIN table_2
INTO new_table_1
ON table_1.id_1 = table_2.id_2;
>> SELECT * FROM new_table_1 INNER JOIN TABLE_3
INTO new_table_2
ON new_table_1.id_1 = table_3.id_3
and then generate the TDE from new_table_2.
However, I have some tables that have over 40 different fields, so this could get messy.
Is this even a possibility with current version of the API?
You can read from as many tables or other sources as you want. Or use complex query with lots of joins, or create a view and read from that. Usually, creating a view is helpful when you have a complex query joining many tables.
The data extract API is totally agnostic about how or where you get the data to feed it -- the whole point is to allow you to grab data from unusual sources that don't have pre-built drivers for Tableau.
Since Tableau has a Postgres driver and can read from it directly, you don't need to write a program with the data extract API at all. You can define your extract with Tableau Desktop. If you need to schedule automated refreshes of the extract, you can use Tableau Server or its tabcmd command.
Many thanks for your replies. I am aware that I could use Tableau Desktop to define my extract. In fact, I have done this many times before. I am just trying to create the extracts using the API, because I need to create some calculated fields, which is near impossible to create using the Tableau Desktop.
At this point, I am hesitant to use JOINs in the SQL query because the resulting table would look too complicated to comprehend (some of these tables also have same field names).
When you say that I could read from multiple tables or sources, does that mean with the Tableau Extract API? At this point, I cannot find anywhere in this API that accommodates multiple sources. For example, I know that when I use multiple tables in the Tableau Desktop, there are icons on the left hand side that tells me that the extract is composed of multiple tables. This just doesn't seem to be happening with the API, which leaves me stranded. Anyways, thank you again for your replies.
Going back to the topic, this is something that I tried few days ago on my python code
try:
tdefile= tde.Extract("extract.tde")
except:
os.remove("extract.tde")
tdefile = tde.Extract("extract.tde")
tableDef = tde.TableDefinition()
# Read each column in table and set the column data types using tableDef.addColumn
# Some code goes here...
for eachTable in tableNames:
tableAdd = tdeFile.addTable(eachTable, tableDef)
# Use SQL query to retrieve bunch_of_rows from eachTable
for some_row in bunch_of_rows:
# Read each row in table, and set the values in each column position of each row
# Some code goes here...
tableAdd.insert(some_row)
some_row.close()
tdefile.close()
When I execute this code, I get the error that eachTable has to be called "Extract".
Of course, this code has its flaws, as there is no where in this code that tells how each table are being joined.
So I am little thrown off here, because it doesn't seem like I can use multiple tables unless I use JOINs to generate one table that contains everything.