I'm new to postgres so this problem is probably a relatively easy one for someone else. However, I have spent many frustrating hours trying to figure out the solution. I have an Access Database of metadata that must be kept updated for sending records to other groups. I also have a database using PostgreSQL and PGAdmin which also has these same metadata tables. Currently these tables in the Postgres database get updated manually by exporting the Access tables as excel files, and then importing them into the SQL tables. It's not the most efficient process and could lead to errors in the SQL database if someone forgets to check before running any queries that they are using the most recent data from Access. So I would like to integrate some of the tables from my Access database with my Postgres database.
Originally I tried just installing drivers to export the Access tables directly to Postgres which worked, but not in the way that I wanted since it just brings in a table which I would still need to manually update. From my understanding, I can create a server connection in postgres to access and that would then bring in updated data using a foreign data wrapper.
I tried to use ogr_fdw.
CREATE EXTENSION ogr_fdw;
When I try:
CREATE SERVER metadata
FOREIGN DATA WRAPPER ogr_fdw
OPTIONS (
datasource 'H:\Databases\20170712.accdb',
format 'ODBC' );
I receive: ERROR: unable to connect to data source "H:\Databases\20170712.accdb"
SQL state: HV00D
When I try:
CREATE SERVER metadata
FOREIGN DATA WRAPPER ogr_fdw
OPTIONS (
datasource 'H:\Databases\20170712.accdb',
format 'ACCDB' );
I receive: ERROR: unable to find format "ADDCB"
HINT: See the formats list at http://www.gdal.org/ogr_formats.html.
I also tried MDB and received the same error. However, MDB is the code name given by the website but it says that it needs JDK/JRE to compile and I'm not really sure if that's another type of driver that I would need or what it is.
When I try:
CREATE SERVER metadata
FOREIGN DATA WRAPPER ogr_fdw
OPTIONS (
datasource 'H:\Databases\20170712.mdb',
format 'ODBC' );
I receive: ERROR: unable to connect to data source "H:\Databases\20170712.mdb"
SQL state: HV00D
Hint: Unable to initialize ODBC connection to DSN for DRIVER=Microsoft Access Driver (*.mdb);DBQ=H:\Databases\20170712.mdb,
[Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified
However I thought after looking at the github help page for ogr_fdw didn't need ODBC and special drivers https://github.com/pramsey/pgsql-ogr-fdw/blob/master/FAQ.md.
A lot of this is probably due to my limited knowledge of the terminology when I'm reading through a lot of this stuff. Also my Access database is an .accdb file but since that wasn't working I tried around with mdb and ODBC as the "format" too. If anyone has any suggestions I would greatly appreciate it.
Thanks!
Related
I'm trying to setup an easily-replicable (or even manual and I do it once a month or so) process for moving data from a large Azure PostgreSQL database to a more manageable Azure SQL database for end users that are most familiar with SQL Server. I've successfully connected to the PostgreSQL database via PGAdmin, so I know all my connection string info.
I started by installing the latest ODBC driver from here.
I then used a connection string which was given to me from the Azure portal, filled in the proper database name and password, and attempted to use the following drivers:
PostgreSQL ODBC Driver(UNICODE)
PostgreSQL ODBC Driver(ANSI)
I am getting the following error with either of them:
ERROR [IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified
What step am I missing in this process? Or how best can I troubleshoot this?
After more research, I attempted to add the ODBC driver here:
And got the following error (I'm not sure why Tableau is relevant to this?):
Thank you.
I am trying to setup a CDC in Talend on an Oracle database. I am following the steps as listed here: https://www.talendforge.org/tutorials/tutorial.php?language=english&idTuto=42
However, when i try to establish the CDC in the main DB conneciton, i get a message saying "Database information must be filled to use the cdc!" and the Create Subscriber button is disabled.
What could I be doing differently? I have created two connections pointing to the same database, and have also retrieved the table schema in my main connection.
I need to export all tables from SQL Server to PostgreSQL.
Try: I tried from SQL Server IDE but at some stage its giving the error about data types are different.
Question:How can I do export of data from SQL Server to PostgreSQL? Is COPY does my job? If yes, then how can I export all tables including records?
You can't export data from MSsql then import to PostgreSql because it is not same syntax, data type, but you can use tool to migration data from mssql to postgreSql,
See more in topic
migrate data from MS SQL to PostgreSQL?
Use https://dbeaver.io/
Create MS SQL and PostgreSQL database connections (login)
Create target tables in PostgreSQL (same structures in MS SQL)
F5 to see new tables
Right-click on new tables -> 'Import Data' -> You will see 'Data Transfer' window
Choose 'Table' type then click 'Next' -> You will see 'Select input object', where you can choose tables from MS SQL connection
Just 'next' and check settings that you need, done :D
First export the schema into a file and run it against PostgreSQL until you've removed all incompatibilities.
You could try to do the same with the data you want to export but you may be better off writing a Python script to migrate it.
There is an absolutely simple way using built-in SSIS tool using Management Studio. You can find the detailed answer here.
Use https://dbeaver.io/ , as An Le mentioned.
After 40 years of DB development, migrating DB data is still a challenge. DBeaver is a free tool to use for data migration. But you still have to migrate the schema.
Exporting data from DBeaver
From contextual menu of your SQLServer database or schema select Tools > Create new Task > Common > Data Export
You will generate SQL insert files or CSV files. For migration between database types use CSV files.
Cons of SQL Server Migration Tool
Unable to migrate rows containing booleans.
Export ended up in errors of migrationg data with Bool columns, complaining that value is not boolean, although both source and destination columns where of boolean type.
Unable to continue with the next tables afer one table migration fails.
SQL Server - A single error stops all migration even for tables that are not related to the initial error.
Configuring the tool over and over again, trying to export your data is a waste of time. SQL Server migration task does not save the configuration of the source and destination connections. And the wizard is not user friendly, spending your time on it is frustrating. I assume the migration project was abandoned for at least 10 years.
I have the following code to generate the schema for a database in Nhibernate
new SchemaExport(configuration).Execute(true, true, false);
but when run against a PostgreSQL database, I end up getting the following error
[NpgsqlException (0x80004005): FATAL: 3D000: database "dbname" does not exist]
If I however create the database manually, the schema is exported without errors. An so the question: Why is Nhibernate SchemaExport unable to create a PostgreSQL database and yet this works against other databases like SQLite, MsSqlCe and MsSql Server.
I have searched for online literature but have been unable to locate any highlighting on this issue.
I am using Nhibernate 3.3.1 with PostgreSQL 9.2.
You must create the database before you can create the tables and other objects within the database.
Do this with a CREATE DATABASE statement on a PostgreSQL connection - either in your app, or via psql or PgAdmin-III.
PostgreSQL doesn't support creating databases on demand / first access. Perhaps that's what your tool is expecting?
If you think the DB does exist and you can see it in other tools, maybe you're not connecting to the same database server? Check the server address and port.
Currently, I have an application that uses Firebird in embedded mode to connect to a relatively simple database stored as a file on my hard drive. I want to switch to using PostgreSQL to do the same thing (Yes, I know it's overkill). I know that PostgreSQL cannot operate in embedded mode and that is fine - I can leave the server process running and that's OK with me.
I'm trying to figure out a connection string that will achieve this, but have been unsuccessful. I've tried variations on the following:
jdbc:postgresql:C:\myDB.fdb
jdbc:postgresql://C:\myDB.fdb
jdbc:postgresql://localhost:[port]/C:\myDB.fdb
but nothing seems to work. PostgreSQL's directions don't include an example for this case. Is this even possible?
You can trick it. If you are running PostGRESQL on a UNIXlike system, then you should be able to create a RAMDISK and use that for the database storage. Here's a pretty good step by step guide for RAMdisks on Linux.
In general though, I would suggest using SQLITE for an SQL db in RAM type of application.
Postgres databases are not a single file. There will be one file for each table and each index in the data directory, inside a directory for the database. All files will be named with the object ID (OID) of db / table / index.
The JDBC urls point to the database name, not any specific file:
jdbc:postgresql:foodb (localhost is implied)
If by "disk that behaves like memory", you mean that the db only exists for the lifetime of your program, there's no reason why you can't create a db at program start and drop it at program exit. Note that this is just DDL to create the DB, not creating the data dir via the init-db program. You could connect to the default 'postgres' db, create your db then connect to it.
Firebird 2.1 onwards supports global temporary tables, which only exist for the duration of the database connection.
Syntax goes something like CREATE GLOBAL TEMPORARY TABLE ... ON COMMIT PRESERVE ROWS