I'm trying to dump the public schema of a database that contains postgis functions and types. The reason why is beyond the question - would be too long to explain why I need to do this.
Anyhow, when I do
pg_dump -d <db> -h <host> -U <user> -n public -Fp > public.sql
the file is essentially empty. I checked (tried different servers too) and all functions and types are in there in public scheams. I'm using the right user, etc.. I even tried to create my own function in there and this one only ended up in the output file
Anyone has a clue why ?
thanks.
Extensions are dumped "by reference" not by listing all of their components.
The creation of the individual components can be found in the installation script.
vi `pg_config --sharedir`/extension/postgis--3.1.1.sql
But you would have to tweak that to get the proper version.
Related
What I want:
I want a pg_dump of a database (let's call the database as 'test').
In this pg_dump I want only the tables without the following: data, triggers, functions, sequences, etc.
What I am doing to get what I want:
The command I run is as follows:
pg_dump -U postgres -s test > dump_test.sql
What I am observing:
Then when I try to restore this dump on another server as follows:
pg_dump -U postgres new_amazing_test < dump_test.sql
I notice that part of the output of running the above command says the following:
CREATE TRIGGER
CREATE FUNCTION
CREATE SEQUENCE
CREATE INDEX
What I actually want:
All I want is the table itself and not these triggers, functions, sequence and indexes. How do I only get the tables only?
Other things I have tried/considered:
I have tried doing this:
pg_dump -U postgres -s -schema=\dtmvE test > dump_test.sql
but it didn't work because the pattern needs to be a name not a \d pattern.
See here: https://www.postgresql.org/docs/13/app-pgdump.html for information on -n pattern option.
One thing that may solve it is to use multiple switches like this:
pgdump -t mytable1 -t mytable2 -t mytable3 ... -t mytableN > dump_test.sql
However, the above solution is impractical because I have some 70+ tables on my database.
Other relevant info:
PostgreSQL version is 13.1
Ubuntu version v16.04 (I have also tried this on Ubuntu v18.04)
I would dump everything with a custom format schema-only dump (-F c -s) and run pg_restore -l on the resulting dump. That gives you a table of contents. Delete everything except the tables from that file and use it as input to pg_restore -L to restore exactly those items from the archive that you need.
This may not be as simple as you have hoped for, but it is certainly simpler than writing tons of -t options, and you may be able to automatize it.
you can use the flag --section as described in the postgres documentation
--section=sectionname
Only dump the named section. The section name can be pre-data, data, or post-data. This option can be specified more than once to select multiple sections. The default is to dump all sections.
The data section contains actual table data, large-object contents, and sequence values. Post-data items include definitions of indexes, triggers, rules, and constraints other than validated check constraints. Pre-data items include all other data definition items.
example:
pg_dump --schema-only --section=pre-data
Is it possible to include the entire dataset into PostGIS? Currently only the nodes/ways having matching tag are imported. Can we make any changes to the style file to make that happen?
Edit:
osm2pgsql -c -d Map -U postgres -H localhost -S C:\osm2pgsql\default.style C:\Users\Saloni\Desktop\map.osm
Map is my database in Postgres
Are you looking for the --hstore option? With this option osm2pgql will store all key-value-pairs in a hash container, see the postgresql documentation for the hstore data type. Although I'm mot sure if this even imports elements that don't have any matching tag in the style file.
I have two databases on the same server. One named A and one named B. Booth databases have the same structure. I want to empty database B and load it with data from database A. Which is the best way to do this?
I have tried to take backup of database A in plain format. Then open the resulting sql-file and replace every occurence of 'A' with 'B' and then run the sql-script. This worked but I think it should be an easier way to move data from one database to another. Is it?
I use 'pgAdmin III' as my tool, but this is not necessary.
This is my first post here, hope the question is relevant and structured well enough. I tried google first but found it hard to find anyone with the same question.
Thanks in advance!
/David
SOLUTION: After help from Craig, this is how I did it
pg_dump -Fc -a -f a.dbbackup A
psql -c 'TRUNCATE table1, table2, ..., tableX CASCADE'
pg_restore dblive.backup -d B -c (not sure if -c was necessary)
Backup:
pg_dump -Fc -f a.dbbackup
Restore:
psql -c 'CREATE DATABASE b;'
pg_restore --dbname b a.dbbackup
Use the -U, -h etc options as required to connect to the correct host as the correct user with permissions to dump, create and restore the DB. See the docs for psql, pg_dump and pg_restore for more info (they all take the same options for connection control).
I run scripts against my database like this...
psql -d myDataBase -a -f myInsertFile.sql
The only problem is I want to be able to specify in this command what schema to run the script against. I could call set search_path='my_schema_01' but the files are supposed to be portable. How can I do this?
You can create one file that contains the set schema ... statement and then include the actual file you want to run:
Create a file run_insert.sql:
set schema 'my_schema_01';
\i myInsertFile.sql
Then call this using:
psql -d myDataBase -a -f run_insert.sql
More universal way is to set search_path (should work in PostgreSQL 7.x and above):
SET search_path TO myschema;
Note that set schema myschema is an alias to above command that is not available in 8.x.
See also: http://www.postgresql.org/docs/9.3/static/ddl-schemas.html
Main Example
The example below will run myfile.sql on database mydatabase using schema myschema.
psql "dbname=mydatabase options=--search_path=myschema" -a -f myfile.sql
The way this works is the first argument to the psql command is the dbname argument. The docs mention a connection string can be provided.
If this parameter contains an = sign or starts with a valid URI prefix
(postgresql:// or postgres://), it is treated as a conninfo string
The dbname keyword specifies the database to connect to and the options keyword lets you specify command-line options to send to the server at connection startup. Those options are detailed in the server configuration chapter. The option we are using to select the schema is search_path.
Another Example
The example below will connect to host myhost on database mydatabase using schema myschema. The = special character must be url escaped with the escape sequence %3D.
psql postgres://myuser#myhost?options=--search_path%3Dmyschema
The PGOPTIONS environment variable may be used to achieve this in a flexible way.
In an Unix shell:
PGOPTIONS="--search_path=my_schema_01" psql -d myDataBase -a -f myInsertFile.sql
If there are several invocations in the script or sub-shells that need the same options, it's simpler to set PGOPTIONS only once and export it.
PGOPTIONS="--search_path=my_schema_01"
export PGOPTIONS
psql -d somebase
psql -d someotherbase
...
or invoke the top-level shell script with PGOPTIONS set from the outside
PGOPTIONS="--search_path=my_schema_01" ./my-upgrade-script.sh
In Windows CMD environment, set PGOPTIONS=value should work the same.
I'm using something like this and works very well:* :-)
(echo "set schema 'acme';" ; \
cat ~/git/soluvas-framework/schedule/src/main/resources/org/soluvas/schedule/tables_postgres.sql) \
| psql -Upostgres -hlocalhost quikdo_app_dev
Note: Linux/Mac/Bash only, though probably there's a way to do that in Windows/PowerShell too.
This works for me:
psql postgresql://myuser:password#myhost/my_db -f myInsertFile.sql
In my case, I wanted to add schema to a file dynamically so that whatever schema name user will provide from the cli, I will run sql file with that provided schema name.
For this, I replaced some text in the sql file. First I added {{schema}} in the file like this
CREATE OR REPLACE FUNCTION {{schema}}.usp_dailygaintablereportdata(
then replace {{schema}} dynamically with user provided schema name with the help of sed command
sed -i "s/{{schema}}/$pgSchemaName/" $filename
result=$(psql -U $user -h $host -p $port -d $dbName -f "$filename" 2>&1)
sed -i "s/$pgSchemaName/{{schema}}/" $filename
First replace is done, then target file is run and then again our replace is reverted back
I was facing similar problems trying to do some dat import on an intermediate schema (that later we move on to the final one). As we rely on things like extensions (for example PostGIS), the "run_insert" sql file did not fully solved the problem.
After a while, we've found that at least with Postgres 9.3 the solution is far easier... just create your SQL script always specifying the schema when refering to the table:
CREATE TABLE "my_schema"."my_table" (...);
COPY "my_schema"."my_table" (...) FROM stdin;
This way using psql -f xxxxx works perfectly, and you don't need to change search_paths nor use intermediate files (and won't hit extension schema problems).
I use following command to dump some structures from server' database to be able to create sample of data on my local hard drive.
pg_dump -h myserver.com -U product_user -s -f ./data/base.structure.postgresql.sql -F p -v -T public.* -T first_product.* -T second_product.* -T another_product.locales mydatabase
I need to exclude some schemas otherwise it would ended up on permissions or other errors. Even that I exclude schema public, it dumps all functions in that schema, like this:
REVOKE ALL ON FUNCTION gin_extract_trgm(text, internal) FROM PUBLIC;
psql:./data/base.structure.postgresql.sql:8482: ERROR: function gin_extract_trgm(text, internal) does not exist
I know this comes from the fulltext or similarity plugin in PostgreSQL, but I don't use it and don't need it on my machine, so I'd like to exclude these functions.
How could I do that?
There is a way to do it. Say your backup is named backup.dump. What you need to do is:
$ pg_restore -l -f out.txt backup.dump
That will create a file out.txt that contains a list of objects that are in the dump. You need to edit the file and delete the items you don't want restored. Then you do this:
$ pg_restore -L out.txt -h your.host.name -U username .... backup.dump
This will use a file out.txt (that you edited) to select the things that will be restored. Pretty handy especially in case the dump is large and you cannot re-dump the database.
I need to exclude some schemas
pg_dump has a switch to exclude schemas:
pg_dump -N schema ...
I quote the manual about pg_dump:
-N schema
--exclude-schema=schema
Do not dump any schemas matching the schema pattern. The pattern is interpreted according to the same rules as for -n. -N can be given
more than once to exclude schemas matching any of several patterns.
...
With PostgreSQL 9.1 or later you have new options to move extensions into a separate schema - even pre-installed old-style modules. You can register old object with your (new-style) extension and then use the new tools. With fulltext and similarity you probably mean fuzzystrmatch and tsearch2. Example:
Register existing old-style objects for the extension fuzzystrmatch:
CREATE EXTENSION fuzzystrmatch SCHEMA public FROM unpackaged;
Drop the extension:
DROP EXTENSION fuzzystrmatch;
Install it to another schema:
CREATE EXTENSION fuzzystrmatch SCHEMA my_schema;
Of course, you cannot drop the extension, if objects from it are in use.
Also, if you install to another schema, you need to schema-qualify its functions in use or add the schema to the search_path.
In addition to the answer from Bartosz above, you can use the following sed command to remove e.g. a certain FUNCTION from the list before restoring:
sed -r -i -e '/FUNCTION public plpgsql_call_handler\(\) postgres/d' /var/backup/${DBNAME}.list