My database is hosted in Amazon and I am using pgAdmin 4 to connect to it.
I copy-pasted snippet from https://www.postgresql.org/docs/11/sql-createprocedure.html
CREATE PROCEDURE insert_data(a integer, b integer)
LANGUAGE SQL
AS $$
INSERT INTO tbl VALUES (a);
INSERT INTO tbl VALUES (b);
$$;
The issue is that I get 'incorrect syntax near 'PROCEDURE' ' error
What is done wrong? Not sure how I check version of postgresql itself
With Postgres 10, you need to use a function:
CREATE function insert_data(a integer, b integer)
returns void
LANGUAGE SQL
AS $$
INSERT INTO tbl VALUES (a), (b);
$$;
According to PostgreSQL documentation, syntax is supported in versions 11 and 12.
PostgreSQL: Documentation: 11: CREATE PROCEDURE
Documentation → PostgreSQL 11
Supported Versions: Current (11)
Development Versions: 12 / devel
Check PostgreSQL version on your server, run this query from PgAdmin:
SELECT version();
I am having an issue with pg_restore --clean not clearing the database.
Or do I misunderstand what the --clean does, I am expecting it to truncate the database tables and reinitialize the indexes/primary keys.
I am using 9.5 on rds
This is the full command we use
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U superuser -d mydatabase backup.dump
Basically what is happening is this.
I do a nightly backup of my production db, and restore it to an analytics db for the analyst to churn and run their reports.
I found out recently that the rails application used to view the reports was complaining that the primary keys were missing from the restored analytics database.
So I started investigating the production db, the analytics db etc. Which was when I realized that multiple rows with the same primary key existed in the analytics database.
I ran a few short experiments and realized that every time the pg_restore script is run it inserts duplicate data into the tables, this leads me to think that the --clean is not dropping and restoring the data. Because if I were to drop the schema beforehand, I don't get duplicate data.
To remove all tables from a database (but keep the database itself), you have two options.
Option 1: Drop the entire schema
You will need to re-create the schema and its permissions. This is usually good enough for development machines only.
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO public;
Applications usually use the "public" schema. You may encounter other schema names when working with a (legacy) application's database.
Note that for Rails applications, dropping and recreating the database itself is usually fine in development. You can use bin/rake db:drop db:create for that.
Option 2: Drop each table individually
Prefer this for production or staging servers. Permissions may be managed by your operations team, and you do not want to be the one who messed up permissions on a shared database cluster.
The following SQL code will find all table names and execute a DROP TABLE statement for each.
DO $$ DECLARE
r RECORD;
BEGIN
FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname = current_schema()) LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(r.tablename) || ' CASCADE'; -- DROP TABLE IF EXISTS instead DROP TABLE - thanks for the clarification Yaroslav Schekin
END LOOP;
END $$;
Original:
https://makandracards.com/makandra/62111-how-to-drop-all-tables-in-postgresql
If I execute single SQL statements in worksheet (eg CREATE ROLE my_user LOGIN PASSWORD 'my_pwd' VALID UNTIL 'infinity';) then is works correctly. Wrapping it in anonymous block like this:
DO
$$
BEGIN
CREATE ROLE my_user LOGIN PASSWORD 'my_pwd' VALID UNTIL 'infinity';
END
$$;
and I get following error message:
Error starting at line : 3 in command -
BEGIN
CREATE ROLE my_user LOGIN PASSWORD 'my_pwd' VALID UNTIL 'infinity';
END
$$;
Error report -
ERROR: syntax error at or near "CREATE"
Position: 10
However, the same script works fine when I execute it in psql shell. Am I missing something obvious?
Using:
PostgreSQL version 9.4
Postgres JDBC driver postgresql-9.3-1102.jdbc41
SQL Developer version 4.0 (jdk 1.7.0_71)
The source of the error is JDBC's inability to deal with dollar-quoting correctly (yet). Related answer:
Exceptions when creating a trigger in PostgreSQL 9.1
You might be able to circumvent the problem in this case with:
DO
'
BEGIN
CREATE ROLE my_user LOGIN PASSWORD ''my_pwd'' VALID UNTIL ''infinity'';
END
';
If that doesn't do the trick, try to set a different query terminator, like advised in the linked answer.
You seem to be aware that you do not need a DO statement for the example code at all. Just:
CREATE ROLE my_user LOGIN PASSWORD 'my_pwd' VALID UNTIL 'infinity';
I am trying to import the spatial file in to my database
Firstly i have created a database using postgis template as below
createdb -T template_postgis database_name;
I have postgis installed already on my machine
POSTGIS="2.1.1 r12113" GEOS="3.3.3-CAPI-1.7.4" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.9.0, released 2011/12/29" LIBXML="2.7.8" LIBJSON="UNKNOWN" TOPOLOGY RASTER
Postgres version : psql (9.3.2, server 9.1.11)
Secondly i have converted the spatial file in to sql file as below
shp2pgsql -s 3425 Aspire.shp test_for_shape_data database_name > shapefile_data.sql
And now i am trying to import this sql file(shapefile_data.sql) in to my database(database_name) like below
psql -d database_name username -f shapefile_data.sql
But i am getting the following error
user#user:~/user/spice$ psql -d psql -d database_name username -f shapefile_data.sql
SET
SET
BEGIN
psql:shapefile_data.sql:30: NOTICE: CREATE TABLE will create implicit sequence "test_for_shape_data_gid_seq" for serial column "test_for_shape_data.gid"
CREATE TABLE
psql:shapefile_data.sql:31: NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "test_for_shape_data_pkey" for table "test_for_shape_data"
ALTER TABLE
psql:shapefile_data.sql:32: ERROR: function addgeometrycolumn(unknown, unknown, unknown, unknown, unknown, integer) is not unique
LINE 1: SELECT AddGeometryColumn('','test_for_shape_data','geom','42...
^
HINT: Could not choose a best candidate function. You might need to add explicit type casts.
psql:shapefile_data.sql:33: ERROR: current transaction is aborted, commands ignored until end of transaction block
psql:shapefile_data.sql:34: ERROR: current transaction is aborted, commands ignored until end of transaction block
...........
.......
Why it is telling me that i don't have AddGeometryColumn function even though i have created the database with postgis template ?
So how to avoid this ?
Also when i try to manually enable the postgis functions its telling that already exists
database_name=# CREATE EXTENSION postgis;
ERROR: type "spheroid" already exists
database_name=# CREATE EXTENSION postgis_topology;
ERROR: required extension "postgis" is not installed
So how to clear this error and insert the shapefile sql file in to database ?
It sounds like you've got an old PostGIS install that was created from an SQL script, pre-extension support. It probably only has some of the functionality and features you expect. This sort of behaviour can occur when you've got a PostGIS 1.5 schema and a PostGIS 2.0 install, etc.
Try creating your DB from template0 instead, and running CREATE EXTENSION postgis; then doing a restore. I suspect your template_postgis contains an old version of the extension schema.
See the PostGIS upgrade guide.
I have dumped a clean, no owner backup for Postgres Database with the command
pg_dump sample_database -O -c -U
Later, when I restore the database with
psql -d sample_database -U app_name
However, I encountered several errors which prevents me from restoring the data:
ERROR: must be owner of extension plpgsql
ERROR: must be owner of schema public
ERROR: schema "public" already exists
ERROR: must be owner of schema public
CREATE EXTENSION
ERROR: must be owner of extension plpgsql
I digged into the plain-text SQL pg_dump generates and I found it contains SQL
CREATE SCHEMA public;
COMMENT ON SCHEMA public IS 'standard public schema';
CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;
COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';
I think the causes are that the user app_name doesn't have the privileges to alter the public schema and plpgsql.
How could I solve this issue?
To solve the issue you must assign the proper ownership permissions. Try the below which should resolve all permission related issues for specific users but as stated in the comments this should not be used in production:
root#server:/var/log/postgresql# sudo -u postgres psql
psql (8.4.4)
Type "help" for help.
postgres=# \du
List of roles
Role name | Attributes | Member of
-----------------+-------------+-----------
<user-name> | Superuser | {}
: Create DB
postgres | Superuser | {}
: Create role
: Create DB
postgres=# alter role <user-name> superuser;
ALTER ROLE
postgres=#
So connect to the database under a Superuser account sudo -u postgres psql and execute a ALTER ROLE <user-name> Superuser; statement.
Keep in mind this is not the best solution on multi-site hosting server so take a look at assigning individual roles instead: https://www.postgresql.org/docs/current/static/sql-set-role.html and https://www.postgresql.org/docs/current/static/sql-alterrole.html.
AWS RDS users if you are getting this it is because you are not a superuser and according to aws documentation you cannot be one. I have found I have to ignore these errors.
For people using Google Cloud Platform, any error will stop the import process.
Personally I encountered two different errors depending on the pg_dump command I issued :
1- The input is a PostgreSQL custom-format dump. Use the pg_restore command-line client to restore this dump to a database.
Occurs when you've tried to dump your DB in a non plain text format. I.e when the command lacks the -Fp or --format=plain parameter. However, if you add it to your command, you may then encounter the following error :
2- SET SET SET SET SET SET CREATE EXTENSION ERROR: must be owner of extension plpgsql
This is a permission issue I have been unable to fix using the command provided in the GCP docs, the tips from this current thread, or following advice from Google Postgres team here. Which recommended to issue the following command :
pg_dump -Fp --no-acl --no-owner -U myusername myDBName > mydump.sql
The only thing that did the trick in my case was manually editing the dump file and commenting out all commands relating to plpgsql.
I hope this helps GCP-reliant souls.
Update :
It's easier to dump the file commenting out extensions, especially since some dumps can be huge :
pg_dump ... | grep -v -E '(CREATE\ EXTENSION|COMMENT\ ON)' > mydump.sql
Which can be narrowed down to plpgsql :
pg_dump ... | grep -v -E '(CREATE\ EXTENSION\ IF\ NOT\ EXISTS\ plpgsql|COMMENT\ ON\ EXTENSION\ plpgsql)' > mydump.sql
Try using the -L flag with pg_restore by specifying the file taken from pg_dump -Fc
-L list-file
--use-list=list-file
Restore only those archive elements that are listed in list-file, and restore them in the order they appear in the file. Note that if filtering switches such as -n or -t are used with -L, they will further restrict the items restored.
list-file is normally created by editing the output of a previous -l operation. Lines can be moved or removed, and can also be commented out by placing a semicolon (;) at the start of the line. See below for examples.
https://www.postgresql.org/docs/9.5/app-pgrestore.html
pg_dump -Fc -f pg.dump db_name
pg_restore -l pg.dump | grep -v 'COMMENT - EXTENSION' > pg_restore.list
pg_restore -L pg_restore.list pg.dump
Here you can see the Inverse is true by outputting only the comment:
pg_dump -Fc -f pg.dump db_name
pg_restore -l pg.dump | grep 'COMMENT - EXTENSION' > pg_restore_inverse.list
pg_restore -L pg_restore_inverse.list pg.dump
--
-- PostgreSQL database dump
--
-- Dumped from database version 9.4.15
-- Dumped by pg_dump version 9.5.14
SET statement_timeout = 0;
SET lock_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET client_min_messages = warning;
SET row_security = off;
--
-- Name: EXTENSION plpgsql; Type: COMMENT; Schema: -; Owner:
--
COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';
--
-- PostgreSQL database dump complete
--
You can probably safely ignore the error messages in this case. Failing to add a comment to the public schema and installing plpgsql (which should already be installed) aren't going to cause any real problems.
However, if you want to do a complete re-install you'll need a user with appropriate permissions. That shouldn't be the user your application routinely runs as of course.
Shorter answer: ignore it.
This module is the part of Postgres that processes the SQL language. The error will often pop up as part of copying a remote database, such as with
a 'heroku pg:pull'. It does not overwrite your SQL processor and warns you about that.
For people using AWS, the COMMENT ON EXTENSION is possible only as superuser, and as we know by the docs, RDS instances are managed by Amazon. As such, to prevent you from breaking things like replication, your users - even the root user you set up when you create the instance - will not have full superuser privileges:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html
When you create a DB instance, the master user system account that you
create is assigned to the rds_superuser role. The rds_superuser role
is a pre-defined Amazon RDS role similar to the PostgreSQL superuser
role (customarily named postgres in local instances), but with some
restrictions. As with the PostgreSQL superuser role, the rds_superuser
role has the most privileges on your DB instance and you should not
assign this role to users unless they need the most access to the DB
instance.
In order to fix this error, just use -- to comment out the lines of SQL that contains COMMENT ON EXTENSION
EDIT 1: As suggested by Dmitrii I., you can also omit comments when dumping: pg_dump --no-comments
For people who have narrowed down the issue to the COMMENT ON statements (as per various answers below) and who have superuser access to the source database from which the dump file is created, the simplest solution might be to prevent the comments from being included to the dump file in the first place, by removing them from the source database being dumped...
COMMENT ON EXTENSION postgis IS NULL;
COMMENT ON EXTENSION plpgsql IS NULL;
COMMENT ON SCHEMA public IS NULL;
Future dumps then won't include the COMMENT ON statements.
Use the postgres (admin) user to dump the schema, recreate it and grant priviledges for use before you do your restore.
In one command:
sudo -u postgres psql -c "DROP SCHEMA public CASCADE;
create SCHEMA public;
grant usage on schema public to public;
grant create on schema public to public;" myDBName
For me, I was setting up a database with pgAdmin and it seems setting the owner during database creation was not enough. I had to navigate down to the 'public' schema and set the owner there as well (was originally 'postgres').
Some of the answers have already provided various approaches related to getting rid of the create extension and comment on extensions. For me, the following command line seemed to work and be the simplest approach to solve the problem:
cat /tmp/backup.sql.gz | gunzip - | \
grep -v -E '(CREATE\ EXTENSION|COMMENT\ ON)' | \
psql --set ON_ERROR_STOP=on -U db_user -h localhost my_db
Some notes
The first line is just uncompressing my backup and you may need to adjust accordingly.
The second line is using grep to get rid of offending lines.
the third line is my psql command; you may need to adjust as you normally would use psql for restore.