Postgres complains about geometry type doesn't exist during docker startup, then decides it exists later - postgresql

I'm trying to use a PostGIS geometry column with a docker build.
When docker starts up the container I get the following logs (abbreviated for clarity):
Loading PostGIS extensions into template_postgis
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
Loading PostGIS extensions into postgres
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/script.sql
where script.sql contains the following:
CREATE TABLE public.spatial_data_wfs (
id int4 NOT NULL,
name varchar(255) NOT NULL,
featureid varchar(255) NOT NULL,
geofeature jsonb NOT NULL,
coordinates geometry NOT NULL
);
Despite loading the PostGIS extensions before the script, it gives the following error:
psql:/docker-entrypoint-initdb.d/script.sql:238: ERROR: type "geometry" does not exist
LINE 6: coordinates geometry NOT NULL,
If I remove the geometry column and wait for the container to start up, I can use an alter table statement to add the geometry column, and it works fine.
The script also defines a function that generates a polygon, which doesn't pose any problem. I have tried the following based on other SO questions:
Create the postgis extension: complains it has already been created.
Use ALTER DATABASE to SET search_path TO include postgis. No complaint, same problem.
I can only assume the following from the logs:
There is some order of operations problem where despite the fact that postgis is installed, I still can't use a geometry column because something else has to complete first. Whatever it is, it has completed by the time postgres accepts connections.
The function is only validated for syntax - the fact the function code refers to the geometry type in the function body doesn't matter until someone calls it.
Has anyone else seen this? We're expecting all our stuff to build using Docker and Jenkins.

I figured out the problem. The database script was originally created by some tool (I think it might be pgdump) and then manual changes were added. The tool that adds a bunch of statements to the beginning of the script that aren't really necessary. One of those statements is the following:
SELECT pg_catalog.set_config('search_path', '', false);
I guess this setting overrides "ALTER DATABASE to SET search_path". Once I removed that statement, the script works fine, and the table is created.

Related

How to automatically generate new UUID in PostgreSQL?

I'm using PostgreSQL version 14.4. I installed the uuid-ossp extension.
I created a table like this:
CREATE TABLE reserved_words
ADD id uuid NOT NULL DEFAULT uuid_generate_v1()
ADD word NOT NULL varchar(20);
Unfortunately, when I try adding a new record, rather than a new UUID being generated, instead the "uuid_generate_v1()" string is added in as the id!
I've scoured the Internet but can't find out how to alter things so that the function itself is executed. Any ideas?
My apologies, it actually does work. What's happening is that in DBeaver, the DB client I user, it does at first show the UUID generation function but then when you save the new record, it creates the UUID correctly.
Note: I don't really understand the difference between uuid_generate_v1 and uuid_generate_v4 but am going to opt to use the latter one.
uuid_generate_v1 () → uuid
Generates a version 1 UUID. This involves the MAC address of the computer and a time stamp. Note that UUIDs of this kind reveal the identity of the computer that created the identifier and the time at which it did so, which might make it unsuitable for certain security-sensitive applications.
uuid_generate_v4 () → uuid
Generates a version 4 UUID, which is derived entirely from random numbers.
source
The foremost point is that data type should be of uuid
The 'uuid-ossp' extension offers functions to generate UUID values.
To add the extension to the database run the following command
CREATE EXTENSION "uuid-ossp";
you can use the core function gen_random_uuid() to generate version-4 UUIDs.
To make use of this function in dbeaver ,follow the steps:
1.Go to table in which you want to generate UUID's
2.In the table properties tab find the column to which you need to apply the
uuid function
3.Double click on the column name and it will show expanded view of it's
properties
4.Under default value of that particular column properties, write the
function name as shown in the image
gen_random_uuid()
Dbeaver illustration

Use a postgres extension as a non-privileged user

The Problem
I have the following trigger function that uses the moddatetime extension:
/* BEFORE UPDATE trigger - Updates "entity_version.updated_on", setting it to the current UTC time. */
CREATE EXTENSION IF NOT EXISTS moddatetime; -- Needs superuser privileges!
DROP TRIGGER IF EXISTS trig_entity_version_before_update
ON entity_version;
CREATE TRIGGER trig_entity_version_before_update
BEFORE UPDATE
ON entity_version
FOR EACH ROW EXECUTE PROCEDURE moddatetime(updated_on); -- updated_on is TIMESTAMP type
The trigger works perfectly fine, but the issue is that the first line (CREATE EXTENSION) requires super-user privileges. Since these databases are going to be created by users (via a script) I don't want them the user that makes these databases and triggers to have super user access.
What I've tried
As a first step, running the script as a super user works fine, as you'd expect.
Naturally, the next step would be to separate the creation of the extension as a superuser from the trigger creation script. But doing that, if I run the above script without CREATE EXTENSION line, I get the following error:
function moddatetime() does not exist
Which I suppose makes sense given that the script never declares moddatetime, but I cannot find anywhere in the documentation how to define the extension as available without using CREATE EXTENSION. Surely there must be a way to import or use an extension without having the create it? Something akin to:
USING EXTENSION moddatetime; -- Does something like this exist?
... rest of the trigger script ...
You need the extension installed if you want to use it.
PostgreSQL v13 introduces the notion of a trusted extension, which can be installed by non-superusers with the CREATE privilege on the database, but this extension is not among them. So upgrading won't fix the problem.
You could connect to template1 as superuser and create the extension there. Then it would automatically be present in all newly created databases.
But frankly, I don't see the point. You can do the same thing with a regular PL/pgSQL trigger – only the performance wouldn't be as good.
As per #laurenz-albe his suggestion, I've just stepped off of using the extension for this particular use-case. Fortunately, writing a function to do what the extension does is relatively trivial. I'll include it here for reference in case someone is looking specifically to replace moddatetime.
DROP FUNCTION IF EXISTS fn_entity_version_before_update() CASCADE;
CREATE FUNCTION fn_entity_version_before_update() RETURNS TRIGGER LANGUAGE PLPGSQL AS
$fn_body$
BEGIN
NEW.updated_on = NOW() AT TIME ZONE 'utc';
RETURN NEW;
END;
$fn_body$;
DROP TRIGGER IF EXISTS trig_entity_version_before_update
ON entity_version;
CREATE TRIGGER trig_entity_version_before_update
BEFORE UPDATE
ON entity_version
FOR EACH ROW EXECUTE PROCEDURE fn_entity_version_before_update();

Change a column participating in a view or rule

I am using a tool called giswater to model our water sewer and storm networks. The tools builds a database from a few simple inputs in a gui one of which is the SRID. the geometry columns are all xy and I want to make them all ZM aware. When I run the alter table command to update the geometry I get an error "Cannot alter column participating in a view or rule"
Anyway to force this change and ignore the error? I tried changing the view to not reference the column and I tried adding a new geometry column to switch the view to temporarily while I make the change. Apparently I cannot drop a column in a view or change it to another column. I also tried writing the schema to SQL then edited the sql lines for linestring and point to linestringzm and pointzm and using psql to run the file to update the schema; all I get is access denied using "psql -U postgres -d utility -1 -f \i Z:......\xyz_test.sql"
Also tried pg_restore.
Anyway to just force the change using pg_admin4? or other suggestions?
I am not familiar with the tool you are using but I would suggest checking all the objects (views and rules) referencing your table by using the below command.
SELECT DISTINCT so.name
FROM syscomments sc
INNER JOIN sysobjects so ON sc.id=so.id
WHERE sc.TEXT LIKE '%tablename%'
Then backing up these object and dropping them completely before attempting to alter the table.

Generating a UUID in Postgres for Insert statement?

My question is rather simple. I'm aware of the concept of a UUID and I want to generate one to refer to each 'item' from a 'store' in my DB with. Seems reasonable right?
The problem is the following line returns an error:
honeydb=# insert into items values(
uuid_generate_v4(), 54.321, 31, 'desc 1', 31.94);
ERROR: function uuid_generate_v4() does not exist
LINE 2: uuid_generate_v4(), 54.321, 31, 'desc 1', 31.94);
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
I've read the page at: http://www.postgresql.org/docs/current/static/uuid-ossp.html
I'm running Postgres 8.4 on Ubuntu 10.04 x64.
uuid-ossp is a contrib module, so it isn't loaded into the server by default. You must load it into your database to use it.
For modern PostgreSQL versions (9.1 and newer) that's easy:
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
but for 9.0 and below you must instead run the SQL script to load the extension. See the documentation for contrib modules in 8.4.
For Pg 9.1 and newer instead read the current contrib docs and CREATE EXTENSION. These features do not exist in 9.0 or older versions, like your 8.4.
If you're using a packaged version of PostgreSQL you might need to install a separate package containing the contrib modules and extensions. Search your package manager database for 'postgres' and 'contrib'.
Without extensions (cheat)
If you need a valid v4 UUID
SELECT uuid_in(overlay(overlay(md5(random()::text || ':' || random()::text) placing '4' from 13) placing to_hex(floor(random()*(11-8+1) + 8)::int)::text from 17)::cstring);
Thanks to #Denis Stafichuk #Karsten and #autronix
Or you can simply get UUID-like value by doing this (if you don't care about the validity):
SELECT uuid_in(md5(random()::text || random()::text)::cstring);
output>> c2d29867-3d0b-d497-9191-18a9d8ee7830
(works at least in 8.4)
PostgreSQL 13 supports natively gen_random_uuid ():
PostgreSQL includes one function to generate a UUID:
gen_random_uuid () → uuid
This function returns a version 4 (random) UUID. This is the most commonly used type of UUID and is appropriate for most applications.
db<>fiddle demo
The answer by Craig Ringer is correct. Here's a little more info for Postgres 9.1 and later…
Is Extension Available?
You can only install an extension if it has already been built for your Postgres installation (your cluster in Postgres lingo). For example, I found the uuid-ossp extension included as part of the installer for Mac OS X kindly provided by EnterpriseDB.com. Any of a few dozen extensions may be available.
To see if the uuid-ossp extension is available in your Postgres cluster, run this SQL to query the pg_available_extensions system catalog:
SELECT * FROM pg_available_extensions;
Install Extension
To install that UUID-related extension, use the CREATE EXTENSION command as seen in this this SQL:
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
Beware: I found the QUOTATION MARK characters around extension name to be required, despite documentation to the contrary.
The SQL standards committee or Postgres team chose an odd name for that command. To my mind, they should have chosen something like "INSTALL EXTENSION" or "USE EXTENSION".
Verify Installation
You can verify the extension was successfully installed in the desired database by running this SQL to query the pg_extension system catalog:
SELECT * FROM pg_extension;
UUID as default value
For more info, see the Question: Default value for UUID column in Postgres
The Old Way
The information above uses the new Extensions feature added to Postgres 9.1. In previous versions, we had to find and run a script in a .sql file. The Extensions feature was added to make installation easier, trading a bit more work for the creator of an extension for less work on the part of the user/consumer of the extension. See my blog post for more discussion.
Types of UUIDs
By the way, the code in the Question calls the function uuid_generate_v4(). This generates a type known as Version 4 where nearly all of the 128 bits are randomly generated. While this is fine for limited use on smaller set of rows, if you want to virtually eliminate any possibility of collision, use another "version" of UUID.
For example, the original Version 1 combines the MAC address of the host computer with the current date-time and an arbitrary number, the chance of collisions is practically nil.
For more discussion, see my Answer on related Question.
pgcrypto Extension
As of Postgres 9.4, the pgcrypto module includes the gen_random_uuid() function. This function generates one of the random-number based Version 4 type of UUID.
Get contrib modules, if not already available.
sudo apt-get install postgresql-contrib-9.4
Use pgcrypto module.
CREATE EXTENSION "pgcrypto";
The gen_random_uuid() function should now available;
Example usage.
INSERT INTO items VALUES( gen_random_uuid(), 54.321, 31, 'desc 1', 31.94 ) ;
Quote from Postgres doc on uuid-ossp module.
Note: If you only need randomly-generated (version 4) UUIDs, consider using the gen_random_uuid() function from the pgcrypto module instead.
Update from 2021,
There is no need for a fancy trick to auto generate uuid on insert statement.
Just do one thing:
Set default value of DEFAULT gen_random_uuid () to your uuid column.
That is all.
Say, you have a table like this:
CREATE TABLE table_name (
unique_id UUID DEFAULT gen_random_uuid (),
first_name VARCHAR NOT NULL,
last_name VARCHAR NOT NULL,
email VARCHAR NOT NULL,
phone VARCHAR,
PRIMARY KEY (unique_id)
);
Now you need NOT to do anything to auto insert uuid values to unique_id column. Because you already defined a default value for it. You can simply focus on inserting onto other columns, and postgresql takes care of your unique_id. Here is a sample insert statement:
INSERT INTO table_name (first_name, last_name, email, phone)
VALUES (
'Beki',
'Otaev',
'beki#bekhruz.com',
'123-456-123'
)
Notice there is no inserting into unique_id as it is already taken care of.
About other extensions like uuid-ossp, you can bring them on if you are not satisfied with postgres's standard gen_random_uuid () function. Most of the time, you should be fine without them on
ALTER TABLE table_name ALTER COLUMN id SET DEFAULT uuid_in((md5((random())::text))::cstring);
After reading #ZuzEL's answer, i used the above code as the default value of the column id and it's working fine.
The uuid-ossp module provides functions to generate universally unique identifiers (UUIDs)
uuid_generate_v1() This function generates a version 1 UUID.
Add Extension
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
Verify Extension
SELECT * FROM pg_extension;
Run Query
INSERT INTO table_name(id, column1, column2 , column3, ...) VALUES
(uuid_generate_v1(), value1, value2, value3...);
Verify table data
SELECT uuid_generate_v5(uuid_ns_url (), 'test');

Table invisible in PostgreSQL - Undefined relation issue at different sessions

I have executed the following create statement using SQLWorkbench at my target postgresql database:
CREATE TABLE Config (
id serial PRIMARY KEY,
pub_ip_range_low varchar(100),
pub_ip_range_high varchar(100)
);
Right after table creation I request the table content by typing 'select * from config;' and see that table could be retrieved. Nevertheless, my java program that uses JDBC type 4 driver cannot access the table when I issue the same select statement in it. An exception is thrown when the program tries to access it which says says "Undefined relation" for the config table.
My questions are:
Why sqlworkbench where I had previously run the create statement recognizes the table while my java program cannot find it?
Where does the postgressql DBMS puts the tables I created? I don't see them neither in public nor in information schema.
NOTE:
I checked target postgres database and cannot see the table Config anywhere although SQL workbench can query it. Then I opened another SQL workbench instance and noticed that the table cannot be queried (i.e. not found). So, my conclusion is that PostgreSQL puts the table I created in the first running SQLBench instance into some location that is bound to that session. Another SQL Workbench instance or my java program is not bound to session, so cannot query the previously created table config.
The only "bloody location" that is session-local in PostgreSQL is the schema pg_temp, in other words: temporary tables. But your CREATE command does not display the keyword TEMP[ORARY]. Of course, as long as the transaction is not commited, nobody sees anything outside the transaction.
It's more likely you are seeing a switcheroo of hosts / databases / ports / or the schema search_path. A mixup with the mixed-case table name is a hot candidate, too. If you don't double-quote "Config", the table ends up all lower case in the system, so: config. If you later double quote the name, it won't match. The manual has the details.
Maybe the create failed on the extra trailing comma?
CREATE TABLE config (
id serial PRIMARY KEY,
pub_ip_range_low varchar(100),
pub_ip_range_high varchar(100) -- >> ,
);