I have a myriad of issues with a seemingly simple process, starting with the tsk described below. But first some background:
Windows 10
QGIS 2.18.5
PgAdmin 4 (v 2.0)
Postgresql 10 installed
-Postgresql database (hosted on Amazon AWS cloud running PostgreSQL 9.6.5)
I'm very new to postgresql and postgis but after following all the basic instructions, I can't seem to be able to view any sql tables in QGIS. Here's what I have set up:
I'm in pgAdmin 4, logged in as admin (I'm the only user anyway), connected to my Amazon AWS server and connected to the new database I've created.
I've enabled the postgis extension (CREATE EXTENSION postgis;) and two others I read I might need...
All good so far?
I have a shapefile called test_poly.shp (created in ArcCatalog, with one simple shape drawn and one string field created) I want to upload to my database, so in the PostGIS Shapefile importer bundled with Postgresql, I connect to my database first:
And then import the shapefile, manually entering 27700 into the SRID field (British National Grid) and setting the 'Shape' field of the shapefile as the Geo Column (am I meant to do this? what is the Geo Column?). I also change the name of the Schema from 'public' to what I renamed it in pgAdmin.
issue: 1- Shapefile import failed...
==============================
Importing with configuration: test_poly, public, geom, D:\PostGIS\Test_poly.shp, mode=c, dump=1, simple=0, geography=0, index=1, shape=1, srid=0
Shapefile type: Polygon
PostGIS type: MULTIPOLYGON[2]
Shapefile import failed.
What's going on here? I've tried changing a few options including changing the input in the Geo Column to 'geom', then MULTIPOLYGON', then 'POLYGON' and back to 'shape', changing the encoding from UTF8 to LATIN1... no help.
Depending on your needs, a nice way to import geometries to PostGIS is using the Import into PostGIS tool from QGIS.
Just go to the toolbox (Advanced Interface) and search for Import into Postgis. The dialog is quite straightforward (The connection to the database needs to be done beforehand) ...
Based on this shapefile it creates the following table structure (Tested with QGIS 2.8.6-Wien):
CREATE TABLE public."tm_world_borders_simpl-0.3"
(
id integer NOT NULL DEFAULT nextval('"tm_world_borders_simpl-0.3_id_seq"'::regclass),
geom geometry(MultiPolygon,4326),
fips character varying(2),
iso2 character varying(2),
iso3 character varying(3),
un integer,
name character varying(50),
area integer,
pop2005 integer,
region integer,
subregion integer,
lon double precision,
lat double precision,
CONSTRAINT "tm_world_borders_simpl-0.3_pkey" PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
);
ALTER TABLE public."tm_world_borders_simpl-0.3"
OWNER TO disco2;
-- Index: public."sidx_tm_world_borders_simpl-0.3"
-- DROP INDEX public."sidx_tm_world_borders_simpl-0.3";
CREATE INDEX "sidx_tm_world_borders_simpl-0.3"
ON public."tm_world_borders_simpl-0.3"
USING gist
(geom);
Another PostGIS extension to import shapefiles is SPIT, but it is no longer maintained and can be quite bumpy with MultiPolygons. There are also other alternatives.
EDIT
How to add a new PostGIS Connection using QGIS:
Go to Add PostGIS Layer and click New
The enter your database address and credentials and click OK
Related
I'm trying to understand data masking in Postgresql and implement it to my database. Firstly i tried on windows and mac but it is unsucessfull. On PostgreSQL Anonymizer 1.0, they said they will not support windows and osx platform. This extension seems like works great on Read Hat and Centos. So i created a virual machine with using Vmware and installed Centos Stream 9. I installed Pgadmin4 and postgresql 14.5... I also installed PostgreSQL Anonymizer 1.0. I activated anon extension successfully on Centos and i am able to use static masking correctly. But it removes data.I need role based data masking like dynamic masking. But when i try to use Dynaming Masking with roles,it is not working. I need to protect personel informations from everyone. I read too many articles but they wont work.Please help me.Thanks.
-Firstly i created 2 roles
CREATE ROLE admin1 LOGIN SUPERUSER;
CREATE ROLE maskedRole1 LOGIN;
-Than,i created Posgresql db which is named testdb with this command on Pgadmin
CREATE DATABASE testdb OWNER admin1;
-Than i activated installed anon extension
ALTER DATABASE testdb SET session_preload_libraries = 'anon';
Than i created a table
CREATE TABLE customer (
id SERIAL PRIMARY KEY,
firstname TEXT,
lastname TEXT,
phone TEXT,
birth DATE,
postcode TEXT
);
Than i add some datas to customer table
INSERT INTO customer(firstname,lastname,phone,birth,postcode) VALUES('Burak','Unal','060-911-0911', '1985-10-10', '90016');
INSERT INTO customer(firstname,lastname,phone,birth,postcode) VALUES ('Şerif','Aydın', NULL, '1981-09-25', '90120');
INSERT INTO customer(firstname,lastname,phone,birth,postcode) VALUES ('Talha','Ersoy','347-515-3423', '1996-04-03', '04520');
INSERT INTO customer(firstname,lastname,phone,birth,postcode) VALUES ('Mehmet','Altunel','347-515-3423', '1995-06-01', '04520');
Than i activated extensions.(anon extension needs pgcrypto is installed)
`CREATE EXTENSION IF NOT EXISTS pgcrypto CASCADE;`
`CREATE EXTENSION IF NOT EXISTS anon CASCADE;`
Than i executed these commands to init extensions.
`SELECT anon.init();`
`SELECT setseed(0);`
Than i activated dynamic masking
SELECT anon.start_dynamic_masking();
Than i made maskedRole1 to masked
SECURITY LABEL FOR anon ON ROLE maskedRole1 IS 'MASKED';
Than i gave select permission to maskedRole1
GRANT SELECT ON supplier TO maskedRole1;
Than i wanted to mask customers phone V1
SECURITY LABEL FOR anon ON COLUMN customer.phone IS 'MASKED WITH VALUE ''CONFIDENTIAL'' '; ($$CONFIDENTIAL$$ is same result)
Note: I found comment on instead of SECURITY LABEL FOR but result is same...Unsuccesfull...masking customers phone V2
comment on column company.name IS 'MASKED WITH VALUE ''CONFIDENTIAL'' '
Now i expect maskedRole1 cant see phone numbers but it is epic fail...Everything is open to see.
set role maskedRole1; select * from company;
I wasted my 1 week to handle this. Read many articles,documantations but nothing changed... Any advice please? I'm completely open any other extensions or solutions
I'm trying to use a PostGIS geometry column with a docker build.
When docker starts up the container I get the following logs (abbreviated for clarity):
Loading PostGIS extensions into template_postgis
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
Loading PostGIS extensions into postgres
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/script.sql
where script.sql contains the following:
CREATE TABLE public.spatial_data_wfs (
id int4 NOT NULL,
name varchar(255) NOT NULL,
featureid varchar(255) NOT NULL,
geofeature jsonb NOT NULL,
coordinates geometry NOT NULL
);
Despite loading the PostGIS extensions before the script, it gives the following error:
psql:/docker-entrypoint-initdb.d/script.sql:238: ERROR: type "geometry" does not exist
LINE 6: coordinates geometry NOT NULL,
If I remove the geometry column and wait for the container to start up, I can use an alter table statement to add the geometry column, and it works fine.
The script also defines a function that generates a polygon, which doesn't pose any problem. I have tried the following based on other SO questions:
Create the postgis extension: complains it has already been created.
Use ALTER DATABASE to SET search_path TO include postgis. No complaint, same problem.
I can only assume the following from the logs:
There is some order of operations problem where despite the fact that postgis is installed, I still can't use a geometry column because something else has to complete first. Whatever it is, it has completed by the time postgres accepts connections.
The function is only validated for syntax - the fact the function code refers to the geometry type in the function body doesn't matter until someone calls it.
Has anyone else seen this? We're expecting all our stuff to build using Docker and Jenkins.
I figured out the problem. The database script was originally created by some tool (I think it might be pgdump) and then manual changes were added. The tool that adds a bunch of statements to the beginning of the script that aren't really necessary. One of those statements is the following:
SELECT pg_catalog.set_config('search_path', '', false);
I guess this setting overrides "ALTER DATABASE to SET search_path". Once I removed that statement, the script works fine, and the table is created.
I am importing my data into QGIS from a local postgreSQL server. The dataset loaded into the database contains 11 columns including a X and Y column which are in the CRS (Coodinate Reference System): EPSG:21781, CH1903 / LV03. I am trying to plot these points as a layer in QGIS but when I import it using the “Add PostGIS layer” I have to click “Also list tables with no geometry” to find it. Once added it appears as an attribute table which I can go into the layer properties and select the correct CRS but it still doesn't appear correctly.
I am still new to QGIS and PostgreSQL, am I doing something wrong or do I need to define the coordinates in the database before I import them to QGIS?
You need to create the geometry on the table. As of now, you are just displaying a table that does not contain any point so the coordinate system does not even apply.
So, first you would create the geometry column via PostGIS AddGeometryColumn
SELECT AddGeometryColumn ('myschema','mytable','geom',21781,'POINT',2);
Then you would update this new column with existing values.
UPDATE mytable SET geom = ST_SETSRID(ST_MakePoint(X, Y), 21781);
it my first time using google cloud.
I am setting up everything and I was going to add a sample database to my google cloud account to test few things, however when I try to import my sampleDB I get this error:
Could not complete the operation.
I have already make a bucket and imported my sql file in there,
this is my sample sql file:
> CREATE DATABASE IF NOT EXISTS student DEFAULT CHARACTER SET utf8mb4
> COLLATE utf8mb4_0900_ai_ci; USE student;
>
> CREATE TABLE person ( id int(11) NOT NULL, name varchar(25) NOT
> NULL, age int(3) NOT NULL, sex text, email text NOT NULL,
> study varchar(20) NOT NULL, birthday date NOT NULL ) ENGINE=InnoDB
> DEFAULT CHARSET=utf8mb4;
>
> INSERT INTO person (id, `name`, age, sex, email, study, birthday)
> VALUES (1, 'Saeed', 30, 'M', 'nakafisarrd#gmail.com', 'computer',
> '1987-04-30');
>
> ALTER TABLE person ADD PRIMARY KEY (id);
this is the tutorial I am following.
further more I have installed google app engine sdk for node. is doesn show me any error so I cant figure out what is going wrong here!
I think you might be mixing two topics here, so I will guide you through both process:
· First, you want to create a Cloud SQL instance importing a *.sql file, as explained in the tutorial you followed. However, bear in mind that this tutorial is using MySQL, so you should be creating a PostgreSQL instance instead if that fits your environment better. I would also recommend you to follow the official documentation on that topic, as it is explained in a clear way how to achieve what you want to do. I have tested the script you provided in a 2nd Generation MySQL instance changing the first line from CREATE DATABASE IF NOT EXISTS student DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci; to CREATE DATABASE IF NOT EXISTS student; and it worked.
· Second, you want to upload your NodeJS application to the Google Cloud Platform, and with that purpose I suggest using App Engine. As of today, NodeJS support is only available in App Engine Flexible, so you can also follow a guided quickstart in the documentation.
So I would recommend you to first clarify if you want to use either MySQL or PostgreSQL, create a Cloud SQL instance that matches your requirements, and then import the .sql file you shared. Everything should work fine.
I'm using a database created in the PostgreSQL. In its schema there are two tables and in one of them I want to add a geometry column.
The problem is that I created the postgis Extension (CREATE EXTENSION postgis;) for the database, but I'm not able to add this data type (geometry) column using pgAdmin.
To do this with pgAdmin's "New Column..." dialog, if you can't find geometry, then you might be able to find public.geometry instead (if PostGIS was installed there, which is normal).
However, I advise against using pgAdmin for creating geometry columns, as it does not understand typmods used to define the geometry type and SRID.
The best way is using DDL to directly manipulate the table, e.g.:
ALTER TABLE locations ADD COLUMN geom geometry(PointZ,4326);
to add a geom column of XYZ points (long, lat, alt).