I've been stuck with this problem for more than a week now, and I hope someone could help me out here.
I was running Postgresql 12 with PostGIS 3.1.1 extension. More than a week ago, I accidentally performed a PostGIS upgrade by running this query
SELECT PostGIS_Extensions_Upgrade(); which returned "Upgrade completed, run SELECT postgis_full_version(); for details".
Although it seemed the upgrade went on successfully, while I check with SELECT postgis_full_version(); it returned POSTGIS="3.2.0 c3e3cc0" [EXTENSION] PGSQL="120" GEOS="3.9.1-CAPI-1.14.2" PROJ="6.3.1" GDAL="GDAL 3.0.4, released 2020/01/28 GDAL_DATA not found" LIBXML="2.9.10" LIBJSON="0.13.1" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY RASTER. Note that GDAL_DATA is no longer found.
This causes functions like ST_Transform breaks with error as following: SQL Error [XX000]: ERROR: could not form projection (LWPROJ) from 'srid=4326' to 'srid=4269'.
So far I have tried reinstalled postgresql-12, postgresql-client-12, postgis, postgresql-12-postgis-3 and postgresql-12-postgis-3-scripts on my Ubuntu 20.04 system, to no avail.
I tried downgrading to PostGIS to 3.1.4 (somehow apt repos only have 3.1.4 not 3.1.1), which returned POSTGIS="3.1.4 ded6c34" [EXTENSION] PGSQL="120" GEOS="3.9.1-CAPI-1.14.2" PROJ="7.2.1" GDAL="GDAL 3.3.2, released 2021/09/01" LIBXML="2.9.10" LIBJSON="0.13.1" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" (core procs from "3.2.0 c3e3cc0" need upgrade) TOPOLOGY (topology procs from "3.2.0 c3e3cc0" need upgrade) RASTER (raster procs from "3.2.0 c3e3cc0" need upgrade) when querying SELECT postgis_full_version();. There didn't seem any error but the ST_Transform query still gave the same error above.
I am also tried to use the pg_upgrade function to upgrade the database to Postgresql 14. While running with the --check option, the results says "Clusters are compatible". But when running the actual upgrade, it breaks at the "Creating dump of database schemas" of the postgres database. Dumps of other databases went through just fine. The "pg_upgrade_dump_13427.log" log shows the following.
pg_dump: error: query was: SELECT classid, objid, refobjid FROM pg_depend WHERE refclassid = 'pg_extension'::regclass AND deptype = 'e' ORDER BY 3
command: "/usr/lib/postgresql/14/bin/pg_dump" --host /var/lib/postgresql --port 50432 --username postgres --schema-only --quote-all-identifiers --binary-upgrade --format=custom --file="pg_upgrade_dump_13427.custom" 'dbname=postgres' >> "pg_upgrade_dump_13427.log" 2>&1
pg_dump: WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
pg_dump: error: query failed: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
pg_dump: error: query was: SELECT c.tableoid, c.oid, c.relname, (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(c.relacl,pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner))) WITH ORDINALITY AS perm(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner))) AS init(init_acl) WHERE acl = init_acl)) as foo) AS relacl, (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner))) WITH ORDINALITY AS initp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(c.relacl,pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner))) AS permp(orig_acl) WHERE acl = orig_acl)) as foo) as rrelacl, CASE WHEN privtype = 'e' THEN (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(pip.initprivs) WITH ORDINALITY AS initp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner)) AS privm(orig_acl) WHERE acl = orig_acl)) as foo) END AS initrelacl, CASE WHEN privtype = 'e' THEN (SELECT pg_catalog.array_agg(acl) FROM (SELECT acl, row_n FROM pg_catalog.unnest(pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner)) WITH ORDINALITY AS privp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(pip.initprivs) AS initp(init_acl) WHERE acl = init_acl)) as foo) END as initrrelacl, c.relkind, c.relnamespace, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = c.relowner) AS rolname, c.relchecks, c.relhastriggers, c.relhasindex, c.relhasrules, 'f'::bool AS relhasoids, c.relrowsecurity, c.relforcerowsecurity, c.relfrozenxid, c.relminmxid, tc.oid AS toid, tc.relfrozenxid AS tfrozenxid, tc.relminmxid AS tminmxid, c.relpersistence, c.relispopulated, c.relreplident, c.relpages, am.amname, CASE WHEN c.relkind = 'f' THEN (SELECT ftserver FROM pg_catalog.pg_foreign_table WHERE ftrelid = c.oid) ELSE 0 END AS foreignserver, CASE WHEN c.reloftype <> 0 THEN c.reloftype::pg_catalog.regtype ELSE NULL END AS reloftype, d.refobjid AS owning_tab, d.refobjsubid AS owning_col, (SELECT spcname FROM pg_tablespace t WHERE t.oid = c.reltablespace) AS reltablespace, array_remove(array_remove(c.reloptions,'check_option=local'),'check_option=cascaded') AS reloptions, CASE WHEN 'check_option=local' = ANY (c.reloptions) THEN 'LOCAL'::text WHEN 'check_option=cascaded' = ANY (c.reloptions) THEN 'CASCADED'::text ELSE NULL END AS checkoption, tc.reloptions AS toast_reloptions, c.relkind = 'S' AND EXISTS (SELECT 1 FROM pg_depend WHERE classid = 'pg_class'::regclass AND objid = c.oid AND objsubid = 0 AND refclassid = 'pg_class'::regclass AND deptype = 'i') AS is_identity_sequence, EXISTS (SELECT 1 FROM pg_attribute at LEFT JOIN pg_init_privs pip ON (c.oid = pip.objoid AND pip.classoid = 'pg_class'::regclass AND pip.objsubid = at.attnum)WHERE at.attrelid = c.oid AND ((SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(at.attacl,pg_catalog.acldefault('c',c.relowner))) WITH ORDINALITY AS perm(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault('c',c.relowner))) AS init(init_acl) WHERE acl = init_acl)) as foo) IS NOT NULL OR (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault('c',c.relowner))) WITH ORDINALITY AS initp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(at.attacl,pg_catalog.acldefault('c',c.relowner))) AS permp(orig_acl) WHERE acl = orig_acl)) as foo) IS NOT NULL OR CASE WHEN privtype = 'e' THEN (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(pip.initprivs) WITH ORDINALITY AS initp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(pg_catalog.acldefault('c',c.relowner)) AS privm(orig_acl) WHERE acl = orig_acl)) as foo) END IS NOT NULL OR CASE WHEN privtype = 'e' THEN (SELECT pg_catalog.array_agg(acl) FROM (SELECT acl, row_n FROM pg_catalog.unnest(pg_catalog.acldefault('c',c.relowner)) WITH ORDINALITY AS privp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(pip.initprivs) AS initp(init_acl) WHERE acl = init_acl)) as foo) END IS NOT NULL))AS changed_acl, pg_get_partkeydef(c.oid) AS partkeydef, c.relispartition AS ispartition, pg_get_expr(c.relpartbound, c.oid) AS partbound FROM pg_class c LEFT JOIN pg_depend d ON (c.relkind = 'S' AND d.classid = c.tableoid AND d.objid = c.oid AND d.objsubid = 0 AND d.refclassid = c.tableoid AND d.deptype IN ('a', 'i')) LEFT JOIN pg_class tc ON (c.reltoastrelid = tc.oid AND c.relkind <> 'p') LEFT JOIN pg_am am ON (c.relam = am.oid) LEFT JOIN pg_init_privs pip ON (c.oid = pip.objoid AND pip.classoid = 'pg_class'::regclass AND pip.objsubid = 0) WHERE c.relkind in ('r', 'S', 'v', 'c', 'm', 'f', 'p') ORDER BY c.oid
command: "/usr/lib/postgresql/13/bin/pg_dump" --host /var/lib/postgresql --port 50432 --username postgres --schema-only --quote-all-identifiers --binary-upgrade --format=custom --file="pg_upgrade_dump_13427.custom" 'dbname=postgres' >> "pg_upgrade_dump_13427.log" 2>&1
pg_dump: WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
pg_dump: error: query failed: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
pg_dump: error: query was: SELECT classid, objid, refobjid FROM pg_depend WHERE refclassid = 'pg_extension'::regclass AND deptype = 'e' ORDER BY 3
It seems the dependencies of postgres extensions are broken. How do I fix that? Can someone please help?
Related
I had created a DB testDb1 on a postgres server. After adding some data to this DB, I created a dump of this data and restored that to a new DB testDb2. After that data was added to second DB for quite some time.
Now, I want to dump data of testDb2 using following command:
pg_dump -U "postgres" --no-privileges -Fd -j 4 -f dump_20230102_db2 testDb2
But this gives me below error:
pg_dump: error: query failed: ERROR: relation "pg_catalog.pg_roles" does not exist
LINE 1: ....tableoid, n.oid, n.nspname, (SELECT rolname FROM pg_catalog...
^
pg_dump: error: query was: SELECT n.tableoid, n.oid, n.nspname, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = nspowner) AS rolname, (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(n.nspacl,pg_catalog.acldefault('n',n.nspowner))) WITH ORDINALITY AS perm(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault('n',n.nspowner))) AS init(init_acl) WHERE acl = init_acl)) as foo) as nspacl, (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault('n',n.nspowner))) WITH ORDINALITY AS initp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(n.nspacl,pg_catalog.acldefault('n',n.nspowner))) AS permp(orig_acl) WHERE acl = orig_acl)) as foo) as rnspacl, NULL as initnspacl, NULL as initrnspacl FROM pg_namespace n LEFT JOIN pg_init_privs pip ON (n.oid = pip.objoid AND pip.classoid = 'pg_namespace'::regclass AND pip.objsubid = 0)
I checked the original DB testDb1, results are generated for above query but not for testDb2.
I also changed the owner of testDb2 to postgres but no luck there.
What can I do to restore or generate pg_catalog.pg_roles for testDb2 ?
Update:
Both DB are on same version 14.4
Output of \d *pg_roles* in testDb2 is:
testDb2-# \d *pg_roles*
Did not find any relation named "*pg_roles*".
Postgres 15 with pgAdmin was installed in Windows 11 and new database was created.
Trying to connect to database in localhost throw error
Request failed with status code 500
Postgres log contains
2022-10-25 09:37:27.159 EEST [10508] ERROR: UNION types "char" and text cannot be matched at character 745
2022-10-25 09:37:27.159 EEST [10508] STATEMENT:
SELECT cl.relkind, COALESCE(cin.nspname, cln.nspname) as nspname,
COALESCE(ci.relname, cl.relname) as relname, cl.relname as indname
FROM pg_catalog.pg_class cl
JOIN pg_catalog.pg_namespace cln ON cl.relnamespace=cln.oid
LEFT OUTER JOIN pg_catalog.pg_index ind ON ind.indexrelid=cl.oid
LEFT OUTER JOIN pg_catalog.pg_class ci ON ind.indrelid=ci.oid
LEFT OUTER JOIN pg_catalog.pg_namespace cin ON ci.relnamespace=cin.oid
WHERE cl.oid IN (SELECT objid FROM pg_catalog.pg_shdepend WHERE refobjid=16395::oid) AND cl.oid > 16383::oid
UNION ALL SELECT 'n', null, nspname, null
FROM pg_catalog.pg_namespace nsp
WHERE nsp.oid IN (SELECT objid FROM pg_catalog.pg_shdepend WHERE refobjid=16395::oid) AND nsp.oid > 16383::oid
UNION ALL SELECT CASE WHEN typtype='d' THEN 'd' ELSE 'y' END, null, typname, null
FROM pg_catalog.pg_type ty
WHERE ty.oid IN (SELECT objid FROM pg_catalog.pg_shdepend WHERE refobjid=16395::oid) AND ty.oid > 16383::oid
UNION ALL SELECT 'C', null, conname, null
FROM pg_catalog.pg_conversion co
WHERE co.oid IN (SELECT objid FROM pg_catalog.pg_shdepend WHERE refobjid=16395::oid) AND co.oid > 16383::oid
UNION ALL SELECT CASE WHEN prorettype=2279 THEN 'T' ELSE 'p' END, null, proname, null
FROM pg_catalog.pg_proc pr
WHERE pr.oid IN (SELECT objid FROM pg_catalog.pg_shdepend WHERE refobjid=16395::oid) AND pr.oid > 16383::oid
UNION ALL SELECT 'o', null, oprname || '('::text || COALESCE(tl.typname, ''::text) || CASE WHEN tl.oid IS NOT NULL
AND tr.oid IS NOT NULL THEN ','::text END || COALESCE(tr.typname, ''::text) || ')'::text, null
FROM pg_catalog.pg_operator op
LEFT JOIN pg_catalog.pg_type tl ON tl.oid=op.oprleft
LEFT JOIN pg_catalog.pg_type tr ON tr.oid=op.oprright
WHERE op.oid IN (SELECT objid FROM pg_catalog.pg_shdepend WHERE refobjid=16395::oid) AND op.oid > 16383::oid
ORDER BY 1,2,3
I installed pgadmin4-6.15-x64.exe from pgadmin.org but problem persists.
Environment:
Version
6.15
Application Mode
Desktop
Current User
pgadmin4#pgadmin.org
NW.js Version
0.62.2
Browser
Chromium 99.0.4844.84
Operating System
Windows-10-10.0.22621-SP0
PgAdmin 4 6.17 has fixed this problem.
I use postgresql 9.6.
I tried create a backup file with pg_dump: pg_dump -U postgres -O -f file.backup database
But I got the error:
SQL Error [XX000]: ERROR: could not read block 270 in file "base/16393/1249": Invalid argument
with this query:
select
c.tableoid,
c.oid,
c.relname,
(
select
pg_catalog.array_agg(acl
order by
row_n)
from
(
select
acl,
row_n
from
pg_catalog.unnest(coalesce(c.relacl, pg_catalog.acldefault(case when c.relkind = 'S' then 's' else 'r' end::"char", c.relowner))) with ordinality as perm(acl,
row_n)
where
not exists (
select
1
from
pg_catalog.unnest(coalesce(pip.initprivs, pg_catalog.acldefault(case when c.relkind = 'S' then 's' else 'r' end::"char", c.relowner))) as init(init_acl)
where
acl = init_acl)) as foo) as relacl,
(
select
pg_catalog.array_agg(acl
order by
row_n)
from
(
select
acl,
row_n
from
pg_catalog.unnest(coalesce(pip.initprivs, pg_catalog.acldefault(case when c.relkind = 'S' then 's' else 'r' end::"char", c.relowner))) with ordinality as initp(acl,
row_n)
where
not exists (
select
1
from
pg_catalog.unnest(coalesce(c.relacl, pg_catalog.acldefault(case when c.relkind = 'S' then 's' else 'r' end::"char", c.relowner))) as permp(orig_acl)
where
acl = orig_acl)) as foo) as rrelacl,
null as initrelacl,
null as initrrelacl,
c.relkind,
c.relnamespace,
(
select
rolname
from
pg_catalog.pg_roles
where
oid = c.relowner) as rolname,
c.relchecks,
c.relhastriggers,
c.relhasindex,
c.relhasrules,
c.relhasoids,
c.relrowsecurity,
c.relforcerowsecurity,
c.relfrozenxid,
c.relminmxid,
tc.oid as toid,
tc.relfrozenxid as tfrozenxid,
tc.relminmxid as tminmxid,
c.relpersistence,
c.relispopulated,
c.relreplident,
c.relpages,
case
when c.reloftype <> 0 then c.reloftype::pg_catalog.regtype
else null
end as reloftype,
d.refobjid as owning_tab,
d.refobjsubid as owning_col,
(
select
spcname
from
pg_tablespace t
where
t.oid = c.reltablespace) as reltablespace,
array_remove(array_remove(c.reloptions, 'check_option=local'), 'check_option=cascaded') as reloptions,
case
when 'check_option=local' = any (c.reloptions) then 'LOCAL'::text
when 'check_option=cascaded' = any (c.reloptions) then 'CASCADED'::text
else null
end as checkoption,
tc.reloptions as toast_reloptions,
exists (
select
1
from
pg_attribute at
left join pg_init_privs pip on
(c.oid = pip.objoid
and pip.classoid = 'pg_class'::regclass
and pip.objsubid = at.attnum)
where at.attrelid = c.oid
and ((
select
pg_catalog.array_agg(acl
order by
row_n)
from
(
select
acl,
row_n
from
pg_catalog.unnest(coalesce(at.attacl, pg_catalog.acldefault('c', c.relowner))) with ordinality as perm(acl,
row_n)
where
not exists (
select
1
from
pg_catalog.unnest(coalesce(pip.initprivs, pg_catalog.acldefault('c', c.relowner))) as init(init_acl)
where
acl = init_acl)) as foo) is not null
or (
select
pg_catalog.array_agg(acl
order by
row_n)
from
(
select
acl,
row_n
from
pg_catalog.unnest(coalesce(pip.initprivs, pg_catalog.acldefault('c', c.relowner))) with ordinality as initp(acl,
row_n)
where
not exists (
select
1
from
pg_catalog.unnest(coalesce(at.attacl, pg_catalog.acldefault('c', c.relowner))) as permp(orig_acl)
where
acl = orig_acl)) as foo) is not null
or null is not null
or null is not null))as changed_acl
from
pg_class c
left join pg_depend d on
(c.relkind = 'S'
and d.classid = c.tableoid
and d.objid = c.oid
and d.objsubid = 0
and d.refclassid = c.tableoid
and d.deptype = 'a')
left join pg_class tc on
(c.reltoastrelid = tc.oid)
left join pg_init_privs pip on
(c.oid = pip.objoid
and pip.classoid = 'pg_class'::regclass
and pip.objsubid = 0)
where
c.relkind in ('r', 'S', 'v', 'c', 'm', 'f')
order by
c.oid;
I think some table is corrupted. Is there a way for me to create a backup without going through this query?
You have data corruption in your pg_attribute table. That table contains the column definitions for all database tables, indexes, composite types and so on.
Check if the size of base/16393/1249 is more than 2220032 bytes (271 blocks).
You should restore your backup. If you don't have a backup, either hire a PostgreSQL expert, who may be able to extract information from the bad block, or cut your losses.
I have been using using DBeaver and I got a new laptop. With a fresh install adding the postgres remote database exactly as it is in the other computer.
I can see all the tables and data in the database. I just cannot take a backup.
When I right click tools->backup and I click start (same process works on old laptop) I am getting the error below. If I do this with my local dev database it works fine as expected. I can do a backup on my laptop both directly before and directly after trying on the new laptop.
C:\Program Files\PostgreSQL\10\bin\pg_dump.exe --format=c -n public --verbose --host=compute-1.amazonaws.com --port=5432 --username=elkjlkjkjyb d5kjhkjhjkht5
Backup started at Sun Sep 02 22:10:20 EDT 2018
pg_dump: last built-in OID is 16383
pg_dump: reading extensions
pg_dump: identifying extension members
pg_dump: reading schemas
pg_dump: reading user-defined tables
pg_dump: [archiver (db)] query failed: SSL error: sslv3 alert bad record mac
pg_dump: [archiver (db)] query was: SELECT c.tableoid, c.oid, c.relname, (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(c.relacl,pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner))) WITH ORDINALITY AS perm(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner))) AS init(init_acl) WHERE acl = init_acl)) as foo) AS relacl, (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner))) WITH ORDINALITY AS initp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(c.relacl,pg_catalog.acldefault(CASE WHEN c.relkind = 'S' THEN 's' ELSE 'r' END::"char",c.relowner))) AS permp(orig_acl) WHERE acl = orig_acl)) as foo) as rrelacl, NULL AS initrelacl, NULL as initrrelacl, c.relkind, c.relnamespace, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = c.relowner) AS rolname, c.relchecks, c.relhastriggers, c.relhasindex, c.relhasrules, c.relhasoids, c.relrowsecurity, c.relforcerowsecurity, c.relfrozenxid, c.relminmxid, tc.oid AS toid, tc.relfrozenxid AS tfrozenxid, tc.relminmxid AS tminmxid, c.relpersistence, c.relispopulated, c.relreplident, c.relpages, CASE WHEN c.reloftype <> 0 THEN c.reloftype::pg_catalog.regtype ELSE NULL END AS reloftype, d.refobjid AS owning_tab, d.refobjsubid AS owning_col, (SELECT spcname FROM pg_tablespace t WHERE t.oid = c.reltablespace) AS reltablespace, array_remove(array_remove(c.reloptions,'check_option=local'),'check_option=cascaded') AS reloptions, CASE WHEN 'check_option=local' = ANY (c.reloptions) THEN 'LOCAL'::text WHEN 'check_option=cascaded' = ANY (c.reloptions) THEN 'CASCADED'::text ELSE NULL END AS checkoption, tc.reloptions AS toast_reloptions, c.relkind = 'S' AND EXISTS (SELECT 1 FROM pg_depend WHERE classid = 'pg_class'::regclass AND objid = c.oid AND objsubid = 0 AND refclassid = 'pg_class'::regclass AND deptype = 'i') AS is_identity_sequence, EXISTS (SELECT 1 FROM pg_attribute at LEFT JOIN pg_init_privs pip ON (c.oid = pip.objoid AND pip.classoid = 'pg_class'::regclass AND pip.objsubid = at.attnum)WHERE at.attrelid = c.oid AND ((SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(at.attacl,pg_catalog.acldefault('c',c.relowner))) WITH ORDINALITY AS perm(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault('c',c.relowner))) AS init(init_acl) WHERE acl = init_acl)) as foo) IS NOT NULL OR (SELECT pg_catalog.array_agg(acl ORDER BY row_n) FROM (SELECT acl, row_n FROM pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault('c',c.relowner))) WITH ORDINALITY AS initp(acl,row_n) WHERE NOT EXISTS ( SELECT 1 FROM pg_catalog.unnest(coalesce(at.attacl,pg_catalog.acldefault('c',c.relowner))) AS permp(orig_acl) WHERE acl = orig_acl)) as foo) IS NOT NULL OR NULL IS NOT NULL OR NULL IS NOT NULL))AS changed_acl, pg_get_partkeydef(c.oid) AS partkeydef, c.relispartition AS ispartition, pg_get_expr(c.relpartbound, c.oid) AS partbound FROM pg_class c LEFT JOIN pg_depend d ON (c.relkind = 'S' AND d.classid = c.tableoid AND d.objid = c.oid AND d.objsubid = 0 AND d.refclassid = c.tableoid AND d.deptype IN ('a', 'i')) LEFT JOIN pg_class tc ON (c.reltoastrelid = tc.oid) LEFT JOIN pg_init_privs pip ON (c.oid = pip.objoid AND pip.classoid = 'pg_class'::regclass AND pip.objsubid = 0) WHERE c.relkind in ('r', 'S', 'v', 'c', 'm', 'f', 'p') ORDER BY c.oid
Backup finished at Sun Sep 02 22:10:21 EDT 2018
Process exit code: 1
UPDATE
I believe this has to do with a large query problem instead of a ssl problem. I did an insert with 50 rows and no problem. As soon as i go over a certain number of characters it seems to break connecting to the server. Also I believe the pg_dump is a problem because the query it tries to do is a very large query. This is a local problem because it doesnt happen on my other laptop.
Probably some max setting in Postgres 10.5-2 or JDBC. My old laptop has the same JDBC version as my new one. Old laptop has Postgres 10.4-1 though.
So this turns out not to be related directly with Postgres.
I have a MSI GS65. It has a hardware network driver called killer network.
The driver has settings you can modify. One of those setting is to optimize network speeds. It seems that this setting will drop larger connections. When I am doing a backup or a large query through dbeaver, it drops the connection and stops the backup/query.
Took me 2 years to find this issue. I hope this helps any devs with a MSI computer with network issues.
I'm looking for a solution to generate postgreSQL database statistics programmatically. The psql commmand "\d+" is exactly what I need (output-wise), but so far I didn't find a way to retrieve this same data programmatically (by using C# and npgsql.dll). Any suggestions? Thanks in advance!
The answer is here: http://wiki.postgresql.org/wiki/Disk_Usage
I find it useful to have a function: https://gist.github.com/ArtemGr/8434901
See also: PostgreSQL "DESCRIBE TABLE"
Apparently the canonical way to find this out is to start psql with the -E option.
Then it spits out the queries it is using (or any query it runs internally). In this case, it spits out this:
=> \d test_int;
********* QUERY **********
SELECT c.oid,
n.nspname,
c.relname
FROM pg_catalog.pg_class c
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE c.relname ~ '^(test_int)$'
AND pg_catalog.pg_table_is_visible(c.oid)
ORDER BY 2, 3;
**************************
********* QUERY **********
SELECT c.relchecks, c.relkind, c.relhasindex, c.relhasrules, c.relhastriggers, c.relhasoids, '', c.reltablespace, CASE WHEN c.reloftype = 0 THEN '' ELSE c.reloftype::pg_catalog.regtype::pg_catalog.text END, c.relpersistence
FROM pg_catalog.pg_class c
LEFT JOIN pg_catalog.pg_class tc ON (c.reltoastrelid = tc.oid)
WHERE c.oid = '57695';
**************************
********* QUERY **********
SELECT a.attname,
pg_catalog.format_type(a.atttypid, a.atttypmod),
(SELECT substring(pg_catalog.pg_get_expr(d.adbin, d.adrelid) for 128)
FROM pg_catalog.pg_attrdef d
WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef),
a.attnotnull, a.attnum,
(SELECT c.collname FROM pg_catalog.pg_collation c, pg_catalog.pg_type t
WHERE c.oid = a.attcollation AND t.oid = a.atttypid AND a.attcollation <> t.typcollation) AS attcollation,
NULL AS indexdef,
NULL AS attfdwoptions
FROM pg_catalog.pg_attribute a
WHERE a.attrelid = '57695' AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum;
**************************
********* QUERY **********
SELECT c.oid::pg_catalog.regclass FROM pg_catalog.pg_class c, pg_catalog.pg_inherits i WHERE c.oid=i.inhparent AND i.inhrelid = '57695' ORDER BY inhseqno;
**************************
********* QUERY **********
SELECT c.oid::pg_catalog.regclass FROM pg_catalog.pg_class c, pg_catalog.pg_inherits i WHERE c.oid=i.inhrelid AND i.inhparent = '57695' ORDER BY c.oid::pg_catalog.regclass::pg_catalog.text;
**************************
Table "name_space.test_int"
Column | Type | Modifiers
--------+---------+-----------
yo | integer |
Which implies that it is getting the oid of the table from the first query, then using it to lookup various attributes in the rest of the queries.