How to identify the total size of the db (Used) and the total size of the db.
for the total size of the db (Used) : pg_database_size('dbName') cmmnd works.
But I am not sure how to calculate the free space size of the db. (total capacity I mean)
I saw pg_spacefree('table'). but it would need GRANT access. Is there any other way?
please guide me here?
As I mentioned in the comment you can create your own function in an untrusted language, for example, plpython
CREATE EXTENSION plpython3u;
CREATE TYPE disk_use AS (Total_GB numeric , Used_GB numeric, free_GB numeric);
CREATE OR REPLACE FUNCTION fn_disk_use(part text default '/') RETURNS disk_use AS
$$
import shutil
hdd = shutil.disk_usage(part)
return (hdd.total / (2**30), hdd.used / (2**30),hdd.free / (2**30))
$$ LANGUAGE plpython3u;
SELECT * from fn_disk_use('/');
total_gb | used_gb | free_gb
-------------------+-------------------+--------------------
455.2938232421875 | 402.6443672180176 | 29.455318450927734
(1 row)
Consider the following xml document that is stored in a PostgreSQL field:
<E_sProcedure xmlns="http://www.minushabens.com/2008/FMSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" modelCodeScheme="Emo_ex" modelCodeSchemeVersion="01" modelCodeValue="EMO_E_PROCEDURA" modelCodeMeaning="Section" sectionID="11">
<tCatSnVsn_Pmax modelCodeScheme="Emodinamica_referto" modelCodeSchemeVersion="01" modelCodeValue="tCat4" modelCodeMeaning="My text"><![CDATA[1]]></tCatSnVsn_Pmax>
</E_sProcedure>
If I run the following query I get the correct result for Line 1, while Line 2 returns nothing:
SELECT
--Line 1
TRIM(BOTH FROM array_to_string((xpath('//child::*[#modelCodeValue="tCat4"]/text()', t.xml_element)),'')) as tCatSnVsn_Pmax_MEANING
--Line2
,TRIM(BOTH FROM array_to_string((xpath('/tCatSnVsn_Pmax/text()', t.xml_element)),'')) as tCatSnVsn_Pmax
FROM (
SELECT unnest(xpath('//x:E_sProcedure', s.XMLDATA::xml, ARRAY[ARRAY['x', 'http://www.minushabens.com/2008/FMSchema']])) AS xml_element
FROM sr_data as s)t;
What's wrong in the xpath of Line 2?
Your second xpath() doesn't return anything because of two problems. First: you need to use //tCatSnVsn_Pmax as the xml_element still starts with <E_sProcedure>. The path /tCatSnVsn_Pmax tries to select a top-level element with that name.
But even then, the second one won't return anything because of the namespace. You need to pass the same namespace definition to the xpath(), so you need something like this:
SELECT (xpath('/x:tCatSnVsn_Pmax/text()', t.xml_element, ARRAY[ARRAY['x', 'http://www.minushabens.com/2008/FMSchema']]))[1] as tCatSnVsn_Pmax
FROM (
SELECT unnest(xpath('//x:E_sProcedure', s.XMLDATA::xml, ARRAY[ARRAY['x', 'http://www.minushabens.com/2008/FMSchema']])) AS xml_element
FROM sr_data as s
)t;
With modern Postgres versions (>= 10) I prefer using xmltable() for anything nontrivial. It makes passing namespaces easier and accessing multiple attributes or elements.
SELECT xt.*
FROM sr_data
cross join
xmltable(xmlnamespaces ('http://www.minushabens.com/2008/FMSchema' as x),
'/x:E_sProcedure'
passing (xmldata::xml)
columns
sectionid text path '#sectionID',
pmax text path 'x:tCatSnVsn_Pmax',
model_code_value text path 'x:tCatSnVsn_Pmax/#modelCodeValue') as xt
For your sample XML, the above returns:
sectionid | pmax | model_code_value
----------+------+-----------------
11 | 1 | tCat4
I am trying to get the index position of a POINT in a MULTILINESTRING.
Here is the whole query I'm stuck with :
SELECT req.id, (dp).geom, netgeo_point_tech.id, ST_Line_Locate_Point(st_lineMerge(geom_cable), (dp).geom)
FROM (SELECT id, ST_DumpPoints(geom) as dp, geom as geom_cable FROM netgeo_cable_test ) as req
JOIN netgeo_point_tech ON ST_dwithin(netgeo_point_tech.geom, (dp).geom, 1)
ORDER BY req.id, (dp).path [ 1] ASC
The error I get is : line_locate_point : 1st arg isnt a line.
The error is due to the return of st_lineMerge() function that is returning LINESTRING but also MULTILINESTRING.
I don't get this. st_lineMerge() is supposed to return only LINESTRING.ST_LineMerge()
When I jsut try a simple query like this :
select st_astext(st_linemerge(geom)) from netgeo_cable_test
The output is :
)
I want to learn from this, so, if possible, explain to me what I'm doing wrong here, or if my approach is lacking insight.
Thanks to #JGH for the suggestion to use ST_Dump I came up with this function:
create or replace function MultiLineLocatePoint(line geometry, point geometry) returns numeric as $$
select (base + extra) / ST_Length(line)
from (
select
sum(ST_Length(l.geom)) over (order by l.path) - ST_Length(l.geom) base,
ST_LineLocatePoint(l.geom, point) * ST_Length(l.geom) extra,
ST_Distance(l.geom, point) dist
from ST_Dump(line) l
) points
order by dist
limit 1;
$$ language SQL;
I get what seems like a bug in Postgresql/Postgis. This is a completely reproducible example that demonstrates the problem:
#create a new srid for local area (formula for proj4text is taken from Quantum GIS)
INSERT INTO spatial_ref_sys (srid, auth_name, auth_srid, proj4text) VALUES (998997, 'EPSG', 998997, '+proj=tmerc +lat_0=0 +lon_0=44.55 +k=1 +x_0=2250000 +y_0=-5714743.504 +ellps=krass +towgs84=24,-123,-94,0.02,-0.25,-0.13,1.1 +units=m +no_defs ');
#create a new table for storing geometry data:
CREATE TABLE layer(
id SERIAL PRIMARY KEY,
geom geometry,
CONSTRAINT enforce_dims_geom_layer CHECK (st_ndims(geom) = 2),
CONSTRAINT enforce_srid_geom_layer CHECK (st_srid(geom) = 998997)
);
#add one polygon to the table:
INSERT INTO layer (geom) VALUES (ST_Force2D(ST_Transform(ST_GeomFromText('POLYGON((4832654.676302 7570323.2813639, 4810946.560269 7597840.6115465, 4836629.4017728 7629944.1634263, 4886772.0923279 7629944.1634263, 4902059.4979849 7591725.6492837, 4864452.4800686 7553507.1351411, 4832654.676302 7570323.2813639),(4845190.3489408 7589891.1606049, 4855585.7847875 7610376.2841853, 4876988.1527074 7604567.0700356, 4874847.9159154 7588362.4200392, 4858031.7696927 7575520.9992872, 4845190.3489408 7589891.1606049))', 3857),998997)));
#check that this polygon is valid!
SELECT ST_IsValid(ST_Force2D(ST_Transform(ST_GeomFromText('POLYGON((4832654.676302 7570323.2813639, 4810946.560269 7597840.6115465, 4836629.4017728 7629944.1634263, 4886772.0923279 7629944.1634263, 4902059.4979849 7591725.6492837, 4864452.4800686 7553507.1351411, 4832654.676302 7570323.2813639),(4845190.3489408 7589891.1606049, 4855585.7847875 7610376.2841853, 4876988.1527074 7604567.0700356, 4874847.9159154 7588362.4200392, 4858031.7696927 7575520.9992872, 4845190.3489408 7589891.1606049))', 3857),998997))) AS is_valid;
#^^^ it returns t. so, the geometry is 100% ok.
#check how it looks like in geojson format:
SELECT ST_AsGeoJSON(ST_Transform(geom, 3857))::json from layer
#^^^ Again it's ok and returns some nice geojson data
#Final step. Check ST_ApproximateMedialAxis function
SELECT ST_ApproximateMedialAxis(ST_Transform(geom, 3857)) from layer;
The final query returns an error message:
ERROR: Polygon is invalid : exterior ring and interior ring 0 have the same orientation : POLYGON((5189023446929109/1073741824 2032143182100511/268435456,5165714534831303/1073741824 31867653268373/4194304,1298322818976529/268435456 8192590163056015/1073741824,26
********** Error **********
So, all ingredients seem to be ok - formula for srid is taken from some standard widely used tool, data is inserted into the table without problems, the data is validated by ST_IsValid, geojson representation of data is also ok, but one library function still does not like something.
I have found this nice thread and came to the solution. I just have to use ST_ForceRHR Postgis function which
forces the orientation of the vertices in a polygon to follow the Right-Hand-Rule
So the right way to insert the data was:
INSERT INTO layer (geom) VALUES (ST_Force2D(ST_Transform(ST_ForceRHR(ST_GeomFromText('POLYGON((4832654.676302 7570323.2813639, 4810946.560269 7597840.6115465, 4836629.4017728 7629944.1634263, 4886772.0923279 7629944.1634263, 4902059.4979849 7591725.6492837, 4864452.4800686 7553507.1351411, 4832654.676302 7570323.2813639),(4845190.3489408 7589891.1606049, 4855585.7847875 7610376.2841853, 4876988.1527074 7604567.0700356, 4874847.9159154 7588362.4200392, 4858031.7696927 7575520.9992872, 4845190.3489408 7589891.1606049))', 3857)),998997)));
Test Case:
drop table master;
create table master(id int primary key, fk1 int, fk2 int, fk3 int, dataS varchar(255), data1 int, data2 int, data3 int, data4 int,data5 int,data6 int,data7 int,data8 int,data9 int,b1 boolean,b2 boolean,b3 boolean,b4 boolean,b5 boolean,b6 boolean,b7 boolean,b8 boolean,b9 boolean,b10 boolean,b11 boolean,b12 boolean,b13 boolean,b14 boolean,b15 boolean,b16 boolean,b17 boolean,b18 boolean,b19 boolean,b20 boolean,b21 boolean,b22 boolean,b23 boolean,b24 boolean,b25 boolean,b26 boolean,b27 boolean,b28 boolean,b29 boolean,b30 boolean,b31 boolean,b32 boolean,b33 boolean,b34 boolean,b35 boolean,b36 boolean,b37 boolean,b38 boolean,b39 boolean,b40 boolean,b41 boolean,b42 boolean,b43 boolean,b44 boolean,b45 boolean,b46 boolean,b47 boolean,b48 boolean,b49 boolean,b50 boolean);
create index idx_comp on master(fk1,fk2,fk3);
#loop 5000000 insert into master values(?, mod(?,100), mod(?,5), ?,'Hello World Hello World Hello World',?, ?, ?,?, ?, ?, ?, ?, ?,true,true,true,true,true,true,false,false,false,true,true,true,true,true,true,true,false,false,false,true,true,true,true,true,true,true,false,false,false,true,true,true,true,true,true,true,false,false,false,true,true,true,true,true,true,true,false,false,false,true);
1.The following select statement takes up to 30seconds. Is there a way to optimize the response time?
SELECT count(*), SUM(CONVERT(b1,INT)) ,SUM(CONVERT(b2,INT)),SUM(CONVERT(b3,INT)),SUM(CONVERT(b4,INT)),SUM(CONVERT(b5,INT)),SUM(CONVERT(b6,INT)),SUM(CONVERT(b7,INT)),SUM(CONVERT(b8,INT)),SUM(CONVERT(b9,INT)),SUM(CONVERT(b10,INT)),SUM(CONVERT(b11,INT)),SUM(CONVERT(b12,INT)),SUM(CONVERT(b13,INT)),SUM(CONVERT(b14,INT)),SUM(CONVERT(b15,INT)),SUM(CONVERT(b16,INT))
FROM master
WHERE fk1=53 AND fk2=3
2.I tried shutdown defrag. But this statement took about 40min for my test case. After shutdown defrag the select takes up to 15seconds. If i execute the statement again it takes under 1sec. Even if stop and start the server, the statement takes about 1sec.
Has H2 a persistent Cache?
Infrastructure: WebBrowser <-> H2 Console Server <-> H2 DB: h2 1.3.158
According to the profiler output, the main problem (93%) is reading from the disk. I ran this in the H2 Console:
#prof_start;
SELECT ... FROM master WHERE fk1=53 AND fk2=3;
#prof_stop;
and got:
Profiler: top 3 stack trace(s) of 48039 ms [build-158]:
4084/4376 (93%):
at java.io.RandomAccessFile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessFile.java:338)
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:397)
at org.h2.store.FileStore.readFully(FileStore.java:285)
at org.h2.store.PageStore.readPage(PageStore.java:1253)
at org.h2.store.PageStore.getPage(PageStore.java:707)
at org.h2.index.PageDataIndex.getPage(PageDataIndex.java:225)
at org.h2.index.PageDataNode.getRowWithKey(PageDataNode.java:269)
at org.h2.index.PageDataNode.getRowWithKey(PageDataNode.java:270)
According to EXPLAIN ANALYZE SELECT it's reading over 55'000 pages from the disk (2 KB each page; 110 MB) for this query. I'm not sure how other databases perform for such a query. But I guess if possible the query should be changed so that it reads less data.
Is it possible to have a temporary table/view that already has the datatype conversions done? If it's feasible to have that update itself from the main table occassionally (once a night or so), then you've got a lot of processing power that goes into the conversion done already.
If that's not feasible, you may want to do multiple sub-selects, one for each "b" column, where you only pull where b# = 1. Then do a COUNT instead of a SUM, which should be faster as well. For instance:
SELECT (count1+count2) AS Count,
(SELECT COUNT(*) FROM master WHERE fk1=53 AND fk2=3 AND b1=1) AS count1
(SELECT COUNT(*) FROM master WHERE fk1=53 AND fk2=3 AND b2=1) AS count2
I'm not sure if that exact syntax works in your program, but hopefully as a generic SQL idea it gets you on the right track.