I return to this topic
I have a database with
1 milion researches (psq)
1 milion publications (pub)
12.5 milion edges between psq and pub
I execute this query
SELECT
psq1.psq_nome AS nomePesquisador, COUNT(pub1) AS qtdPub
FROM
(MATCH
{class:Pesquisador, as:psq1}.outE("PUBLICOU").inV(){as:pub1}
RETURN psq1, pub1)
GROUP BY
psq1
ORDER BY
qtdPub DESC, nomePesquisador
LIMIT 1000;
OR
SELECT out.psq_nome AS nomePesquisador, COUNT(in.#rid) AS qtdPub
FROM Publicou
GROUP BY out.psq_nome
ORDER BY qtdPub DESC, nomePesquisador
LIMIT 1000;
Then there are errors of memory heap
How I configure the OrientDb to permit I execute this?
The most important thing on tuning is assuring the memory settings are correct. What can make the real difference is the right balancing between the heap and the virtual memory used by Memory Mapping, specially on large datasets (GBs, TBs and more) where the in memory cache structures count less than raw IO.
For example if you can assign maximum 8GB to the Java process, it's usually better assigning small heap and large disk cache buffer (off-heap memory). So rather than:
java -Xmx8g ...
You could instead try this:
java -Xmx800m -Dstorage.diskCache.bufferSize=7200 ...
Reference: Performance Tuning
Related
I have a very simple query:
SELECT count(*) FROM link_list ll
WHERE ll.tsvector ## to_tsquery('english', 'advertising|slogan|publicity|ads|advertise|brand|promo|ad|propaganda|sales|promote|logo|hype|commercial|consumerism|pamphlet|viral|salesperson|selling|blurb|advert|merchandise|marking|products|campaign|packaging|billboard|advertisement|promotion|questionnaire|marketing')
It runs fast (Here's the output from EXPLAIN ANALYZE). It uses the GIN index, and works exactly as expected.
Life is good.
But now, let me tweak the query slightly, and now it takes 2x as long!
WITH query AS
(
SELECT to_tsquery('english',('advertising|slogan|publicity|ads|advertise|brand|promo|ad|propaganda|sales|promote|logo|hype|commercial|consumerism|pamphlet|viral|salesperson|selling|blurb|advert|merchandise|marking|products|campaign|packaging|billboard|advertisement|promotion|questionnaire|marketing')) query
)
SELECT count(*) FROM link_list ll
WHERE ll.tsvector ## (SELECT query FROM query);
(Output from EXPLAIN ANALYZE)
I would just use the first query... but unfortunately, the list of synonyms has to be dynamically generated, which I pull from another table.
For some strange reason, putting the tsquery inside WITH makes Postgresql not use the index as efficiently (It thinks it'll be a quick and dirty job that doesn't need an index, and it ends up being dead wrong).
Why in the world is this happening??
Neither one of those executions seems all that fast.
Your use of the WITH inhibits the parallel plan. If your bottleneck is IO (which seems likely) you can get parallel IO without parallel query by setting effective_io_concurrency to a value > 1.
The time spent setting up JIT is over 10% of the faster plan, and is probably a complete waste. You can set jit = off (or turn it off globally in the .conf file) to spare that time.
Your recheck of lossy blocks also wastes time. You should increase work_mem to get rid of those. But the waste is of mostly CPU time, so the effect will be small if the bottleneck is IO, not CPU. It still has to visit the same set of blocks. (An exception to this is if TOAST is heavily used, then rows not rechecked don't need to be assembled from TOAST, so those TOAST block reads are avoided.)
I am investigating a few slow queries and I need some help reading the data I got.
We have this one particular query which uses an index and runs pretty fast most of the time, however from time to time it runs slow (700ms+), not sure why.
Limit (cost=8.59..8.60 rows=1 width=619) (actual time=5.653..5.654 rows=1 loops=1)
-> Sort (cost=8.59..8.60 rows=1 width=619) (actual time=5.652..5.652 rows=1 loops=1)
Sort Key: is_main DESC, id
Sort Method: quicksort Memory: 25kB
-> Index Scan using index_pictures_on_imageable_id_and_imageable_type on pictures (cost=0.56..8.58
rows=1 width=619) (actual time=3.644..5.587 rows=1 loops=1)
Index Cond: ((imageable_id = 12345) AND ((imageable_type)::text = 'Product'::text))
Filter: (tag = 30)
Rows Removed by Filter: 2
Planning Time: 1.699 ms
Execution Time: 5.764 ms
If I understand that correctly, I would say that almost the entire cost of the query is on index scan, right? which sounds good to me, so why does the same query run pretty slow sometimes?
I started to think that maybe our instance is not being able to keep the entire index in memory, so it is using disk from time to time. That would explain the slow queries. However, that is way over my head. Does that make sense?
That table has around 15 million rows and 5156 MB in size. Index is 1752 MB. BTW, it is a btree index.
Our PostgreSQL is on a "Highly available" Google Cloud SQL instance. It has 2 vCPUs and 7.5 GB of RAM. Our entire database is around 35 GB in size.
CPU consumption almost never goes beyond 40%. It usually settles around 20-30%.
Checking instance memory graph, I noticed that consumption grows until ~4 GB, then it drops down ~700 MB and it starts growing again. That is a repetitive pattern.
In theory, the instance has 7.5 GB of RAM, but I don't know if all of it is supposed to be available for PostgreSQL. Anyway, ~3.5 GB just for OS sounds pretty high, right?
Memory graph
I read that these configs are important, so throwing them here (Cloud SQL defaults):
shared_buffers | 318976
temp_buffers | 1024
work_mem | 4096
Considering that we have a bunch of other tables and indexes, is it reasonable to assume that if one index alone is 1.7 GB, 7.5 GB for the entire instance is too low?
Is there any way I can assert whether we have a memory issue or not?
I appreciate your help.
Three things that can help you:
This function do a "prewarm" on table permanently on your memory. This reduces drastically your disk access, helping a lot on performance. The limitation for prewarm is resources. So, not all tables can be put on memory. If the table is small or not constantly accessed, it's not recommended. Every time that your database is stopped, on the next up of database, you need to run pg_prewarm() again
https://www.postgresql.org/docs/current/pgprewarm.html
Create a CLUSTER on your index. You can create one cluster per table. Clustering your index is a great way to get a good access of the data. The way that data is stored is related with cluster, so, to access a determined position on previously ordered data is very faster.
CLUSTER [VERBOSE] table_name [ USING index_name ]
Reference: https://www.postgresql.org/docs/current/sql-cluster.html
Run periodically VACUUM ANALYZE on table. Postgresql collect statistics about your queries and classifies the information in vacuum with analyze option focused on optimize your queries.
I think is more a memory problem as you say. Checking your graph I can say that most of the time your database is using the 4GB of memory assigned and when you run your query postgres has to use the disk.
I suppose your query runs faster is when is under the memory limit. Another thing to consider is that maybe, time ago, your database was not big as now and with the dafult memory assign (4 GB) was ok.
You can modify your memory assigned to postgres configuring the flags, in particular the work_mem flag. I suggest to assign 2GB of extra memory and check the results. If you see your database uses again the 100% of the memory, consider increasing the whole memory and the memory assigned to the database.
As I understand it (after a fair amount of searching online)...
1- If a component of a query (sort, join, etc.) uses more RAM/memory than my work_mem setting or the total memory used by all current operations on the server exceeds available OS memory, the query will start writing to disk.
Is that true?
2- Postgres (and many other good DB engines) use memory to cache a lot so queries go faster; therefore, the server should indicate low free memory even if the server isn't really starved for memory. So low free memory doesn't really indicate anything other than a good DB engine and healthy utilization.
Is that true?
3- If both #1 and #2 above are true, holding everything else content, if I want a board indicator of a work_mem setting that is too low or not enough overall OS memory, I should look to see if the server free disk space is going down?
Am I thinking about this correctly?
links:
https://www.postgresql.org/docs/current/static/runtime-config-resource.html
http://patshaughnessy.net/2016/1/22/is-your-postgres-query-starved-for-memory
https://www.enterprisedb.com/monitor-cpu-and-memory-percentage-used-each-process-postgresqlppas-9
https://dba.stackexchange.com/questions/18484/tuning-postgresql-for-large-amounts-of-ram
I know I can set log_temp_files and look at individual temp files to tune the work_mem setting, but I wanted an overall gauge I could use to determine if possibly work_mem is too low before I start digging around looking at temp file sizes that exceed my work_mem setting.
I have PostgreSQL 10.
Processing a query takes a number of steps:
generate (all)possible plans
estimate the cost of execution of these plans (in terms of resources: disk I/O,buffers,memory,CPU), based on tuning constants and statistics.
pick the "optimal" plan , based on tuning constants
execute the chosen plan.
In most cases, a plan that is expected (step2) to need more work_mem than your work_mem setting will not be chosen in step3. (because "spilling to disk" is considered very expensive)
Once step4 detects that it is needing more work_mem, its only choice is to spill to disk. Shit happens... At least this doesn't rely on the OS's page-swapping the the overcommitted memory.)
The rules are very simple:
hash-joins are often optimal but will cost memory
don't try to use more memory than you have
if there is a difference between expected(step2) and observed(step4) memory, your statistics are wrong. You will be punished by spill-to-disk.
a lack of usable indexes will cause hash joins or seqscans.
sorting uses work_mem, too. The mechanism is similar :bad estimates yield bad plans.
CTE's are often/allways(?) materialized. This will splill to disk once your bufferspace overflows.
CTE's don't have statistics, and don't have indices.
A few guidelines/advice:
use a correct data model (and don't denormalize)
use the correct PK/FK's and secundary indices.
run ANALYZE the_table_name; to gather fresh statistics after huge modifications to the table's structure or data.
Monitoring:
check the Postgres logfile
check the query plan, compare observed <--> expected
monitor the system resource usage (on Linux: via top/vmstat/iostat)
Is there a reason why the same query executed a number of times have huge variance in response times? from 50% - 200% what the projected response time is? They range from 6 seconds to 20 seconds even though it is the only active query in the database.
Context:
Database on Postgres 9.6 on AWS RDS (with Provisioned IOPS)
Contains one table comprising five numeric columns, indexed on id, holding 200 million rows
The query:
SELECT col1, col2
FROM calculations
WHERE id > 0
AND id < 100000;
The query's explain plan:
Bitmap Heap Scan on calculation (cost=2419.37..310549.65 rows=99005 width=43)
Recheck Cond: ((id > 0) AND (id <= 100000))
-> Bitmap Index Scan on calculation_pkey (cost=0.00..2394.62 rows=99005 width=0)
Index Cond: ((id > 0) AND (id <= 100000))
Is there any reasons why a simple query like this isn't more predictable in response time?
Thanks.
When you see something like this in PostgreSQL EXPLAIN ANALYZE:
(cost=2419.37..310549.65)
...it doesn't mean the cost is between 2419.37 and 310549.65. These are in fact two different measures. The first value is the startup cost, and the second value is the total cost. Most of the time you'll care only about the total cost. The times that you should be concerned with startup cost is when that component of the execution plan is in relation to (for example) an EXISTS clause, where only the first row needs to be returned (so you only care about startup cost, not total, as it exits almost immediately after startup).
The PostgreSQL documentation on EXPLAIN goes into greater detail about this.
A query may be (and should be, excluding special cases) more predictable in response time when you are a sole user of the server. In the case of a cloud server, you do not know anything about the actual server load, even if your query is the only one performed on your database, because the server most likely supports multiple databases at the same time. As you asked about response time, there may be also various circumstances involved in accessing a remote server over the network.
After investigation of the historical load, we have found out that the provisioned IOPS we originally configured had been exhausted during the last set of load tests performed on the environment.
According to Amazon's documentation #http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html, after this point, Amazon does not guarantee consistency in execution times and the SLAs are no longer applicable.
We have confirmed that replicating the database onto a new instance of AWS RDS with same configuration yields consistent response times when executing the query multiple times.
I have an RDS PostgreSQL instance that's running simple queries, much slower than I would expect - particularly sequential scans, like copying a table or counting a table.
Eg. create table copied_table as (select * from original_table) or select count(*) from some_table
Running count(*) on a 30GB table takes ~15 minutes (with indexes, immediately following a vaccuum).
It's an RDS db.r3.large, 15 GB memory, 400GB SSD. Watching the metrics logs, I've never seen Read IOPS exceed 1,400 and it's usually around 500, well below my expected base.
Configuration:
work_mem: 2GB,
shared_buffers: 3GB,
effective_cache_size: 8GB
wal_buffers: 16MB,
checkpoint_segments: 16
Is this the expected timing? Should I be seeing higher IOPS?
There is not much you can do around plain count queries like that in Postgres, except in 9.6 that implemented parallel sequential scans, which is not available yet in RDS.
Event though, there is a some tips that you can find here. Generally, it's recommended to try to make Postgres to use Index Only Scan, by creating indexes and it's columns in the projection.
SELECT id FROM table WHERE id > 6 and id <100;
-- or
SELECT count(id) FROM table ...
Table should have an index on that column.
The queries that you exposed as example, won't avoid the sequential scan. For the CREATE TABLE, if you don't care about the order in the table, you can open a few backends and import in parallel by filtering by a key range. Also, the only way to speed up this on RDS is increasing IOPs.