Could anyone explains to me what is the MEM%, displayed by pg_activity?
I would like to know if MEM% represents \postgres total memory usage\, or 'the system memory usage'.
Thanks,
Related
I was learning to on how to snapshot of javascript heap memory and I came across 2 terms
"Summery View" and "Containment view".
The definition provided by there documentation is not clear and doesn't helps in understanding there usage
Can any one please help me to get clarity on following questions please?
When to use summary view and containment view, with a small example
In containment view the "retained" size of all element doesn't sum up to be total size of snapshot, then were i can find rest of memory being used
In summery view, total retained memory size looks greater then total heap size, why?
Good day,
There are a lot of the same themes, but i wonder how i can get Ram usage for the exact process, for instance, notepad++ (need to search by name). The output shouldn't be in the table view, just MB and %
Thank you for your help
As I understand it (after a fair amount of searching online)...
1- If a component of a query (sort, join, etc.) uses more RAM/memory than my work_mem setting or the total memory used by all current operations on the server exceeds available OS memory, the query will start writing to disk.
Is that true?
2- Postgres (and many other good DB engines) use memory to cache a lot so queries go faster; therefore, the server should indicate low free memory even if the server isn't really starved for memory. So low free memory doesn't really indicate anything other than a good DB engine and healthy utilization.
Is that true?
3- If both #1 and #2 above are true, holding everything else content, if I want a board indicator of a work_mem setting that is too low or not enough overall OS memory, I should look to see if the server free disk space is going down?
Am I thinking about this correctly?
links:
https://www.postgresql.org/docs/current/static/runtime-config-resource.html
http://patshaughnessy.net/2016/1/22/is-your-postgres-query-starved-for-memory
https://www.enterprisedb.com/monitor-cpu-and-memory-percentage-used-each-process-postgresqlppas-9
https://dba.stackexchange.com/questions/18484/tuning-postgresql-for-large-amounts-of-ram
I know I can set log_temp_files and look at individual temp files to tune the work_mem setting, but I wanted an overall gauge I could use to determine if possibly work_mem is too low before I start digging around looking at temp file sizes that exceed my work_mem setting.
I have PostgreSQL 10.
Processing a query takes a number of steps:
generate (all)possible plans
estimate the cost of execution of these plans (in terms of resources: disk I/O,buffers,memory,CPU), based on tuning constants and statistics.
pick the "optimal" plan , based on tuning constants
execute the chosen plan.
In most cases, a plan that is expected (step2) to need more work_mem than your work_mem setting will not be chosen in step3. (because "spilling to disk" is considered very expensive)
Once step4 detects that it is needing more work_mem, its only choice is to spill to disk. Shit happens... At least this doesn't rely on the OS's page-swapping the the overcommitted memory.)
The rules are very simple:
hash-joins are often optimal but will cost memory
don't try to use more memory than you have
if there is a difference between expected(step2) and observed(step4) memory, your statistics are wrong. You will be punished by spill-to-disk.
a lack of usable indexes will cause hash joins or seqscans.
sorting uses work_mem, too. The mechanism is similar :bad estimates yield bad plans.
CTE's are often/allways(?) materialized. This will splill to disk once your bufferspace overflows.
CTE's don't have statistics, and don't have indices.
A few guidelines/advice:
use a correct data model (and don't denormalize)
use the correct PK/FK's and secundary indices.
run ANALYZE the_table_name; to gather fresh statistics after huge modifications to the table's structure or data.
Monitoring:
check the Postgres logfile
check the query plan, compare observed <--> expected
monitor the system resource usage (on Linux: via top/vmstat/iostat)
I'm getting an access violation in a program. Windbg shows that the program is trying to read at 0x09015000. It shows question marks (??) next to the address. My question is, what do these question marks indicate. Do they mean the memory location was never allocated, i.e. it's not backed by any physical memory (or page file)? Or is it something else?
It means that the virtual address is bad. Possibly a bogus pointer (i.e. uninitialized garbage), freed memory, etc.
Do they mean the memory location was never allocated
That's one possibility. Other options:
it was allocated before, but has been freed (VirtualFree())
it's not included in the crash dump you analyze. This may depend on the MINIDUMP_TYPE. Also, Procdump has an option ( -mp) to exclude memory regions larger than 512 MB.
Can somebody give some clear explanation of the meaning of the SIZE and RSS values we get from prstat in Solaris?
I wrote a testing C++ application that allocates memory with new[], fills it and frees it with delete[].
As I understood, the SIZE value should be related to how much virtual memory has been "reserved" by the process, that is memory "malloced" or "newed".
That memory doesn't sum up in the RSS value unless I really use it (filling with some values). But then even if I free the memory, the RSS doesn't drop.
I don't understand what semantic I can correctly assign to those 2 values.
RSS is (AFAIK reliably) representing how much physical memory a process is using. Using Solaris default memory allocator, freeing memory doesn't do anything about RSS as it just changes some pointers and values to tell that memory is free to be reused.
If you don't use again that memory by allocating it again, it will eventually be paginated out and the RSS will drop.
If you want freed memory to be returned immediately after a free, you can use the Solaris mmap allocator like this:
export LD_PRELOAD=libumem.so
export UMEM_OPTIONS=backend=mmap
Size is the total virtual memory size of the process, including all mapped files and devices, and RSS should be the resident set size, but is completely unreliable, you should try to get that information from pmap.
As a general rule once memory is allocated to a process it will never be given back to the operating system. On Unix systems the sbrk() call is used to extend the processes address space, and there is not analogous call to go in the other direction.