I'm on RHEL6 and have installed PostgreSQL. Now whenever I want to start development I need to run the following command to start PostgreSQL
/opt/PostgreSQL/9.5/bin/postgres -D /opt/PostgreSQL/9.5/data
Then it halts for that terminal and I need to start another session of postgresql into another terminal. Whats wrong in Installation? and How to rectify this problem?
Image of practical for better understanding
https://www.postgresql.org/docs/current/static/app-postgres.html
The utility command pg_ctl can be used to start and shut down the
postgres server safely and comfortably.
If at all possible, do not use SIGKILL to kill the main postgres
server. Doing so will prevent postgres from freeing the system
resources (e.g., shared memory and semaphores) that it holds before
terminating. This might cause problems for starting a fresh postgres
run.
use pg_ctl -D /opt/PostgreSQL/9.5/data start instead, otherwise one day your database will tell you about corrupted data
I test now an web application at my job. I use Debian. I don't know what kind of project is, just know that is built in Perl and uses PostgreSQL. The project where Back-End is built, uses CARTON, a Perl module dependency manager (aka Bundler for Perl) http://search.cpan.org/~miyagawa/Carton-v1.0.12/lib/Carton.pm. To run the Back-End I have to start PostgreSQL sudo su postgres and then execute command carton exec foo and Back-End start to work. But today, after some updates and upgrades, I executed it and got this error message Segmentation fault. I found that to check what was going on I had to get a backtrace so I found and read this article:
https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD
but still don't understand how to run the project with GDB.
Thanks
but still didn't find out how to get a backtrace. Once I start in gdb mode, I don't know what to type to run the back-end in gdb mode.
It's hard to answer your question because it's not clear exactly where you are getting stuck.
Use the first link you provided, to attach GDB to the running back end:
sudo gdb -p pid
(gdb) continue
Now execute whatever command causes your backend to crash. Once you do, GDB will stop and print something like:
Program received signal SIGSEGV, Segmentation fault.
0x00000000004004c0 in foo (p=0x0) at t.c:1
(gdb)
Now you are ready to obtain the crash stack trace, by using where GDB command.
(gdb) where
#0 0x00000000004004c0 in foo (p=0x0) at t.c:1
#1 0x00000000004004dc in bar (p=0x0) at t.c:2
#2 0x00000000004004ec in main () at t.c:4
You will likely not get file/line info and parameter values (you'll need to install debuginfo packages described in your second link for that), but you should get function names, which may be sufficient to find the relevant bug.
I am trying to terminate a session (a specific session or all sessions, doesnt matter) in postgresql 8.3 and am having trouble doing that. I know in newer versions (8.4 and later) there is a pg_terminate_backend command that will do the trick but this is not available in postgresql 8.3. If I use pg_stat_activity, I can see all the sessions that are active but have no way of terminating them.
The solution does not have to necessarily be sql commands but I would like it to be independent of the OS that is being used (i.e. no DOS/UNIX commands).
Stopping and starting the postgres service in windows services works perfectly but this is an OS specific approach. Using 'pg_ctl restart -D DATA_DIR' does not stop the service however. Actually using pg_ctl to try and restart the service at the time I am trying to do it causes some weird behavior. If there is a way I can somehow use pg_ctl to force shutdown the process like I assume windows does, then I can probably use that.
Anyways, I am looking for a way to terminate one or all sessions in postgresql 8.3 that is not platform specific. Any help would be great!
You can use pg_cancel_backend():
select pg_cancel_backend(55555);
You can use this with pg_stat_activity. For example:
select pg_cancel_backend(procpid)
from pg_stat_activity where current_query='<IDLE>';
If that doesn't work you can try this:
pg_ctl kill -TERM pid
That should be OS independent. I'm not sure if there's any real difference in behaviour.
Other than that you could try stopping and starting the server, but you indicated odd behaviour from that. (What kind?)
Finally, for an OS specific option, on linux you can of course try using the kill command. kill -15 (SIGTERM) is safe; that's basically what pg_terminate_backend uses: kill -15 <pid>. kill -9 is moderately unsafe and you should use it only as a last resort.
su - posgres
psql
SELECT pg_terminate_backend(pg_stat_activity.procpid) FROM pg_stat_activity WHERE procpid <> pg_backend_pid() AND datname = 'dbname' ;
drop database "database name";
Sometimes I run a Postgres query and it takes 30 seconds. Then, I immediately run the same query and it takes 2 seconds. It appears that Postgres has some sort of caching. Can I somehow see what that cache is holding? Can I force all caches to be cleared for tuning purposes?
I'm basically looking for a Postgres version of the following SQL Server command:
DBCC FREEPROCCACHE
DBCC DROPCLEANBUFFERS
But I would also like to know how to see what is actually contained in that buffer.
You can see what's in the PostgreSQL buffer cache using the pg_buffercache module. I've done a presentation called "Inside the PostgreSQL Buffer Cache" that explains what you're seeing, and I show some more complicated queries to help interpret that information that go along with that.
It's also possible to look at the operating system cache too on some systems, see [pg_osmem.py] for one somewhat rough example.
There's no way to clear the caches easily. On Linux you can stop the database server and use the drop_caches facility to clear the OS cache; be sure to heed the warning there to run sync first.
I haven't seen any commands to flush the caches in PostgreSQL. What you see is likely just normal index and data caches being read from disk and held in memory. by both postgresql and the caches in the OS. To get rid of all that, the only way I know of:
What you should do is:
Shutdown the database server (pg_ctl, sudo service postgresql stop, sudo systemctl stop postgresql, etc.)
echo 3 > /proc/sys/vm/drop_caches
This will clear out the OS file/block caches - very important though I don't know how to do that on other OSs. (In case of permission denied, try sudo sh -c "echo 3 > /proc/sys/vm/drop_caches" as in that question)
Start the database server (e.g. sudo service postgresql start, sudo systemctl start postgresql)
Greg Smith's answer about drop_caches was very helpful. I did find it necessary to stop and start the postgresql service, in addition to dropping the caches. Here's a shell script that does the trick. (My environment is Ubuntu 14.04 and PostgreSQL 9.3.)
#!/usr/bin/sudo bash
service postgresql stop
sync
echo 3 > /proc/sys/vm/drop_caches
service postgresql start
I tested with a query that took 19 seconds the first time, and less than 2 seconds on subsequent attempts. After running this script, the query once again took 19 seconds.
I use this command on my linux box:
sync; /etc/init.d/postgresql-9.0 stop; echo 1 > /proc/sys/vm/drop_caches; /etc/init.d/postgresql-9.0 start
It completely gets rid of the cache.
I had this error.
psql:/cygdrive/e/test_insertion.sql:9: ERROR: type of parameter 53
(t_stat_gardien) does not match that when preparing the plan
(t_stat_avant)
I was looking for flushing the current plan and a found this:
DISCARD PLANS
I had this between my inserts and it solves my problem.
Yes, it is possible to clear both the shared buffers postgres cache AND the OS cache. Solution bellow is for Windows... others have already given the linux solution.
As many people already said, to clear the shared buffers you can just restart Postgres (no need to restart the server). But just doing this won't clear the OS cache.
To clear the OS cache used by Postgres, after stopping the service, use the excelent RamMap (https://technet.microsoft.com/en-us/sysinternals/rammap), from the excelent Sysinternals Suite.
Once you execute RamMap, just click "Empty"->"Empty Standby List" in the main menu.
Restart Postgres and you'll see now your next query will be damm slow due to no cache at all.
You can also execute the RamMap without closing Postgres, and probably will have the "no cache" results you want, since as people already said, shared buffers usually gives little impact compared to the OS cache. But for a reliable test, I would rather stop postgres as all before clearing the OS cache to make sure.
Note: AFAIK, I don't recommend clearing the other things besides "Standby list" when using RamMap, because the other data is somehow being used, and you can potentially cause problems/loose data if you do that. Remember that you are clearing memory not only used by postgres files, but any other app and OS as well.
Regards, Thiago L.
Yes, postgresql certainly has caching. The size is controlled by the setting shared_buffers. Other than that, there is as the previous answer mentions, the OS file cache which is also used.
If you want to look at what's in the cache, there is a contrib module called pg_buffercache available (in contrib/ in the source tree, in the contrib RPM, or wherever is appropriate for how you installed it). How to use it is listed in the standard PostgreSQL documentation.
There are no ways to clear out the buffer cache, other than to restart the server. You can drop the OS cache with the command mentioned in the other answer - provided your OS is Linux.
There is pg_buffercache module to look into shared_buffers cache. And at some point I needed to drop cache to make some performance tests on 'cold' cache so I wrote an pg_dropcache extension that does exactly this. Please check it out.
this is my shortcut
echo 1 > /proc/sys/vm/drop_caches; echo 2 > /proc/sys/vm/drop_caches; echo 3 > /proc/sys/vm/drop_caches; rcpostgresql stop; rcpostgresql start;
If you have a dedicated test database, you can set the parameter: shared buffers to 16. That should disable the cache for all queries.
The original heading was "See and Clear" buffers.
Postgres 13 with pg_buffercache extension provides a way to see doc page
On OSX there is a purge command for that:
sync && sudo purge
sync - force completion of pending disk writes (flush cache)
purge - force disk cache to be purged (flushed and emptied)
Credit goes to kenorb answering echo 3 > /proc/sys/vm/drop_caches on Mac OSX
Occasionally in our lab, our postgres 8.3 database will get orphaned from the pid file, and we get this message when trying to shut down the database:
Error: pid file is invalid, please manually kill the stale server process postgres
When this happens, we immediately do a pg_dump so we can restore the database later. But, if we just kill -9 the orphan postgres process and then start it, the database starts only with the data from the last successful shutdown. But if you psql to it before killing it, the data is all available, thus why the pg_dump works.
Is there a way to gracefully shutdown the orphaned postgres process so we don't have to go through the pg_dump and restore? Or is there a way to have the database recover after killing the orphaned process?
According to the documentation you could either send SIGTERM or SIGQUIT. SIGTERM is preferred. Either way never use SIGKILL (as you know from personal experience).
Edit: on the other hand what you experience is not normal and could indicate a mis-configuration or a bug. Please, ask for assistance on the pgsql-admin mailing list.
Never use kill -9.
And I would strongly advice you to try to figure out exactly how this happens. Where exactly does the error message come from? It's not a PostgreSQL error message. Are you by any chance mixing different ways to start/stop the server (initscripts sometimes and pg_ctl sometimes, for example)? That could probably cause things to go out of sync.
But to answer the direct question - use a regular kill (no -9) on the process to shut it down. Make sure you kill all the postgres processes if there is more than one running.
The database will always do an automatic recovery whenever it's shut down. This shuold happen with kill -9 as well - any data that is committed should be up there. This almost sounds like you have two different data directories mounted on top of each other or something like that - this has been a known issue with NFS at least before.
I use a script like the following run by cron every minute.
#!/bin/bash
DB="YOUR_DB"
# Here's a snippet to watch how long each connection to the db has been open:
# watch -n 1 'ps -o pid,cmd,etime -C postgres | grep $DB'
# This program kills any postgres workers/connections to the specified database
# which have been running for 2 or 3 minutes. It actually kills workers which
# have an elapsed time including "02:" or "03:". That'll be anything running
# for at least 2 minutes and less than 4. It'll also cover anything that
# managed to stay around until an hour and 2 or 3 minutes, etc.
#
# Run this once a minute via cron and it should catch any connection open
# between 2 and 3 minutes. You can temporarily disable it if if you need to run
# a long connection once in a while.
#
# The check for "03:" is in case there's a little lag starting the cron job and
# the timing is really bad and it never sees a worker in the 1 minute window
# when it's got "02:".
old=$(ps -o pid,cmd,etime -C postgres | grep "$DB" | egrep '0[23]:')
if [ -n "$old" ]; then
echo "Killing:"
echo "$old"
echo "$old" | awk '{print $1}' | xargs -I {} kill {}
fi