I tried installing Sphinx as a service on windows, and followed the steps mentioned in the following tutorial:
http://sphinxsearch.com/forum/view.html?id=2972
But the service doesn't start and gives Error #1067
So, as mentioned in the tutorial, I run searchd from console, and this is what I get.
Can someone guide me what am I doing wrong?
c:\sphinx\bin>searchd --console
Sphinx 2.0.7-release (r3759)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file './sphinx.conf'...
WARNING: compat_sphinxql_magics=1 is deprecated; please update your application
and config
listening on all interfaces, port=9312
listening on all interfaces, port=9306
precaching index 'test1'
precached 1 indexes in 0.004 sec
binlog: replaying log C:/sphinx/data/binlog.001
binlog: replay stats: 0 rows in 0 commits; 0 updates; 0 indexes
binlog: finished replaying C:/sphinx/data/binlog.001; 0.0 MB in 0.000 sec
binlog: replaying log C:/sphinx/data/binlog.001
binlog: replay stats: 0 rows in 0 commits; 0 updates; 0 indexes
binlog: finished replaying C:/sphinx/data/binlog.001; 0.0 MB in 0.000 sec
binlog: finished replaying total 2 in 0.003 sec
accepting connections
What working for me
Ensure that folders log and data exist in #confdir#
Use full path to config file, not relative
I cannot give you a complete answer because I have got the same "problem". However, I can probably give you a hint.
Basically, you have already started the daemon which is good news. When you run searchd from console and get the message accepting connections you can start asking questions! searchd is ready to answer (as long as it runs in a console window). I personally ask questions from VB.NET using Sphinx.Client. searchd listens on ports 9306 and 9312 (you can change it in a config), so I send him questions on port 9312. He answers and I process those answers. That's all.
When I finish searching and close my app, the console is still on and I have to close it manually (Ctrl+Break). And that is what annoys me a little bit. I would imagine that it runs somewhere in the background without me even notice it (service).
Until now I could not figure out why my Sphinx didn't run as a service. Now I have got it working. The problem was that I blindly used a command from Sphinx on-line manual and didn't change path to config accordingly.
Originally (the manual): C:\Sphinx\bin\searchd --install --config C:\Sphinx\sphinx.conf.in --servicename SphinxSearch
Working for me: C:\Sphinx\bin\searchd --install --config C:\Sphinx\sphinx.conf --servicename SphinxSearch
Correct is sphinx.conf and not sphinx.conf.in (in my case).
Note: If you have already tried to install searchd as a service you might want to delete it first as you can't add another service with the same name. Have a look at e.g. http://www.howtogeek.com/howto/windows-vista/how-to-delete-a-windows-service-in-vista-or-xp/. Then issue correct command with correct path to your config. You get desired behaviour, i.e. service runs in the background and will not remain in opened console window.
Related
I tried finding solutions, but nothing helps.
I need to do a backup of my pgsql data from the app, I haven't used for months now. I have discovered, that the postgresql server is not running. But cannot start it.
I run pg_ctl -D /usr/local/pgsql/data -l logging.log -w -s start as pgsql user (su pgsql). Output says that it couldn't start a server and tells me to check logs. But logging.log is an empty file. Any default logging file I have found on the web about is modified months ago or empty or even doesn't exist.
I have no idea how to find the error, since logs are empty or I just don't know where to look for them.
Important note: it was working few months ago, but there were almost no changes in that time (possible hostname change).
Postgres is v9.1
System: FreeBSD 10.0-RC4
Some versions of FreeBSD ports installed PostgreSQL with syslog logging enabled. You can confirm this by looking at /usr/local/pgsql/data/postgresql.conf for log_destination = 'syslog'
If that is the case, the logging output should be visible in /var/log/messages
Default syslog logging enabled (log_destination = 'syslog') and logging output should be visible in /var/log/messages.
If you want to make a log in a separate file:
1) Create log file:
touch /var/log/postgresql/postgresql.log
2) Edit /etc/syslog.conf, append lines
!postgres
*.* /var/log/postgresql/postgresql.log
!*
After editing, you need to restart the service
service syslogd restart
4) do not forget to rotate postgresql.log (edit /etc/newsyslog.conf)
5) Perhaps in order to see something you will need to set the logging level. As an example, add to your postgresql.conf
client_min_messages = log
log_min_messages = info
log_checkpoints = on
log_connections = on
log_disconnections = on
I use following command to restart HAProxy, when changing the configurration file:
/usr/local/sbin/haproxy -f /etc/haproxy.cfg -p /var/run/haproxy.pid -sf $(</var/run/haproxy.pid)
Sadly after the HAProxy is back all stats of the previous launch are away.
Is there a possibility in HAProxy to restore stats from a previous HAProxy start?
As of version 1.6, you can dump server states into a flat file right before performing the reload and let the new process know where the states are stored.
See example here:seamless_reload
The "show servers state" command is used to keep servers uptime and healthy status cross reload, but it doesn't give session data, or bytes in/out, etc. "show stat" command can dump these stats to a file that you can use to create a report later, although HAproxy doesn't have a feature to reload this information.
Can't be done unfortunately. HAProxy's stats are all in memory, so when restarting (even gracefully with -sf), those stats get lost.
Might you can export data to CSV file before doing reload/restart
"http://localhost:8080/haproxy?stats;csv"
or
curl -u <USER>:<MyPASSWORD> "http://localhost:8080/haproxy?stats;csv"
according to HAproxy 1.5 doc you can clear all stats using the unix socket.
clear counters all
Clear all statistics counters in each proxy (frontend & backend) and in each
server. This has the same effect as restarting. This command is restricted
and can only be issued on sockets configured for level "admin".
I have a web app that uses postgresql 9.0 with some plperl functions that call custom libraries of mine. So, when I want to start fresh as if just released, my build process for my development area does basically this:
dumps data and roles from production
drops dev data and roles
restores production data and roles onto dev
restarts postgresql so that any cached versions of my custom libraries are flushed and newly-changed ones will be picked up
applies my dev delta
vacuums
Since switching my app's stack from win32 to CentOS, I now sometimes (i.e., it seems, only if and only if I haven't run this build process in "a while"--perhaps at least a day) get an error when my build script tries to apply the delta:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
Specifically, what's failing to execute at the shell level is this:
psql --host=$host -U $superuser -p $port -d $db -f "$delta_filename.sql"
If, immediately after seeing this error, I try to connect to the dev database with psql, I can do so with no trouble. Also, if I just re-run the build script, it works fine the second time, every time I've encountered this. Acceptable workaround, but is the underlying cause something to be concerned about?
So far in my attempts to debug this, I inserted a step just after the server restart (which of course reports OK shutdown, OK startup) whereby I check the results of service postgresql-dev status in a loop, waiting 2 seconds between tries if it fails. On my latest build script run, said loop succeeds on the first try--status returns "is running"--but then applying the delta still fails with the above connection error. Again, second try succeeds, as does connecting via psql outside the script just after it fails.
My next debug attempt was to sleep for 5 seconds before the first status check and see what happens. So far this seems to solve the problem.
So why is pgsql not listening on the socket after it starts [OK] and also has status running ok, for up to 5 seconds, unless it has "recently" been restarted?
The status check only checks whether the process is running. It doesn't check whether you can connect. There can be any amount of time between starting the process and the process being ready to accept connections. It's usually a few seconds, but it could be longer. If you need to cope with this, you need to script it so that it checks whether it is possible to connect before proceeding. You could argue that the CentOS package should do this for you, but it doesn't.
Actually, I think in your case there is no reason to do a full restart. Unless you are loading libraries with shared_preload_libraries, it is sufficient to restart the connection to pick up new libraries.
I want to write a cron job which will check if sphinx search is running or not. If not running, it will restart. How can I do this? I am assuming I will have to use the search.pid( which has the process id) and some bash script.
thanks
Tip: use a service designed for this, eg: http://mmonit.com/monit/
It will take care of the gritty details for you.
Ready to use plugins for sphinx are available
http://capitate.rubyforge.org/recipes/sphinx-monit.html
(otherwise - yes, inspect the pidfile to find the expected pid of searchd. Check if searchd is actully running on that pid. If the pid file is not found - or searchd is not running, start it. If that fails, send the administator a panicky sounding email)
Sometimes I run a Postgres query and it takes 30 seconds. Then, I immediately run the same query and it takes 2 seconds. It appears that Postgres has some sort of caching. Can I somehow see what that cache is holding? Can I force all caches to be cleared for tuning purposes?
I'm basically looking for a Postgres version of the following SQL Server command:
DBCC FREEPROCCACHE
DBCC DROPCLEANBUFFERS
But I would also like to know how to see what is actually contained in that buffer.
You can see what's in the PostgreSQL buffer cache using the pg_buffercache module. I've done a presentation called "Inside the PostgreSQL Buffer Cache" that explains what you're seeing, and I show some more complicated queries to help interpret that information that go along with that.
It's also possible to look at the operating system cache too on some systems, see [pg_osmem.py] for one somewhat rough example.
There's no way to clear the caches easily. On Linux you can stop the database server and use the drop_caches facility to clear the OS cache; be sure to heed the warning there to run sync first.
I haven't seen any commands to flush the caches in PostgreSQL. What you see is likely just normal index and data caches being read from disk and held in memory. by both postgresql and the caches in the OS. To get rid of all that, the only way I know of:
What you should do is:
Shutdown the database server (pg_ctl, sudo service postgresql stop, sudo systemctl stop postgresql, etc.)
echo 3 > /proc/sys/vm/drop_caches
This will clear out the OS file/block caches - very important though I don't know how to do that on other OSs. (In case of permission denied, try sudo sh -c "echo 3 > /proc/sys/vm/drop_caches" as in that question)
Start the database server (e.g. sudo service postgresql start, sudo systemctl start postgresql)
Greg Smith's answer about drop_caches was very helpful. I did find it necessary to stop and start the postgresql service, in addition to dropping the caches. Here's a shell script that does the trick. (My environment is Ubuntu 14.04 and PostgreSQL 9.3.)
#!/usr/bin/sudo bash
service postgresql stop
sync
echo 3 > /proc/sys/vm/drop_caches
service postgresql start
I tested with a query that took 19 seconds the first time, and less than 2 seconds on subsequent attempts. After running this script, the query once again took 19 seconds.
I use this command on my linux box:
sync; /etc/init.d/postgresql-9.0 stop; echo 1 > /proc/sys/vm/drop_caches; /etc/init.d/postgresql-9.0 start
It completely gets rid of the cache.
I had this error.
psql:/cygdrive/e/test_insertion.sql:9: ERROR: type of parameter 53
(t_stat_gardien) does not match that when preparing the plan
(t_stat_avant)
I was looking for flushing the current plan and a found this:
DISCARD PLANS
I had this between my inserts and it solves my problem.
Yes, it is possible to clear both the shared buffers postgres cache AND the OS cache. Solution bellow is for Windows... others have already given the linux solution.
As many people already said, to clear the shared buffers you can just restart Postgres (no need to restart the server). But just doing this won't clear the OS cache.
To clear the OS cache used by Postgres, after stopping the service, use the excelent RamMap (https://technet.microsoft.com/en-us/sysinternals/rammap), from the excelent Sysinternals Suite.
Once you execute RamMap, just click "Empty"->"Empty Standby List" in the main menu.
Restart Postgres and you'll see now your next query will be damm slow due to no cache at all.
You can also execute the RamMap without closing Postgres, and probably will have the "no cache" results you want, since as people already said, shared buffers usually gives little impact compared to the OS cache. But for a reliable test, I would rather stop postgres as all before clearing the OS cache to make sure.
Note: AFAIK, I don't recommend clearing the other things besides "Standby list" when using RamMap, because the other data is somehow being used, and you can potentially cause problems/loose data if you do that. Remember that you are clearing memory not only used by postgres files, but any other app and OS as well.
Regards, Thiago L.
Yes, postgresql certainly has caching. The size is controlled by the setting shared_buffers. Other than that, there is as the previous answer mentions, the OS file cache which is also used.
If you want to look at what's in the cache, there is a contrib module called pg_buffercache available (in contrib/ in the source tree, in the contrib RPM, or wherever is appropriate for how you installed it). How to use it is listed in the standard PostgreSQL documentation.
There are no ways to clear out the buffer cache, other than to restart the server. You can drop the OS cache with the command mentioned in the other answer - provided your OS is Linux.
There is pg_buffercache module to look into shared_buffers cache. And at some point I needed to drop cache to make some performance tests on 'cold' cache so I wrote an pg_dropcache extension that does exactly this. Please check it out.
this is my shortcut
echo 1 > /proc/sys/vm/drop_caches; echo 2 > /proc/sys/vm/drop_caches; echo 3 > /proc/sys/vm/drop_caches; rcpostgresql stop; rcpostgresql start;
If you have a dedicated test database, you can set the parameter: shared buffers to 16. That should disable the cache for all queries.
The original heading was "See and Clear" buffers.
Postgres 13 with pg_buffercache extension provides a way to see doc page
On OSX there is a purge command for that:
sync && sudo purge
sync - force completion of pending disk writes (flush cache)
purge - force disk cache to be purged (flushed and emptied)
Credit goes to kenorb answering echo 3 > /proc/sys/vm/drop_caches on Mac OSX