I want to write a cron job which will check if sphinx search is running or not. If not running, it will restart. How can I do this? I am assuming I will have to use the search.pid( which has the process id) and some bash script.
thanks
Tip: use a service designed for this, eg: http://mmonit.com/monit/
It will take care of the gritty details for you.
Ready to use plugins for sphinx are available
http://capitate.rubyforge.org/recipes/sphinx-monit.html
(otherwise - yes, inspect the pidfile to find the expected pid of searchd. Check if searchd is actully running on that pid. If the pid file is not found - or searchd is not running, start it. If that fails, send the administator a panicky sounding email)
Related
I have a collection with about 3,000,000 entries that I need to reindex. This whole thing began when I tried to add a 2d index. To do this, I created an ssh tunnel, opened the mongo shell and tried to use ensureIndex. I'm in a place with a somewhat unreliable internet connection, and an hour in it ended up breaking the pipe. I then tunneled back in, opened the mongo shell and tried to look at the number of indexes using getIndexes; the new index I created showed up, but I wasn't confident it had finished, so I decided to use reIndex. In retrospect, this was stupid. The pipe broke again. Now when I open the shell and try to issue getIndexes, the shell doesn't respond.
So what should I do? Do I need to repair my database? Can I issue reIndex when I have a more reliable internet connection? Is there a way to issue reIndex without keeping the shell open, but without doing it in the background and having it take eons? (I'll check the mongod shell options to see if I can find anything, then check the node.js mongo api so I can try running something as a service on server)
And also, if I end up running reIndex as a service on the server, is there any way to check if it's working? The most frustrating part of this right now is I have no idea if my database is ok, if reIndex is still running, etc. Any help would be much appreciated. Thanks.
You don't have a problem. Mongo runs commands and only stops them if you explicitly kill the operation (db.killOp()).
You do not need to wait for the index operation to finish!
Regarding the connection problems, try using the screen command.
It enables you to create a "persistent" screen - not in the way of disk persistence, but in the means of connection-loss.
I have a web app that uses postgresql 9.0 with some plperl functions that call custom libraries of mine. So, when I want to start fresh as if just released, my build process for my development area does basically this:
dumps data and roles from production
drops dev data and roles
restores production data and roles onto dev
restarts postgresql so that any cached versions of my custom libraries are flushed and newly-changed ones will be picked up
applies my dev delta
vacuums
Since switching my app's stack from win32 to CentOS, I now sometimes (i.e., it seems, only if and only if I haven't run this build process in "a while"--perhaps at least a day) get an error when my build script tries to apply the delta:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
Specifically, what's failing to execute at the shell level is this:
psql --host=$host -U $superuser -p $port -d $db -f "$delta_filename.sql"
If, immediately after seeing this error, I try to connect to the dev database with psql, I can do so with no trouble. Also, if I just re-run the build script, it works fine the second time, every time I've encountered this. Acceptable workaround, but is the underlying cause something to be concerned about?
So far in my attempts to debug this, I inserted a step just after the server restart (which of course reports OK shutdown, OK startup) whereby I check the results of service postgresql-dev status in a loop, waiting 2 seconds between tries if it fails. On my latest build script run, said loop succeeds on the first try--status returns "is running"--but then applying the delta still fails with the above connection error. Again, second try succeeds, as does connecting via psql outside the script just after it fails.
My next debug attempt was to sleep for 5 seconds before the first status check and see what happens. So far this seems to solve the problem.
So why is pgsql not listening on the socket after it starts [OK] and also has status running ok, for up to 5 seconds, unless it has "recently" been restarted?
The status check only checks whether the process is running. It doesn't check whether you can connect. There can be any amount of time between starting the process and the process being ready to accept connections. It's usually a few seconds, but it could be longer. If you need to cope with this, you need to script it so that it checks whether it is possible to connect before proceeding. You could argue that the CentOS package should do this for you, but it doesn't.
Actually, I think in your case there is no reason to do a full restart. Unless you are loading libraries with shared_preload_libraries, it is sufficient to restart the connection to pick up new libraries.
I tried installing Sphinx as a service on windows, and followed the steps mentioned in the following tutorial:
http://sphinxsearch.com/forum/view.html?id=2972
But the service doesn't start and gives Error #1067
So, as mentioned in the tutorial, I run searchd from console, and this is what I get.
Can someone guide me what am I doing wrong?
c:\sphinx\bin>searchd --console
Sphinx 2.0.7-release (r3759)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file './sphinx.conf'...
WARNING: compat_sphinxql_magics=1 is deprecated; please update your application
and config
listening on all interfaces, port=9312
listening on all interfaces, port=9306
precaching index 'test1'
precached 1 indexes in 0.004 sec
binlog: replaying log C:/sphinx/data/binlog.001
binlog: replay stats: 0 rows in 0 commits; 0 updates; 0 indexes
binlog: finished replaying C:/sphinx/data/binlog.001; 0.0 MB in 0.000 sec
binlog: replaying log C:/sphinx/data/binlog.001
binlog: replay stats: 0 rows in 0 commits; 0 updates; 0 indexes
binlog: finished replaying C:/sphinx/data/binlog.001; 0.0 MB in 0.000 sec
binlog: finished replaying total 2 in 0.003 sec
accepting connections
What working for me
Ensure that folders log and data exist in #confdir#
Use full path to config file, not relative
I cannot give you a complete answer because I have got the same "problem". However, I can probably give you a hint.
Basically, you have already started the daemon which is good news. When you run searchd from console and get the message accepting connections you can start asking questions! searchd is ready to answer (as long as it runs in a console window). I personally ask questions from VB.NET using Sphinx.Client. searchd listens on ports 9306 and 9312 (you can change it in a config), so I send him questions on port 9312. He answers and I process those answers. That's all.
When I finish searching and close my app, the console is still on and I have to close it manually (Ctrl+Break). And that is what annoys me a little bit. I would imagine that it runs somewhere in the background without me even notice it (service).
Until now I could not figure out why my Sphinx didn't run as a service. Now I have got it working. The problem was that I blindly used a command from Sphinx on-line manual and didn't change path to config accordingly.
Originally (the manual): C:\Sphinx\bin\searchd --install --config C:\Sphinx\sphinx.conf.in --servicename SphinxSearch
Working for me: C:\Sphinx\bin\searchd --install --config C:\Sphinx\sphinx.conf --servicename SphinxSearch
Correct is sphinx.conf and not sphinx.conf.in (in my case).
Note: If you have already tried to install searchd as a service you might want to delete it first as you can't add another service with the same name. Have a look at e.g. http://www.howtogeek.com/howto/windows-vista/how-to-delete-a-windows-service-in-vista-or-xp/. Then issue correct command with correct path to your config. You get desired behaviour, i.e. service runs in the background and will not remain in opened console window.
Hi Guys I start Mongo remotely on Putty SSH and after mongod, it says "listening for ports" but I can't then leave without shutting down the server. how do I do that?
On linux, look up the Screen module. There are other alternatives that do the same thing as well. It basically makes a saved session that you can reattach to later -- I use it extensively to run long tests/services that I can reattach to quickly.
The Starting and Stopping Mongo documentation article explains several options, one of which is using the --fork command line parameter to start mongod as a background process. Additionally, you can look into using service controls provided by your operating system (e.g. Windows services, init.d, upstart).
You can make a quick Mongo service script in either Windows or Linux.
You can also press CTRL+Z which will basically switch it to a background process. This basically gives you back temrinal (or at least does for me).
I also like "Screen". It is quite powerful and easy to navigate so I would personally give that a try.
I'd like to monitor the queries getting sent to my database from an application. To that end, I've found pg_stat_activity, but more often then not, the rows which are returned read " in transaction". I'm either doing something wrong, am not fast enough to see the queries come through, am confused, or all of the above!
Can someone recommend the most idiot-proof way to monitor queries running against PostgreSQL? I'd prefer some sort of easy-to-use UI based solution (example: SQL Server's "Profiler"), but I'm not too choosy.
PgAdmin offers a pretty easy-to-use tool called server monitor
(Tools ->ServerStatus)
With PostgreSQL 8.4 or higher you can use the contrib module pg_stat_statements to gather query execution statistics of the database server.
Run the SQL script of this contrib module pg_stat_statements.sql (on ubuntu it can be found in /usr/share/postgresql/<version>/contrib) in your database and add this sample configuration to your postgresql.conf (requires re-start):
custom_variable_classes = 'pg_stat_statements'
pg_stat_statements.max = 1000
pg_stat_statements.track = top # top,all,none
pg_stat_statements.save = off
To see what queries are executed in real time you might want to just configure the server log to show all queries or queries with a minimum execution time. To do so set the logging configuration parameters log_statement and log_min_duration_statement in your postgresql.conf accordingly.
pg_activity is what we use.
https://github.com/dalibo/pg_activity
It's a great tool with a top-like interface.
You can install and run it on Ubuntu 21.10 with:
sudo apt install pg-activity
pg_activity
If you are using Docker Compose, you can add this line to your docker-compose.yaml file:
command: ["postgres", "-c", "log_statement=all"]
now you can see postgres query logs in docker-compose logs with
docker-compose logs -f
or if you want to see only postgres logs
docker-compose logs -f [postgres-service-name]
https://stackoverflow.com/a/58806511/10053470
I haven't tried it myself unfortunately, but I think that pgFouine can show you some statistics.
Although, it seems it does not show you queries in real time, but rather generates a report of queries afterwards, perhaps it still satisfies your demand?
You can take a look at
http://pgfouine.projects.postgresql.org/