Does MongoDB create a file I can poll for in order to determine when prealloc is done? Right now I have a script to run rs.init(..config..), but I need to wait with triggering it until mongod is up and running.
Since tail -f | grep .. | xarg.. the log file is a bit of a flaky hack, I wondered if there is any other way to determine that mongod is done with prealloc?
We have the same problem for testing replica sets with the PHP driver. Here we use the mongo shell's ReplSetTest() functionality to get around this. You can see here how that works:
https://github.com/mongodb/mongo-php-driver/blob/master/tests/utils/myconfig.js#L9
However, I am not sure how well this works for non-test environments as the amount of options you can give are rather limited (such as, you can't set a data dir properly as things are hardcoded). All the functions and code for this is all in JavaScript at https://github.com/mongodb/mongo/blob/master/src/mongo/shell/replsettest.js — this should give you an overview how it works and allows you to rewrite it in your preferred language.
try to use inotify (i am not sure exactly), for example, if it is necessary to determine that the file is closed after writing:
[maverick#mutabor ~]$ pyinotify -e IN_CLOSE_WRITE /tmp/testfile
Related
We're currently using chef to provision our servers and we want our recipe/cookbook to automatically add some data to the mongo database once its installed and running.
This is where we start to run into problems. We were using an execute resource to run the mongo script like this:
execute "install-mongodb-config" do
command "mongo #{node[:mongodb][:mongo_db_host]}/#{node[:mongodb][:mongo_db]} \"#{node[:mongodb][:mongo_add_config_script]}\""
action :run
end
This part of the recipe always failed no matter what we tried! I won't get into the details of everything we tried here (unless i need to) but lets just say that i've exhausted all possibilities of subscribes and notifies (i think).
The problem originates from the fact that we are using the mongodb::10gen_repo to install mongodb. The recipe exits when apt-get installs the package and then chef continues on to execute more resourses.
We have tried executing the above resource directly after mongodb::10gen_repo but it doesn't seem like mongodb is available and the mongo shell cannot connect and run the script. The error we see is somewhat like this:
MongoDB shell version: 2.0.2
Thu Sep 6 18:40:45 ReferenceError: setTimeout is not defined mongotest.js:2
failed to load: mongoAddConfig.js
Nothing we have tried has been able to get around this in a nice chef way. The thing that we resorted to was to replace the execute resource with the following:
execute "install-mongodb-config" do
command "sleep 60; mongo #{node[:mongodb][:mongo_db_host]}/#{node[:mongodb][:mongo_db]} \"#{node[:mongodb][:mongo_add_config_script]}\""
action :run
end
Which just makes the command sleep for 60 seconds before the mongo script is run. I know this isn't the Right way to do this but it works for now.
Can anyone suggest the Right way to do this? I have a feeling that I will need to talk to the guys that created the mongodb chef script and request a feature!
First of all. Remove this "sleep 60". This can be done by chef: All resources have common attributes and "retries" and "retry_delay" are part of them. So the easiest way would be:
execute "install-mongodb-config" do
command "mongo some_command"
action :run
retries 6
retry_delay 10
end
If you have more than 2-3 places, where you have to run some command on mongo database, consider creating LWRP, similar to one created in this mongodb cookbook. (Particularly check the libraries/mongodb.rb file). You can hide the logic that waits for the server to respond there.
Is it important that the same Chef run that installs the software also injects the initial configuration? The 'chefly' method to constructing cookbooks and recipes is to guard against idempotency in order to ensure that they can be run over and over again without producing unintended results.
In this particular case, I would limit the first recipe to only just installing and starting up mongodb. This recipe would do nothing if it saw that mongodb was already running on the host. Then, I'd have another recipe that would run only if it saw that mongo had been setup and was running. It would query the mongodb to see if the initial configuration had been done. If so, it would simply return. If not, it would run your configuration routine.
In this way, these recipes could run all the time, anytime, on your machine. Even if someone uninstalled mongodb, chef would get around to ensuring that it was set back up again and pristine.
So, I don't know much at all about chef. But your problem seems to be that you try to immediately connect after bringing the server up.
Server's are not immediately available when you bring them up since there is a bit of overhead that goes into electing a primary, getting all the server status's etc.
You can recreate this without chef by trying to bring up a replica set and immediately trying to connect to it in a simple script. So it's not chef specific.
Not sure if there is a way around the server startup lag since bringing up a primary is expected to be a relatively infrequent occurrence compared to just adding nodes to a set.
The only potential solution I see that is cleaner is adding a longer Timeout for the connection to be formed in the configuration. You can find how to do this in the mongodb documentation here: http://www.mongodb.org/display/DOCS/Connections
The flag of interest for you is likely connectTimeoutMS
I have a shell script which archives log files based on the whether the process is running or not. If the log file is not used by the process then I archive it. Until now, I'm using lsof to get the log file being used but in future, I have decided to use perl to do this function.
Is there a perl module similar to what lsof in linux can perform ?
There is a perl module, which wraps around lsof. See Unix::Lsof.
As I see it, the big problem with not using lsof is that one would need to work in a way that is independent of the operating system. Using lsof allows the perl programmer to work with a consistent application allowing for operating system independence.
To have a perl module developer to write lsof would, in effect, be writing lsof as a library and then link that into perl - which is much more work than just using the existing binary.
One could also use the fuser command, which shows the process IDs with the file handle. There is also a module which seeks to implement the same functionality. Note from the perldoc:
The way that this works is highly unlikely to work on any other OS
other than Linux and even then it may not work on other than 2.2.*
kernels.
One might try walking /proc/*/fd and looking at the file descriptors in there to see if any are pointing to the file in question. If it is known what the process ID of a running process that would be opening the log file, it would be just as easy to look at that process. Note, that this is how the fuser module works.
That said, it should be asked "why do you want to move away from lsof"?
Lets say I have an Emacs-Server running on some remote server, with all the libraries and software necessary for running my application.
Then I want several clients to connect to that remote machine, using Emacs-client. Does each client need a full Emacs installation, or is there a minimal installation that is just enough to communicate with the remote server, where all the action is?
Could this (Emacs-)client installation be so minimal, that almost all software-updates can be done on the server, without affecting the Emacs-clients?
Is there a reason not to run the clients remotely as well, and simply use a local display? That way, pretty much all you need on the local machines is the ssh client and the X Window server.
ssh -X (user)#(server) "emacsclient -c"
Edits for the comments:
This command starts a new client to connect to an existing Emacs server (which it assumes is already running). You can use "emacsclient -a '' -c" to automatically start emacs --daemon if there is no existing server, but I don't know whether you want the connecting user to be starting the server.
In fact, I'm pretty unsure about the whole multi-user side of this to be honest, as I've never done that before. Authentication for the above is handled by ssh, but there may well be subsequent permission issues to deal with, or similar, when the server and the clients are started by different users.
This approach should be possible with Windows/Cygwin as client and/or server, as Cygwin provides Emacs, OpenSSH, and X.org packages. (I regularly use Windows/Cygwin as a local display for Emacs running on Linux.) It may be harder to set up, though, and any permissions issues are probably different when you're using Cygwin.
I'm less sure how this would work without Cygwin. NTEmacs certainly won't talk to X.org, so I imagine you'd be terminal based in that instance. (There are probably other options, but Cygwin sounds to me like the best-integrated approach to using all of Emacs, SSH, and X on Windows).
Lastly, I imagine you're probably getting your "Connection refused" error because localhost is not running a sshd daemon? I would say that configuration of ssh is outside the scope of this question, but there are lots of resources online for that.
Depending on what you're trying to achieve, you may be able to use a combination of Emacs and Screen. By starting up Emacs from Screen on the remote machine and detaching from it, you can subsequently re-attach from a different machine that doesn't have Emacs. Again, whether this will work for you or not depends on what you're trying to do; however, for many Emacs use-cases, this can be very effective. If you're not familiar with using Screen in this manner, here is some reading material:
screen - The Terminal Multiplexer
I am not sure that would be possible. emacsclient uses tramp to connect to a remote server, and just by looking at the number of requires in the tramp elisp files (41) it seems very unlikely. You can try it yourself with the following:
zgrep -oE "\(require '[a-z-]+\)" *el.gz | sed -e 's%[a-z0-9-]\+\.el\.gz:%%g' | sort | uniq -cu | wc -l
I'm not an expert in emacsclient, but I don't think is was designed to do what you're looking for. I think the general use case is that emacsclient allows you to redirect new requests to open a file with emacs to a persistent emacs process to avoid what may be a bit of an overhead in startup time. You seem to be looking for more of a true client/server relationship.
I think to meet the goal you're aiming at you'll probably need to look a little outside emacs, probably a project unto itself - 'emacsRemoteClient. It boils down to one or two models; the file you want to edit would need to have it's path sent over to the server machine so that emacs could do some sort of remote tramp access & then spawn the xwindow locally (using the local X env or requiring an x server on windows)... or two, transferring the file to some temp location on the server box and again spawning the remote x window locally (followed by syncing the changes between the tmp & local file).
Would be cool to have something like that... but suspecting it'll involve a bit of work. Maybe we just need a version of emacs written in javascript and it can live in the cloud or on your browser... oh to have emacs keybindings in the browser ;-)
-Steve
I'm running an apache2 / mod_perl2 combo on our development server.
When I'm developing, my changes are instantly reflected in the webpage I'm working on. I assumed mod_perl was being clever and was reloading files when they were changed.
But now another developer is working on a different part of the system and their changes are not picked up by mod_perl. He has to restart apache before he can see his changes.
Is there a way to disable caching on our development server, or get mod_perl to pick up his changes?
Thanks.
EDIT: I'm editing file directly on the dev server using VI, the other developer has mounted their dev directory via samba, and is editing their files in windows. This seems to be the difference that prevents mod_perl picking up changes.
I just read a nice blog post that sums up all possible ways to achieve this: How not to restart mod_perl servers by Jonathan Swartz
What exactly is the other developer changing?
To reload modules when they have changed you would use Apache2::Reload. (Though see Performance Issues before thinking about using this in production.)
Even without that, mod_perl will reload cgi scripts when they change; I don't know of any way the other developer could have turned that off if you are talking about cgi scripts.
Since it's just for development, how about just killing all of the child processes and letting the parent apache process respawn?
kill -9 $(ps axf | grep httpd | egrep -e ' S ' | cut -b1-5 | paste -s -d ' ')
The above command works for me on my box in my environment, your mileage will vary.
It's not an elegant solution by any means, but hey, it's faster than a full apache restart.
I have a Perl Expect.pm script that does some moderately complex stuff like packaging applications, deploying the application, checking for logs, etc. on multiple remote unix hosts.
My predecessor had written similar scripts using rsh.
Is there a better approach between the two? Or should I use something all together different?
I am guessing somebody will bring up SSH; it's basically replacement for rsh, right? Unfortunately, however SSH is not an option for me right now.
Another thing I should add is that after logging in I need to be able to SUDO to a particular user to do most of the actions on the remote hosts.
Another thing I should add is that after logging in I need to be able to SUDO to a particular user to do most of the actions on the remote hosts.
To address this one particular point: using rsh (or ssh) you can specify which user to become in the remote session:
$ rsh -l username hostname
There's no need to use sudo in this case. Now would definitely be the time to look into ssh due to security issues. The syntax is the same, but ssh also allows a slightly different (and I'd say better) syntax:
$ ssh username#hostname
I found expect to be too finicky, but my experience with it is not substantial.
They do different things. Expect is a way to script what would otherwise be manual responses. rsh -- remote shell, not restricted shell, an unfortunate name clash -- allows you to run commands remotely on another system.
That said, the security holes and other disadvantages of using rsh to do remote commands, run sudo, etc, are immense.
I can see multiple ways of doing this:
Expect over telnet, rsh, or ssh
pros: single connection, fewer escaping issues
cons: fragility in a changing environment
rsh/ssh each command individually
pros: fewer escaping issues, more reliable in a changing environment
cons: each connection takes time for authentication, and, for ssh, handshaking the encryption
rsh/ssh all commands at once
pros: single connection (less overhead), more reliable than expect
cons: fragility in maintenance especially as you get more than a handful of statements in there, escaping issues are more prevalent (escape in perl so that it's still escaped by rsh/ssh so that it's still escaped by the remote shell so that it's properly handled by the sudo'd remote shell?)
rsh/ssh and run a script
pros: single connection, more reliable, more maintainable
cons: finding a way to get it over there (rcp/scp work, NFS works, you need to determine the best way for you).
All things considered, this is the most minor con as you could simply do something like
open my $fh, "|ssh user#host 'cat > /tmp/myscript'";
print $fh $script;
system qw(ssh user#host), "chmod u+x /tmp/myscript; /tmp/myscript; rm /tmp/myscript";
Of course, you'd add in some error handling (failed open, what if /tmp/myscript exists, etc.), but that's the idea.
Given a choice between rsh and telnet via expect, I would chose rsh. Expect scripts are fragile. All it takes to break one is someone changing the value of PS1 on the remote machine. Using rsh also prepares you for the day you will finally enter the '90s and start using ssh (since you can mostly just change rcp to scp and rsh to ssh and everything still works).