We are trying to build CRAN reposiotry locally in windows using cwrsync. we are using rsync command to download all the packages etc. Since the machine where we are running is behind proxy we are also using RSYNC_PROXY variable. when we execute rsync command we are getting "rsync : did not see server greeting". Can you please help us.
rsync -av --chmod u+rwx -e "ssh -i d:\r-download" "/cygdrive/d/r-download"
rsync -rtlzv --delete cran.r-project.org::CRAN "/cygdrive/d/download"
Not sure whether its because of -e ssh.
Regards...Srini
Yes,
with daemon you cannot use '-e' ('--rsh') option.
From man page:
CONNECTING TO AN RSYNC DAEMON
It is also possible to use rsync without a remote shell as the
transport. In this case you will directly connect to a remote rsync
daemon, typically using TCP port 873. (This obviously requires the
daemon to be running on the remote system, so refer to the STARTING AN
RSYNC DAEMON TO ACCEPT CONNECTIONS section below for information on
that.)
Using rsync in this way is the same as using it with a remote shell
except that:
o you either use a double colon :: instead of a single colon to
separate the hostname from the path, or you use an rsync:// URL.
o the first word of the "path" is actually a module name.
o the remote daemon may print a message of the day when you
connect.
o if you specify no path name on the remote daemon then the list
of accessible paths on the daemon will be shown.
o if you specify no local destination then a listing of the
specified files on the remote daemon is provided.
o you must not specify the --rsh (-e) option.
Check last line.
Regards.
Related
I have a weird problem and don't really know where it's coming from. I have machine A, B and C. I want to connect my VSCode to machine C with the remote-ssh extension for vscode, here's my config:
# Jump box
Host jump-box
HostName machineB
User myuser
# Target machine
Host target-box
HostName machineC
User myuser
ProxyCommand ssh -q -W %h:%p jump-box
The machine C is a weird server used by a lot of people, when I try to connect, the connection to machine B is fine but then, the extension tries to ssh -D -T some5XXXXport machineC bash from B.
The last command passes fine, and I have tested it manually, however the bash at the end makes it run the root bash or something, because I lose my home directory and get an admin one.
So in consequence, the extension fails because it tries to mkdir /some/admin/home/.vscode-server/bin/somecommithash: Permission Denied. My ~doesn't belong to me anymore when the ssh command is bash.
Any ideas how to overwrite or even hack the command?
Do you know why when you ssh address bash it breaks everything?
I also don't think the B->C ssh connection is picking any ~/.bash_profile, ~/.bashrc nor ~/.profile from machine C, perhaps because ~ points to another home.
Solved it. Added all what I need directly in the extension.js.
I want a fast and flexible file server but I don't need encryption or authentication. How can I use SFTP for this on Linux systems?
SFTP happens to be used by SSH servers but it's a well-developed protocol that works well on its own. The sftp-server developed by OpenSSH has no dependency on an SSH server; sftp-server uses standard input/output. (Other SFTP servers are similar.)
It is trivial to share a filesystem via SFTP, similar to what you might do with NFS but without the need for root access. I'll use socat as the daemon for this ad-hoc example, but xinetd would make a more permanent solution. The location of sftp-server is from my Ubuntu installation of the openssh-sftp-server package.
On the server:
$ mkdir shared_to_the_world
$ cd shared_to_the_world
$ socat tcp-listen:1234,reuseaddr,fork exec:/usr/lib/openssh/sftp-server
On the client:
$ mkdir /tmp/sftp_test
$ sshfs -o reconnect,ssh_command="nc my_sftp_server_address 1234 --" : /tmp/sftp_test
$ cd /tmp/sftp_test
Now your client (and anyone else's!) can seamlessly work with the files in the shared directory on the server. Both read and write are enabled, so be careful.
Consider using socat listen's "bind" and "range" options to limit the access to your server.
You can’t use SFTP without SSH. Take a look at : https://www.ssh.com/ssh/sftp/ (emphasis mine) :
“ SFTP (SSH File Transfer Protocol) is a secure file transfer protocol. It runs over the SSH protocol.. It supports the full security and authentication functionality of SSH.
...
SFTP port number is the SSH port 22 ... It is basically just an SSH server. Only once the user has logged in to the server using SSH can the SFTP protocol be initiated. There is no separate SFTP port exposed on servers. “
I want to run simulations in parallel in a remote Cluster calling them from matlab.
I manage to run them in my local Ubuntu machine using.
unix('parallel -j4 flow > /dev/null :::: Pool.txt');
But when I want it to run it in a remote cluster I really did not mange to make the parallel command to work.
The first problem was to avoid entering the password.
For that I used the sshpass as this
unix('sshpass -p password ssh user#cluster.example.com')
That get me in to the server but it does not continue to the next command line.
I try so many commands that I do not want to reference here.
But basically can some one that understands well the parallel GNU command usage tell me how can I connect to a cluster. and run the simulations there. is it better just to make a script at the server and run it from matlab?
Any expert advice is highly appreciated.
Your problem is not with GNU Parallel but with configuring ssh. First, you must get ssh set up, then the rest is easy.
So, on your local Ubuntu machine, you need to create your keys:
ssh-keygen -t rsa -b 2048
That will make some files in $HOME/.ssh. You now need to copy the public part of those keys to each and every node of the remote cluster where you want to run your parallel jobs:
ssh-copy-id -i $HOME/.ssh/id_rsa.pub CLUSTERUSERNAME#NODE-0
...
ssh-copy-id -i $HOME/.ssh/id_rsa.pub CLUSTERUSERNAME#NODE-15
e.g.
ssh-copy-id -i $HOME/.ssh/id_rsa.pub fred#192.168.0.100
Now, test you can ssh into each node:
ssh fred#node2
Then, on your local Ubuntu box, set up your config file for ssh, so it will be $HOME/.ssh/config
Host node0
Hostname 192.168.0.100
User fred
...
...
Host node15
Hostname 192.168.0.115
User fred
Now you can just use:
ssh node0
and it will know that means fred#192.168.0.100
Now GNU Parallel will work with:
parallel -S node0,node1,node2
I was trying to setup an SSH connection with Github following this tutorial:
Testing your SSH connection
I came across the following command:
$ ssh -T git#github.com
# Attempts to ssh to github
Curious, I looked at the ssh manual. It said the following:
-T Disable pseudo-tty allocation.
What is tty allocation? What does tty stand for? Why are we disabling it?I earnestly tried to look it up but I was unable to find even a definition.
As explained in "gitolite: PTY allocation request failed on channel 0", it is important to do ssh test connection with -T, because some server could abort the transaction entirely if a text-terminal (tty) is requested.
-T avoids requesting said terminal, since GitHub has no intention of giving you an interactive secure shell, where you could type command.
GitHub only wants to reply to your ssh request, in order to ascertain that the ssh command does work (you have the right public/private keys, and the public one has been registered to your GitHub account)
PuTTy would be an example of a terminal emulator, serial console and network file transfer application. It supports several network protocols, including SCP, SSH, Telnet and rlogin.
The name "PuTTY" has no definitive meaning, though "tty" is the name for a terminal in the Unix tradition, usually held to be short for Teletype.
Other use-cases for -T (beside testing)
Transferring binary files
Execute commands on a remote server
SSH tunneling: ssh -fnT -L port:server:port user#server (-f for background: you don't want to execute command, don't need a TTY and just want to establish a tunnel)
I'm stuck in a bit of annoying situation.
There's a chain of machines between my desktop and the production servers. Something like this:
desktop -> firewall 1 -> firewall 2 -> prod_box 1
-> prod_box 2
-> ...
I'm looking for a way to automate deployment to the prod boxes via ssh.
I'm aware there are a number of solutions in general, but my restrictions are:
No changes permitted to firewall 2
No config changes permitted to prod boxes (only content)
firewall 1 has a local user account for me
firewall 2 and prod are accessed as root
port 22 is the only open port between each link
So, in general the command sequence I do to deploy is:
scp archive.tar user#firewall1:archive.tar
ssh user#firewall1
scp archive.tar root#firewall2:/tmp/archive.tar
ssh root#firewall2
scp /tmp/archive.tar root#prod1:/tmp/archive.tar
ssh root#prod1
cd /var/www/
tar xvf /tmp/archive.tar
Its a bit more complex than that in reality, but that's a basic summary of the tasks to do.
I've put my ssh key in firewall1:/home/user/.ssh/authorized_keys, so that's no problem.
However, I can't do this for firewall2 or prod boxes.
It'd be great if I could run this (commands above) from a shell script locally, type my password in 4 times and be done with it. Sadly I cannot figure out how to do that.
I need some way to chain ssh commands. I've spent all afternoon trying to use python to do this and eventually given up because the ssh libraries don't seem to support password-entry-style login.
What can I do here?
There must be some kind of library I can use to:
login via ssh using either a key file OR a dynamically entered password
remote remote shell commands through the chain of ssh tunnels
I'm not really sure what to tag this question, so I've just left it as ssh, deployment for now.
NB. It'd be great to use ssh tunnels and a deployment tool to push these changes out, but I'd still have to manually login to each box to setup the tunnel, and that wont work anyway, because of the port blocking.
I am working on Net::OpenSSH::Gateway, an extension for my other Perl module Net::OpenSSH that does just that.
For instance:
use Net::OpenSSH;
use Net::OpenSSH::Gateway;
my $gateway = Net::OpenSSH::Gateway->find_gateway(
proxies => ['ssh://user#firewall1',
'ssh://password:root#firewall2'],
backend => 'perl');
for my $host (#prod_hosts) {
my $ssh = Net::OpenSSH->new($host, gateway => $gateway);
if ($ssh->error) {
warn "unable to connect to $host\n";
next;
}
$ssh->scp_put($file_path, $destination)
or warn "scp for $host failed\n";
}
It requires Perl available in both firewalls, but no write permissions or installing any additional software there.
Unfortunately this isn't possible to do as one shell script. I did try, but ssh's password negotiation requires an interactive terminal, which you don't get with chained ssh commands. You could do it with passwordless keys, but since that's highly insecure and you can't do it anyway, nevermind.
The basic idea is that each server sends a bash script to the next one, which is then activated and sends the next one (and so on) until it reaches the last one, which does the distribution.
However, since this requires an interactive terminal at each stage, you're going to need to follow the payload down the chain manually executing each script as you go, somewhat as you do now but with less typing.
Obviously, you will need to customise them a bit, but try these scripts:
script1.sh
#!/bin/bash
user=doug
firewall1=firewall_1
#Minimise password entries across the board.
tar cf payload1.tar script3.sh archive.tar
tar cf payload2.tar script2.sh payload1.tar
scp payload2.tar ${user}#${firewall1}:payload2.tar
ssh ${user}#${firewall1} "tar xf payload2.tar;chmod +x script2.sh"
echo "Now connect to ${firewall1} and run ./script.sh"
script2.sh
#!/bin/bash
user=root
firewall2=firewall_2
# Minimise password entries
scp payload1.tar ${user}#${firewall2}:/tmp/payload1.tar
ssh ${user}#${firewall2} "cd /tmp;tar xf payload1.tar;chmod +x script3.sh"
echo "Now connect to ${firewall2} and run /tmp/script3.sh"
script3.sh
#!/bin/bash
user=root
hosts="prod1 prod2 prod3 prod4"
for host in $hosts
do
echo scp archive.tar ${user}#${host}:/tmp/archive.tar
echo ssh ${user}#${host} "cd /var/www; tar xvf /tmp/archive.tar"
done
It does require 3 password entries per firewall which is a bit annoying, but such is life.
This do you any good?