I have a few simple scripts that are intended to daisy chain together to run a specific script on a set of servers all listed out in a file, one per line.
The single server deploy script contains the following:
1 #!/bin/bash
2
3 file=$1
4 host=$2
5
6 scp ${file} ${host}:/tmp/
7 USER=`whoami`
8 ssh -t -t $USER#${host} /tmp/${file}
9
10 ssh "${host}" /bin/rm /tmp/${file}
11 exit
It works fine on a script I have that yum installs tomcat and symlinks hadoop/hbase configs to the shared class directory.
The second major file is deploy-all.sh which is intended to parse a list of hosts and run the deploy script to all of them:
1 #!/bin/bash
2
3 script=$1
4
5 cat dumbo-hosts | while read fileline
6 do
7 echo ${fileline}
8 ./deploy.sh ${script} ${fileline}
9
10 sleep 10
11 done
What happens is that the script runs once, and then the for loop is broken... I got something like the following output:
$ ./deploy-all.sh setup-tomcat.sh
line is hadoop01.myhost
setup-tomcat.sh 100% 455 0.4KB/s 00:00
tcgetattr: Inappropriate ioctl for device
hadoop02.myhost
hadoop03.myhost
hadoop04.myhost
<succesful output of hadoop01 task>
...
Connection to hadoop01.myhost closed.
If I comment out the ssh commands the loop runs succesfully through all 4 hosts so I presume it's something involving stdio getting cut off once the ssh occurs. In addition the tcgatattr error concerns me somewhat.
How can I get around this? What exactly is causing the tcgetattr error(I'm not even sure if it's related)?
Haven't really done much with shell scripts so sorry if I'm missing something really obvious here, any help would be appreciated.
It's a problem with ssh reusing the stdin file descriptor when running as part of the subprocess.
The workaround is to use '-n' when invoking ssh from a non-terminal context.
option=-n
tty -s 2>/dev/null && option=
scp ${file} ${host}:/tmp/
ssh $option -t ${host} /tmp/${file}
ssh $option ${host} rm /tmp/${file}
I solved this by using bash arrays to temporarily store the lines into an array to avoid the stdin interruption... But it feels wrong... If anyone has a better way of getting around this, please let me know.
Here's my solution:
1 #/bin/bash
2
3 #myscript = $1
4 count=0
5
6 declare -a lines
7
8 while read line
9 do
10 lines[$count]=$line
11 ((count++))
12 done < dumbo-hosts
13
14 for i in "${lines[#]}"
15 do
16 echo "$i"
17 ./deploy.sh "$1" "${i}"
18 done
Related
There is a server that starts and listens to a SOCK_SEQPACKET. I think a SOCK_SEQPACKET has a name, say, #Foo. It's per user, so initiated by each Linux user account, and there is only one service per user.
The goal is to make an external tool that forcefully kill the service when something goes wrong. The language is C++.
When I have to kill that service, I use netstat:
$ netstat -lnp | egrep Foo
I would not like to have a dependency on the net-tools package, though. I would like to do this with minimal dependencies on external tools and/or even libraries.
I've tried Google search, and could not find how. I guess I could perhaps read netstat source code to see how they do; I would like to defer that to the last resort, though. I have learned how I could kill the service, using netstat, which gives me dependency on net-tools. The service somehow runs as /proc/self or so. I could visit every process ID in /proc, and see if that looks like the service: see the executable name, etc. However, that is not necessarily sufficient indication to narrow down to that one single process, which uses the #Foo SOCK_SEQPACKET. Now, as non-expert in socket/network programming, I am running out of ideas about how I should proceed.
As #user253751 says, that's not actually a socket type but more of a socket behavior type. As they suggested, it is probably a unix domain socket. Given the '#' sign in it, it is almost certainly using the unix domain socket's "abstract" namespace (which means the socket doesn't have an endpoint directly visible in the filesystem). At the system call level, such endpoints are created with a null byte as the first character (the # is commonly substituted in command line utilities or wherever the socket is surfaced to user space for user convenience). See the unix(7) man page for more details.
Since the socket doesn't appear in the filesystem anywhere, it can be difficult to find the association of process to socket. One way you can find that is through use of the lsof utility -- but that is probably little improvement over the netstat utility.
It appears that the abstract namespace sockets do show up in the /proc/net/unix pseudo-file. In association with that, there is an inode column (second to the last). In conjunction with looking at each process's /proc/<pid>/fd directory, I think you can make the association.
For example, I created a small python program that creates a Unix domain SOCK_SEQPACKET socket and binds it to '#Foo'. That program is here:
import socket, time
sock = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET)
sock.bind('\0Foo') # Note the \0 for the initial null byte
time.sleep(10000) # Just hang around while we do other stuff
In a separate terminal window:
gh $ grep '#Foo' /proc/net/unix
0000000000000000: 00000002 00000000 00000000 0005 01 7733102 #Foo
gh $ sudo bash -c 'for proc in /proc/[0-9]*; do ls -l $proc/fd | grep 7733102 && echo $proc; done'
lrwx------ 1 gh gh 64 Feb 2 17:13 3 -> socket:[7733102]
/proc/339264
gh $ ps -fp 339264
UID PID PPID C STIME TTY TIME CMD
gh 339264 22444 0 17:11 pts/3 00:00:00 python3
gh $ ls -l /proc/339264/fd
total 0
lrwx------ 1 gh gh 64 Feb 2 17:13 0 -> /dev/pts/3
lrwx------ 1 gh gh 64 Feb 2 17:13 1 -> /dev/pts/3
lrwx------ 1 gh gh 64 Feb 2 17:13 2 -> /dev/pts/3
lrwx------ 1 gh gh 64 Feb 2 17:13 3 -> 'socket:[7733102]'
Explaining:
First, we do a grep on /proc/net/unix for the '#Foo' socket. That gives us the inode number 7733102 (second to last column). The inode number is unique to every socket on the system.
Next, as root, we do an ls -l /proc/<pid>/fd for each process on the system and grep for the inode number we found in the previous step. When found, we printed out the /proc/<pid> entry. (On linux, that pseudo-directory shows a symlink to a representation of what each file descriptor open in the process is. For sockets, that representation is always socket:[<inode>].) Here we found the process ID is 339264.
The remaining two steps are just confirming that that is in fact, our python process, and you can see its four open files (the first three [stdin, stdout, stderr] all pointing to its terminal's pseudo-tty, the fourth to the socket).
To make this into a more foolproof program, you'd need to account for the fact that the inode number you found in step 1 could be a substring of some other socket's inode number but that is left as an exercise for the reader. :)
When I use comm command to compare the files with 2 GB and 1.7GB I got the following error.
Value too large for defined data type
I tried the following command.
comm -23 file1.txt file2.txt
Solaris Generic_150401-32 i86pc
Kindly help
As Sathiyadasan writes, Solaris 10 comm can't handle large files (>2GB).
I offer 3 options:
1) download the GNU version of comm and use that on solaris 10
2) move to Solaris 11 and use the /usr/gnu/bin/comm
3) write a more complicated script, depending on what you're trying to accomplish:
Reducing your data might make the problem more manageable. If the files have lots of duplicate entries, this workds well. If you're trying to find lines that are unique to the first file, but don't care about the order of the lines within the file, you could use:
sort -o file1.smaller -u file1.txt
sort -o file2.smaller -u file2.txt
comm -23 file1.smaller file2.smaller
Really, how you handle this depends on the nature of your data and what you're trying to discover.
Good luck!
how to use "grep" command to find a match and to print followup of 10 lines from the match. this i need to get some error statements from log files. (else need to download use match for log time and then copy the content). Instead of downloading bulk size files i need to run a command to get those number of lines.
A default install of Solaris 10 or 11 will have the /usr/sfw/bin file tree. Gnu grep - /usr/sfw/bin/ggrep is there. ggrep supports /usr/sfw/bin/ggrep -A 10 [pattern] [file] which does what you want.
Solaris 9 and older may not have it. Or your system may not have been a default install. Check.
Suppose, you have a file /etc/passwd and want to filter user "chetan"
Please try below command:
cat /etc/passwd | /usr/sfw/bin/ggrep -A 2 'chetan'
It will print the line with letter "chetan" and the next two lines as well.
-- Tested in Solaris 10 --
How can i make my driver to act on following commands cat & echo.
Does cat call read() system call of a device driver ?
Does echo call write() system call of a device driver ?
I want to implement these two command cat & echo for my driver controlling 8 led.
If i echo it gives glows led 3 :-----
echo "3=1" > /dev/led_node
If i cat it gives following out-put status :-----
cat /dev/led_node
0 0
1 0
2 0
3 1
4 0
5 0
6 0
7 0
Please suggest what part or system call of driver interact with cat & echo system calls ?
You can easily check how cat reads files using, for exampl, strace. Here is an example:
$ echo '123' >/tmp/test.txt
$ strace cat /tmp/test.txt
In the output, you can spot the open() call:
open("/tmp/test.txt", O_RDONLY) = 3
Which returns 3 - a file descriptor associated with /tmp/test.txt. Further down the output, you can see:
read(3, "123\n", 65536) = 4
Which takes file descriptor 3 and reads from it (using a buffer size of 65536 bytes and getting back 4 bytes). basically answers your first question - cat does call read(). You can do the same thing for echo and figure out that it calls write().
In your character device driver, you would have to implement those calls. For a great explanation on how it works along with useful examples, check out Linux Device Drivers, Chapter 3.
Hope it helps. Good Luck!
cat: system call interacts with read function of your driver. echo: system call interacts with write function of your driver. Thing is if you cat on /dev/led_node, device file/node is opened i.e. calling open system call, then read system call is called, keeps looping in the read unless zero is returned (no data present to read) and atlast close system call is called, which close the device node/file.
I want to create a directory structure in Windows from within SAS. Preferably using a method that will allow me to specify a UNC naming convention such as:
\\computername\downloads\x\y\z
I have seen many examples for SAS on the web using the DOS mkdir command called via %sysexec() or the xcommand. The nice thing about the mkdir command is that it will create any intermediate folders if they also don't exist. I successfully tested the below commands from the prompt and it behaved as expected (quoting does not seem to matter as I have no spaces in my path names):
mkdir \\computername\downloads\x\y\z
mkdir d:\y
mkdir d:\y\y
mkdir "d:\z"
mkdir "d:\z\z"
mkdir \\computername\downloads\z\z\z
mkdir "\\computername\downloads\z\z\z"
The following run fine from SAS:
x mkdir d:\x;
x 'mkdir d:\y';
x 'mkdir "d:\z"';
x mkdir \\computername\downloads\x;
x 'mkdir \\computername\downloads\y';
But these do not work when run from SAS,eg:
x mkdir d:\x\x;
x 'mkdir d:\y\y';
x 'mkdir "d:\z\z"';
x mkdir \\computername\downloads\x\y\z ;
x 'mkdir "\\computername\downloads\z"';
** OR **;
%sysexec mkdir "\\computername\downloads\x\y\z ";
** OR **;
filename mkdir pipe "mkdir \\computername\downloads\x\y\z";
data _null_;
input mkdir;
put infile;
run;
It does not work. Not only this but the window closes immediately even though I have options xwait specified so there is no opportunity to see any ERROR messages. I have tried all methods with both the UNC path and a drive letter path, ie. D:\downloads\x\y\z.
If I look at the error messages being returned by the OS:
%put %sysfunc(sysrc()) %sysfunc(sysmsg());
I get the following:
-20006 WARNING: Physical file does not exist, d:\downloads\x\x\x.
Looking at the documentation for the mkdir command it appears that it only supports creating intermediate folders when 'command extensions' are enabled. This can be achieved with adding the /E:ON to cmd.exe. I've tried changing my code to use:
cmd.exe /c /E:ON mkdir "\\computername\downloads\x\y\z"
And still no luck!
Can anyone tell me why everyone else on the internet seems to be able to get this working from within SAS except for me? Again, it works fine from a DOS prompt - just not from within SAS.
I'd prefer an answer that specifically address this issue (I know there are other solutions that use multiple steps or dcreate()).
I'm running WinXP 32Bit, SAS 9.3 TS1M2. Thanks.
Here is a trick that uses the LIBNAME statement to make a directory
options dlcreatedir;
libname newdir "/u/sascrh/brand_new_folder";
I believe this is more reliable than an X statement.
Source: SAS trick: get the LIBNAME statement to create folders for you
You need to use the mkdir option -p which will create all the sub folders
i.e.
x mkdir -p "c:\newdirectory\level 1\level 2";
I'm on WinXP as well, using SAS 9.3 TS1M1. The following works for me as advertised:
122 options noxwait;
123 data _null_;
124 rc = system('mkdir \\W98052442n3m1\public\x\y\z');
125 put rc=;
126 run;
rc=0
NOTE: DATA statement used (Total process time):
real time 1.68 seconds
cpu time 0.03 seconds
That's my actual log file; "public" is a Windows shared folder on that network PC and the entire path was created. Perhaps using the SYSTEM function did the trick. I never ever use the X command myself.
You need to quote your x commands, e.g.
x 'mkdir "c:\this\that\something else"' ;
Also, I've never had a problem using UNC paths, e.g.
x "\\server.domain\share\runthis.exe" ;
This seems to work just fine with the dos window remaining open. You may need the XSYNC option. I am using 9.3 TS1M1 64 bit under VMWARE on a MAC:
options xwait xsync;
x mkdir c:\newdirectory;