Read file lines into list - fish

How come $VARS doesn't contain ('testing', 'second') in the following example:
~ ❯❯❯ /bin/cat t
testing
second
~ ❯❯❯ /bin/cat t | read -L -a VARS
~ ❯❯❯ echo $VARS
testing
~ ❯❯❯
I thought that -L is supposed to split on lines and then -a stores into a list. fish seems to stop after the first line, however.

It's unclear why this works based on the documentation, but this is the solution:
~ ❯❯❯ /bin/cat t
testing
second
~ ❯❯❯ /bin/cat t | read -za VARS
~ ❯❯❯ echo $VARS
testing second
See this issue.

Related

Why "-n" is commonly used for dry-run?

Well known commands like make, rsync, and, git use -n option for dry-run.
What does -n stand for in this context?
My guess is that it's because dry-run contains the letter n and because d and r are already used:
$ make --help | grep '^ *-[dr]'
-d Print lots of debugging information.
-r, --no-builtin-rules Disable the built-in implicit rules.
$ rsync --help | grep '^ *-[dr]'
-r, --recursive recurse into directories
-d, --dirs transfer directories without recursing

Why is this docopt string not working either with or without optional args?

Here is the complete docopt string I used:
foo.
Usage:
foo [options] <file> -o <output>
foo --help | --version
Options:
-h, --help print this help message.
--target <target> target.
--version print the version.
According to the official parser, either foo a -o b or foo --target abc a -o b is not correctly parsed. What could be possible reasons for this case? Any help would be appreciated.
I'm not entirely sure about the allowed combinations of options for your script, but here's something that should be close.
Just for fun, I wrote a script that has similar options to yours to test this out with the latest docopts.
I found it simplest to write just [options] in the main Usage section, and have all the options below as alternatives, with no specific combinations required.
I'm on macOS so I'm using bash 3.2 (with patched docopts.sh to fix some Mac issues). You can avoid some of the code in this script if you're on bash 4.x - see the commented-out section with --auto option and docopt_print_ARGS function. Currently you would need bash 4.x to avoid patching docopts.sh.
#!/bin/bash
#
# foo.
#
# Usage:
# foo [options] <file>
# foo --help | --version
#
# Options:
# -t, --target <target> target.
# -o, --output <output> output.
# -h, --help print this help message.
# --version print the version.
#
# bash 3.2 (patched docopts.sh) - include set -x to see the args easily
source docopts.sh
usage=$(docopt_get_help_string "$0")
set -x
eval "$(docopts -G ARGS -V "$VERSION" -h "$usage" : "$#")"
# On bash 4.x, replace preceding section with this, or use -A instead of -G above
# source docopts.sh --auto "$#"
# docopt_print_ARGS
This parses the Usage section OK and processes command lines such as:
foo --target a -o b file1
foo --target a --output b file1
foo --target a file1
Partial output with set -x to show args processed correctly:
$ ./foo --target a file1 --output b
...
++ ARGS_target=a
++ ARGS_output=b
++ ARGS_file=file1
++ ARGS_help=false
++ ARGS_version=false
Thanks for #RichVel's efforts. Yesterday I finally found out the underlying (stupid) cause for this problem.
In the official online parser the first part, i.e. foo shouldn't be used. --target abc a -o b works fine in the online example.
Regarding my question, the bug actually comes from that docopt.rs stores --target abc in flag_target instead of arg_target.

Using /proc/<pid>, how can I identify a network port number's application?

I'm trying to identify what application is running on port 56474 without having root access. I know the application was started by me.
Example:
netstat -tunap
tcp 0 0 0.0.0.0:56474 0.0.0.0:* LISTEN -
I've tried using /proc/pid scripts to walk all using grep on ls -l /proc/pid/fd results. Here is my attempt. NOTE: Not sure if I was heading the right direction
for I in `find /proc/*/fd -exec ls -l {} \; 2>/dev/null | awk -F"->|:" '/socket/ {print $4}' | sort -u | sed -e 's/\[//g' -e 's/\]//g'`; do grep $I /proc/*/net/tcp; done
I had no success. Not sure if there is a way. Thanks.
NOTE: Added another answers as lsof was not satisfactory.
This should work:
#! /bin/bash
port=56474
hex_port=$(echo "obase=16; $port" | bc )
inode=$(cat /proc/net/tcp | grep ":$hex_port" | awk '{print $10}')
for i in $(ps axo pid); do
ls -l /proc/$i/fd 2> /dev/null | grep -q ":\[$inode\]" && echo $i
done
Explanation:
Once we have the port number converted to Hexadecimal, we can get the inode number from /proc/net/tcp (10th field), then we loop through /proc/pids/fd and find a symlink pointing to the inode.
If you're sure the application was started by you then you can use lsof:
/usr/sbin/lsof -nP | grep :56474 | awk '{print $2}'
Another technique to resolve pids and ports of all running apps without root:
1.) Get the pids of running apps. Either use the ActivityManager or parse a ps console output.
2.) iterate through /proc/$pid/net/status files and get the matching uid for a pid.
cat /proc/*pid*/net/status | grep Uid:
3.) Call and parse the output of tcp, tcp6,udp, udp6 files to match ports and uids:
cat /proc/net/tcp
...
4.) match the uids of both matchings, get a port-to-pid map without su access.
Cheers,
goethe

Inconsistent External Command Output

The terminal transcript speaks for itself:
iMac:~$ echo -n a | md5
0cc175b9c0f1b6a831c399e269772661
iMac:~$ perl -e 'system "echo -n a | md5"'
c3392e9373ccca33629d82b17699420f
Note that the MD5 hash of a is 0cc175b9c0f1b6a831c399e269772661, the first
result. Why does it turns out to be different when the same command is called
by perl?
By the way, perl is perl 5, version 12, subversion 4 (v5.12.4) built for darwin-thread-multi-2level. And the system: Mac OS 10.8, Darwin 12.0
When in the /bin/sh shell on mac, echo -n doesn't not print out the newline like it does in /bin/bash. You can see this if you drop into /bin/sh and run echo -n a, your output should look like this:
sh-3.2$ echo -n a
-n a
so you're literally getting -n a instead of the desired a. As perl system runs /bin/sh to evaluate your command, -n a is being passed into md5 instead of your desired a
The specific question has already been answered, but I want to point out that od is useful to help understand exactly what any command outputs or file contains. This is useful especially to show otherwise non-printing characters.
$ echo -n a | od -tc
0000000 a
0000001
$ perl -e 'system "echo -n a | od -tc";'
0000000 - n a \n
0000005

Map sd?/sdd? names to Solaris disk names?

Some commands in Solaris (such as iostat) report disk related information using disk names such as sd0 or sdd2. Is there a consistent way to map these names back to the standard /dev/dsk/c?t?d?s? disk names in Solaris?
Edit: As Amit points out, iostat -n produces device names such as eg c0t0d0s0 instead of sd0. But how do I found out that sd0 actually is c0t0d0s0? I'm looking for something that produces a list like this:
sd0=/dev/dsk/c0t0d0s0
...
sdd2=/dev/dsk/c1t0d0s4
...
Maybe I could run iostat twice (with and without -n) and then join up the results and hope that the number of lines and device sorting produced by iostat is identical between the two runs?
Following Amit's idea to answer my own question, this is what I have come up with:
iostat -x|tail -n +3|awk '{print $1}'>/tmp/f0.txt.$$
iostat -nx|tail -n +3|awk '{print "/dev/dsk/"$11}'>/tmp/f1.txt.$$
paste -d= /tmp/f[01].txt.$$
rm /tmp/f[01].txt.$$
Running this on a Solaris 10 server gives the following output:
sd0=/dev/dsk/c0t0d0
sd1=/dev/dsk/c0t1d0
sd4=/dev/dsk/c0t4d0
sd6=/dev/dsk/c0t6d0
sd15=/dev/dsk/c1t0d0
sd16=/dev/dsk/c1t1d0
sd21=/dev/dsk/c1t6d0
ssd0=/dev/dsk/c2t1d0
ssd1=/dev/dsk/c3t5d0
ssd3=/dev/dsk/c3t6d0
ssd4=/dev/dsk/c3t22d0
ssd5=/dev/dsk/c3t20d0
ssd7=/dev/dsk/c3t21d0
ssd8=/dev/dsk/c3t2d0
ssd18=/dev/dsk/c3t3d0
ssd19=/dev/dsk/c3t4d0
ssd28=/dev/dsk/c3t0d0
ssd29=/dev/dsk/c3t18d0
ssd30=/dev/dsk/c3t17d0
ssd32=/dev/dsk/c3t16d0
ssd33=/dev/dsk/c3t19d0
ssd34=/dev/dsk/c3t1d0
The solution is not very elegant (it's not a one-liner), but it seems to work.
One liner version of the accepted answer (I only have 1 reputation so I can't post a comment):
paste -d= <(iostat -x | awk '{print $1}') <(iostat -xn | awk '{print $NF}') | tail -n +3
Try using the '-n' switch. For eg. 'iostat -n'
As pointed out in other answers, you can map the device name back to the instance name via the device path and information contained in /etc/path_to_inst. Here is a Perl script that will accomplish the task:
#!/usr/bin/env perl
use strict;
my #path_to_inst = qx#cat /etc/path_to_inst#;
map {s/"//g} #path_to_inst;
my ($device, $path, #instances);
for my $line (qx#ls -l /dev/dsk/*s2#) {
($device, $path) = (split(/\s+/, $line))[-3, -1];
$path =~ s#.*/devices(.*):c#$1#;
#instances =
map {join("", (split /\s+/)[-1, -2])}
grep {/$path/} #path_to_inst;
*emphasized text*
for my $instance (#instances) {
print "$device $instance\n";
}
}
I found the following in the Solaris Transistion Guide:
"Instance Names
Instance names refer to the nth device in the system (for example, sd20).
Instance names are occasionally reported in driver error messages. You can determine the binding of an instance name to a physical name by looking at dmesg(1M) output, as in the following example.
sd9 at esp2: target 1 lun 1
sd9 is /sbus#1,f8000000/esp#0,800000/sd#1,0
<SUN0424 cyl 1151 alt 2 hd 9 sec 80>
Once the instance name has been assigned to a device, it remains bound to that device.
Instance numbers are encoded in a device's minor number. To keep instance numbers consistent across reboots, the system records them in the /etc/path_to_inst file. This file is read only at boot time, and is currently updated by the add_drv(1M) and drvconf"
So based upon that, I wrote the following script:
for device in /dev/dsk/*s2
do
dpath="$(ls -l $device | nawk '{print $11}')"
dpath="${dpath#*devices/}"
dpath="${dpath%:*}"
iname="$(nawk -v dpath=$dpath '{
if ($0 ~ dpath) {
gsub("\"", "", $3)
print $3 $2
}
}' /etc/path_to_inst)"
echo "$(basename ${device}) = ${iname}"
done
By reading the information directly out of the path_to_inst file, we are allowing for adding and deleting devices, which will skew the instance numbers if you simply count the instances in the /devices directory tree.
I think simplest way to find descriptive name having instance name is:
# iostat -xn sd0
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
4.9 0.2 312.1 1.9 0.0 0.0 3.3 3.5 0 1 c1t1d0
#
The last column shows descriptive name for provided instance name.
sd0 sdd0 are instance names of devices.. you can check /etc/path_to_inst to get instance name mapping to physical device name, then check link in /dev/dsk (to which physical device it is pointing) it is 100% sure method, though i dont know how to code it ;)
I found this snippet on the internet some time ago, and it does the trick. This was on Solaris 8:
#!/bin/sh
cd /dev/rdsk
/usr/bin/ls -l *s0 | tee /tmp/d1c |awk '{print "/usr/bin/ls -l "$11}' | \
sh | awk '{print "sd" substr($0,38,4)/8}' >/tmp/d1d
awk '{print substr($9,1,6)}' /tmp/d1c |paste - /tmp/d1d
rm /tmp/d1[cd]
A slight variation to allow for disk names that are longer than 8 characters (encountered when dealing with disk arrays on a SAN)
#!/bin/sh
cd /dev/rdsk
/usr/bin/ls -l *s0 | tee /tmp/d1c | awk '{print "/usr/bin/ls -l "$11}' | \
sh | awk '{print "sd" substr($0,38,4)/8}' >/tmp/d1d
awk '{print substr($9,1,index($9,"s0)-1)}' /tmp/d1c | paste - /tmp/d1d
rm /tmp/d1[cd]