I'm trying to get a Value out of a information block.
I'm creating the following output:
RSSI: -21
RSSI: -12
RSSI: -13
RSSI: -13
RSSI: -16
RSSI: -13
with following command:
hcidump -a | egrep 'RSSI|bdaddr' | grep -A1 --no-group-separator 'bdaddr XX:XX:XX:XX:XX' | egrep 'RSSI'
But i would like to get only the Value at the end to be displayed.
But I'm struggling with the right pattern, I found many hints online, but not one that will work, e.g.:
sed 's/RSSI: //'
Further I would like to know if there is a smoother way of my command on the top, because it seems with every | the output is getting slower, like it fills a buffer before it outputs the values.
PS: i wanna use this command in C with the system() function, is there an option to get the value to be stored in a C-variable?
Give this a try
hcidump -a |.. whatever...:XX:XX'|grep -Po 'RSSI:\s+\K.*'
or
hcidump -a |.. whatever...:XX:XX'|awk '/RSSI/{print $2}'
No need for sed, you can use cut:
cut -d' ' -f2 myfile
This will output only the 2nd column, considering one whitespace as column delimiter.
Edit: to delete whitespaces at the beginning of each line in myfile, do this first:
sed -i 's/^ *//' myfile
Or, if you have variable number of spaces in between, you can use following:
rev myfile | cut -d' ' -f1 | rev
Related
In a part of my script I am trying to generate a list of the year and month that a file was submitted. Since the file contains the timestamp, I should be able to cut the filenames to the month position, and then do a sort+uniq filtering. However sed is generating an outlier for one of the files.
I am using this command sequence
ls -1 service*json | sed -e "s|\(.*201...\).*json$|\1|g" | sort |uniq
And this works for most of time except in some cases it outputs the whole timestamp:
$ ls
service-parent-20181119092630.json service-parent-20181123134132.json service-parent-20181202124532.json service-parent-20190121091830.json service-parent-20190125124209.json
service-parent-20181119101003.json service-parent-20181126104300.json service-parent-20181211095939.json service-parent-20190121092453.json service-parent-20190128163539.json
service-parent-20181120095850.json service-parent-20181127083441.json service-parent-20190107035508.json service-parent-20190122093608.json
service-parent-20181120104838.json service-parent-20181129155835.json service-parent-20190107042234.json service-parent-20190122115053.json
$ ls -1 service*json | sed -e "s|\(.*201...\).*json$|\1|g" | sort |uniq
service-parent-201811
service-parent-201811201048
service-parent-201812
service-parent-201901
I have also tried this variation but the second output line is still returned:
ls -1 service*json | sed -e "s|\(.*201.\{3\}\).*json$|\1|g" | sort |uniq
Can somebody explain why service-parent-201811201048 is returned past the requested 3 characters?
Thanks.
service-parent-201811201048 happens to have 201048 to match 201....
Might try ls -1 service*json | sed -e "s|\(.*-201...\).*json$|\1|g" | sort |uniq to ask for a dash - before 201....
It is not recommended to parse the output of ls. Please try instead:
for i in service*json; do
sed -e "s|^\(service-.*-201[0-9]\{3\}\).*json$|\1|g" <<< "$i"
done | sort | uniq
Your problem is explained at https://stackoverflow.com/a/54565973/1745001 (i.e. .* is greedy) but try this:
$ ls | sed -E 's/(-[0-9]{6}).*/\1/' | sort -u
service-parent-201811
service-parent-201812
service-parent-201901
The above requires a sed that supports EREs via -E, e.g. GNU sed and OSX/BSD sed.
In my script, have a possible version number: 15.03.2 set to variable $STRING. These numbers always change. I want to strip it down to: 15.03 (or whatever it will be next time).
How do I remove everything after the second . using sed?
Something like:
$(echo "$STRING" | sed "s/\.^$\.//")
(I don't know what ^, $ and others do, but they look related, so I just guessed.)
I think the better tool here is cut
echo '15.03.2' | cut -d . -f -2
This might work for you (GNU sed):
sed 's/\.[^.]*//2g' file
Remove the second or more occurrence of a period followed by zero or non-period character(s).
$ echo '15.03.2' | sed 's/\([^.]*\.[^.]*\)\..*/\1/'
15.03
More generally to skip N periods:
$ echo '15.03.2.3.4.5' | sed -E 's/(([^.]*\.){2}[^.]*)\..*/\1/'
15.03.2
$ echo '15.03.2.3.4.5' | sed -E 's/(([^.]*\.){3}[^.]*)\..*/\1/'
15.03.2.3
$ echo '15.03.2.3.4.5' | sed -E 's/(([^.]*\.){4}[^.]*)\..*/\1/'
15.03.2.3.4
I have a multi-line string, downloaded from the Web:
toast the lemonade
blend with the lemonade
add one tablespoon of the lemonade
grill the spring onions
add the lemonade
add the raisins to the saucepan
rinse the horseradish sauce
I have assigned this to $INPUT, like this:
INPUT=$(lynx --dump 'http://example.net/recipes' \
| python -m json.tool \
| awk '/steps/,/]/' \
| egrep -v "steps|]" \
| sed 's/[",]\|^ *//g; $d')
At this point, $INPUT is ready for substitution into my target file as follows:
sed -i "0,/OLDINPUT/s//$INPUT/" /home/test_file
Of course, sed complains about an unterminated s command - herein lies the problem.
The current workaround I am using is to echo $INPUT prior to giving it to sed, but then the newlines are not preserved. echo strips newlines - which is the problem.
The correct output should maintain its newlines. How can sed be instructed to preserve the newlines?
The hacky direct answer is to replace all newlines with \n, which you can do by adding
| sed ':a $!{N; ba}; s/\n/\\n/g'
to the long command above. A better answer, because substituting shell variables into code is always a bad idea and with sed you wouldn't have a choice, is to use awk instead:
awk -i inplace -v input="$INPUT" 'NR == 1, /OLDINPUT/ { sub(/OLDINPUT/, input) } 1' /home/test_file
This requires GNU awk 4.1.0 or later for the -i inplace.
If you're using Bash, you can substitute \n for the newlines:
INPUT="${INPUT//
/\\n}"
If you don't like the literal linefeed in your parameter expansion, you might prefer
INPUT="${INPUT//$'\n'/\\n}"
Side note - you probably mean to change the matched lines to your input, not substitute each of them. In which case, you don't want to quote the newlines, after all...
To clean up your code some.
This:
lynx --dump 'http://somesite.net/recipes' | python -m json.tool | awk '/steps/,/]/' | egrep -v "steps|]" | sed 's/"//g' |sed 's/,//g' | sed 's/^ *//g' | sed '$d'
Can be replaced with this:
lynx --dump 'http://somesite.net/recipes' | python -m json.tool | awk '/]/ {f=0} f {if (c--) print line} /steps/{f=1} {gsub(/[",]|^ */,"");line=$0}'
It may be shorten more, but I do not now what this does: python -m json.tool
This:
awk '/]/ {f=0} f {if (c--) print line} /steps/{f=1} {gsub(/[",]|^ */,"");line=$0}'
Does:
Print line after pattern steps to line before ] - awk '/steps/,/]/' | egrep -v "steps|]"
Removes ", , and all space in front of all lines. - sed 's/"//g' |sed 's/,//g' | sed 's/^ *//g'
Then remove last line of this group. - sed '$d'
Example:
cat file
my data
steps data
more
do not delet this
hei "you" , more data
extra line
here is end ]
this is good
awk '/]/ {f=0} f {if (c--) print line} /steps/{f=1} {gsub(/[",]|^ */,"");line=$0}' file
more
do not delet this
hei you more data
Assuming your input JSON fragment looks something like this:
{ "other": "random stuff",
"steps": [
"toast the lemonade",
"blend with the lemonade",
"add one tablespoon of the lemonade",
"grill the spring onions",
"add the lemonade",
"add the raisins to the saucepan",
"rinse the horseradish sauce"
],
"still": "yet more stuff" }
you can extract just the steps member with
jq -r .steps
To interpolate that into a sed statement, you'd need to escape any regex metacharacters in the result. A less intimidating and hopefully slightly less hacky solution would be to read static text from standard input:
lynx ... | jq ... |
sed -i -e '/OLDINPUT/{s///; r /dev/stdin' -e '}' /home/test_file
The struggle to educate practitioners to use structure-aware tools for structured data has reached epic heights and continues unabated. Before you decide to use the quick and dirty approach, at least make sure you understand the dangers (technical and mental).
You'll want to use an editor instead of sed's substitution:
$ input="toast the lemonade
blend with the lemonade
add one tablespoon of the lemonade
grill the spring onions
add the lemonade
add the raisins to the saucepan
rinse the horseradish sauce"
$ seq 10 > file
$ ed file <<END
1,/5/d
1i
$input
.
w
q
END
$ cat file
toast the lemonade
blend with the lemonade
add one tablespoon of the lemonade
grill the spring onions
add the lemonade
add the raisins to the saucepan
rinse the horseradish sauce
6
7
8
9
10
I have a file with three columns. I would like to delete the 3rd column(in-place editing). How can I do this with awk or sed?
123 abc 22.3
453 abg 56.7
1236 hjg 2.3
Desired output
123 abc
453 abg
1236 hjg
try this short thing:
awk '!($3="")' file
With GNU awk for inplace editing, \s/\S, and gensub() to delete
1) the FIRST field:
awk -i inplace '{sub(/^\S+\s*/,"")}1' file
or
awk -i inplace '{$0=gensub(/^\S+\s*/,"",1)}1' file
2) the LAST field:
awk -i inplace '{sub(/\s*\S+$/,"")}1' file
or
awk -i inplace '{$0=gensub(/\s*\S+$/,"",1)}1' file
3) the Nth field where N=3:
awk -i inplace '{$0=gensub(/\s*\S+/,"",3)}1' file
Without GNU awk you need a match()+substr() combo or multiple sub()s + vars to remove a middle field. See also Print all but the first three columns.
This might work for you (GNU sed):
sed -i -r 's/\S+//3' file
If you want to delete the white space before the 3rd field:
sed -i -r 's/(\s+)?\S+//3' file
It seems you could simply go with
awk '{print $1 " " $2}' file
This prints the two first fields of each line in your input file, separated with a space.
Try using cut... its fast and easy
First you have repeated spaces, you can squeeze those down to a single space between columns if thats what you want with tr -s ' '
If each column already has just one delimiter between it, you can use cut -d ' ' -f-2 to print fields (columns) <= 2.
for example if your data is in a file input.txt you can do one of the following:
cat input.txt | tr -s ' ' | cut -d ' ' -f-2
Or if you better reason about this problem by removing the 3rd column you can write the following
cat input.txt | tr -s ' ' | cut -d ' ' --complement -f3
cut is pretty powerful, you can also extract ranges of bytes, or characters, in addition to columns
excerpt from the man page on the syntax of how to specify the list range
Each LIST is made up of one range, or many ranges separated by commas.
Selected input is written in the same order that it is read, and is
written exactly once. Each range is one of:
N N'th byte, character or field, counted from 1
N- from N'th byte, character or field, to end of line
N-M from N'th to M'th (included) byte, character or field
-M from first to M'th (included) byte, character or field
so you also could have said you want specific columns 1 and 2 with...
cat input.txt | tr -s ' ' | cut -d ' ' -f1,2
Try this :
awk '$3="";1' file.txt > new_file && mv new_file file.txt
or
awk '{$3="";print}' file.txt > new_file && mv new_file file.txt
Try
awk '{$3=""; print $0}'
If you're open to a Perl solution...
perl -ane 'print "$F[0] $F[1]\n"' file
These command-line options are used:
-n loop around every line of the input file, do not automatically print every line
-a autosplit mode – split input lines into the #F array. Defaults to splitting on whitespace
-e execute the following perl code
I can chain multiple sed substitutions and a awk operation to achieve this, but is there a single sed substitution that can do it?
Also is there any other tool that is more suitable for this parsing task?
You could try:
sed -r 's!rec:id=(.*?)&name=(.*?)&age=(.*?)!\1 \2 \3!' input_file
If you don't know the rec:id etc in advance but you know there's three, you could try:
sed -r 's![^=]+=(.*?)&[^=]+=(.*?)&[^=]+=(.*?)!\1 \2 \3!' input_file
If you don't know how many &name=value pairs you're after in advance but want to output all the values, you could try something like:
grep -P -o '(?<==)([^&]*)(?=&|$)' | xargs
where the -P means 'perl regex', the regex says "find the string followed by an & (or end of string) and preceded by and equals sign", the -o means to print just the matches (ie the 1, zz, and 21) each on their own line, and the | xargs moves these from their own line to one line and space separated (ie 1\nzz\n21 to 1 zz 21).
This might work for you:
echo "rec:id=1&name=zz&age=21" | sed 's/[^=]*=\([^&]*\)/\1 /g'
1 zz 21
However this leaves an extra space at the end, to solve this use:
echo "rec:id=1&name=zz&age=21"|sed 's/[^=]*=\([^&]*\)/\1 /g:;s/ $//'
1 zz 21
How about parsing the values directly into variables?
inbound="rec:id=1&name=zz&age=21"
eval $(echo $inbound | cut -c5- | tr \& "\n")
echo "Name:$name, ID:$id, Age:$age"
Or even better, though slightly more arcane:
inbound="rec:id=1&name=zz&age=21"
IFS=\& eval $(cut -c5- <<< $inbound)
echo "Name:$name, ID:$id, Age:$age"