I have a function block that runs an Expect script, pulls the output, tweaks it through a combination of tr and grep to pull out the specific line of text I am looking for and then displays the text in a Whiptail textbox. Everything works fine except the beginning of the output in the textbox; it contains characters that are not in the actual variable that is being outputted.
The function block I have is:
do_view_URL() {
NODE="$PROPERTY_IP""$TOTE_BOARD_IP"
OUTPUT=""
OUTPUT=$(/usr/bin/expect /home/pi/ssh_tote_board_url_grab.sh $NODE)
CURRENT_URL=$(echo $OUTPUT | tr -d '\r' | tr ' ' '\n' | grep https: | head -1)
echo $CURRENT_URL
echo $CURRENT_URL > ./textbox_file
whiptail --title "Current Production URL for Tote Board #$TOTE_BOARD_NUM" --textbox ./textbox_file 10 150
}
The Whiptail textbox output looks like this:
Whiptail Textbox
At the beginning of the output, the following characters appear; the characters are always the same no matter how many times I run the script:
^[[?2004l
As you can see in the function block, I am also outputting the value of "CURRENT_URL" to the CLI. The results of that as well as a cat of the ./textbox_file are as follows; clearly the garbage characters do not appear in either the echo output or the cat output:
pi#XXXXXX:~ $ ./tote_board_updater.sh
https://<REMOVED>
pi#XXXXXX:~ $ cat ./textbox_file
https://<REMOVED>
pi#XXXXXX:~ $
Any help, assistance, or guidance as to where these characters could be coming from and how I can get rid of them in the output so that the end result is that the output is just the full URL would be greatly appreciated. Thank you in advance.
Related
This is a Bash/.bat terminal script for Mac.
I'm trying to add text ("!!XX!!") into a group of tab-delimited .txt files in a folder, but I only want to add it into the 4th and all following incidents of the tab in each .txt file, and then only if those cels have text in them. So, the end result would be something like (assuming the 7th cel/field/bit of info is blank). So turn this:
text01
text02
text03
text04
text05
text06
... into this:
text01 [TAB] text02 [TAB] text03 [TAB] text04!!XX!! [TAB] text05!!XX!! [TAB] text06!!XX!! [TAB]
The text marker "!!XX!!" is so that another script in a different system can run on the file and perform special system-compatible/custom line formatting at each incident of "!!XX!!", but I don't want to populate the first three fields/tab-delimited text (because it's not needed there) or in the empty fields (because it's not wanted there).
I'm already replacing each line return with a tab, so it is possible to do it there, though my preference is to do it later to the tab-delimited text b/c of weird issues with the line returns/formatting coming in from .rtf files. Below is what I am to replace each line return and replace it with a TAB (and, yes, that is an actual line return and tab in there, which seems to work best because... Macs?):
perl -pi -w -e 's/
/ /g' *.txt;
Thanks in advance :)
This post assumes an input file that has lines with tab-separated fields, where each field starting from (and including) the fourth need be edited if it has something.
One way
perl -F"\t" -wlane'
for (3..$#F) { $F[$_] .= "!XX!" if defined $F[$_] }; print join("\t", #F)
' file
(In tcsh shell need to escape those ! with a backslash.) Once you've tested enough add -i switch to change input file in place (-i.bak keeps a backup).
This uses Perl's -a switch to break input lines by what is given under -F switch (or by whitespace by default), and the resulting array is in #F. See switches in perlrun.
Then it iterates from the fourth field to the last. I use syntax $#ary for the index of the last element of array #ary.
I don't know what counts for cells that "have text in them" so above I test a field for defined-ness; thus this will append even for an empty string. Adjust as suitable.
Or use a regex, which allows more flexibility here. For example,
for (3..$#F) { $F[$_] =~ s/.+\K/!XX!/ }
This matches all characters and then adds !XX! (keeping what it matched, by \K assertion). Using regex allows and demands to specify more precisely what is accepted there; the shown pattern will match even for whitespace alone, but not for empty string. To not touch fields with whitespace only, and to strip trailing spaces if any
for (3..$#F) { $F[$_] =~ s/.+\S\K\s*/XX/ };
Again, adjust to your details.
I don't quite understand the discussion of newlines and what is wanted of them; the above one-liner goes line by line. If that's not what you need please clarify. I don't have Macs to test, so I can't comment on all that.
A self-contained example for ready testing and tweaking
echo "t1\tt2\tt3\tt4\t\tt6 \t " |
perl -F"\t" -wlanE'for (3..$#F) { $F[$_] =~ /.+\S\K\s*/XX/ } say for #F'
where I print each field on a separate line for easier inspection. The last tab in input is followed by trailing spaces only -- this results in an empty field, but with no text marker added (as asked for in a comment).
with GNU sed
$ echo text{01..07}$'\t' | sed -E 's/([^\t]+)(\t|$)/\1!!xx!!/4g'
text01 text02 text03 text04!!xx!! text05!!xx!! text06!!xx!! text07!!xx!!
or
$ echo text{01..07}$'\t' | sed -E 's/\t([^\t]+)/\1!!xx!!/3g'
Assuming each text file contains 7 lines, you can do
paste -s *.txt | awk '
BEGIN {FS=OFS="\t"}
{for (i=4; i<=NF; i++) if ($i != "") $i = $i "!!XX!!"; print}
'
Here is an awk:
echo text{01..10}$'\t' |
awk -v OFS=$'\t' '{for(i=1;i<=NF;i++) printf "%s%s", $i, i>=4 ? "XXX\t" : i<NF ? OFS : ORS }'
With perl, I would do this:
echo text{01..10}$'\t' |
perl -lpE '$cnt=0; s/\h+/++$cnt>=4 ? "XXX\t" : "\t"/ge;'
Both print:
text01 text02 text03 text04XXX text05XXX text06XXX text07XXX text08XXX text09XXX text10XXX
I've been trying to extract the substring in between parentheses (including parentheses) from:
"WHITE-TAILED TROPIC-BIRD _Phaëthon lepturus_ (Hawaiian name—koae)"
I tried this:
str=$(echo $1 | sed 's/.*\(\([^)]*\)\).*/\1/');
echo $str
What I wanted to get was:
"(Hawaiian name—koae)"
However, I've been getting an error called:
bash: syntax error near unexpected token `('
What do I do wrong?
You may use
sed -n 's/.*\(([^()]*)\).*/\1/p'
Here,
-n - suppresses default line output
.*\(([^()]*)\).* - matches any text, then captures into Group 1 a (, then 0 or more chars other than ( and ), then a ), and then again .* matches any text to the end of the string
/\1/ - replaces the whole match with the contents of Group 1
p - prints the result.
See online demo
In a script called script.sh:
#!/bin/bash
str=$(echo "$1" | sed -n 's/.*\(([^()]*)\).*/\1/p');
echo "$str"
Called like
bash ./script.sh "WHITE-TAILED TROPIC-BIRD _Phaëthon lepturus_ (Hawaiian name—koae)"
Result: (Hawaiian name—koae)
echo "WHITE-TAILED TROPIC-BIRD _Phaëthon lepturus_ (Hawaiian name—koae)" | \
cut -d'_' -f3 | sed s'#^ ##'
If you've got good delimiters, then it is much easier to get parts of lines with cut, than with sed. I only really use sed for replacing characters when I know exactly where they are. For more complex operations, I will usually have to get a piece of text out of the file by inserting carriage returns before and after it with ed, (thus putting it on its' own line, which I can then rip out easily) make the minor changes to it that I want with sed, and then use ed to put it back in.
I am developing a tomcat application and would like to be able to search for specific things and highlight it when viewing the log. I want something like an alias that takes a parameter (regex) as input and highlight the matching string.
So far, I've figured this works, but its not practical enough to have to change a small part of it for every time I want something new:
tail -n 100 -f /opt/apache-tomcat-6.0.26/logs/catalina.out | perl -pe 's/null/\e[1;31m$&\e[0m/g'
This is what I thought would work:
logColor(){
x="'s/"
y="/\e[1;31m$&\e[0m/g'"
tail -n 100 -f /opt/apache-tomcat-6.0.26/logs/catalina.out | perl -pe $x$1$y
}
alias logC=logColor
I've tested that this prints out the two same lines:
logColorTest(){
x="'s/"
y="/\e[1;31m$&\e[0m/g'"
echo $x$1$y
echo "'s/null/\e[1;31m$&\e[0m/g'"
}
alias logCT=logColorTest
logCT null
So I am lost on why this does not work and would appreciate input from someone who knows how this works :)
Problem with grep is that, you get only matching lines & other lines are filtered out. (That's what is grep supposed to do anyway.) Many times however, we need all the output, but with some particular strings highlighted.
I have this small bash function in my .bashrc for such requirement:
mark ()
{
local searchExpr=${1/\//\\\/};
sed "s/$searchExpr/"`echo -n -e "\e[91;1m"`'&'`echo -n -e "\e[0m"`'/gi' $2
}
Usage:
command | mark some_string # OR
mark some_string some_file
Rename to suitable function name if required.
NOTE: There is a great command called highlight. Hence I could not use that as my function name.
As #fedorqui pointed out, you can use grep to do this:
grep --colour 'null\|$'
This will match and highlight null or the end of a line, meaning all lines are shown.
Using the GREP_COLORS environment variable you can control how different parts are highlighted, e.g mark matched text in yellow:
export GREP_COLORS='ms=1;33'
I am doing some calculations using gaussian. From the gaussian output file, I need to extract the input structure information. The output file contains more than 800 structure coordinates. What I did so far is, collect all the input coordinates using some combinations of the grep, awk and sed commands, like so:
grep -A 7 "Input orientation:" test.log | grep -A 5 "C" | awk '/C/{print "structure number"}1' | sed '/--/d' > test.out
This helped me to grep all the input coordinates and insert a line with "structure number". So now I have a file that contains a pattern which is being repeated in a regular fashion. The file is like the following:
structure Number
4.176801 -0.044096 2.253823
2.994556 0.097622 2.356678
5.060174 -0.115257 3.342200
structure Number
4.180919 -0.044664 2.251182
3.002927 0.098946 2.359346
5.037811 -0.103410 3.389953
Here, "Structure number" is being repeated. I want to write a number like "structure number:1", "structure number 2" in increasing order.
How can I solve this problem?
Thanks for your help in advance.
I am not familiar at all with a program called gaussian, so I have no clue what the original input looked like. If someone posts an example I might be able to give an even shorter solution.
However, as far as I got it the OP is contented with the output of his/her code besided that he/she wants to append an increasing number to the lines inserted with awk.
This can be achieved with the following line (adjusting the OP's code):
grep -A 7 "Input orientation:" test.log | grep -A 5 "C" | awk '/C/{print "structure number"++i}1' | sed '/--/d' > test.out
Addendum:
Even without knowing the actual input, I am sure that one can at least get rid of the sed command leaving that piece of work to awk. Also, there is no need to quote a single character grep pattern:
grep -A 7 "Input orientation:" test.log | grep -A 5 C | awk '/C/{print "structure number"++i}!/--/' > test.out
I am not sure since I cannot test, but it should be possible to let awk do the grep's work, too. As a first guess I would try the following:
awk '/Input orientation:/{li=7}!li{next}{--li}/C/{print "structure number"++i;lc=5}!lc{next}{--lc}!/--/' test.log > test.out
While this might be a little bit longer in code it is an awk-only solution doing all the work in one process. If I had input to test with, I might come up with a shorter solution.
Using shell script I want to read a word from text file and return next column word.
For eg, my input file will be like
AGE1 PERSON1
AGE2 PERSON2
AGE3 PERSON3
AGE4 PERSON4
I have variable in Sh file having PERSON's name.
I want read input text file and get value of person's age.
Please help, i'm beginner in Shell Scripting
A slightly simpler solution is:
age=$( awk '$2==name { print $1 }' name="$name" input-file )
Building upon shellter's comment:
age=$(grep "$person_name" people_file.txt | cut -f1 -d' ')
I'll try to explain everything. First, I assume somethings (but you can change them on your script):
Your file with the data you entered is called people_file.txt.
The person's name you want to find is in the variable $person_name.
The variable you want to store the result is $age.
Firstly, because we need to use commands to generate the value of the $age variable, we must use $( and ) to run a command (or a series of commands), and replace itself with the text it captures from executing the command (or commands).
We first need to find the line which contains the person's name. For that we use grep: grep regex file. Grep will search file line by line until it finds a line that matches the regular expression regex. In our case we can simply search for the person's name directly (assuming it doesn't contain special characters, like the period or an asterisk). Note that we must place the variable between double quotes, otherwise a person's name that has a space in it might be split in the command line so that its first name is used as the regular expression and the surname as the file. If you want to search in a case insensitive manner (like for example: John will find a line with JOHN or john), you can use the -i flag: grep -i regex file. The selected lines will be printed by grep into its output, but we will pump those lines into the input of the next command with the pipe operator |.
Finally, we have a line (or many lines) with the results. Now we must extract the age. The cut command will split each line it reads from the input into fields, and only print the fields you ask it to. In this case, we ask for the first field with the -f1 option. Also, we specify that the space character is to be used as the delimeter (ie. the character that separates the fields) with the -d1 command.
If you have more than one line with the same person's name, we need to pipe the output of grep into a head command, so that we can have only the number of lines we want. We can tell head how many lines we want with the -n N option. So if you only want the first match:
age=$(grep "$person_name" people_file.txt | head -n 1 | cut -f1 -d' ')
Hope this helps a little =)
age=`
perl -nle'
BEGIN { $n = shift(#ARGV); }
print $1 if /^(\S+)\s+\Q$n\E$/;
' "$name" file
`
Tested with bash in sh mode.