I have a script that performs many XML edit operations with XMLStarlet.
For instance, it removes all foo nodes if any are present:
xmlstarlet ed -d '//foo'
(except that in my script, the name of the element is not foo).
When no foo node is present, the following message is printed:
None of the XPaths matched; to match a node in the default namespace
use '_' as the prefix (see section 5.1 in the manual).
For instance, use /_:node instead of /node
But there is nothing wrong if no foo nodes are present in the input document.
So for this particular operation, I do want to avoid this particular warning,
while I do not want to disable such warnings in general.
How can I achieve this?
At the moment, deleting a node that does not exist is categorized by xmlstarlet as an error. The return code will be 1 which you will have to understand as "no nodes were removed".
The error message appears if you happen to use a document which has a default namespace:
No namespace:
echo '<foo />' | xmlstarlet sel -t -m foo -v joe
With a default namespace xmlstarlet prints the error message
echo '<foo xmlns="urn:foo" />' | xmlstarlet sel -t -m foo -v joe
None of the XPaths matched; to match a node in the default namespace
use '_' as the prefix (see section 5.1 in the manual).
For instance, use /_:node instead of /node
No default namespace:
echo '<ns:foo xmlns:ns="urn:foo" />' | xmlstarlet sel -t -m foo -v joe
In all of these cases, no nodes are found, and xmlstarlet thus exit with return code 1 (i.e. an error). The error message was to explain the error in the case where the user forgot that the document had a default namespace. I have discussed this with the author and more recently changes have been introduced to reduce the chances of these messages, and a way to inhibit them.
xmlstarlet has (yet undocumented) support for using the document's namespaces, without needing to declare them up-front:
Compare:
echo '<ns:foo xmlns:ns="urn:foo" />' | xmlstarlet sel -t -m ns:foo -v 'count(.)'
echo '<ns:foo xmlns:ns="urn:foo" />' | xmlstarlet sel -N xx=urn:foo -t -m xx:foo -v 'count(.)'
Technically, the commands are identical, except the first one depends on the document binding to the 'ns' prefix, while the second does not.
To inhibit the message, you have to redirecting stderr to null:
echo '<foo xmlns="urn:xxx"/>' |
xmlstarlet sel -t -m foo -v joe 2> /dev/null
The downside of this is that it suppresses legitimate error messages, not just this bogus one which is caused by the fact that the source document uses namespaces.
In recent builds --no-doc-namespace has been added to inhibit this behaviour
The change was introduced in this changeset and the changeset contains a long exchange regarding this error message, all caused by this StackOverflow question!
Related
I am attempting to automate the following series of commands which work correctly into a BASH script:
kubectl exec -it mongo-pod -- bash
mongo DBNAME
db.auth("theUser", "thePw")
db.theCollection.find()
The script I am using is as follows:
#!/bin/bash
kubectl exec -it mongo-pod -- bash -c "mongo DBNAME && /
db.auth("theUser", "thePw") && /
db.theCollection.find()"
I have tried the following:
Executing multiple commands( or from a shell script) in a kubernetes pod
but any commands that are added after the first using & or && are not executed. For example just using "mongo DBNAME" correctly opens the prompt and sets it to the correct db, but adding any other command with && causes all commands to fail with the following:
bash: -c line 0: syntax error near unexpected token 'theUser'
All of the comments are spot on, but at least two things I have the highest confidence I have the answer to:
First, you have the line continuation character wrong; it should be \ and not /. It actually wouldn't even be required if you switched bash into "exit on error" mode, with
kubectl exec -it mongo-pod -- bash -ec "mongo DBNAME
echo 'this command only runs if mongo exits a-ok'
exit 1
and this never will run
"
However, the other mistake is around the quoting characters used: if you have bash -c " then you must either use the single-quote for the interior string literals, or escape them with \". You can actually see what I'm talking about by looking at the syntax highlighting of the shell snippet in your question. Observe that the string literal is red, but then the text theUser as well as thePw are both black -- that's because they are outside the string literal since the string stopped at the first " it encountered -- the one present in db.auth("
It is almost always the case that you'll want to use single quotes when invoking bash remotely like that, for several reasons but the most relevant is that you can then use db.auth("something") without having to unnecessarily escape the double quotes.
Since mongo (like many interpreters such a node and python) wants you to either type in it interactively, provide the input on its "standard input", or give it a local file containing commands, you will want to change the invocation to one of those strategies depending on your needs.
A very convenient way of redirecting standard input without having to use echo or printf and its associated quoting hell is to use what are called "here documents" (abbreviated "heredocs") in bash:
kubectl exec -it mongo-pod -- bash -ec 'mongo DBNAME<<"FOO"
db.auth("theUser", "thePw")
printjson(db.theCollection.find())
FOO
'
That causes bash to transmit almost all characters between the two "heredoc delimiters" to the standard input of the command. If you quote the delimiter, as I have with the [arbitrary] word FOO, then the contents are not subject to variable expansion, command interpolation, etc, which can be one more mechanism to avoid backtick and dollarsign weirdness.
As the title suggests, I'm trying to compile and subsequently link a binary, and I want the sections (text, data, bss, etc.) to be paged-aligned. To achieve that, I'm trying to pass the --no-omagicoption to ld. The literal command I'm using (in my Makefile) is:
ld --no-omagic -o build/initlink.out $^
ld --help says the option should have the following effect:
--no-omagic Page align data, make text readonly
But if I do an nm -n on the output, the first symbol in each section is not aligned. For example:
$ nm -n build/initlink.out | grep ' [tT] ' | head -n 1
00000000004000f0 T main
$ nm -n build/initlink.out | grep ' [dD] ' | head -n 1
00000000006d74b0 d vsscanf.basefix
$ nm -n build/initlink.out | grep ' [bB] ' | head -n 1
00000000006e7200 B __bss_start
If it's important, one thing with which I'm trying to link is a static library (.a extension).
Is there anything that I could be doing wrong? Is there another way to ensure page alignment of sections, without resorting to a linker script? I've tried using the --section-alignment option, but when I do, I'm told that it's unrecognized. The static library defines some strange symbols (involving "linker sets," which I've found to be woefully undocumented) with which I was running into issues, so I want to try to do this without having to do anything but tell the linker to put sections in certain places.
I am searching for swipl the similar feature as perl -e
In particular, I want to run prolog code in this fashion:
swipl --wanted-flag "fact(a). message:-writeln('hello')." -g "message" -t halt
This is possible to do with
swipl -f file -g "message" -t halt
where the prolog clauses are written in file
I am running swipl on the server side that takes user input as prolog clauses, therefore writing a file on the server is not a good idea.
One thing you can do is to use load_files/2 with the option stream, and load from standard input, not from an argument (you can still pass the entry point as an argument, I guess):
Say in a file fromstdin.pl you have:
main :-
load_files(stdin, [stream(user_input)]),
current_prolog_flag(argv, [Goal|_]),
call(Goal),
halt.
main :- halt(1).
and with this you can do:
$ echo 'message :- format("hello~n").' | swipl -q -t main fromstdin.pl -- message
|: hello
The comments by #false to this answer and the question will tell you what this |: is, if you are wondering, but if it annoys you, just do:
$ echo 'message :- format("hello~n").' \
| swipl -q -t main fromstdin.pl -- message \
| cat
hello
instead.
This will let you read any Prolog from standard input and call an arbitrary predicate from it. Whether this is a clever thing to do, I don't know. I would also not be surprised if there is a much easier way to achieve the same.
can xmlstarlet be used with a String instead of a xml file?
e.g.:
xmlstarlet sel -t -v "/*" "<pathlist><path>C:\file.txt</path></pathlist>"
instead of
xmlstarlet sel -t -v "/*" pathlist.xml
or how else could i realize with a string ?
when i echo the string and pipe it to xmlstarlet it does not work:
SET "_var=^<pathlist^>^<path^>C:\file.txt^</path^> ^</pathlist^>"
&
call echo %^_var% | xmlstarlet sel -t -v "//*"
gives error:
< was unexpected at this time.
-:1.1: Document is empty
^
-:1.1: Start tag expected, '<' not found
^
this is a simple task actually, but i cant get it to work. i just want to echo a string to xmlstarlet within a One-Liner.
cmd.exe syntax is weird, the following trick using set /p seems to work:
C:\tmp><nul (set /p ="<pathlist><path>C:\file.txt</path></pathlist>") | xmlstarlet sel -t -v /*
C:\file.txt
/* may get glob expanded (depending on what files you have). Unfortunately, there is no way to quote it from cmd.exe (the expansion is performed by libc on behalf of xmlstarlet), so you will have to rewrite the XPath in that case, e.g. /pathlist instead.
Source: https://groups.google.com/d/msg/alt.msdos.batch.nt/RNug94fXI5s/BdgYJfNmXysJ via http://www.netikka.net/tsneti/info/tscmd047.htm
I found no explanation of why escaping <> doesn't work with | redirection??
C:\tmp> echo ^<^>
<>
C:\tmp> echo ^<^> | more
> was unexpected at this time.
I have a bash script in which I have a few qsubs. Each of them are waiting for a preivous qsub to be done before starting.
My first qsub consist of sending files in a certain directory to a perl program and having the outfiles printed in a new directory. At the end, I echo the array with all my jobs names. This script works as intented.
mkdir -p /perl_files_dir
for ID_FILES in `ls Infiles_dir/*.txt`;
do
JOB_ID=`echo "perl perl_scirpt.pl $ID_FILES" | qsub -j oe `
JOB_ID_ARRAY="${JOB_ID_ARRAY}:$JOB_ID"
done
echo $JOB_ID_ARRAY
My second qsub is meant to sort all my previous files made with my perl script in a new outfile and to start after all these jobs are done (about 100 jobs) with depend=afterany. Again, this part is working fine.
SORT_JOB=`echo "sort -m -n perl_files_dir/*.txt >>sorted_file.txt" | qsub -j oe -W depend=afterany$JOB_ID_ARRAY`
SORT_ARRAY="${SORT_ARRAY}:$SORT_JOB"
My issue is that in my sorted file, I have a few columns I wish to remove (2 to 6), so I came up with this last line using awk piped to sed with another depend=afterany
SED=`echo "awk '{\$2="";\$3="";\$4="";\$5="";\$6=""; print \$0}' sorted_file.txt \
| sed 's/ //g' >final_file.txt" | qsub -j oe -W depend=afterany$SORT_ARRAY`
This last step creates final_file.txt, but leaves it empty. I added SED= before my echo because it would otherwise give me Command not found.
I tried without the pipe so it would just print everything. Unfortunately it prints nothing.
I assume it is not opening my sorted file and this is why my final file is empty after my sed. If it's the case, then why won't awk read it?
In my script, I am using variables to define my directories and files (with the correct path). I know my issue is not about find my files or directories since they are perfectly defined at the beginning and used throughout the script. I tried to write the whole path instead of a variable and I get the same results.
for ID_FILES in `ls Infiles_dir/*.txt`
Simplify this to
for ID_FILES in Infiles_dir/*.txt
ls lists the files you pass it (except when you pass it directories, then it lists their content). Rather than telling it to display a list of files and parse the output, use the list of files you already have! This is more reliable (parsing the output of ls will fail if the file names contain whitespace or wildcard characters), clearer and faster. Don't parse the output of ls.
SORT_JOB=`echo "sort -m -n perl_files_dir/*.txt >>sorted_file.txt" | qsub -j oe -W depend=afterany$JOB_ID_ARRAY`
You'd make your life simpler if you used the right form of quoting in the right place. Don't use backquotes, because it's difficult to know how to quote things inside. Use $(…) instead, it's exactly equivalent except that it is parsed in a sane way.
I recommend using a here document for the shell snippet that you're feeding to qsub. You have fewer quoting issues to worry about, and it's more readable.
While we're at it, always put double quotes around variable substitutions and command substitutions: "$some_variable", "$(some_command)". Annoyingly, $var in shell syntax doesn't mean “take the value of the variable var”, it means “take the value of the variable var, parse it as a list of wildcard patterns, and replace each pattern by the list of matching files if there are matching files”. This extra stuff is turned off if the substitution happens inside double quotes (or in a here document, by the way): "$var" means “take the value of the variable var”.
SORT_JOB=$(qsub -j oe -W depend="afterany$JOB_ID_ARRAY" <<'EOF'
sort -m -n perl_files_dir/*.txt >>sorted_file.txt
EOF
)
We now get to the snippet where the quoting was actually causing a problem.
SED=`echo "awk '{\$2="";\$3="";\$4="";\$5="";\$6=""; print \$0}' sorted_file.txt \
| sed 's/ //g' >final_file.txt" | qsub -j oe -W depend=afterany$SORT_ARRAY`
The string that becomes the argument to the echo command is:
awk '{$2=;$3=;$4=;$5=;$6=; print $0}' sorted_file.txt | sed 's/ //g' >final_file.txt
This is syntactically incorrect, and that's why you're not getting any output.
You didn't escape the double quotes inside what was meant to be the awk snippet. It's a lot clearer if you use a here document. Also, you don't need the SED= part. You added it because you had a command substitution (a command between …), which substitutes the output of a command. But since you aren't interested in the output of the qsub command, don't take its output, just execute it.
qsub -j oe -W depend="afterany$SORT_ARRAY" <<'EOF'
awk '{$2="";$3="";$4="";$5="";$6=""; print $0}' sorted_file.txt |
sed 's/ //g' >final_file.txt
EOF
I'm not familiar with qsub, but presumably there's a way to get the error output and the return status of the commands it runs. Inspect that error output, you should have seen the errors from awk.
The version of awk that I am using, does not like the character escapes
awk --version
GNU Awk 3.1.7
spuder#cent64$ awk '{\$2="";\$3="";\$4=""; print \$0}' foo.txt
awk: {\$2="";\$3="";\$4=""; print \$0}
awk: ^ backslash not last character on line
Try the following syntax
awk '{for(i=2;i<=7;i++) $i="";print}' foo.txt
As a side note, if you are using Torque 4.x you may not be able to use a comma separated list of jobs with -W depend=, instead you may need to create a new PBS declarative (-W) for each job.
eg...
#Invalid syntax in newer versions of torque
qsub -W depend=foo,bar
Resources
backslash in gawk fields
Print all but the first three columns
http://docs.adaptivecomputing.com/torque/help.htm#topics/commands/qsub.htm#-W