I'm trying to use jq's --arg syntax to pass a variable from the shell to a jq filter.
If I run the following command, the jq query works as expected:
TS_SEC_START=1534574204 \
kubectl -n istio-system logs deployment/flagger --tail=50 | grep '^{' | \
jq --arg TS_SEC_START 1534574204 --arg SVC "${SERVICE_NAME}.${RELEASE_NAMESPACE}" \
-s 'map(select(.ts | (split(".")[0] + "Z") | fromdateiso8601 > ($TS_SEC_START | tonumber)) | select(.canary == "mysvc.prod"))'
-->
Found 3 pods, using pod/flagger-6dc6fd7d85-z294g
[
{
"level": "error",
"ts": "2021-08-16T02:41:34.128Z",
"caller": "controller/scheduler.go:163",
"msg": "Canary mysvc.prod not found",
"canary": "mysvc.prod",
"stacktrace": "github.com/fluxcd/flagger/pkg/controller.(*Controller).advanceCanary\n\t/workspace/pkg/controller/scheduler.go:163\ngithub.com/fluxcd/flagger/pkg/controller.CanaryJob.Start.func1\n\t/workspace/pkg/controller/job.go:39"
}
]
However, when I try to pass mysvc.prod as an arg, I receive 0 responses.
TS_SEC_START=1534574204 SERVICE_NAME=mysvc RELEASE_NAMESPACE=prod \
kubectl -n istio-system logs deployment/flagger --tail=50 | grep '^{' | \
jq --arg TS_SEC_START 1534574204 --arg SVC "${SERVICE_NAME}.${RELEASE_NAMESPACE}" \
-s 'map(select(.ts | (split(".")[0] + "Z") | fromdateiso8601 > ($TS_SEC_START | tonumber)) | select(.canary == $SVC))'
I've tried a few different options here, e.g. .canary == "$SVC" and `.canary == "$SERVICE_NAME.$RELEASE_NAMESPACE" but I can't get it to work.
What am I doing wrong?
-->
Found 3 pods, using pod/flagger-6dc6fd7d85-z294g
[]
In general, you can't do this:
MYVAR=12345 printf "%s\n" "$MYVAR"
When you prefix a shell command with MYVAR=12345 you are modifying the environment that is passed to that process, in this case printf. However, the $MYVAR expression is expanded in the shell and no MYVAR variable ever exists in the shell.
Furthermore, when you set variables like this, they are only passed to that command, not the entire pipeline.
So in this example:
TS_SEC_START=1534574204 \
SERVICE_NAME=mysvc \
RELEASE_NAMESPACE=prod \
kubectl -n istio-system logs deployment/flagger --tail=50 | \
grep '^{' | \
jq \
--arg TS_SEC_START 1534574204 \
--arg SVC "${SERVICE_NAME}.${RELEASE_NAMESPACE}" \
-s \
'
map(
select(
.ts |
(split(".")[0] + "Z") |
fromdateiso8601 > ($TS_SEC_START | tonumber)
) |
select(.canary == $SVC)
)
'
The three environment variables are visible to kubectl and nothing else. They are not visible to grep or jq (although neither of them are looking for such environment variables in the first place) and they are not visible to the shell itself as it expands those commands.
Related
I registered the script to crontab to run every 10 minutes. When I run the script, the log is printed, but if I put it on the cron tab and wait, the log does not appear, so it seems that it does not work. If you have anything I need to add or edit, please let me know :(
my sh script
#!/bin/sh
pslist=`ps -ef | grep ffprobe | awk '{print $2}'`
pscount=`ps -ef | grep ffprobe | wc -l`
logs='/apps/kMobile/kEncoderPy/batch/kill_ffprobe.log'
timestamp=`date +%Y%m%d--%H:%M`
echo "[$timestamp] Kill the Batch process Start :: "$timestamp >> $logs
echo "[$timestamp] process ffprobe running count :: "$pscount >> $logs
for pid in $pslist
do
rtime=`ps -p $pid -o etime | tail -1`
rtime=$rtime | tr -d ' '
minutes=${rtime:6:2}
if [$rtime != "ELAPSED" ]; then
if [ $minutes -gt 10 ]; then
`kill -9 $pid`
echo "[$timestamp] passed 10 minute kill process id : "$pid >> $logs
echo "[$timestamp] process kill after ffprobe running count : "$psount >> $logs
fi
fi
done
my crontab -e
*/10 * * * * /apps/kMobile/batch/kill_ffprobe.sh
Bash works as expected
$ echo '{"foo": "0"}' | jq 'select(.foo == "0")'
{
"foo": "0"
}
But in Powershell,
PS> echo '{"foo": "0"}' | jq 'select(.foo == "0")'
PS>
Why powershell results this?
You have to backslash the doublequotes unfortunately.
'{"foo": "0"}
{"foo": "1"}' | jq -c 'select(.foo == \"0\")'
{"foo":"0"}
My code
function foo
argparse -n foo 'u/user=' 'g/group=' 's/shell=' -- $argv
bar $_flag_u $_flag_g $_flag_s
end
function bar -a user group shell
echo $user
echo $group
echo $shell
end
foo $argv
Execute and Result
> fish test.fish -u user_name -g group_name -s shell_name
user_name
group_name
shell_name
> fish test.fish -g group_name -s shell_name
group_name
shell_name
> fish test.fish -u user_name -s shell_name
user_name
shell_name
>
Expectation I want
> fish test.fish -u user_name -g group_name -s shell_name
user_name
group_name
shell_name
> fish test.fish -g group_name -s shell_name
group_name
shell_name
> fish test.fish -u user_name -s shell_name
user_name
shell_name
>
Can I get the expectation?
If -u option is not used, my expectation is that $user in bar function is empty.
(it means that result of "echo $status" is 1 after "set -q user".)
If I have no choice, I'll do bellow:
(Reference: https://github.com/fish-shell/fish-shell/issues/3926)
function foo
argparse -n foo 'u/user=' 'g/group=' 's/shell=' -- $argv
set -q _flag_u; or set _flag_u ''
set -q _flag_g; or set _flag_g ''
set -q _flag_s; or set _flag_s ''
bar $_flag_u $_flag_g $_flag_s
end
function bar -a user group shell
if test -n $user
# do something
end
if test -n $group
# do something
end
if test -n $shell
# do something
end
end
foo $argv
Your issue is this:
function bar -a user group shell
This means bar calls its first argument "user", the second "group" and the third "shell".
When you then run
bar $u $g $s
fish expands the variables. If they are set, they'll expand to as many arguments as they have elements (in fish, all variables are lists!).
So you'll get $u into $user, $g into $group and $s into $shell.
However, if one isn't set (because the option wasn't given in the call to foo), then the variable will expand to nothing, so if e.g. --group wasn't given, this works like
bar $u $s
which will assign the contents of $u to $user and $s to $group - because it is the second argument that bar sees.
To inhibit this, either use set -q or double-quote the variable:
bar "$_flag_u" "$_flag_g" "$_flag_s"
Want to find all mkv files without having same-name ass/srt file in the same folder.
How can I do that?
for example, I have following directory:
folder_1
|----folder_2
| |-----a.mkv
| |-----a.srt
|----folder_3
| |-----b.mkv
|----folder_4
|-----c.mkv
|-----c.ass
The search result should be: folder_1/folder_3/b.mkv.
Many Thanks.
Get answer from my friends, share it:
find . -name "*.mkv" -o -name "*.ass" -o -name "*.srt"| sort |rev|uniq -s 3 -u| rev|rgrep ".mkv"
BTW, if you are using synology nas, which does not have 'rev' command, you can walkaround it by using a python script(rev.py):
import sys
if __name__ == '__main__':
if len(sys.argv) >= 2:
for arg in sys.argv[1:]:
print '"' + arg[::-1] + '"'
and the script will be changed to:
find . -name "*.mkv" -o -name "*.ass" -o -name "*.srt"| sort |awk '{print "\"", $0,"\""}' OFS=""|xargs python rev.py |uniq -s 3 -u| xargs python rev.py | grep ".mkv\""
I have a few filenames in a directory
blabla.01
blabla.02
...
I'm trying to make a new file with the following format:
01 new stuff here
02 more new stuff
...
I wrote a script and dumbed it down a bit:
#!/bin/bash
FILES=$(find . -type f -name "blabla*" | awk -F'[.]' '$(NF-1)>=1' | sort)
for f in $FILES
do
echo -n $f | cut -d "." -f 3
echo "test"
done
the 'test' will be the output of another code..
However in this example i get something like:
01
test02
test
Thanks
Try
printf '%s%s\n' "$(echo "$f" | cut -d . -f3)" "test"
The echo | cut could probably be replaced with something like "${f##*.}" if you always want the laat field.