I have this String value which is a Map of user to password
'{"userJohn":"1234","userLinda":"9876"}'
the string has a single quate from each side.
How can I extract the password of say, userLinda
# fishell
~ echo '\'{"userJohn":"1234","userLinda":"9876"}\'' | sed 's/^\'//' | sed 's/\'$//' | jq '.userLinda'
"9876"
# bash zsh
~ echo "'"'{"userJohn":"1234","userLinda":"9876"}'"'" | sed "s/^'//" | sed "s/'$//" | jq '.userLinda'
"9876"
Related
I have a shell script that uses output from a sql query and based on the value of one column sends out an alert. However i don't think it's capturing the value. although the value is not greater than 0 yet it still sends out an email.
Any idea where i am going wrong? Thanks.
............................................................................
#!/bin/sh
psql -d postgres -U postgres -c "select pid,application_name,pg_wal_lsn_diff(pg_current_wal_lsn(), sent_lsn) sending_lag,pg_wal_lsn_diff(sent_lsn,flush_lsn) receiving_lag,pg_wal_lsn_diff(flush_lsn, replay_lsn) replaying_lag,pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn) total_lag from pg_stat_replication;"| while read total_lag;
do
echo $total_lag
lag1=$(echo $total_lag)
done
if [[ $lag1 -ge 0 ]]
then echo "Current replication lag is $lag1" |mail -s "WARNING!" abcd#mail.com
else
echo "No issue"
fi
............................................................................
this is the output of above query
pid | application_name | sending_lag | receiving_lag | replaying_lag | total_lag
-------+------------------+-------------+---------------+---------------+-----------
27823 | db123 | 0 | 0 | 0 | 0
27824 | db023 | 0 | 0 | 0 | 0
I already used grep sed command to do something (cat service.yaml | grep -Po '(service_[^:]*|version:.+)'| sed -z 's/\nversion//g'), but now I need to create a "table" of content of this file in the output like this:
Here's a variant of #jq170727's solution that only uses:
a yaml-to-json translator; and
jq
To illustrate an alternative to yq for (1), I'll use npm's yaml2json.
yaml2json service.yaml | jq -r '
keys_unsorted as $keys
| (["service", "version", "replica"] | map("\"\(.)\"")), # quote the headers
($keys[] as $k | .[$k] | [$k, .version, .replica])
| #tsv'
Notice also that keys_unsorted is used, mainly to respect the order of the keys in the original YAML file.
Here is a filter which may help
keys[] as $k | .[$k] | [$k, .version, .replica]
Here is an example using it. For reference this is the yq I have from brew
$ yq --help | grep github
See https://github.com/kislyuk/yq for more information.
and/or https://stedolan.github.io/jq
First convert yaml to json
$ yq . service.yaml
{
"service-1": {
"replica": 2,
"version": "0.6.24"
},
"service-2": {
"replica": 3,
"version": "0.21.14"
}
}
Examine top level object
$ yq . service.yaml \
| jq -r 'keys'
[
"service-1",
"service-2"
]
Flatten into a stream of arrays
$ yq . service.yaml \
| jq -c 'keys[] as $k | .[$k] | [$k, .version, .replica]'
["service-1","0.6.24",2]
["service-2","0.21.14",3]
Convert to csv and format as a table
$ yq . service.yaml \
| jq -cr 'keys[] as $k | .[$k] | [$k, .version, .replica] | #csv' \
| column -s, -t
"service-1" "0.6.24" 2
"service-2" "0.21.14" 3
Add titles
$ yq . service.yaml \
| jq -cr 'keys[] as $k | .[$k] | [$k, .version, .replica] | #csv' \
| (echo '"service","version","replica"'; cat -) \
| column -s, -t
"service" "version" "replica"
"service-1" "0.6.24" 2
"service-2" "0.21.14" 3
If you want no quotes around the strings in the final rows this may be better
$ yq . service.yaml \
| jq -cr 'keys[] as $k | .[$k] | "\($k),\(.version),\(.replica)"' \
| (echo '"service","version","replica"'; cat -) \
| column -s, -t
"service" "version" "replica"
service-1 0.6.24 2
service-2 0.21.14 3
I have following content in my csv file(with 3 columns):
141413,"\"'/x=/></script></title><x><x/","Mountain View, CA\"'/x=/></script></title><x><x/"
148443,"CLICK LINK BELOW TO ENTER^^^^^^^^^^^^^^","model\
\
xxx lipsum as it is\
\
100 sometimes unknown\
\
travel evening market\
"
When I import above mentioned csv in mysql using following command, it treats the backslash() as new line; which is the expected behavior.
LOAD DATA INFILE '1.csv' INTO TABLE users FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\n';
MYSQL Output
But when I try to import to psql using copy command, it treats \ as a normal character.
copy users from '1.csv' WITH (FORMAT csv, DELIMITER ',', ENCODING 'utf8', NULL "\N", QUOTE E'\"', ESCAPE '\');
postgres Output
Try parsing these \ before importing the CSV file, e.g. using perl -pe or sed and the STDIN from psql:
$ cat 1.csv | perl -pe 's/\\\n/\n/g' | psql testdb -c "COPY users FROM STDIN WITH (FORMAT csv, DELIMITER ',', ENCODING 'utf8', NULL "\N", QUOTE E'\"', ESCAPE '\');"
This is how it looks like after the import:
testdb=# select * from users;
id | company | location
--------+-----------------------------------------+-------------------------------------------------
141413 | "'/x=/></script></title><x><x/ | Mountain View, CA"'/x=/></script></title><x><x/
148443 | CLICK LINK BELOW TO ENTER^^^^^^^^^^^^^^ | model +
| | +
| | xxx lipsum as it is +
| | +
| | 100 sometimes unknown +
| | +
| | travel evening market +
| |
(2 Zeilen)
I'd like to print the value of members and id from below jq output:
$ cat test_|jq -r '.[] | select(.name=="AAA") | .'
{
"name": "AAA",
"members": 10,
"profiles": 0,
"templates": 0,
"ldapGroups": 0,
"ldapMembers": 0,
"id": "20"
}
Unfortunately it works for one of each only:
$ cat test_|jq -r '.[] | select(.name=="AAA") | .members'
10
with +" "+ I get error:
$ cat test_|jq -r '.[] | select(.name=="AAA") | .members+" "+.id'
jq: error: number and string cannot be added
I recommend to use string interpolation. It will automatically cast input to a string if necessary:
jq -r '.[]|select(.name=="AAA")|"\(.members) \(.id)"' file.json
Convert number to string with tostring function:
jq -r '.[] | select(.name=="AAA") | (.members|tostring) +" "+ .id test'
You have not specified precisely how you want the two values to appear, so it is worth pointing out that you could write:
.... | (.members, .id)
or
.... | [.members, .id]
or
.... | [.members, .id] | E
where E could be #csv or #tsv or etc. In jq 1.5, join/1 will also do the type conversion.
You could start by just projecting the members of interest. e.g
$ jq -Mc '.[] | select(.name=="AAA") | {members,id}' data.json
{"members":10,"id":"20"}
Now you can see .members is a number and .id is a string. You can't add them directly with + but you can choose any of the options explained in the other answers.
i wish to write a fish shell script to automatically initialize JAVA_HOME to current configured java-alternative.
In bash it would look like this (sorry for the ugly double dirname)
j=`update-alternatives --query javac | grep Value:`
JAVA_HOME=`dirname ${j#Value:}`
JAVA_HOME=`dirname $JAVA_HOME`
export JAVA_HOME
what about fish?
set j (update-alternatives --query javac | grep Value:)
set JAVA_HOME (dirname ${j#Value:}) <-- this won't work!!
set JAVA_HOME (dirname $JAVA_HOME)
set --export JAVA_HOME
The fish shell now has a string builtin command for string manipulation. This was added in version 2.3.0 (May 2016).
E.g. in this case, we could use string replace to remove the Value: substring:
set j (update-alternatives --query javac | grep Value: | string replace 'Value: ' '')
set --export JAVA_HOME (dirname (dirname $j))
There's lots more that string can do. From the string command documentation:
Synopsis
string length [(-q | --quiet)] [STRING...]
string sub [(-s | --start) START] [(-l | --length) LENGTH] [(-q | --quiet)]
[STRING...]
string split [(-m | --max) MAX] [(-r | --right)] [(-q | --quiet)] SEP
[STRING...]
string join [(-q | --quiet)] SEP [STRING...]
string trim [(-l | --left)] [(-r | --right)] [(-c | --chars CHARS)]
[(-q | --quiet)] [STRING...]
string escape [(-n | --no-quoted)] [STRING...]
string match [(-a | --all)] [(-i | --ignore-case)] [(-r | --regex)]
[(-n | --index)] [(-q | --quiet)] [(-v | --invert)] PATTERN [STRING...]
string replace [(-a | --all)] [(-i | --ignore-case)] [(-r | --regex)]
[(-q | --quiet)] PATTERN REPLACEMENT [STRING...]
Bash:
j=$(update-alternatives --query javac | sed -n '/Value: /s///p')
export JAVA_HOME=${j%/*/*}
Fish:
set j (update-alternatives --query javac | sed -n '/Value: /s///p')
set --export JAVA_HOME (dirname (dirname $j))
or
set --export JAVA_HOME (dirname (dirname (update-alternatives --query javac | sed -n '/Value: /s///p')))
Instead of sed, u could make use of expr with a regexp, for example:
$ set a /path/to/some/folder/file.extension
the command:
$ expr "//$a" : '.*/\([^.]*\)\..*$'
file
extract the file basename without extension.
See man expr
Fish shell:
~> set JAVA_HOME (readlink -f /usr/bin/javac | sed "s:/bin/javac::")
~> echo $JAVA_HOME
Output (example):
/usr/lib/jvm/java-8-openjdk-amd64
Also u can add to ~/.config/fish/config.fish this line:
set JAVA_HOME (readlink -f /usr/bin/javac | sed "s:/bin/javac::")
WBR