Ive got the following output from jq which I would like to following to happen:
each URL should be on its own line
remove the double quotes
remove The http and https if they exist
remove any :ports
The output is assigned to a variable OUTPUT and contains the following:
"test.test.io:1337" "https://www.test.io"
I tried the following
echo $ENDPOINTS | tr " " "\n" | sed 's/^http\(\|s\):\/\///g'
Which gives me:
"test.test.io:1337"
"https://www.test.io"
Need to mention this is on OSX
Try:
endpoints='"test.test.io:1337" "https://www.test.io"'
echo "$endpoints" | tr " " "\n" | sed 's/^"//; s/"$//; s~^https\?://~~; s/:[0-9]\+$//'
remove the double quotes
s/^"//; - remove " on beginning of the line
s/"$//; - remove " on ending of the line
remove The http and https if they exist
s~^https\?://~~ - remove http optionally s followed by :// on begining of thelin
remove any :ports
s/:[0-9]\+$// - remove : followed by at least one number on ending of theline
Note: upper case variables are by convention used for environment variable, like PWD UID COLUMNS LINES SHELL LC_CTYPE USER etc. Prefer lowercase variables in custom scripts.
Related
How can I retrieve the "#" and "(" inside of the PowerShell script.
When I'm trying to npm config get proxy the proxy displayed is http://username:ZA%40(testpassword#domain:8080/ but I want to display is http://username:ZA#(testpassword#domain:8080/
TEST: git config --global http.proxy http://username:ZA%40%28testpassword#domain:8080 --replace-all
I am trying to display the "#" by using %40 and for "(" is %28.
I'm thinking if the reserved character "#" is in the side of "("
Please help me out of this problem.
I would like to join strings for file name and shell script so I can run the command line in kdb for ftp transfer.
But there are quotations in quotation marks. not sure how to add / in there.
This is the code I have:
host:"abc.com";
usr:"def";
path:"get /home/eddie/abc.csv /home/terry/";
cmd:" " sv ("/home/kdb/eddie/ftp.sh";host;usr;path);
system cmd;
So the path will not have quotation mark and will be running error. How can I solve this problem?
You can escape quotes with \ e.g. "\"Matt\"" but I don't think that's your issue. It looks like you are attempting to use get in the system command. This is a kdb keyword and your OS will not recognise it. You should just be passing the location of the csv to your ftp script.
Edit:
You may also need sh in the system command.
cat test.sh
echo $1
system "test.sh hello"
sh: ./test.sh: Permission denied
'os
system "sh test.sh hello"
"hello"
Assuming that you simply want quotes within a string, it may just be as simple as using .Q.s1 aka -3!, see https://code.kx.com/q/ref/dotq/#qs1-string-representation
q)" " sv ("/home/kdb/eddie/ftp.sh";host;usr;.Q.s1 path)
"/home/kdb/eddie/ftp.sh abc.com def \"get /home/eddie/abc.csv /home/terry/\""
q)" " sv ("/home/kdb/eddie/ftp.sh";host;usr;-3!path)
"/home/kdb/eddie/ftp.sh abc.com def \"get /home/eddie/abc.csv /home/terry/\""
How to remove line if characters do not exist after a symbol (e.g. #)?
E.g.
hello#lawyer
B#b
smith#
Nac#gyo
treat#
Lines smith# and treat# will be removed as there are no characters after #.
I would post sample of my experimentation -- but have been so far off the mark that would be unhelpful.
using the delete command d and the $ anchor that matches the end of the line:
sed '/#$/d' file
/#$/: when this pattern succeeds, the d command is executed.
You could use grep -v:
grep -v '#$'
to exclude all lines that match the pattern "line ends with #".
I am trying to replace the string #STYLESHEET present in a text file with <xsl:stylesheet using sed.
I am getting error saying
The filename,directory name, or volumelabel syntax is incorrect
Below is my bat script:
ECHO replacing strings in Generated.txt
sed s/#STYLESHEET/<xsl:stylesheet'/g Generated.txt > UpdatedGenerated.txt
PAUSE
type UpdatedGenerated.txt
The text file has the below line:
#STYLESHEET version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" #END
The < character is fine. However, you have an apostrophe at the end of replacement, not sure why. Moreover, you have to pass the sed command as a string: sed '...' or sed "...". The required quote type depends on your console too.
So, this will make your script work well:
sed "s/#STYLESHEET/<xsl:stylesheet/g" Generated.txt > UpdatedGenerated.txt
^^^ ^^^
It worked after putting the options in double quotes. The problem was not with the character <.
sed "s/#STYLESHEET/<xsl:stylesheet/g" Generated.txt > UpdatedGenerated.txt
I am trying to use a sed command to replace specials characters in my file.
The characters are %> to replace by ].
I'am using sed -r s/\%>\/\]\/g but i have this error bash: /]/g: No such file or directory, looks like sed doesn't like it.
Put your sed code inside quotes and also add the file-path you want to work with and finally don't escape the sed delimiters.
$ echo '%>' | sed 's/%>/]/g'
]
ie,
sed 's/%>/]/g' file
To complement Avinash Raj's correct and helpful answer:
Since you were using an overall unquoted string (neither single- nor double-quoted), you were on the right track by \-escaping individual characters in your sed command.
However, you neglected to \-quote >, which is what caused your problem:
> is one of the shell's so-called metacharacters
Metacharacters have special meaning and separate words
Thus, s/\%>\/\]\/g is mistakenly split into 2 arguments by >:
s/\% is passed to sed - as s/%, because the shell removes the \ instances (a process called quote removal).
As you can see, this is not a valid sed command, but that doesn't even come into play - see below.
>\/\]\/g is interpreted by the shell (bash), because it starts with output-redirection operator >; after quote removal, the shell sees >/]/g, tries to open file /]/g for writing, and fails, because your system doesn't have a subdirectory named ] in its root directory.
bash tries to open an output file specified by a redirection before running the command and, if it fails to open the file, does not run the command - which is what happened here:
bash complained about the nonexistent target directory and aborted processing of the command - sed was never even invoked.
Upshot:
In a string that is neither enclosed in single nor in double-quotes, you must \-quote:
all metacharacters: | & ; ( ) < > space tab
additionally, to prevent accidental pathname expansion (globbing): * ? [
Also note that if you need to quote (escape) characters for sed,you need to add an extra layer of quoting; for instance to instruct sed to use a literal . in the regex, you must pass \\. - two backslashes - so that sed sees the properly escaped \..
Given the above, it is much simpler to (habitually) use single quotes around your sed command, because it ensures that the string is passed as is to sed.
Let's compare a working version of your command to the one from Avinash Raj's answer (leaving out the -r for brevity):
sed s/\%\>\/\]\/g # ok - all metachars. \-quoted, others are, but needn't be quoted
sed s/%\>/]/g # ok - minimum \-quoting
sed 's/%>/]/g' # simplest: single-quoted command
I'm not sure whether I got the question correctly. If you want to replace either % or > by ] then sed is not required here. Use tr in this case:
tr '%>' ']' < input.txt
If you want to replace the sequence %> by ] then the sed command as shown by #AvinashRaj is the way to go.