I want to edit content of pg_hba.conf using sed :
# "local" is for Unix domain socket connections only
local all all peer
to
# "local" is for Unix domain socket connections only
local all all trust
For know, I use this command that works:
sed -i 's/local all all peer/local all all trust/' pg_hba.conf
But I'm looking for a way to bypass all theses spaces
You can use
sed -E -i 's/local([[:space:]]+)all([[:space:]]+)all([[:space:]]+)peer/local\1all\2all\3trust/' test.conf
Or, since it seems like you have a GNU sed:
sed -E -i 's/local(\s+)all(\s+)all(\s+)peer/local\1all\2all\3trust/' test.conf
And certainly you can do as potong did in the comments and reduce this to
sed -E -i 's/(local\s+all\s+all\s+)peer/\1trust/' test.conf
Note:
-E enables the POSIX ERE syntax (no need to escape + and (...))
([[:space:]]+) / (\s+) defines capturing groups with IDs starting with 1 that match one or more whitespaces
\1, \2 and \3 are placeholders, backreferences to the appropriate group values.
In s/(local\s+all\s+all\s+)peer/\1trust/, you capture the whole part before peer and match peer, then the whole match is replaced with the part before peer (with \1) + trust.
You could use this sed if the string that needs to be changed is not always going to be peer
$ sed -i 's/\S*$/trust/' input_file
# local is for Unix domain socket connections trust
local all all trust
Related
I am trying to create a filter command to reduce the lines from a log file, assume each line contains partition made of date,
/iamthepath01/20200301/file01.txt
/iamthepath02/20200302/file02.txt
....
/iamthepathxx/20210619/filexx.txt
then from thousands of lines I only want to keep the ones with two string in the path
/202106
/202105
and remove any other lines
I have tried following command
sed -i -e '\(/202105\|/202106\)!d' ~/log.txt
above command threw
sed: -e expression #1, char 24: unterminated address regex
You can use
sed -i '/\/20210[56]/!d' ~/log.txt
Or, if you need to use more specific alternatives and further enhance the pattern:
sed -i -E '/\/(202105|202106)/!d' ~/log.txt
Details:
-i - GNU sed option for inline file replacement
-E - option enabling POSIX ERE regex syntax
/\/20210[56]/ - regex that matches /20210 and then either 5 or 6
\/(202105|202106) - the POSIX ERE pattern that matches / and then either 202105 or 202106
!d - removes the lines not matching the pattern.
See the online demo:
#!/bin/bash
s='/iamthepath01/20200301/file01.txt
/iamthepath02/20200302/file02.txt
/iamthepathxx/20210619/filexx.txt'
sed '/\/20210[56]/!d' <<< "$s"
Output:
/iamthepathxx/20210619/filexx.txt
sed is the wrong tool for this. If you want a script that's as fragile as the sed one then use grep as it's the tool that exists solely to do a simple g/re/p (hence the name) like you're doing:
$ grep '/20210[56]' file
/iamthepathxx/20210619/filexx.txt
or if you want a more robust solution that focuses just on the part of the line you want to match and so will avoid false matches, then use awk:
$ awk -F '/' '$3 ~ /^20210[56]/' file
/iamthepathxx/20210619/filexx.txt
This might work for you (GNU sed):
sed -ni '\#/20210[56]#p' file
This uses seds -n grep-like option to turn off implicit printing and -i option to edit the file in place.
Normally sed uses the /.../ to match but other delimiters may be used if the first is escaped e.g. \#...#.
So the above solution will filter the existing file down to lines that contain either /202105 or /202106.
N.B. grep will almost certainly be faster in finding the above lines however the use of the -i option may be the ultimate reason for choosing sed (although the same outcome can be achieved by tacking on the > tmpFile && mv tmpFile file to a grep solution).
I have a bash script that needs to modify .ssh/config. My goal is to change the HostName value of server1 using sed and I have already managed to do it, but in the file there are more HostName and sed modifies them all. I have tried specifying to sed to stop at the first occurrence but continues to modify them all.
This is the file where I need to change the HostName of server1
Host server1
HostName 172.160.189.196
User admin
Port 353
Host server2
HostName 254.216.34.18
User user
Port 22
This is the command I give:
sed -i '0,/RE/s/HostName .*/HostName 14.208.54.132/' .ssh/config
Try using a sed range:
sed -i '/Host server1/,/HostName/ s/HostName .*/HostName 14.208.54.132/' .ssh/config
This will replace HostName in the range of lines between Host server1 and the first occurrence of HostName, which I think is what you want.
While awk is generally known to work on lines, it actually works with records, by default lines. A record can be defined by a record separator RS. If this variable is empty, it assumes that a record is given by text-blocks separated by one or more empty lines. With this you can do the following:
awk 'BEGIN{RS="";FS=OFS="\n";ORS="\n\n"}
($1~/server1/) {sub(/Hostname[^\n]*\n/,"Hostname 14.208.54.132" OFS)}
1' file
This is not short, but conceptually clean.
Obviously, you have to update the regex to match the hostname such that it is unique. If you also have a hostname server1a, then you will have to make sure that ($1~/server1/) does not match that.
You can use awk like this:
awk '$2=="server1" {f=1} f && /HostName/ {$0=" HostName 14.208.54.132";f=0} 1' file
Host server1
HostName 14.208.54.132
User admin
Port 353
Host server2
HostName 254.216.34.18
User user
Port 22
This might work for you (GNU sed):
sed '/\<server1\>/{:a;n;/HostName/!ba;s/\S\+/14.208.54.132/2}' file
Focus on a line containing server1 then read additional lines until one containing HostName and substitute the second field for the desired result.
I'm trying to replace a.mysql.com in a file using sed for the following line
'ENGINE': 'a.mysql.com', # MySQL host
How can I replace the text without removing the comment entry?
I tried the below but it isn't working
sed -i -e "s/\('ENGINE': ').*\([^']*\)/\1new.mysql.com\2/g" file.py
Following sed may help you in same.
sed "s/\(.*: \)\('.*'\)\(.*\)/\1'my_next_text'\3/" Input_file
Output will be as follows.
'ENGINE': 'my_next_text', # MySQL host
EDIT: Tested it with edited Input_file of OP too as follows.
cat Input_file
'ENGINE': 'a.mysql.com', # MySQL host
sed "s/\(.*: \)\('.*'\)\(.*\)/\1'my_next_text'\3/" Input_file
'ENGINE': 'my_next_text', # MySQL host
EDIT2: IN case OP wants to check for line which has string ENGINE in it then following could be done.
sed "/ 'ENGINE/s/\(.*: \)\('.*'\)\(.*\)/\1'my_next_text'\3/" Input_file
You can use the following logic using GNU sed to achieve your requirement.
sed "/ENGINE/s/'[^']*'/'new.mysql.com'/2" file
The s/ENGINE/ matches any lines containing ENGINE and does the following substitution s/'[^']*'/'new.mysql.com'/2 which:
s/ # Substitute
' # Match a single quote
[^']* # Match anything not a single quote
' # Match the closing single quote
/ # Replace with
'new.mysql.com' # The literal value
/2 # The 2 here matches the second quoted string, not the first.
Add the -i extension once the file is modified appropriately.
Currently I am working on a project where I have a file full of IP addresses I want to delete the IP adddresses that are private (whole range) and using bash to accomplish it. I do make a match of private addresses but unable to delete it.
I have tried:
sed -i '/(192)\.(168)(\.([2][0-5][0-5]|[1][0-9][0-9]|[1-9][0-9]|[0-9])){2}/d' validIPOnly
same for
127.x.x.x,172.16.x.X,10.x.x.x
When using capture groups with sed you either need to escape the ( ) or specify option -E:
$ sed -E -i '/(...)...'
or
$ sed -i '/\(...\)...`
For GNU sed only use the -r option instead of -E.
I am trying to write a sed expression that can remove urls from a file
example
http://samgovephotography.blogspot.com/ updated my blog just a little bit ago. Take a chance to check out my latest work. Hope all is well:)
Meet Former Child Star & Author Melissa Gilbert 6/15/09 at LA's B&N https://hollywoodmomblog.com/?p=2442 Thx to HMB Contributor #kdpartak :)
But I dont get it:
sed 's/[\w \W \s]*http[s]*:\/\/\([\w \W]\)\+[\w \W \s]*/ /g' posFile
FIXED!!!!!
handles almost all cases, even malformed URLs
sed 's/[\w \W \s]*http[s]*[a-zA-Z0-9 : \. \/ ; % " \W]*/ /g' positiveTweets | grep "http" | more
The following removes http:// or https:// and everything up until the next space:
sed -e 's!http\(s\)\{0,1\}://[^[:space:]]*!!g' posFile
updated my blog just a little bit ago. Take a chance to check out my latest work. Hope all is well:)
Meet Former Child Star & Author Melissa Gilbert 6/15/09 at LA's B&N Thx to HMB Contributor #kdpartak :)
Edit:
I should have used:
sed -e 's!http[s]\?://\S*!!g' posFile
"[s]\?" is a far more readable way of writing "an optional s" compared to "\(s\)\{0,1\}"
"\S*" a more readable version of "any non-space characters" than "[^[:space:]]*"
I must have been using the sed that came installed with my Mac at the time I wrote this answer (brew install gnu-sed FTW).
There are better URL regular expressions out there (those that take into account schemes other than HTTP(S), for instance), but this will work for you, given the examples you give. Why complicate things?
The accepted answer provides the approach that I used to remove URLs, etc. from my files. However it left "blank" lines. Here is a solution.
sed -i -e 's/http[s]\?:\/\/\S*//g ; s/www\.\S*//g ; s/ftp:\S*//g' input_file
perl -i -pe 's/^'`echo "\012"`'${2,}//g' input_file
The GNU sed flags, expressions used are:
-i Edit in-place
-e [-e script] --expression=script : basically, add the commands in script
(expression) to the set of commands to be run while processing the input
^ Match start of line
$ Match end of line
? Match one or more of preceding regular expression
{2,} Match 2 or more of preceding regular expression
\S* Any non-space character; alternative to: [^[:space:]]*
However,
sed -i -e 's/http[s]\?:\/\/\S*//g ; s/www\.\S*//g ; s/ftp:\S*//g'
leaves nonprinting character(s), presumably \n (newlines). Standard sed-based approaches to remove "blank" lines, tabs and spaces, e.g.
sed -i 's/^[ \t]*//; s/[ \t]*$//'
do not work, here: if you do not use a "branch label" to process newlines, you cannot replace them using sed (which reads input one line at a time).
The solution is to use the following perl expression:
perl -i -pe 's/^'`echo "\012"`'${2,}//g'
which uses a shell substitution,
'`echo "\012"`'
to replace an octal value
\012
(i.e., a newline, \n), that occurs 2 or more times,
{2,}
(otherwise we would unwrap all lines), with something else; here:
//
i.e., nothing.
[The second reference below provides a wonderful table of these values!]
The perl flags used are:
-p Places a printing loop around your command,
so that it acts on each line of standard input
-i Edit in-place
-e Allows you to provide the program as an argument,
rather than in a file
References:
perl flags: Perl flags -pe, -pi, -p, -w, -d, -i, -t?
ASCII control codes: https://www.cyberciti.biz/faq/unix-linux-sed-ascii-control-codes-nonprintable/
remove URLs: sed to remove URLs from a file
branch labels: How can I replace a newline (\n) using sed?
GNU sed manual: https://www.gnu.org/software/sed/manual/sed.html
quick regex guide: https://www.gnu.org/software/sed/manual/html_node/Regular-Expressions.html
Example:
$ cat url_test_input.txt
Some text ...
https://stackoverflow.com/questions/4283344/sed-to-remove-urls-from-a-file
https://www.google.ca/search?dcr=0&ei=QCsyWtbYF43YjwPpzKyQAQ&q=python+remove++citations&oq=python+remove++citations&gs_l=psy-ab.3...1806.1806.0.2004.1.1.0.0.0.0.61.61.1.1.0....0...1c.1.64.psy-ab..0.0.0....0.-cxpNc6youY
http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html
https://bbengfort.github.io/tutorials/2016/05/19/text-classification-nltk-sckit-learn.html
http://datasynce.org/2017/05/sentiment-analysis-on-python-through-textblob/
https://www.google.ca/?q=halifax&gws_rd=cr&dcr=0&ei=j7UyWuGKM47SjwOq-ojgCw
http://www.google.ca/?q=halifax&gws_rd=cr&dcr=0&ei=j7UyWuGKM47SjwOq-ojgCw
www.google.ca/?q=halifax&gws_rd=cr&dcr=0&ei=j7UyWuGKM47SjwOq-ojgCw
ftp://ftp.ncbi.nlm.nih.gov/
ftp://ftp.ncbi.nlm.nih.gov/1000genomes/ftp/alignment_indices/20100804.alignment.index
Some more text.
$ sed -e 's/http[s]\?:\/\/\S*//g ; s/www\.\S*//g ; s/ftp:\S*//g' url_test_input.txt > a
$ cat a
Some text ...
Some more text.
$ perl -i -pe 's/^'`echo "\012"`'${2,}//g' a
Some text ...
Some more text.
$