using sed with ? (question mark) special character - sed

I have an infected website, and I am trying to clean it out using sed. Unfortunately I am unable to escape the question mark sign in the URL and I am really stuck here. I've searched over the web for a possible solution, but unfortunately I didn't found a proper way to do so.
Just an explanation:
The injected code is similar to this one:
< iframe src=http://test.com/index.html?i=23123>< /iframe>
Note that I am not a pro, and there is why I need your help!
so my way to clear the code is :
sed -i '/< iframe src=http:\/\/test.com\/index.html\?i=23123>/,/< \/iframe>/d' index.html
Unfortunately that didn't help as well as all others.
All help will be gratefully appreciated.

echo "< iframe src=http://test.com/index.html?i=23123>< /iframe>" \
| sed 's#< iframe src=http://test.com/index.html?i=23123>< /iframe>##'
Produces no output, which to me means this is successfully deleting your problem string.
Note that most seds will accept an alternate regex-replacement character, here I am using # because there are no #s in the search target. On some seds, you have to tell it 'hey I'm using an alternate, and escape the char, like s\#.....##.
I don't see why your attempt to quote the ? is failing. Did you try [?] and (worst case) [\?]. Are there 2nd level evaluations happening by the shell that you're not mentioning here? Does my simple example also fail?
As others will certainly tell you, your approach is strictly a bandaid, you need to figure out what the security hole is in your system and fix it. Your pages will get corrupted again. :-(
IHTH

Related

Using SED to remove a domain name starting with http from result

I am very new to shell scripting, the command line, sed, awk, etc so bear with me.
I have a script that outputs -
Reseller: iwantmyname http://iwantmyname.com
I want it to read -
Reseller: iwantmyname
Dropping anything starting with http
I figured SED would be a good tool but I only have a basic knowledge of it, and the tutorials I've found online seem advanced and difficult for me.
I know the basic is sed 's/find_this/replace_with_this/' and I figured I'd replaced the found http with // or nothing. But I don't know how to search for something that starts with http and include EVERYTHING after it. I've looked up regex but that seems quite difficult as well.
Replace a white space followed by http and rest of row with nothing:
's/ http.*//'

Less ugly way to use sed to simply include a new line?

There are a lot of guides, handbooks, fast-guides, question/answers about it: no one are simple and objective...
It is a classical problem, near all text editors crashes with big files XML or HTML "all in one line", so we need to decide what tag will recive the \n and replace all occurences of <tag by \n<tag ... so simple. Why it is not simple to do by terminal?
The best question/answer about this case not solves: Bash: How can I replace a string by new line in osx bash? Example using that solution: sed 's/<article/\'$'\n\n<article/g' file.htm not works, need some more exotical syntax, so it is not simple as I solicitated in this question.
So, this quetion is not about "any solution", but about "some simple/elegant solution".
If I understand what you are looking for you could try something like the following:
sed 's/<tag>/\n<tag>/g' file.htm
which is very close to the anwser you linked.
It already looks quite simple to me, it replaces the tag with a new line character and writes the tag again.
However I don't get the need for this '$' in your case.

sed seems to match pattern properly only when newline inserted

I am currently running the following sed command:
sed 's/P(\(.*\))\\mid(\(.*\))/\\condprob{\1}{\2}/g' myfile.tex
Essentially, I have inherited an oddly formatted tex file, and want to replace everything like this:
P(<foo>)\mid(<bar>)
With this
\condprob{<foo>}{<bar>}
The file I am trying to run sed on contains the following line:
P(\vec{m}_i)\mid(t,h,\alpha) = \prod_{u\in\mathcal{U}} P(\vec{m}_{iu})\mid(t,h,\alpha)
Which I would like to change to this:
\condprob{\vec{m}_i}{t,h,\alpha} = \prod_{u\in\mathcal{U}}\condprob{\vec{m}_{iu}}{t,h,\alpha}
However, sed keeps missing the first \mid and instead gives me this:
\condprob{\vec{m}_i)\mid(t,h,\alpha) = \prod_{u\in\mathcal{U}} P(\vec{m}_{iu}}{t,h,\alpha}
If I add a line break at the = sign it matches everything fine
Can someone please a) help me resolve this, and b) perhaps tell me why it is happening?
Thanks.
Edit: thanks choroba and Sloopjon, you've both answered my why, and Sloopjon's solution is actually exactly what I was needing. choroba: I guess I will have to wait another day to learn perl.
For those that are interested Sloopjon's solution when translated into my problem looks like this (match everything that isn't a closing parenthesis):
sed 's/P(\([^)]*\))\\mid(\([^)]\))/\\condprob{\1}{\2}/g' myfile.tex
It looks like you expect P(\(.*\)) to match only P(\vec{m}_i), but the * quantifier is greedy, so it actually matches P(\vec{m}_i)\mid...P(\vec{m}_{iu}). There are two common fixes for this: use a non-greedy quantifier if your tool supports it, or change the pattern so that it only matches what you expect. For example, if you know that parentheses won't nest in this P() construct, change .* to [^)]*.
Edit: I also suggest that you look for a regex visualizer or debugger when you have a problem like this. For example, pasting your example into debuggex.com makes it clear what's happening.
The problem is the greediness of the * quantifier. It matches as many times as it can, i.e. it doesn't stop at the first ).
You can try Perl, that features "non-greedy" (frugal, lazy) *?:
perl -pe 's/P\((.*?)\)\\mid\((.*?)\)/\\condprob{$1}{$2}/g'

Execute Commands in the Linux Commandline [Lazarus / Free Pascal]

I have a problem. I want to execute some commands in the Commandline of linux. I tested TProcess (So i am using Lazarus) but now when i am starting the programm, there is nothing, wich the Program do.
Here is my Code:
uses [...], unix, process;
[...]
var LE_Path: TLabeledEdit;
[...]
Pro1:=TProcess.Create(nil);
Pro1.CommandLine:=(('sudo open'+LE_Path.Text));
Pro1.Options := Pro1.Options; //Here i used Options before
Pro1.Execute;
With this Program, i want to open Files with sudo (The Programm is running on the User Interface)
->Sorry for my Bad English; Sorry for fails in the Question: I am using StackOverflow the first time.
I guess the solution was a missing space char?
Change
Pro1.CommandLine:=(('sudo open'+LE_Path.Text));
to
Pro1.CommandLine:=(('sudo open '+LE_Path.Text));
# ----------------------------^--- added this space char.
But if you're a beginner programmer, my other comments are still worth considering:
trying to use sudo in your first bit of code may be adding a whole extra set of problems. SO... Get something easier to work first, maybe
/bin/ls -l /path/to/some/dir/that/has/only/a/few/files.
find out how to print a statement that will be executed. This is the most basic form of debugging and any language should support that.
Your english communicated your problem well enough, and by including sample code and reasonable (not perfect) problem description "we" were able to help you. In general, a good question contains the fewest number of steps to re-create the problem. OR, if you're trying to manipulate data,
a. small sample input,
b. sample output from that same input
c. your "best" code you have tried
d. your current output
e. your thoughts about why it is not working
AND comments to indicate generally other things you have tried.

General help for deciphering/explaining sed one-liners?

I've just stumbled upon some cryptic sed expression in a legacy script. Could you give me some hints how to start decoding it?
Best thing would be some automatic tool translating sed incantations to English, but for a close runner up, I'd be very grateful for some nice index of (all) sed commands. Otherwise, I'm certainly highly interested in any help at all on how to quickly attack the problem (other than having to read the manual cover to cover...).
(Side note: as you may have guessed, I don't want to just paste the expression here, as I'd like to be able to do it easier and faster next time I stumble on some similar line noise...)
I'd be very grateful for help!
Edit: regexps themselves aren't problem, by the way, I'm good enough at them.
i don't think there is automatic tool that can 'transalte' sed commands to english. however you may want to check http://aurelio.net/sedsed/ . it will help you to understand one sed script, what it does, and how.
anyway, if you list some examples would be good.
This might work for you.
Unix in a Nutshell by Robbins has a very nice chapter on sed. Clear and concise descriptions of the commands.
Your best bet would be to learn the sed language in-depth. Unforunately, the sed documentation is more like a reference. Here's a nice step by step guide that doesn't take too long to read.
I found "Sed One-Liners Explained" to be very informative as well as fun.