mitmdump replacement pattern construction - mitmproxy

I'm trying to specify replacements in mitmdump but am having trouble getting the syntax right. What I want to do is replace the entire path in a request with a fixed string.
I've tried -R :~bq:/*:/example.html but that results in "Invalid filter pattern: ~bq"
Any pointers?

Try to use ~q not ~bq as filter pattern. Because ~bq needs regex itself like ~bq regex. The error "Invalid filter pattern" is caused by using ~bq with regex part.
More details https://github.com/mitmproxy/mitmproxy/pull/2589#issuecomment-340426254

Related

match string pattern by certain characters but exclude combinations of those characters

I have the following sample string:
'-Dparam="x" -f hello-world.txt bye1.txt foo_bar.txt -Dparam2="y"'
I am trying to use RegEx (PowerShell, .NET flavor) to extract the filenames hello-world.txt, bye1.txt, and foo_bar.txt.
The real use case could have any number of -D parameters, and the -f <filenames> argument could appear in any position between these other parameters. I can't easily use something like split to extract it as the delimiter positioning could change, so I thought RegEx might be a good proposition here.
My attempt is something like this in PowerShell (can be opened on any Windows system and copy pasted into it):
'-Dparam="x" -f hello-world.txt bye1.txt foo_bar.txt -Dparam2="y"' -replace '^.* -f ([a-zA-Z0-9_.\s-]+).*$','$1'
Desired output:
hello-world.txt bye1.txt foo_bar.txt
My problem is that I either only take hello-world.txt, or I get hello-world.txt all the way to the end of the string or next = symbol (as in the example above).
I am having trouble expressing that \s is allowed, since I need to capture multiple space-delimited filenames, but that the combination of \s-[a-zA-Z] is not allowed, as that indicates the start of the next argument.

When I run a file which is begin with "#!/usr/bin/perl -w", I get a error: "syntax error at line 153, near "=~ ?""

When I run a file which is begin with #!/usr/bin/perl -w, I get a error:
syntax error at line 153, near "=~ ?"
I try to add "#!/bin/bash", this error is not append, but I get another
error:
"line 34: syntax error near unexpected token `('"
line 153 in my file:
($output_volume =~ ?^([\S]+).mnc?) && ($base_name = $1) ||
die "sharpen_volume failed: output volume does not appear to be"
." a minc volume.\n";
line34 in my file:
use MNI::Startup qw(nocputimes);
$output_volume =~ ?^([\S]+).mnc?
This used to be valid perl and thus might appear in old code and instructional material.
From perlop:
In the past, the leading m in m?PATTERN? was optional, but omitting it would produce a deprecation warning. As of v5.22.0, omitting it produces a syntax error. If you encounter this construct in older code, you can just add m.
That is Perl code so the first error message is meaningful.
With delimiters other than // in the match operator you must have the explicit m for it, so
$output_volume =~ m?^([\S]+).mnc?
It is only with // delimiters that the m may be omitted; from Regex Quote-Like Operators (perlop)
If "/" is the delimiter then the initial m is optional.
See perlretut for a tutorial introduction to regex and perlre for reference.
Also note that the particular delimiters of ? trigger specific regex behavior in a special case. This is discussed by the end of the documentation section in perlop linked above.
You already have two answers that explain the problem.
? ... ? is no longer valid syntax for a match operator. You need m? ... ? instead.
Until Perl 5.22, your syntax generated a warning. Now it's a fatal error (which is what you are seeing). So I assume you're now running this on a more recent version of Perl.
There are, however, a few other points it is probably worth making.
You say you tried to investigate this by changing the first line of your file from #!/usr/bin/perl -w to #!/bin/bash. I'm not sure how you think this was going to help. This line defines the program that is used to run your code. As you have Perl code, you need to run it with Perl. Trying to run it with bash is very unlikely to be useful.
The m? ... ? (or, formerly, ? ... ?) syntax triggers an obscure and specialised behaviour. It seems to me that this behaviour isn't required in your case, so you can probably change it to the more usual / ... /.
Your regex contains an unescaped dot character. Given that you seem to be extracting the basename from a filename that has an extension, it seems likely that this should be escaped (using \.) so that it matches an actual dot (rather than any character).
If you are using this code to extract a file's basename, then using a regex probably isn't the best approach. Perhaps take a look at File::Basename instead.

PHP: Using preg_replace to replace an unknown string between two known strings

I have $stringF. Contained within $stringF is the following (the string is all one line, not word-wrapped as below):
http://news.google.com/news/url?sa=t&fd=R&ct2=us&usg=
AFQjCNHWQk0M4bZi9xYO4OY4ZiDqYVt2SA&clid=
c3a7d30bb8a4878e06b80cf16b898331&cid=52779892300270&ei=
H4IAW6CbK5WGhQH7s5SQAg&url=https://abcnews.
go.com/Lifestyle/wireStory/latest-royal-wedding-thousands-streets-windsor-55280649
I want to locate that string and make it look like this:
https://abcnews.go.com/Lifestyle/wireStory/latest-royal-
wedding-thousands-streets-windsor-55280649
Basically I need to use preg_replace to find the following string:
http://news.google.com/news/url?sa= ***SOME UNKNOWN CONTENT*** &url=http
and replace it with the following string:
http
I'm a little rusty with my php, and even rustier with regular expressions, so I'm struggling to figure this one out. My code looks like this:
$stringG = preg_replace('http://news.google.com/news/url?sa=*&url=http','http',$stringH);
except I know I can't use wildcards and I know I need to specially deal with the special characters (colon, forward slash, question mark, and sign, etc). Hoping someone can help me out here.
Also of note is that my $stringF contains multiple instances of such strings, so I need the preg_replace to be not greedy - otherwise it will replace a huge chunk of my string unnecessarily.
PHP has tools for that, no need to use a regex. parse_url to get the components of an url (scheme, host, path, anchor, query, ...) and parse_str to get the keys/values of the query part.
$url = 'http://news.google.com/news/url?sa=t&fd=R&ct2=us&usg=AFQjCNHWQk0M4bZi9xYO4OY4ZiDqYVt2SA&clid=c3a7d30bb8a4878e06b80cf16b898331&ci=52779892300270&ei=H4IAW6CbK5WGhQH7s5SQAg&url=https://abcnews.go.com/Lifestyle/wireStory/latest-royal-wedding-thousands-streets-windsor-55280649';
parse_str(parse_url($url, PHP_URL_QUERY), $arr);
echo $arr['url'];

Rewrite another users message

I use MIRC and this is my goal, i'm sure it's simple! thanks guys.
on $*:text:*test*:#: { msg $chan "Entire message containing test" }
You dont need the $ after the on, it's only used when you use regex to match (see here).
And indeed it's quite simple, use $1- to match he whole message. $1 will match the first 'token'.
In your example sentence Entire message containing test, $1 will rerturn Entire assuming you did not use a tokenize before and because the default delimiter is a space in mSL. $2 will retrurn message and so on.
Here is a great article about token manipulation and in your mIRC client you can use the command /help $1- to learn more about remote identifiers.

Using Github API code search, find the exact string `throw "`

I want to find the exact strings, throw ' and throw " in javascript files in a given repo using the Github API.
It says:
You can't use the following wildcard characters as part of your search
query: . , : ; / \ ` ' " = * ! ? # $ & + ^ | ~ < > ( ) { } [ ]. The
search will simply ignore these symbols.
I'm supposing there is no way to find these exact strings using the API?
I have tried various searches with no luck. Trying to escape the " with \ doesn't work either.
https://api.github.com/search/code?q=%22throw+%27%22+in:file+language:js+repo:angular/angular.js
All of the queries I try return, for instance, https://github.com/angular/angular.js/blob/master/docs/config/tag-defs/tutorial-step.js which just finds the throw and disregards the '.
An alternative to this strategy is to find where there is NOT Error on the line, so that the search is throw NOT Error to try to find where someone is throwing a string and not throw new Error(. This strategy doesn't work for some reason. For instance,
https://api.github.com/search/code?q=%22throw%20NOT%20Error%22+in:file+language:js+repo:driverdan/node-XMLHttpRequest
There are many times that a plain string is thrown in https://github.com/driverdan/node-XMLHttpRequest/blob/master/lib/XMLHttpRequest.js but, the file does contain the string Error; so, I guess this is enough to make the result empty. I'm seeing a way to "search" in my client program as a workaround; but, I would rather get the results from the API.
I'm supposing there is no way to find these exact strings using the API?
Correct, that's not possible currently.