I'm trying to generate some traces that can trigger snort's alert to test the performance of snort. But there are some pcre options in some rules that contain /R, which I don't understand.
For example, there's a pcre option in one snort rule pcre: "/^(\x75|\x2d|\x2f|\x73|\xa2|\x2e|\x24|\x74)/sR", and I don't know what "R" means there. I know "s" is a pcre modifier that can set PCRE_DOTALL. But what about "R"? Is it a modifier too or something else?
I've searched the pcre doc, but I didn't find there's an "R" modifier. So I think it's not a modifier.
Here's an example rule that contains pcre option with /R in it, I got this rule from snort3's community rules.
alert udp $EXTERNAL_NET any -> $HOME_NET 138 ( msg:"OS-WINDOWS
Microsoft Windows SMB unicode andx invalid server name share access";
content:"|11|",depth 1; content:"|00|",distance 13;
content:"|00|",distance 0; content:"|FF|SMB",within 4,distance 3;
pcre:"/^(\x75|\x2d|\x2f|\x73|\xa2|\x2e|\x24|\x74)/sR";
byte_test:1,&,128,6,relative; content:"u",depth 1,offset 39;
byte_jump:2,0,little,relative; byte_jump:2,7,little,relative;
content:"|5C 00 5C 00|",distance 2,nocase;
pcre:!"/^([^\x5C\x00].|[\x5c\x00][^\x00])+\x5C\x00/sR";
metadata:policy max-detect-ips drop; reference:cve,2010-0022;
reference:url,technet.microsoft.com/en-us/security/bulletin/MS10-012;
classtype:protocol-command-decode; sid:16403; rev:12; )
The R modifier is not a native PCRE modifier, it is a Snort specific modifier for PCRE regex, that enables Snort3 to force specific pattern behavior.
See Snort3 "3.5.26.1 Format" documentation:
R Match relative to the end of the last pattern match. (Similar to distance:0;)
...
The modifiers R (relative) and B (rawbytes) are not allowed with any of the HTTP modifiers such as U, I, P, H, D, M, C, K, S and Y.
Note that acc. to Rules Authors Introduction to Writing Snort 3 Rules:
In Snort 2, the post-re modifiers (B, U, P, H, M, C, I, D, K, S, Y) set compile time flags for the regular expression. For example, the Snort specific modifier for pcre U is used to match the decoded URI buffers.
In Snort 3, some of post-re modifiers (B, U, P, H, M, C, I, D, K, S, Y) have been deleted in favor of sticky
buffers.
Related
I wonder what is the most elegant way of rearranging associative terms in Coq. I think it is a well-defined problem for some existing operators, such as plus.
I have defined new associative operater and operands with corresponding types. In short, an associative lemma states that
Lemma assoc: forall A B C: sometype, A # (B # C) = A # B # C
Surely, all chains organized in the same order are equal no matter how to put the brackets. e.g.
A # B # C # D # E = (A # B) # (C # (D # E)) = (A # (B # (C # (D # E))))
I know one can use repeat rewrite assoc. or repeat rewrite <- assoc.to achieve LHS or RHS from the middle expression. Now I want to apply a lemma about (B # C). How can I quickly rewrite the expression to something like,
A # (B # C) # D # E
One solution is to use replace ... with ... by now rewrite assoc. However, it can be tedious when the expressions are getting longer. Is there any other effective way of rearranging terms/bracket? Is the aac-tactics library helpful for the problem? I cannot find a tutorial for using aac over the customized operators.
Yes AAC-Tactics is the recommended tactic for such problems. It provides an extensible tactic for associative and commutative rewriting. The tutorial is here (https://github.com/coq-community/aac-tactics/blob/master/theories/Tutorial.v).
AAC-Tactics is included in Coq Platform, so you can expect long term support for it.
I'm trying to understand this snippet code from:
https://code.kx.com/q/kb/loading-from-large-files/
to customize it by myself (e.x partition by hours, minutes, number of ticks,...):
$ cat fs.q
\d .Q
/ extension of .Q.dpft to separate table name & data
/ and allow append or overwrite
/ pass table data in t, table name in n, : or , in g
k)dpfgnt:{[d;p;f;g;n;t]if[~&/qm'r:+en[d]t;'`unmappable];
{[d;g;t;i;x]#[d;x;g;t[x]i]}[d:par[d;p;n];g;r;<r f]'!r;
#[;f;`p#]#[d;`.d;:;f,r#&~f=r:!r];n}
/ generalization of .Q.dpfnt to auto-partition and save a multi-partition table
/ pass table data in t, table name in n, name of column to partition on in c
k)dcfgnt:{[d;c;f;g;n;t]*p dpfgnt[d;;f;g;n]'?[t;;0b;()]',:'(=;c;)'p:?[;();();c]?[t;();1b;(,c)!,c]}
\d .
r:flip`date`open`high`low`close`volume`sym!("DFFFFIS";",")0:
w:.Q.dcfgnt[`:db;`date;`sym;,;`stats]
.Q.fs[w r#]`:file.csv
But I couldn't find any resources to give me detail explain. For example:
if[~&/qm'r:+en[d]t;'`unmappable];
what does it do with the parameter d?
(Promoting this to an answer as I believe it helps answer the question).
Following on from the comment chain: in order to translate the k code into q code (or simply to understand the k code) you have a few options, none of which are particularly well documented as it defeats the purpose of the q language - to be the wrapper which obscures the k language.
Option 1 is to inspect the built-in functions in the .q namespace
q).q
| ::
neg | -:
not | ~:
null | ^:
string | $:
reciprocal| %:
floor | _:
...
Option 2 is to inspect the q.k script which creates the above namespace (be careful not to edit/change this):
vi $QHOME/q.k
Option 3 is to lookup some of the nuggets of documentation on the code.kx website, for example https://code.kx.com/q/wp/parse-trees/#k4-q-and-qk and https://code.kx.com/q/basics/exposed-infrastructure/#unary-forms
Options 4 is to google search for reference material for other/similar versions of k, for example k2/k3. They tend to be similar-ish.
Final point to note is that in most of these example you'll see a colon (:) after the primitives....this colon is required in q/kdb to use the monadic form of the primitive (most are heavily overloaded) while in k it is not required to explicitly force the monadic form. This is why where will show as &: in the q reference but will usually just be & in actual k code
When ran in the command line, this
swipl -g "write(42)" -t "halt"
prints 42 to STDOUT as expected.
However, this
swipl -g "X = 42" -t "halt"
does not print anything, it simply returns.
How do I get it to print what it prints in the REPL (that is, X = 42)?
Note: this is in a Windows terminal. Let me know if this actually works in a Linux terminal.
As expected, X = 42 by itself produces no output whatsoever, because (=)/2 is a completely pure predicate that does not yield any side effects by itself. This is the case on Window, OSX and all other operating systems.
Even if there were a way to obtain and redirect the toplevel output itself, the fact remains that the SWI toplevel is subject to change and you cannot rely on future versions to behave in the same way as it does now. Long term, you will likely be better off to roll your own toplevel and produce exactly the output you want.
It is not so hard to roll your own toplevel. The trick is mainly to use the variable_names/1 option when reading terms, so that you can keep track of the variable names that you want to show in answers. Here is a very simplistic start:
repl :-
read_line_to_codes(current_input, Codes),
read_term_from_codes(Codes, Term, [variable_names(NameVars)]),
call(Term),
report_bindings(NameVars).
repl :- repl.
report_bindings(NameVars) :-
phrase(bindings(NameVars), Bs),
format("~s", [Bs]).
bindings([]) --> [].
bindings([E]) --> name_var(E).
bindings([E1,E2|Rest]) --> name_var(E1), ",\n", bindings([E2|Rest]).
name_var(Name=Var) -->
format_("~w = ~q", [Name,Var]).
format_(Format, Ls) -->
call(format_codes(Format, Ls)).
format_codes(Format, Ls, Cs0, Cs) :-
format(codes(Cs0,Cs), Format, Ls).
Example:
?- repl.
|: X = 4, between(1, 3, Y).
X = 4,
Y = 1
true ;
X = 4,
Y = 2
true ;
X = 4,
Y = 3
true ;
|: X = 7.
X = 7
It is easy to modify this so that it works on terms that are specified as arguments.
Note that the variable_names/1 option is essential for reading terms in such a way, and thanks to the ISO standardization effort an increasing number of implementations provide it for read_term/2 and related predicates.
This ability to read variable names is a requirement for implementing a portable Prolog toplevel!
The main exercise that I leave for you is to check if the quoting is right in all cases and (if desired) to produce answers in such a way that they can always be pasted back on the terminal. To extend this to residual constraints, use copy_term/3 and call_residue_vars/2 to collect pending constraints that you can append to the bindings.
I am trying to learn Coq by working through the online Software Foundations book: http://www.cis.upenn.edu/~bcpierce/sf/
I'm using the interactive command line Coq interpreter coqtop.
In the induction chapter (http://www.cis.upenn.edu/~bcpierce/sf/Induction.html), I am following the instructions exactly. I compile Basics.v using coqc Basics.v. I then start coqtop and type exactly:
Require Export Basics.
Theorem andb_true_elim1 : forall b c : bool,
andb b c = true -> b = true.
Proof.
intros b c H.
destruct b.
Case "b = true".
Everything works until that last line, at which point I get the following error:
Toplevel input, characters 5-15:
> Case "b = true".
> ^^^^^^^^^^
Error: No interpretation for string "b = true".
I'm too new to Coq to start to unpack why this isn't working. I found something online suggesting I needed to do Require String. first, however, this didn't work either. Has anyone worked through this book or encountered this problem? How do I get the code to work properly?
This Case keyword (tactic?) seems to be dependent on something else that the SF book is not making clear is needed, but I can't figure out what.
What's missing is a string datatype which hooks into the "..." notation; the String module contains such a notation and datatype, but you have to tell Coq to use that notation via Open Scope string_scope. What's also missing, however, is an implementation of Case, which will only show up after you fix the string problem. All of this is provided for you in the Induction.v file inside the "Download" tarball, but it is not included in the output Induction.html, possibly due to a typo in the .v file. The relevant code, which would be the second paragraph of the "Naming Cases" section (right after "…but a better way is to use the Case tactic," and right before "Here's an example of how Case is used…") is:
(* [Case] is not built into Coq: we need to define it ourselves.
There is no need to understand how it works -- you can just skip
over the definition to the example that follows. It uses some
facilities of Coq that we have not discussed -- the string
library (just for the concrete syntax of quoted strings) and the
[Ltac] command, which allows us to declare custom tactics. Kudos
to Aaron Bohannon for this nice hack! *)
Require String. Open Scope string_scope.
Ltac move_to_top x :=
match reverse goal with
| H : _ |- _ => try move x after H
end.
Tactic Notation "assert_eq" ident(x) constr(v) :=
let H := fresh in
assert (x = v) as H by reflexivity;
clear H.
Tactic Notation "Case_aux" ident(x) constr(name) :=
first [
set (x := name); move_to_top x
| assert_eq x name; move_to_top x
| fail 1 "because we are working on a different case" ].
Tactic Notation "Case" constr(name) := Case_aux Case name.
Tactic Notation "SCase" constr(name) := Case_aux SCase name.
Tactic Notation "SSCase" constr(name) := Case_aux SSCase name.
Tactic Notation "SSSCase" constr(name) := Case_aux SSSCase name.
Tactic Notation "SSSSCase" constr(name) := Case_aux SSSSCase name.
Tactic Notation "SSSSSCase" constr(name) := Case_aux SSSSSCase name.
Tactic Notation "SSSSSSCase" constr(name) := Case_aux SSSSSSCase name.
Tactic Notation "SSSSSSSCase" constr(name) := Case_aux SSSSSSSCase name.
(A side note: When I worked through Software Foundations, I found using the provided .v files as my work material to be very helpful. You don't have to worry about elided code, you don't have to retype the definitions, and all the problems are right there. Your mileage may vary, of course.)
I'm reading a file in scala using
def fileToString(that:String):String= {
var x:String=""
for(line <- Source.fromFile(that).getLines){
x += line + "\n"
}
x
}
This works fine for a scala file. But on a txt file it adds spaces between every character. For example. I read in a .txt file and get this:
C a l l E v e n t L o g ( E r r o r $ , E r r N u m , E r r O b j )
' E n d E r r o r h a n d l i n g b l o c k .
E n d S u b
and I read in the scala file for the program and it comes out normally
EDIT: It seems to be something to do with Encoding. When I change it to UTF-16, it reads the .txt file, but not the scala file. Is there a way to make it universally work?
No it can't work for all files. To read/interpret a file/data you need to know the format/encoding unless you're treating it as a binary blob.
Either save all files in the usual unicode format (UTF-8) or specify the encoding when reading the file.
FromFile takes an implicit codec, you can pass it explicitly.
io.Source.fromFile("123.txt")(io.Codec("UTF-16"))
In general, if you read from a file you need to know its encoding in order to correctly read the characters. I am not sure what the default encoding is that Scala assumes, probably UTF8, but you can either pass a Codec to fromFile, or specify the encoding as a string:
io.Source.fromFile("file.txt", "utf-8")
It's hard to be sure, but it sounds like the two files were written with different encodings. On any Unix system (including Mac) you can use the command od to look at the actual bytes in the file.
UTF-8 is the standard for ordinary text files on most systems, but if you have a mix of UTF-8 and UTF-16, you'll have to know which encoding to use for which files and correctly specify the encoding.
Or be more careful when you create the files to insure that they are all in the same format.