Related
I am changing pg-pool to cater cypher queries for Apache AGE and I have the following code, which takes the select statement. From that it gets the FROM clause, then from the FROM clause it gets the function names. So now I want that function arguments also. How would I do that. The code is as follows:
if (IsA(node, SelectStmt))
{
SelectStmt *stmt = (SelectStmt *) node;
List* fromClause = stmt->fromClause;
ListCell *fl;
//Match the first cypher function call in the FROM clause. Could be multiple tables
// e.g. FROM table1, table2, cypher(),table3....
foreach(fl, fromClause)
{
Node *n = lfirst(fl);
if (IsA(n, RangeFunction))
{
RangeFunction *rf = (RangeFunction*) n;
List* functions = rf->functions;
if (functions->length == 1)
{
List *sublist = (List *) lfirst(list_head(functions));
FuncCall *fntree = (FuncCall *) lfirst(list_head(sublist));
StringInfoData str;
initStringInfo(&str);
_outNode(&str,fntree->funcname);
if (!strcmp("\"cypher\"",str.data)){
return true;
}
}
}
}
}
The arguments are stored in FuncCall. Since you already have that, you can get them with
fntree->args
Note that your code only covers function calls in the top FROM list. What if there are subqueries?
I currently try to filter calls to a function by command. I try to do so with the following code where ##REPLACE_comm## is replaced by python by the command name. The double backslash are cause I am using bcc. The following code throws an error when loading:
if(1){
char filter[TASK_COMM_LEN] = "##REPLACE_comm##";
char command[TASK_COMM_LEN];
bpf_get_current_comm(&command, sizeof(command));
for(u16 i = 0; i<=TASK_COMM_LEN; i++){
if(command[i] == '\\0' && filter[i] == '\\0'){
break;
}
if(command[i] == filter[i]){
continue;
}
return 0;
}
}
The error is:
unknown opcode 70
HINT: The 'unknown opcode' can happen if you reference a global or static variable, or data in read-only section. For example, 'char *p = "hello"' will result in p referencing a read-only section, and 'char p[] = "hello"' will have "hello" stored on the stack.
I feel like I already made sure the variables are on the stack by allocating space and not just having a pointer but it doesnt work. What am I missing?
I'm trying to write a DSL via Xtext with which I can customize SQL.
First I want to have a string representation of my query.
There I got stuck on the expressions. I created them after this example: https://typefox.io/parsing-expressions-with-xtext.
Expression returns Expression:
OrExpression;
OrExpression returns Expression:
AndExpression ({OrExpression.left=current} name="OR" right=AndExpression)*;
AndExpression returns Expression:
NotExpression({AndExpression.left=current} name="AND" right=NotExpression)*;
NotExpression returns Expression:
ComparisonExpression({NotExpression.left=current} name='NOT' right=ComparisonExpression)*;
ComparisonExpression returns Expression:
BitwiseOR({ComparisonExpression.left=current} name=cmpop right=BitwiseOR)*;
BitwiseOR returns Expression:
BitwiseAND ({BitwiseOR.left=current} name='|' right=BitwiseAND)*;
BitwiseAND returns Expression:
BitwiseXOR ({BitwiseAND.left=current} name='&' right=BitwiseXOR)*;
BitwiseXOR returns Expression:
Addition ({BitwiseXOR.left=current} name='^' right=Addition)*;
Addition returns Expression:
Substraction ({Addition.left=current} name='+' right=Substraction)*;
Substraction returns Expression:
Multiplication ({Substraction.left=current} name='-' right=Multiplication)*;
Multiplication returns Expression:
Division ({Multiplication.left=current} name='*' right=Division)*;
Division returns Expression:
Modulo ({Division.left=current} name='/' right=Modulo)*;
Modulo returns Expression:
Primary ({Modulo.left=current} name='%' right=Primary)*;
Primary returns Expression:
=> count=count
| => func=funccall
| {Bracket} '(' inner=Expression ')'
| (name=unop) expr=Expression
| (name=unop)? ID=Literal ('IS' 'NOT'? 'NULL')?;
/*Values that a Expression can have */
Literal returns Expression:
value=values;
I can manage to get to get a single value but im unable to get the left or the right side of a expression.
My Generator (unfinished and with my testings looks like this)
class TsqlGenerator extends AbstractGenerator {
StringBuilder st = new StringBuilder();
TimeConditionHandler tch = new TimeConditionHandler
ExpressionHandler exprH = new ExpressionHandler
String expression = ""
int fromIndex = 0;
int whereIndex
def clear() {
st.delete(0, st.length);
fromIndex = 0;
whereIndex =0;
}
override beforeGenerate(Resource input, IFileSystemAccess2 fsa, IGeneratorContext context) { clear() }
override void doGenerate(Resource resource, IFileSystemAccess2 fsa, IGeneratorContext context) {
for (e : resource.allContents.toIterable.filter(ComplexSelect)) {
/*SELECT */
// TODO: String Value ohne Anführungszeichen?
st.append("SELECT ");
for (selectArgument : e.left.selectArguments.arguments) {
if (selectArgument.expr.name == null) {
//hier später in eine Liste speichern?
st.append(" " + selectArgument.expr.ID.value.name);
}else {
// TODO: Generate a Sting of the Expression!
}
}
/*FROM */
st.append(" FROM ")
//Hier ebenso in eine Liste speichern?
for (fromSource : e.left.from.sources) {
st.append(e.left.from.sources.get(fromIndex).name);
}
/*WHERE */
if (e.left.where !== null || e.left.timedef !== null) {
st.append(" WHERE");
if (e.left.where !== null) {
for (whereArgument : e.left.where.predicates.expr) {
if (whereIndex == 0) {
st.append(" " + whereArgument.ID.value.name);
} else {
st.append("AND " + whereArgument.ID.value.name);
}
whereIndex++;
}
}
/*TIMEINTERVALL
*TODO: timestamp as columname hardcoded*/
if (e.left.timedef !== null) {
if (whereIndex > 0) {
st.append(" AND")
}
st.append(tch.toIso8601(e.left.timedef))
//if (e.left.timedef.name != null) {
//st.append(" ").append(e.left.where.name);
}
}
fsa.generateFile("query.txt", st.toString());
}
}
Given the following simplified grammar
grammar org.xtext.example.mydsl1.MyDsl with org.eclipse.xtext.common.Terminals
generate myDsl "http://www.xtext.org/example/mydsl1/MyDsl"
Model:
(expressions+=Expression ";")*;
Expression returns Expression:
OrExpression;
OrExpression returns Expression:
PrimaryExpression ({OrExpression.left=current} name="OR" right=PrimaryExpression)*;
PrimaryExpression returns Expression:
'(' Expression ')'
| {Negation} "NOT" expr=Expression
| {IntLiteral} value=INT
| {StringLiteral} value=STRING
| {Variable} name=ID;
the input NOT x or y can be parsed in two ways: Either as
Or(Negation(Variable("x")), Variable("y"))
or as
Negation(Or(Variable("x"), Variable("y")))
When compiling the Xtext grammer it tells you so (somewhat vaguely) with warnings likes this:
warning(200): ../org.xtext.example.mydsl1/src-gen/org/xtext/example/mydsl1/parser/antlr/internal/InternalMyDsl.g:195:3: Decision can match input such as "'OR'" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
The important part here is: [...]alternative(s) 2 were disabled for that input. So parts of your grammar are dead code and effectively ignored.
Solution: Put all operators into the precedence hierarchy properly. Do not put anything complex into PrimaryExpression. In fact ( Expression )
should be the only place which calls a rule higher in the hierarchy. The result is a conflict-free grammar like this:
OrExpression returns Expression:
IsNullExpression ({OrExpression.left=current} name="OR" right=IsNullExpression)*;
IsNullExpression returns Expression:
CompareExpression ({IsNullExpression.expr=current} 'IS' not?='NOT'? 'NULL')?;
CompareExpression returns Expression:
NegationExpression ({CompareExpression.left=current} name=("NOT"| "=" | "!=") right=NegationExpression)*;
NegationExpression returns Expression:
{NegationExpression} name=("NOT" | "-") expr=PrimaryExpression
| PrimaryExpression;
PrimaryExpression returns Expression:
'(' Expression ')'
| {IntLiteral} name=INT
| {StringLiteral} name=STRING
| {Variable} name=ID;
Note that this grammar is conflict free despite several keywords are used multiple times in distinct places (e.g. NOT, -).
I'm trying to implement a macro to allow MATLAB-esque matrix creation. I've got a basic working macro but I still have a long way to go.
I want to be able to enforce the right structure (same number of elements in each row) but I'm not sure how to do this within the macro. I think I want to enforce that each internal repetition has the same length - is this something I can do?
Here is my code so far:
pub struct Matrix<T> {
pub cols: usize,
pub rows: usize,
pub data: Vec<T>
}
macro_rules! mat {
( $($( $x:expr ),*);* ) => {
{
let mut vec = Vec::new();
let mut rows = 0;
$(
$(
vec.push($x);
)*
rows += 1;
)*
Matrix { cols : vec.len()/rows, rows: rows, data: vec}
}
};
}
It works but as you can see isn't very safe. It has no restrictions on the structure.
I want to do a lot more with this macro but I think this is a good start!
Update:
Here is some playground code for a crappy implementation I worked out. If anyone has any better suggestions please let me know! Otherwise I'll close this myself.
macro_rules! count {
() => (0usize);
( $x:tt $($xs:tt)* ) => (1usize + count!($($xs)*));
}
macro_rules! mat {
( $( $x:expr ),* ) => { {
let vec = vec![$($x),*];
Matrix { cols : vec.len(), rows: 1, data: vec }
} };
( $( $x0:expr ),* ; $($( $x:expr ),*);* ) => { {
let mut _assert_width0 = [(); count!($($x0)*)];
let mut vec = Vec::new();
let rows = 1usize;
let cols = count!($($x0)*);
$( vec.push($x0); )*
$(
let rows = rows + 1usize;
let _assert_width = [(); count!($($x)*)];
_assert_width0 = _assert_width;
$( vec.push($x); )*
)*
Matrix { cols : cols, rows: rows, data: vec }
} }
}
playground
The count! macro expands to a constant expression that represents the number of arguments it got as input. It's just a helper for the mat! macro. If you need to count a lot of items and the compiler can't cope with it, see the Counting chapter in The Little Book of Rust Macros, which has more complex macros for counting.
My version of the macro uses dummy variables and assignments to verify that the width of all rows are the same. First off, I changed the macro's pattern to handle the first row separately from the subsequent rows. The first variable, _assert_width0, is initialized with an array of units ((), which makes the array take no memory), with the size of the array being the number of items in the first row. Then, _assert_width is also initialized with an array of units, with the size of the array being the number of items in each subsequent row. Then, _assert_width is assigned to _assert_width0. The magic here is that this line will raise a compiler error if the width of a row doesn't match the width of the first row, since the types of the array won't match (you might have e.g. [(); 3] and [(); 4]). The error isn't super clear if you don't know what's going on in the macro, though:
<anon>:38:24: 38:37 error: mismatched types:
expected `[(); 3]`,
found `[(); 4]`
(expected an array with a fixed size of 3 elements,
found one with 4 elements) [E0308]
<anon>:38 _assert_width0 = _assert_width;
^~~~~~~~~~~~~
<anon>:47:13: 47:44 note: in this expansion of mat! (defined in <anon>)
<anon>:38:24: 38:37 help: see the detailed explanation for E0308
First, to quickly address the title of your question: see the Counting chapter in The Little Book of Rust Macros. To summarise: there is no direct way, you need to write a macro that expands to something you can count in regular code.
Now, to address your actual question: hoo boy.
It's not so much counting that you want, it's to fail at compile time if the sub-sequences have different lengths.
First of all, there's no clean way to trigger a compilation failure from a macro. You can trigger some other pre-existing error, but you can't control the actual error message.
Secondly, there's no easy way to do "variable" comparisons in macros at all. You can sometimes compare against a fixed token sequence, but you're not doing that here.
So it's doubly not-really-doable.
The simplest thing to do is check the lengths during construction at runtime, and return an error or panic if they don't match.
Is it actually impossible? I don't believe so. If you're willing to accept inscrutable error messages and a massive jump in complexity, you can check for length equality between two token sequences like so:
macro_rules! tts_equal_len {
(($_lhs:tt $($lhs_tail:tt)*), ($_rhs:tt $($rhs_tail:tt)*)) => {
tts_equal_len!(($($lhs_tail)*), ($($rhs_tail)*))
};
(($($_lhs_tail:tt)+), ()) => { do_something_bad!() };
((), ($($_rhs_tail:tt)+)) => { do_something_bad!() };
((), ()) => { do_something_good!() };
}
macro_rules! do_something_bad { () => { { println!("kaboom!") } } }
macro_rules! do_something_good { () => { { println!("no kaboom!") } } }
fn main() {
tts_equal_len!((,,,), (,,,));
tts_equal_len!((,,,), (,,));
tts_equal_len!((,), (,,));
}
Again, the real problem is finding some way to fail at compile time such that the user will understand why compilation failed.
Update: there's a new way of doing things
As of the day on which this was written, the feature of rust which enables the following (count) to be done, in still unstable and is available in nightly builds.
You can check out the github issues and test cases for further understanding of what's given below
To enable this feature, you need to add the line #![feature(macro_metavar_expr)] to the top of the crate's root module (usually main.rs or lib.rs), and also set your repo to use nightly builds, which is easily done by creating a file rust-toolchain.toml in the root directory (alongside Cargo.toml) and add the folllowing lines to it:
[toolchain]
channel = "nightly"
Now, instead of providing a solution to you specific problem, I'd like to share a generic solution I created to better illustrate most situations.
I highly recommend studying the code AND the comments, by pasting the following two code blocks in a file (main.rs).
The macro_rules
#[derive(Eq, PartialEq, Debug, Copy, Clone)]
struct SumLen {
sum: i32,
len: u32
}
/// currently one `i32` type is available
///
/// # Examples
///
/// The output of the following:
/// ```ignore
/// sumnarr!(i32 => 5 ; 6, 7, 8)
/// ```
/// will be `[(5, 1), (21, 3)]`
macro_rules! sumnarr {
( $type:ty => $( $( $x: expr ),* );* ) => {
{
// `${count(x,0)}` will give you "length" (number of iterations)
// in `$( )*` loop that you are IMMEDIATELY OUTSIDE OF (e.g.: the `$( )*` loop below)
// `${count(x,1)}` will give you TOTAL number of iterations that the `$( )*` loop
// INSIDE of the IMMEDIATE `$( )*` loop will make. i.e. it is similar to converting
// [ [i1,i2], [i1,i2,i3] ] TO [ i1,i2,i3,i4,i5 ] i.e. flatten the nested iteration.
// In case of `[ [i1,i2], [i1,i2,i3] ]`, `${count(x,0)}` is 2 and `${count(x,1)}` is 5
let mut arr: [SumLen; ${count(x,0)}] = [SumLen{ sum:0, len:0}; ${count(x,0)}];
$(
// `${index()}` refers to the iteration number within the `$( )*` loop
arr[${index()}] = {
let mut sum = 0;
//let mut len = 0;
// THe following will give us length is the loop it is IMMEDIATELY OUTSIDE OF
// (the one just below)
let len = ${count(x,0)};
$(
sum += $x;
// If you were NOT using `$x` somewhere else inside `$( )*`,
// then you should use `${ignore(x)};` to inform the compiler
//You could use the below method, where `${length()}` will give you
//"length" or "number of iterations" in current loop that you are in
// OR
// you could go with my method of `${count(x,0)}` which is explained above
//len = ${length()};
)*
SumLen {
sum,
len
}
};
)*
arr
}
};
}
The #[test] (unit test)
#[test]
fn sumnarr_macro() {
let (a, b, c, d, e) = (4, 5, 6, 9, 10);
let sum_lens = [
SumLen {
sum: a + e,
len: 2
},
SumLen {
sum: b + c + d,
len: 3
}
];
assert_eq!(sum_lens, sumnarr!(i32 => a,e;b,c,d));
}
I hope this helps
I have been using CouchDB for quite sometime without any issues. That is up until now. I recently saw something in my map/reduce results which I had overlooked!
This is before performing a sum on the "avgs" variable. I'm basically trying to find the average of all values pertaining to a particular key. Nothing fancy. The result is as expected.
Note the result for timestamp 1308474660000 (4th row in the table):
Now I sum the "avgs" array. Now here is something that is peculiar about the result. The sum for the key with timestamp 1308474660000 is a null!! Why is CouchDB spitting out nulls for a simple sum? I tried with a custom addition function and its the same problem.
Can someone explain to me why is there this issue with my map/reduce result?
CouchDB version: 1.0.1
UPDATE:
After doing a rereduce I get a reduce overflow error!
Error: reduce_overflow_error
Reduce output must shrink more rapidly: Current output: '["001,1,1,1,1,1,11,1,1,1,1,1,1,11,1,1,1,1,1,1,11,1,1,1,1,1,1,11,1,1,1,1,1,101,1,1,1,1,1,1,11,1,1,1,1'... (first 100 of 396 bytes)
This is my modified reduce function:
function (key, values, rereduce) {
if(!rereduce) {
var avgs = [];
for(var i=values.length-1; i>=0 ; i--) {
avgs.push(Number(values[i][0])/Number(values[i][1]));
}
return avgs;
} else {
return sum(values);
};
}
UPDATE 2:
Well now it has gotten worse. Its selectively rereducing. Also, the ones it has rereduced show wrong results. The length of the value in 4th row for timestamp (1308474660000) should be 2 and not 3.
UPDATE 3:
I finally got it to work. I hadn't understood the specifics of rereduce properly. AFAIK, Couchdb itself decides how to/when to rereduce. In this example, whenever the array was long enough to process, Couchdb would send it to rereduce. So I basically had to sum twice. Once in reduce, and again in rereduce.
function (key, values, rereduce) {
if(!rereduce) {
var avgs = [];
for(var i=values.length-1; i>=0 ; i--) {
avgs.push(Number(values[i][0])/Number(values[i][1]));
}
return sum(avgs);
} else {
return sum(values); //If my understanding of rereduce is correct, it only receives only the avgs that are large enough to not be processed by reduce.
}
}
Your for loop in the reduce function is probably not doing what you think it is. For example, it might be throwing an exception that you did not expect.
You are expecting an array of 2-tuples:
// Expectation
values = [ [value1, total1]
, [value2, total2]
, [value3, total3]
];
During a re-reduce, the function will get old results from itself before.
// Re-reduce values
values = [ avg1
, avg2
, avg3
]
Therefore I would begin by examining how your code works if and when rereduce is true. Perhaps something simple will fix it (although often I have to log() things until I find the problem.)
function(keys, values, rereduce) {
if(rereduce)
return sum(values);
// ... then the same code as before.
}
I will elaborate on my count/sum comment, just in case you are curious.
This code is not tested, but hopefully you will get the idea. The end result is always a simple object {"count":C, "sum":S} and you know the average by computing S / C.
function (key, values, rereduce) {
// Reduce function
var count = 0;
var sum = 0;
var i;
if(!rereduce) {
// `values` stores actual map output
for(i = 0; i < values.length; i++) {
count += Number(values[i][1]);
sum += Number(values[i][0]);
}
return {"count":count, "sum":sum};
}
else {
// `values` stores count/sum objects returned previously.
for(i = 0; i < values.length; i++) {
count += values[i].count;
sum += values[i].sum;
}
return {"count":count, "sum":sum};
}
}
I use the following code to do average. Hope it helps.
function (key, values) {
return sum(values)/values.length;
}