Related
I am beginning my journey of learning Rust. I came across this line in Rust by Example:
However, unlike macros in C and other languages, Rust macros are expanded into abstract syntax trees, rather than string preprocessing, so you don't get unexpected precedence bugs.
Why is an abstract syntax tree better than string preprocessing?
If you have this in C:
#define X(A,B) A+B
int r = X(1,2) * 3;
The value of r will be 7, because the preprocessor expands it to 1+2 * 3, which is 1+(2*3).
In Rust, you would have:
macro_rules! X { ($a:expr,$b:expr) => { $a+$b } }
let r = X!(1,2) * 3;
This will evaluate to 9, because the compiler will interpret the expansion as (1+2)*3. This is because the compiler knows that the result of the macro is supposed to be a complete, self-contained expression.
That said, the C macro could also be defined like so:
#define X(A,B) ((A)+(B))
This would avoid any non-obvious evaluation problems, including the arguments themselves being reinterpreted due to context. However, when you're using a macro, you can never be sure whether or not the macro has correctly accounted for every possible way it could be used, so it's hard to tell what any given macro expansion will do.
By using AST nodes instead of text, Rust ensures this ambiguity can't happen.
A classic example using the C preprocessor is
#define MUL(a, b) a * b
// ...
int res = MUL(x + y, 5);
The use of the macro will expand to
int res = x + y * 5;
which is very far from the expected
int res = (x + y) * 5;
This happens because the C preprocessor really just does simple text-based substitutions, it's not really an integral part of the language itself. Preprocessing and parsing are two separate steps.
If the preprocessor instead parsed the macro like the rest of the compiler, which happens for languages where macros are part of the actual language syntax, this is no longer a problem as things like precedence (as mentioned) and associativity are taken into account.
This question already has answers here:
Is there a way to count with macros?
(4 answers)
Counting length of repetition in macro
(3 answers)
Using a macro to initialize a big array of non-Copy elements
(3 answers)
Closed 6 years ago.
Say we need to declare a fixed size array with values, where the size of the array is defined by a constant that may change depending on compile time settings.
So for example:
let my_array = expand_into_array!(j, ARRAY_SIZE, -v0[j] * f);
Where ARRAY_SIZE is a constant, for example:
const ARRAY_SIZE: usize = 3;
Could expand into something like...
let my_array = [
{let j = 0; {-v0[j] * f}},
{let j = 1; {-v0[j] * f}},
{let j = 2; {-v0[j] * f}},
];
Since the expression is a fixed size array, it may be possible to use pattern matching, for a limited number of items ... up to 32 for example.
Is it possible to write a macro that expands an expression N times, based on a constant integer?
Details...
Looking into this, I wrote a macro which defines an array, then fills it in, eg:
const ARRAY_SIZE: usize = 3;
macro_rules! expand_into_array {
($index_var:ident, $const_size:expr, $body:expr) => {
{
let mut tmp: [_; $const_size] = [0.0; $const_size];
for $index_var in 0..$const_size {
tmp[$index_var] = $body;
}
// TODO, check $body _never_ breaks.
tmp
}
}
}
pub fn negated_array(v0: &[f64; ARRAY_SIZE]) -> [f64; ARRAY_SIZE] {
expand_into_array!(j, ARRAY_SIZE, {
-v0[j]
})
}
This works as expected, and besides the wrinkle (that the $body expression could include a break). this works without problems.
However initializing the array to 0.0 isn't getting optimized out (changing this value shows up as changes when run with: cargo rustc --release -- --emit asm
I'd rather not use unsafe { std::mem::uninitialized }.
Update, from asking another question, it seems macros can only match against literals, and not constants.
So this is simply not possible with macros in Rust.
I'm trying to write a DAC macro that gets as input the name of list of bits and its size, and the name of integer variable. Every element in the list should be constrained to be equal to every bit in the variable (both of the same length), i.e. (for list name list_of_bits and variable name foo and their length is 4) the macro's output should be:
keep list_of_bits[0] == foo[0:0];
keep list_of_bits[1] == foo[1:1];
keep list_of_bits[2] == foo[2:2];
keep list_of_bits[3] == foo[3:3];
My macro's code is:
define <keep_all_bits'exp> "keep_all_bits <list_size'exp> <num'name> <list_name'name>" as computed {
for i from 0 to (<list_size'exp> - 1) do {
result = appendf("%s keep %s[%d] == %s[%d:%d];",result, <list_name'name>, index, <num'name>, index, index);
};
};
The error I get:
*** Error: The type of '<list_size'exp>' is 'string', while expecting a
numeric type
...
for i from 0 to (<list_size'exp> - 1) do {
Why it interprets the <list_size'exp> as string?
Thank you for your help
All macro arguments in DAC macros are considered strings (except repetitions, which are considered lists of strings).
The point is that a macro treats its input purely syntactically, and it has no semantic information about the arguments. For example, in case of an expression (<exp>) the macro is unable to actually evaluate the expression and compute its value at compilation time, or even to figure out its type. This information is figured out at later compilation phases.
In your case, I would assume that the size is always a constant. So, first of all, you can use <num> instead of <exp> for that macro argument, and use as_a() to convert it to the actual number. The difference between <exp> and <num> is that <num> allows only constant numbers and not any expressions; but it's still treated as a string inside the macro.
Another important point: your macro itself should be a <struct_member> macro rather than an <exp> macro, because this construct itself is a struct member (namely, a constraint) and not an expression.
And one more thing: to ensure that the list size will be exactly as needed, add another constraint for the list size.
So, the improved macro can look like this:
define <keep_all_bits'struct_member> "keep_all_bits <list_size'num> <num'name> <list_name'name>" as computed {
result = appendf("keep %s.size() == %s;", <list_name'name>, <list_size'num>);
for i from 0 to (<list_size'num>.as_a(int) - 1) do {
result = appendf("%s keep %s[%d] == %s[%d:%d];",result, <list_name'name>, i, <num'name>, i, i);
};
};
Why not write is without macro?
keep for each in list_of_bits {
it == foo[index:index];
};
This should do the same, but look more readable and easier to debug; also the generation engine might take some advantage of more concise constraint.
I'm working on a verification-tool for some VHDL-Code in MATLAB/Octave. Therefore I need data types which generate "real" overflows:
intmax('int32') + 1
ans = -2147483648
Later on, it would be helpful if I can define the bit width of a variable, but that is not so important right now.
When I build a C-like example, where a variable gets increased until it's smaller than zero, it spins forever and ever:
test = int32(2^30);
while (test > 0)
test = test + int32(1);
end
Another approach I tried was a custom "overflow"-routine which was called every time after a number is changed. This approach was painfully slow, not practicable and not working in all cases at all. Any suggestions?
In MATLAB, one option you have is to overload the methods that handle arithmetic operations for integer data types, creating your own custom overflow behavior that will result in a "wrap-around" of the integer value. As stated in the documentation:
You can define or overload your own
methods for int* (as you can for any
object) by placing the appropriately
named method in an #int* folder within
a folder on your path. Type help
datatypes for the names of the methods
you can overload.
This page of the documentation lists the equivalent methods for the arithmetic operators. The binary addition operation A+B is actually handled by the function plus(A,B). Therefore, you can create a folder called #int32 (placed in another folder on your MATLAB path) and put a function plus.m in there that will be used instead of the built-in method for int32 data types.
Here's an example of how you could design your overloaded plus function in order to create the overflow/underflow behavior you want:
function C = plus(A,B)
%# NOTE: This code sample is designed to work for scalar values of
%# the inputs. If one or more of the inputs is non-scalar,
%# the code below will need to be vectorized to accommodate,
%# and error checking of the input sizes will be needed.
if (A > 0) && (B > (intmax-A)) %# An overflow condition
C = builtin('plus',intmin,...
B-(intmax-A)-1); %# Wraps around to negative
elseif (A < 0) && (B < (intmin-A)) %# An underflow condition
C = builtin('plus',intmax,...
B-(intmin-A-1)); %# Wraps around to positive
else
C = builtin('plus',A,B); %# No problems; call the built-in plus.m
end
end
Notice that I call the built-in plus method (using the BUILTIN function) to perform addition of int32 values that I know will not suffer overflow/underflow problems. If I were to instead perform the integer addition using the operation A+B it would result in a recursive call to my overloaded plus method, which could lead to additional computational overhead or (in the worst-case scenario where the last line was C = A+B;) infinite recursion.
Here's a test, showing the wrap-around overflow behavior in action:
>> A = int32(2147483642); %# A value close to INTMAX
>> for i = 1:10, A = A+1; disp(A); end
2147483643
2147483644
2147483645
2147483646
2147483647 %# INTMAX
-2147483648 %# INTMIN
-2147483647
-2147483646
-2147483645
-2147483644
If you want to get C style numeric operations, you can use a MEX function to call the C operators directly, and by definition they'll work like C data types.
This method is a lot more work than gnovice's overrides, but it should integrate better into a large codebase and is safer than altering the definition for built-in types, so I think it should be mentioned for completeness.
Here's a MEX file which performs the C "+" operation on a Matlab array. Make one of these for each operator you want C-style behavior on.
/* c_plus.c - MEX function: C-style (not Matlab-style) "+" operation */
#include "mex.h"
#include "matrix.h"
#include <stdio.h>
void mexFunction(
int nlhs, mxArray *plhs[],
int nrhs, const mxArray *prhs[]
)
{
mxArray *out;
/* In production code, input/output type and bounds checks would go here. */
const mxArray *a = prhs[0];
const mxArray *b = prhs[1];
int i, n;
int *a_int32, *b_int32, *out_int32;
short *a_int16, *b_int16, *out_int16;
mxClassID datatype = mxGetClassID(a);
int n_a = mxGetNumberOfElements(a);
int n_b = mxGetNumberOfElements(b);
int a_is_scalar = n_a == 1;
int b_is_scalar = n_b == 1;
n = n_a >= n_b ? n_a : n_b;
out = mxCreateNumericArray(mxGetNumberOfDimensions(a), mxGetDimensions(a),
datatype, mxIsComplex(a));
switch (datatype) {
case mxINT32_CLASS:
a_int32 = (int*) mxGetData(a);
b_int32 = (int*) mxGetData(b);
out_int32 = (int*) mxGetData(out);
for (i=0; i<n; i++) {
if (a_is_scalar) {
out_int32[i] = a_int32[i] + b_int32[i];
} else if (b_is_scalar) {
out_int32[i] = a_int32[i] + b_int32[0];
} else {
out_int32[i] = a_int32[i] + b_int32[i];
}
}
break;
case mxINT16_CLASS:
a_int16 = (short*) mxGetData(a);
b_int16 = (short*) mxGetData(b);
out_int16 = (short*) mxGetData(out);
for (i=0; i<n; i++) {
if (a_is_scalar) {
out_int16[i] = a_int16[0] + b_int16[i];
} else if (b_is_scalar) {
out_int16[i] = a_int16[i] + b_int16[0];
} else {
out_int16[i] = a_int16[i] + b_int16[i];
}
}
break;
/* Yes, you'd have to add a separate case for every numeric mxClassID... */
/* In C++ you could do it with a template. */
default:
mexErrMsgTxt("Unsupported array type");
break;
}
plhs[0] = out;
}
Then you have to figure out how to invoke it from your Matlab code. If you're writing all the code, you could just call "c_plus(a, b)" instead of "a + b" everywhere. Alternately, you could create your own numeric wrapper class, e.g. #cnumeric, that holds a Matlab numeric array in its field and defines plus() and other operations that invoke the approprate C style MEX function.
classdef cnumeric
properties
x % the underlying Matlab numeric array
end
methods
function obj = cnumeric(x)
obj.x = x;
end
function out = plus(a,b)
[a,b] = promote(a, b); % for convenience, and to mimic Matlab implicit promotion
if ~isequal(class(a.x), class(b.x))
error('inputs must have same wrapped type');
end
out_x = c_plus(a.x, b.x);
out = cnumeric(out_x);
end
% You'd have to define the math operations that you want normal
% Matlab behavior on, too
function out = minus(a,b)
[a,b] = promote(a, b);
out = cnumeric(a.x - b.x);
end
function display(obj)
fprintf('%s = \ncnumeric: %s\n', inputname(1), num2str(obj.x));
end
function [a,b] = promote(a,b)
%PROMOTE Implicit promotion of numeric to cnumeric and doubles to int
if isnumeric(a); a = cnumeric(a); end
if isnumeric(b); b = cnumeric(b); end
if isinteger(a.x) && isa(b.x, 'double')
b.x = cast(b.x, class(a.x));
end
if isinteger(b.x) && isa(a.x, 'double')
a.x = cast(a.x, class(b.x));
end
end
end
end
Then wrap your numbers in the #cnumeric where you want C-style int behavior and do math with them.
>> cnumeric(int32(intmax))
ans =
cnumeric: 2147483647
>> cnumeric(int32(intmax)) - 1
ans =
cnumeric: 2147483646
>> cnumeric(int32(intmax)) + 1
ans =
cnumeric: -2147483648
>> cnumeric(int16(intmax('int16')))
ans =
cnumeric: 32767
>> cnumeric(int16(intmax('int16'))) + 1
ans =
cnumeric: -32768
There's your C-style overflow behavior, isolated from breaking the primitive #int32 type. Plus, you can pass a #cnumeric object in to other functions that are expecting regular numerics and it'll "work" as long as they treat their inputs polymorphically.
Performance caveat: because this is an object, + will have the slower speed of a method dispatch instead of a builtin. If you have few calls on large arrays, this'll be fast, because the actual numeric operations are in C. Lots of calls on small arrays, could slow things down, because you're paying the per method call overhead a lot.
I ran the following snippet of code
test = int32(2^31-12);
for i = 1:24
test = test + int32(1)
end
with unexpected results. It seems that, for Matlab, intmax('int32')+1==intmax('int32'). I'm running 2010a on a 64-bit Mac OS X.
Not sure that this as an answer, more confirmation that Matlab behaves counterintuitively. However, the documentation for the intmax() function states:
Any value larger than the value returned by intmax saturates to the intmax value when cast to a 32-bit integer.
So I guess Matlab is behaving as documented.
Hm, yes...
Actually, I was able to solve the problem with my custom "overflow"-Subroutine... Now it runs painfully slow, but without unexpected behaviour! My mistake was a missing round(), since Matlab/Octave will introduce small errors.
But if someone knows a faster solution, I would be glad to try it!
function ret = overflow_sg(arg,bw)
% remove possible rounding errors, and prepare returnvalue (if number is inside boundaries, nothing will happen)
ret = round(arg);
argsize = size(ret);
for i = 1:argsize(1)
for j = 1:argsize(2)
ret(i,j) = flow_sg(ret(i,j),bw);
end
end
end%function
%---
function ret = flow_sg(arg,bw)
ret = arg;
while (ret < (-2^(bw-1)))
ret = ret + 2^bw;
end
% Check for overflows:
while (ret > (2^(bw-1)-1))
ret = ret - 2^bw;
end
end%function
If 64 bits is enough to not overflow, and you need a lot of these, perhaps do this:
function ret = overflow_sg(arg,bw)
mask = int64(0);
for i=1:round(bw)
mask = bitset(mask,i);
end
topbit = bitshift(int64(1),round(bw-1));
subfrom = double(bitshift(topbit,1))
ret = bitand( int64(arg) , mask );
i = (ret >= topbit);
ret(i) = int64(double(ret(i))-subfrom);
if (bw<=32)
ret = int32(ret);
end
end
Almost everything is done as a matrix calculation, and a lot is done with bits, and everything is done in one step (no while loops), so it should be pretty fast. If you're going to populate it with rand, subtract 0.5 since it assumes it should round to integer values (rather than truncate).
I'm not a Java expert, but underlying Java classes available in Matlab should allow handling of overflows like C would. One solution I found, works only for single value, but it converts a number to the int16 (Short) or int32 (Integer) representation. You must do your math using Matlab double, then convert to Java int16 or int32, then convert back to Matlab double. Unfortunately Java doesn't appear to support unsigned types in this way, only signed.
double(java.lang.Short(hex2dec('7FFF')))
<br>ans = 32767
double(java.lang.Short(hex2dec('7FFF')+1))
<br>ans = -32768
double(java.lang.Short(double(intmax('int16'))+1))
<br>ans = -32768
double(java.lang.Integer(hex2dec('7FFF')+1))
<br>ans = 32768
https://www.tutorialspoint.com/java/lang/java_lang_integer.htm
I have a HTTP connector in my iPhone project and queries must have a parameter set from the username using the Fowler–Noll–Vo (FNV) Hash.
I have a Java implementation working at this time, this is the code :
long fnv_prime = 0x811C9DC5;
long hash = 0;
for(int i = 0; i < str.length(); i++)
{
hash *= fnv_prime;
hash ^= str.charAt(i);
}
Now on the iPhone side, I did this :
int64_t fnv_prime = 0x811C9DC5;
int64_T hash = 0;
for (int i=0; i < [myString length]; i++)
{
hash *= fnv_prime;
hash ^= [myString characterAtIndex:i];
}
This script doesn't give me the same result has the Java one.
In first loop, I get this :
hash = 0
hash = 100 (first letter is "d")
hash = 1865261300 (for hash = 100 and fnv_prime = -2128831035 like in Java)
Do someone see something I'm missing ?
Thanks in advance for the help !
In Java, this line:
long fnv_prime = 0x811C9DC5;
will yield in fnv_prime the numerical value -2128831035, because the constant is interpreted as an int, which is a 32-bit signed value in Java. That value is then sign-extended when written in a long.
Conversely, in the Objective-C code:
int64_t fnv_prime = 0x811C9DC5;
the 0x811C9DC5 is interpreted as an unsigned int constant (because it does not fit in a signed 32-bit int), with numerical value 2166136261. That value is then written into fnv_prime, and there is no sign to extend since, as far as the C compiler is concerned, the value is positive.
Thus you end up with distinct values for fnv_prime, which explains your distinct results.
This can be corrected in Java by adding a "L" suffix, like this:
long fnv_prime = 0x811C9DC5L;
which forces the Java compiler to interpret the constant as a long, with the same numerical value than what you get with the Objective-C code.
Incidentally, 0x811C9DC5 is not a FNV prime (it is not even prime); it is the 32 bit FNV "offset basis". You will get incorrect hash values if you use this value (and more hash collisions). The correct value for the 32 bit FNV prime is 0x1000193. See http://www.isthe.com/chongo/tech/comp/fnv/index.html
It is a difference in sign extension assigning the 32-bit value 0x811C9DC5 to a 64-bit var.
Are the characters in Java and Objective-c the same? NSString will give you unichars.