I need to compute some fairly long expressions that contain common subexpressions. For example, consider the following two expressions:
double dfdx1 = 2 * (-x2 + x1 - sin(b2)*n34 + cos(b2)*sin(c2)*n24 - cos(b2)*cos(c2)*n14 + sin(b1)*m34 - cos(b1)*sin(c1)*m24 + cos(b1)*cos(c1)*m14);
double dfdx2 = -2 * (-x2 + x1 - sin(b2)*n34 + cos(b2)*sin(c2)*n24 - cos(b2)*cos(c2)*n14 + sin(b1)*m34 - cos(b1)*sin(c1)*m24 + cos(b1)*cos(c1)*m14);
Aside from eliminating all the trigonometric functions, one obvious elimination is dfdx2 = -dfdx1. The question is whether the compiler will recognise this. I found that using MATLAB's MuPad generate::optimize() function does not, which rather surprised me.
More generally, will the compiler recognise that f2 = -f1 in the example below:
double f1 = a*a + b*b - c*a - c*b;
double f2 = c*a + c*b - a*a - b*b;
Or will it just eliminate the terms a*a, b*b, c*a and c*b?
I am using the MSVC compiler, but I guess that they all do pretty much the same thing.
Normally compilers should recognize this and carry out the asked transformation if you enable "fast math" (-ffast-math for gcc). The reason is that floating point operations are not perfectly exact and the order of the evaluation of an expression might matter.
Example (for doubles, all constants are actually considered being results of other operations):
"1e100"+"1.0"-"1e100" results in 0.0
"1e100"-"1e100"+"1.0" results in 1.0
So the compiler will only reorder expressions if you explicitly allow such transformations.
Related
I've been stuck on this for quite some time. Is there a command in R that will create a Box Cox Linear transformation that has a different a parameter for my independent variable and dependent variable?
chicago.reg <- lm(data = Chichagodata, sprice ~ nrooms + lvarea + hage + lsize + ptaxes + sspend + mspend +
medinc + dfcl + particle + sulfur+pctwht+ dfni +aircon+ garage+ nbath + cook+ohare)
Note that I do not want to transform my variables pctwht, dfni, aircon, garage, nbath, cook,and ohare as they are dummy variables.
Thank you if you can provide me insight on this issue.
You should have a look into coxBox and boxcox functions. The former allow to transform individual variables/lists, while the latter works on lm (or aov) objects. In addition you might want to read this and this posts...
I am trying to take the derivative of a function including a boolean variable with sympy.
My expected result:
Two different derivatives, depending on the boolean being either True or False (i.e. 1 or 0).
Example:
import sympy as sy
c, x = sy.symbols("c x", positive=True, real=True)
bo = sy.Function("bo")
fct1 = sy.Function("fct1")
fct2 = sy.Function("fct2")
FOC2 = sy.Function("FOC2")
y = 5
a = 2
b = 4
def fct1(x):
return -0.004*x**2 + 0.25*x + 4
# the following gives the smaller positive intercept with the x-axis)
# this intercept is the threshold value for the boolean function, bo
min(sy.solve(fct1(x)-y, x))
def bo(x):
if fct1(x) <= y:
return 1
else:
return 0
def fct2(c, x):
return a + b*c + bo(x)*c
def FOC2(c, x):
return sy.diff(fct2(c, x), c)
print(FOC2(c, x))
The min-function after the comments shows me the threshold of x for bo being True or False would be 4.29..., thus positive and real.
Output:
TypeError: cannot determine truth value of Relation
I understand that the truth value depends on x, which is a symbol. Thus, without knowing x one cannot determine bo.
But how would I get my expected result, where bo is symbolic?
First off, I would advise you to carefully consider what is going on in your code the way it is pasted above. You first define a few sympy functions, e.g.
fct1 = sy.Function("fct1")
So after this, fct1 is an undefined sympy.Function - undefined in the sense that it is neither specified what its arguments are, nor what the function looks like.
However, then you define same-named functions explicitly, as in
def fct1(x):
return -0.004*x**2 + 0.25*x + 4
Note however, that at this point, fct1 ceases to be a sympy.Function, or any sympy object for that matter: you overwrite the old definition, and it is now just a regular python function!
This is also the reason that you get the error: when you call bo(x), python tries to evaluate
-0.004*x**2 + 0.25*x + 4 <= 5
and return a value according to your definition of bo(). But python does not know whether the above is true (or how to make that comparison), so it complains.
I would suggest 2 changes:
Instead of python functions, as in the code, you could simply use sympy expressions, e.g.
fct1 = -0.004*x**2 + 0.25*x + 4
To get the truth value of your condition, I would suggest to use the Heaviside function (wiki), which evaluates to 0 for a negative argument, and to 1 for positive. Its implementation in sympy is sympy.Heaviside.
Your code could then look as follows:
import sympy as sy
c, x = sy.symbols("c x", positive=True, real=True)
y = 5
a = 2
b = 4
fct1 = -0.004*x**2 + 0.25*x + 4
bo = sy.Heaviside(y - fct1)
fct2 = a + b*c + bo * c
FOC2 = sy.diff(fct2, c)
print(FOC2)
Two comments on the line
bo = sy.Heaviside(y - fct1)
(1) The current implementation does not evaluate sympy.Heaviside(0)by default; this is beacause there's differing definitions around (some define it to be 1, others 1/2). You'd want it to be 1, to be in accordance with the (weak) inequality in the OP. In sympy 1.1, this can be achieved by passing an additional argument to Heaviside, namely whatever you want Heaviside(0) to evaluate to:
bo = sy.Heaviside(y - fct1, 1)
This is not supported in older versions of sympy.
(2) You will get your FOC2, again involving a Heaviside term. What I like about this, is that you could keep working with this expression, say if you wanted to take a second derivative and so on. If, for the sake of readability, you would prefer a piecewise expression - no problem. Just replace the according line with
bo = sy.Heaviside(y - fct1)._eval_rewrite_as_Piecewise(y-fct1)
Which will translate to a piecewise function automatically. (note that under older versions, this automatically implicitly uses Heaviside(0) = 0.5 - best to use (1) and (2) together:
bo = sy.Heaviside(y - fct1, 1)._eval_rewrite_as_Piecewise(y-fct1)
Unfortunately, I don't have a working sympy 1.1 at my hands right now and can only test the old code.
One more noteconcerning sympy's piecewise functions: they are much more readable if using sympy's latex printing, by inserting
sy.init_printing()
early in the code.
(Disclaimer: I am by no means an expert in sympy, and there might be other, preferable solutions out there. Just trying to make a suggestion!)
So, I have a really long symbolic exrpression in Matlab that I want to copy/paste into a JavaScript-code to animate a numerical solution for it. The problem is that some places in my code i get exponents (mostly ^2), when I'd rather have Matlab express it as A*A.
I have mulitple (and different) expressions like
cos(th2t)^2
That I would rather have expressed as
cos(th2t)*cos(th2t)
Any way I can do this? The alternative is to use a text editor afterward and search for powers of 2 and replace it, but there are multiple different expressions, so that would take some time...
This is an example of one of the exrpressions I end up with:
(J31*(2*ddth2t*cos(th1t) - 2*w12*w13 - 4*dth1t*dth2t*sin(th1t) - 4*dth2t*w13*sin(th1t) + dth1t^2*sin(2*th2t)*cos(th1t) - w12^2*sin(2*th2t)*cos(th1t) + w13^2*sin(2*th2t)*cos(th1t) + 2*dth2t*w11*sin(2*th2t) + 2*w12*w13*cos(th2t)^2 + ddth1t*sin(2*th2t)*sin(th1t) + 4*dth1t*dth2t*cos(th2t)^2*sin(th1t) + 2*dth1t*w13*sin(2*th2t)*cos(th1t) + 4*dth2t*w13*cos(th2t)^2*sin(th1t) + w11*w12*sin(2*th2t)*sin(th1t) + 4*dth1t*w12*cos(th1t)^2*cos(th2t)^2 - 2*dth1t*w11*sin(2*th1t)*cos(th2t)^2 - 2*dth2t*w11*sin(2*th2t)*cos(th1t)^2 + 2*w12*w13*cos(th1t)^2*cos(th2t)^2 - w11*w13*sin(2*th1t)*cos(th2t)^2 - 4*dth2t*w12*cos(th1t)*cos(th2t)*sin(th1t)*sin(th2t)))/(2*(J11 + J31))
Steve's answer should work fine if you want to rely on the ** operator. However, since that operator is not officially supported, and that solution doesn't directly answer the OP's question, here is a function that can expand out the exponents in a symbolic expression.
function [ text ] = remove_powers( exp )
%Cleans the powers from T, and return expanded text representation
% Define functions
enclose =#(t) ['(' t ')'];
expand_pow=#(b,p) strjoin(strcat(cell(1,p),enclose(char(b))),'*');
count_pow=#(s) arrayfun(#(k) count(char(s(k)),'^'), 1:length(s));
sym2str = #(s) strrep(char(s), ' ', '');
% Check for fractions
[N,D]=numden(exp);
if ~isequal(D,sym(1))
% pass up the num and den
text = [remove_powers(N) '/' enclose(remove_powers(D))];
else
% Split a into subterms
Ts = children(exp);
% Clean children
has_pow=count_pow(Ts)>0;
text = sym2str(exp);
if sum(count_pow(Ts))<count_pow(exp)
% We have removed a power, clean it, expand it, and pass up
text = expand_pow(remove_powers(Ts(1)),Ts(2));
else
% Just clean the unclean children and pass up
for t=Ts(has_pow)
text = strrep(text,sym2str(t),remove_powers(t));
end
end
end
end
The function uses the children function in Matlab to recursively clean each subexpression and replace it (as text) in the parent. This method is better than using regex because it avoids the issue of parsing the syntax, as Steve mentioned in the comment with respect to cos(x^2+2*y)^2.
Which makes for a good example:
syms x y real
exp = cos(x^2+2*y)^2;
cleaned_exp = remove_powers(exp)
Outputs: (cos(2*y+(x)*(x)))*(cos(2*y+(x)*(x)))
Notice that since Matlab is doing the parsing, there was no need to parse the order of precedence for the '^' operators, which could be difficult to accomplish with regex.
To test the OP's example:
syms ddth1t dth1t th1t ddth2t dth2t th2t w11 w12 w13 J11 J31 real
exp = (J31*(2*ddth2t*cos(th1t) - 2*w12*w13 - 4*dth1t*dth2t*sin(th1t) - 4*dth2t*w13*sin(th1t) + dth1t^2*sin(2*th2t)*cos(th1t) - w12^2*sin(2*th2t)*cos(th1t) + w13^2*sin(2*th2t)*cos(th1t) + 2*dth2t*w11*sin(2*th2t) + 2*w12*w13*cos(th2t)^2 + ddth1t*sin(2*th2t)*sin(th1t) + 4*dth1t*dth2t*cos(th2t)^2*sin(th1t) + 2*dth1t*w13*sin(2*th2t)*cos(th1t) + 4*dth2t*w13*cos(th2t)^2*sin(th1t) + w11*w12*sin(2*th2t)*sin(th1t) + 4*dth1t*w12*cos(th1t)^2*cos(th2t)^2 - 2*dth1t*w11*sin(2*th1t)*cos(th2t)^2 - 2*dth2t*w11*sin(2*th2t)*cos(th1t)^2 + 2*w12*w13*cos(th1t)^2*cos(th2t)^2 - w11*w13*sin(2*th1t)*cos(th2t)^2 - 4*dth2t*w12*cos(th1t)*cos(th2t)*sin(th1t)*sin(th2t)))/(2*(J11 + J31));
cleaned_exp = remove_powers(exp);
isequal(sym(cleaned_exp),exp) % This should be 1
count(cleaned_exp,'^') % This should be 0
As expected, the new expression is equivalent to the original, but has no '^' symbols.
This answer suggests that you could call the exponential operator in Javascript with e.g. A**2. As such you could replace all instances of ^ with **.
(Solution from comments)
If I start with the following symbolic expression:
a^2 + 2*a*b + b^2
Then run simplify (or factor), I get the expected result:
>> simplify(a^2 + 2*a*b + b^2)
(a + b)^2
Now when I run the same example, but adding another term, no factorisation occurs:
>> simplify(a^2 + 2*a*b + b^2 + 1)
a^2 + 2*a*b + b^2 + 1
How can I get these functions to return the more practical version of this expression ((a + b)^2 + 1)? I have tried all of the obvious options with these functions (like 'Steps', 'IgnoreAnalyticConstraints', etc.) but to no avail.
Context: I have the expression ax^2 - 2*ax*bx + bx^2 + ay^2 - 2*ay*by + by^2 which I need to convert back into (ax - bx)^2 + (ay - by)^2 so it can then be treated correctly as r^2. I know I could use some blunt substitution rules, but for something so simple I feel like I'm missing an obvious 'non-hack' solution.
you can run simplify on the two terms separately.
simplify(ax^2 - 2*ax*bx + bx^2) + simplify(ay^2 - 2*ay*by + by^2)
It seems like you already know how it should be simplified anyway.
Also, you eventually want to write it as r^2. This is not generally possible for all second-order expressions, so don't bother trying to find a general solution.
So Mathematica is different from other dialects of lisp because it blurs the lines between functions and macros. In Mathematica if a user wanted to write a mathematical function they would likely use pattern matching like f[x_]:= x*x instead of f=Function[{x},x*x] though both would return the same result when called with f[x]. My understanding is that the first approach is something equivalent to a lisp macro and in my experience is favored because of the more concise syntax.
So I have two questions, is there a performance difference between executing functions versus the pattern matching/macro approach? Though part of me wouldn't be surprised if functions were actually transformed into some version of macros to allow features like Listable to be implemented.
The reason I care about this question is because of the recent set of questions (1) (2) about trying to catch Mathematica errors in large programs. If most of the computations were defined in terms of Functions, it seems to me that keeping track of the order of evaluation and where the error originated would be easier than trying to catch the error after the input has been rewritten by the successive application of macros/patterns.
The way I understand Mathematica is that it is one giant search replace engine. All functions, variables, and other assignments are essentially stored as rules and during evaluation Mathematica goes through this global rule base and applies them until the resulting expression stops changing.
It follows that the fewer times you have to go through the list of rules the faster the evaluation. Looking at what happens using Trace (using gdelfino's function g and h)
In[1]:= Trace#(#*#)&#x
Out[1]= {x x,x^2}
In[2]:= Trace#g#x
Out[2]= {g[x],x x,x^2}
In[3]:= Trace#h#x
Out[3]= {{h,Function[{x},x x]},Function[{x},x x][x],x x,x^2}
it becomes clear why anonymous functions are fastest and why using Function introduces additional overhead over a simple SetDelayed. I recommend looking at the introduction of Leonid Shifrin's excellent book, where these concepts are explained in some detail.
I have on occasion constructed a Dispatch table of all the functions I need and manually applied it to my starting expression. This provides a significant speed increase over normal evaluation as none of Mathematica's inbuilt functions need to be matched against my expression.
My understanding is that the first approach is something equivalent to a lisp macro and in my experience is favored because of the more concise syntax.
Not really. Mathematica is a term rewriter, as are Lisp macros.
So I have two questions, is there a performance difference between executing functions versus the pattern matching/macro approach?
Yes. Note that you are never really "executing functions" in Mathematica. You are just applying rewrite rules to change one expression into another.
Consider mapping the Sqrt function over a packed array of floating point numbers. The fastest solution in Mathematica is to apply the Sqrt function directly to the packed array because it happens to implement exactly what we want and is optimized for this special case:
In[1] := N#Range[100000];
In[2] := Sqrt[xs]; // AbsoluteTiming
Out[2] = {0.0060000, Null}
We might define a global rewrite rule that has terms of the form sqrt[x] rewritten to Sqrt[x] such that the square root will be calculated:
In[3] := Clear[sqrt];
sqrt[x_] := Sqrt[x];
Map[sqrt, xs]; // AbsoluteTiming
Out[3] = {0.4800007, Null}
Note that this is ~100× slower than the previous solution.
Alternatively, we might define a global rewrite rule that replaces the symbol sqrt with a lambda function that invokes Sqrt:
In[4] := Clear[sqrt];
sqrt = Function[{x}, Sqrt[x]];
Map[sqrt, xs]; // AbsoluteTiming
Out[4] = {0.0500000, Null}
Note that this is ~10× faster than the previous solution.
Why? Because the slow second solution is looking up the rewrite rule sqrt[x_] :> Sqrt[x] in the inner loop (for each element of the array) whereas the fast third solution looks up the value Function[...] of the symbol sqrt once and then applies that lambda function repeatedly. In contrast, the fastest first solution is a loop calling sqrt written in C. So searching the global rewrite rules is extremely expensive and term rewriting is expensive.
If so, why is Sqrt ever fast? You might expect a 2× slowdown instead of 10× because we've replaced one lookup for Sqrt with two lookups for sqrt and Sqrt in the inner loop but this is not so because Sqrt has the special status of being a built-in function that will be matched in the core of the Mathematica term rewriter itself rather than via the general-purpose global rewrite table.
Other people have described much smaller performance differences between similar functions. I believe the performance differences in those cases are just minor differences in the exact implementation of Mathematica's internals. The biggest issue with Mathematica is the global rewrite table. In particular, this is where Mathematica diverges from traditional term-level interpreters.
You can learn a lot about Mathematica's performance by writing mini Mathematica implementations. In this case, the above solutions might be compiled to (for example) F#. The array may be created like this:
> let xs = [|1.0..100000.0|];;
...
The built-in sqrt function can be converted into a closure and given to the map function like this:
> Array.map sqrt xs;;
Real: 00:00:00.006, CPU: 00:00:00.015, GC gen0: 0, gen1: 0, gen2: 0
...
This takes 6ms just like Sqrt[xs] in Mathematica. But that is to be expected because this code has been JIT compiled down to machine code by .NET for fast evaluation.
Looking up rewrite rules in Mathematica's global rewrite table is similar to looking up the closure in a dictionary keyed on its function name. Such a dictionary can be constructed like this in F#:
> open System.Collections.Generic;;
> let fns = Dictionary<string, (obj -> obj)>(dict["sqrt", unbox >> sqrt >> box]);;
This is similar to the DownValues data structure in Mathematica, except that we aren't searching multiple resulting rules for the first to match on the function arguments.
The program then becomes:
> Array.map (fun x -> fns.["sqrt"] (box x)) xs;;
Real: 00:00:00.044, CPU: 00:00:00.031, GC gen0: 0, gen1: 0, gen2: 0
...
Note that we get a similar 10× performance degradation due to the hash table lookup in the inner loop.
An alternative would be to store the DownValues associated with a symbol in the symbol itself in order to avoid the hash table lookup.
We can even write a complete term rewriter in just a few lines of code. Terms may be expressed as values of the following type:
> type expr =
| Float of float
| Symbol of string
| Packed of float []
| Apply of expr * expr [];;
Note that Packed implements Mathematica's packed lists, i.e. unboxed arrays.
The following init function constructs a List with n elements using the function f, returning a Packed if every return value was a Float or a more general Apply(Symbol "List", ...) otherwise:
> let init n f =
let rec packed ys i =
if i=n then Packed ys else
match f i with
| Float y ->
ys.[i] <- y
packed ys (i+1)
| y ->
Apply(Symbol "List", Array.init n (fun j ->
if j<i then Float ys.[i]
elif j=i then y
else f j))
packed (Array.zeroCreate n) 0;;
val init : int -> (int -> expr) -> expr
The following rule function uses pattern matching to identify expressions that it can understand and replaces them with other expressions:
> let rec rule = function
| Apply(Symbol "Sqrt", [|Float x|]) ->
Float(sqrt x)
| Apply(Symbol "Map", [|f; Packed xs|]) ->
init xs.Length (fun i -> rule(Apply(f, [|Float xs.[i]|])))
| f -> f;;
val rule : expr -> expr
Note that the type of this function expr -> expr is characteristic of term rewriting: rewriting replaces expressions with other expressions rather than reducing them to values.
Our program can now be defined and executed by our custom term rewriter:
> rule (Apply(Symbol "Map", [|Symbol "Sqrt"; Packed xs|]));;
Real: 00:00:00.049, CPU: 00:00:00.046, GC gen0: 24, gen1: 0, gen2: 0
We've recovered the performance of Map[Sqrt, xs] in Mathematica!
We can even recover the performance of Sqrt[xs] by adding an appropriate rule:
| Apply(Symbol "Sqrt", [|Packed xs|]) ->
Packed(Array.map sqrt xs)
I wrote an article on term rewriting in F#.
Some measurements
Based on #gdelfino answer and comments by #rcollyer I made this small program:
j = # # + # # &;
g[x_] := x x + x x ;
h = Function[{x}, x x + x x ];
anon = Table[Timing[Do[ # # + # # &[i], {i, k}]][[1]], {k, 10^5, 10^6, 10^5}];
jj = Table[Timing[Do[ j[i], {i, k}]][[1]], {k, 10^5, 10^6, 10^5}];
gg = Table[Timing[Do[ g[i], {i, k}]][[1]], {k, 10^5, 10^6, 10^5}];
hh = Table[Timing[Do[ h[i], {i, k}]][[1]], {k, 10^5, 10^6, 10^5}];
ListLinePlot[ {anon, jj, gg, hh},
PlotStyle -> {Black, Red, Green, Blue},
PlotRange -> All]
The results are, at least for me, very surprising:
Any explanations? Please feel free to edit this answer (comments are a mess for long text)
Edit
Tested with the identity function f[x] = x to isolate the parsing from the actual evaluation. Results (same colors):
Note: results are very similar to this Plot for constant functions (f[x]:=1);
Pattern matching seems faster:
In[1]:= g[x_] := x*x
In[2]:= h = Function[{x}, x*x];
In[3]:= Do[h[RandomInteger[100]], {1000000}] // Timing
Out[3]= {1.53927, Null}
In[4]:= Do[g[RandomInteger[100]], {1000000}] // Timing
Out[4]= {1.15919, Null}
Pattern matching is also more flexible as it allows you to overload a definition:
In[5]:= g[x_] := x * x
In[6]:= g[x_,y_] := x * y
For simple functions you can compile to get the best performance:
In[7]:= k[x_] = Compile[{x}, x*x]
In[8]:= Do[k[RandomInteger[100]], {100000}] // Timing
Out[8]= {0.083517, Null}
You can use function recordSteps in previous answer to see what Mathematica actually does with Functions. It treats it just like any other Head. IE, suppose you have the following
f = Function[{x}, x + 2];
f[2]
It first transforms f[2] into
Function[{x}, x + 2][2]
At the next step, x+2 is transformed into 2+2. Essentially, "Function" evaluation behaves like an application of pattern matching rules, so it shouldn't be surprising that it's not faster.
You can think of everything in Mathematica as an expression, where evaluation is the process of rewriting parts of the expression in a predefined sequence, this applies to Function like to any other head