I would like to find optimal hyperparamters for a specific function, I am using bayesopt routine in MATLAB.
I can set the variables to optimize like the following:
a = optimizableVariable('a',[0,1],'Type','integer');
But I have coupled variables, i.e, variables whose value depend on the existence of other variables, e.g., a={0,1}, b={0,1} iff a=1.
Meaning that b has an influence on the function if a==1.
I thought about creating a unique variables that encompasses all the possibilities, i.e., c=1 if a=0, c=2 if a=1,b=0, c=3 if a=1,b=0. The problem is that I am interested in optimizing continuous variables and the above approach does not hold anymore.
I tried something alone the line of
b = a * optimizableVariable('b',[0,1],'Type','integer');
But MATLAB threw an error.
Undefined operator '*' for input arguments of type 'optimizableVariable'.
After three months almost to the day, buried deep down in MATLAB documentation, the answer was to use constrained variables.
https://www.mathworks.com/help/stats/constraints-in-bayesian-optimization.html#bvaw2ar
Related
I have a differential equation that's a function of around 30 constants. The differential equation is a system of (N^2+1) equations (where N is typically 4). Solving this system produces N^2+1 functions.
Often I want to see how the solution of the differential equation functionally depends on constants. For example, I might want to plot the maximum value of one of the output functions and see how that maximum changes for each solution of the differential equation as I linearly increase one of the input constants.
Is there a particularly clean method of doing this?
Right now I turn my differential-equation-solving script into a large function that returns an array of output functions. (Some of the inputs are vectors & matrices). For example:
for i = 1:N
[OutputArray1(i, :), OutputArray2(i, :), OutputArray3(i, :), OutputArray4(i, :), OutputArray5(i, :)] = DE_Simulation(Parameter1Array(i));
end
Here I loop through the function. The function solves a differential equation, and then returns the set of solution functions for that input parameter, and then each is appended as a row to a matrix.
There are a few issues I have with my method:
If I want to see the solution to the differential equation for a different parameter, I have to redefine the function so that it is an input of one of the thirty other parameters. For the sake of code readability, I cannot see myself explicitly writing all of the input parameters as individual inputs. (Although I've read that structures might be helpful here, but I'm not sure how that would be implemented.)
I typically get lost in parameter space and often have to update the same parameter across multiple scripts. I have a script that runs the differential-equation-solving function, and I have a second script that plots the set of simulated data. (And I will save the local variables to a file so that I can load them explicitly for plotting, but I often get lost figuring out which file is associated with what set of parameters). The remaining parameters that are not in the input of the function are inside the function itself. I've tried making the parameters global, but doing so drastically slows down the speed of my code. Additionally, some of the inputs are arrays I would like to plot and see before running the solver. (Some of the inputs are time-dependent boundary conditions, and I often want to see what they look like first.)
I'm trying to figure out a good method for me to keep track of everything. I'm trying to come up with a smart method of saving generated figures with a file tag that displays all the parameters associated with that figure. I can save such a file as a notepad file with a generic tagging-number that's listed in the title of the figure, but I feel like this is an awkward system. It's particularly awkward because it's not easy to see what's different about a long list of 30+ parameters.
Overall, I feel as though what I'm doing is fairly simple, yet I feel as though I don't have a good coding methodology and consequently end up wasting a lot of time saving almost-identical functions and scripts to solve fairly simple tasks.
It seems like what you really want here is something that deals with N-D arrays instead of splitting up the outputs.
If all of the OutputArray_ variables have the same number of rows, then the line
for i = 1:N
[OutputArray1(i, :), OutputArray2(i, :), OutputArray3(i, :), OutputArray4(i, :), OutputArray5(i, :)] = DE_Simulation(Parameter1Array(i));
end
seems to suggest that what you really want your function to return is an M x K array (where in this case, K = 5), and you want to pack that output into an M x K x N array. That is, it seems like you'd want to refactor your DE_Simulation to give you something like
for i = 1:N
OutputArray(:,:,i) = DE_Simulation(Parameter1Array(i));
end
If they aren't the same size, then a struct or a table is probably the best way to go, as you could assign to one element of the struct array per loop iteration or one row of the table per loop iteration (the table approach would assume that the size of the variables doesn't change from iteration to iteration).
If, for some reason, you really need to have these as separate outputs (and perhaps later as separate inputs), then what you probably want is a cell array. In that case you'd be able to deal with the variable number of inputs doing something like
for i = 1:N
[OutputArray{i, 1:K}] = DE_Simulation(Parameter1Array(i));
end
I hesitate to even write that, though, because this almost certainly seems like the wrong data structure for what you're trying to do.
I know that in Matlab, there is a 'lazy' evaluation when a new variable is assigned to an existing one. Such as:
array1 = ones(1,1e8);
array2 = array1;
The value of array1 won't be copied to array2 unless the element of array2 is modified.
From this I supposed that all the variables in Matlab are actually value-type and are all passed by values (although lazy evaluation is used). This also implies that the variables are created on the call stack.
Well, I am not judging the way it treats the variables, although I have never seen a second programming language doing this way. I mean, for possibly large data structures such as arrays, treating it as value type and passing it by values does not seem to be a good idea. Though the lazy evaluation saves the space and time, it just seems strange to me. You may have an expression for mutating (instead of initialization or assignment) of a variable leading to an out-of-memory error. As far as I know, in C array names are actually pointers, and in Fortran, arrays are passed by reference. Most modern languages retreat arrays as reference type.
So, can anyone tell me why Matlab use such a not-so-common way to implement the arrays. Is it true that in Matlab, nothing is or can be created on the heap?
By the way, I have asked some experienced Matlab users about it. They simply say that they never change the variable once it is created, and use function call to create new variables. That means all the mutable data are treated immutable. Is there any gain or loss for programming in this way?
You're phrasing your question in a confusing way, using terms from programming languages such as C and FORTRAN that are misleading when applied to other languages.
There is a distinction between variables being passed by value or by reference, and variables having value semantics or reference semantics.
In C, variables can be passed by value, or they can be passed by reference using a pointer.
MATLAB does not have pointers. Whatever you've been told, MATLAB always passes variables by value. Since it does not have pointers, it doesn't make sense to ask whether it is passing variables by value or by reference - it must be by value.
Nevertheless, MATLAB variables can have either value semantics or reference semantics. In MATLAB, a variable with reference semantics is called a handle variable.
To emphasise - even if the variable is being passed by value, it can have either value or reference semantics.
When you create a regular variable:
>> a = 1;
The variable a has value semantics. What this means is that if you create another variable from it and then change the original, the new variable does not change.
>> b = a;
>> b
b =
1
>> a = 2;
>> b
b =
1
But if you create, for example, a figure:
>> f = figure;
The variable f has reference, or handle semantics. What this means is that if you create another variable from it and then change the original, the new variable also changes.
>> get(f, 'Name')
ans =
''
>> g = f;
>> set(f, 'Name', 'hello')
>> get(g, 'Name')
ans =
hello
When you define your own variable types using MATLAB OO classes, you can specify whether the objects of that class will have value or reference/handle semantics by inheriting the class from the built-in class handle.
Objects that are instances of value classes will behave similarly to a above; objects that are instances of handle classes will behave similarly to f above.
And they are both, always, passed by value.
I'm guessing at the underlying reason for your question: but I would recommend that you take a look into how to create handle classes. They will probably provide you with the variable behaviour that you're hoping to achieve (i.e. being able to pass it around, take a copy of it without increasing memory significantly, and it always refers to the same underlying thing).
If the "experienced MATLAB users" you have spoken to are using only value variables then they are losing a great deal - it is very often much more convenient to use handle variables. And I would actually bet that they are using them without realising it - pretty much all of MATLAB Handle Graphics relies on handle variables, like f above.
I believe the above is a complete explanation of the semantics of MATLAB variables. There are a couple of other wrinkles that confuse people, but they do not contradict the above:
Although MATLAB has pass-by-value behaviour (which, as explained above is different from whether variables have value or reference semantics), it also has lazy or copy-on-write behaviour. You describe this in your question, so you obviously get what it's doing, but it's simply an optimization that is a separate issue from the passing behaviour or variable semantics.
As mentioned in a comment by #Bernhard, if you implement functions using a syntax similar to x = myfun(x) rather than the more normal y = myfun(x), MATLAB can perform in-place optimizations on your code (i.e. overwriting the original variable rather than making a temporary copy) in some circumstances (in particular, the operations carried out on x within myfun have to be capable of being done in-place, such as arithmetic or trigonometric functions, not matrix operations like ' that would change the dimensions). But again, this is just an optimization, it doesn't change the semantics of the variables.
PS One more thing - stop thinking about the stack and the heap as well; there's not really an analogue in MATLAB, because you don't really have control over what area of memory your variables are stored in.
In my code, I have a line that looks like this:
f=#(test) bf{i}(5);
where bf is a cell array with functions from str2func() stored in it, i is a variable storing an integer, and the 5 is the argument to pass to the function. How can I get matlab to evaluate the line using the current value of i? Right now when I display f it outputs:
#(test)bf{i}(5)
Lets say i=1, I want it to output:
#(test)bf{1}(5)
Although technically the bf{1} should also be replaced with whatever function is stored in bf{1}. How can I force matlab to evaluate the variables in this statement?
When you create a function handle, the workspace variables are copied and the expression is evaluated when you call the function handle (Typically not a problem in memory consumption, matlab stores only changes).
Now the problem is, to tell Matlab when to evaluate what part of the expression.
If you are aiming for a better performance, pre-evaluate all constant parts of the function. Let's say your function is #(x)(g(3).*f(x)), in this case matlab would evaluate g(3) on every call.
Instead use:
f=#(x)(x.^2)
g_3=g(3)
h=#(x)(g_3.*f(x))
Now having the constant parts evaluated, you want to see the constants instead of the variabe name. I know two ways to achieve this.
You can use the symbolic toolbox, basically converting the function handle to a symbolic function, then to a function handle again. This not only displays the constants, but also substitutes f. This is not possible for all functions.
>> matlabFunction(h(sym('x')))
ans =
#(x)x.^2.*4.2e1
Another possibility is to use eval:
h=eval(['#(x)',sprintf('%e',g_3),'.*f(x)'])
Pre-evaluating constant parts of the expressions as I did in the first step is typically recommendable, but both solutions to get the constant visible in your function handle aren't really recommendable. The first solution using matlabFunction only applies to some functions, while the second comes with all the disadvantages of eval.
This question may initially appear similar to this other question but my situation is a little bit different.
I have a function 'deriv' that takes a symbolic expression as its input, then takes the first derivative of that symbolic expression. That derivative is then converted into an anonymous function using matlabFunction(), and is then evaluated over an array of points. I also use the anonymous function later in some other code.
The problem I'm having is that sometimes the input symbolic expression happens to be linear, and thus the derivative is constant; therefore the anonymous function is also a constant. When I evaluate the anonymous function over the array of points, I only get one output instead of an array of outputs.
Here's some code showing what I'm doing. For the sake of simplicity here, let's assume that the symbolic input expressions will involve only one symbolic variable called q.
function[derivFun,derivVals] = deriv(input)
derivSym = diff(input,q);
derivFun = matlabFunction(derivSym,'vars',q);
evalPoints = [1;2;3;4;5]; %in my true application, a much larger array
derivVals = derivFun(evalPoints);
end
So if the input is q^2, then the output derivVals will be [2;4;6;8;10]. But if the input happens to be, say, 3*q, then derivVals will be 3 (just a single scalar). What I'd like is for derivVals to be [3;3;3;3;3].
That is, I'd like derivVals to be the same size as evalPoints even if the input function happens to be linear (or constant). And I don't know ahead of time what the input expression will be.
Can anyone give suggestions for a scheme that would do that? I understand that a constant anonymous function will just return a single constant scalar, regardless of the size of its input. What I'm hoping for is perhaps some way to recognize when the anonymous function is constant and then still cause derivVals to be the same size as evalPoints.
I know that I could use a for loop to evaluate derivFun for every row of evalPoints, but I'd like to avoid using such a loop if possible.
Thank you for your time and consideration.
I think that this is a slightly simpler solution. The issue is that you're using matlabFunction, which simplifies down the equations and doesn't allow much customization. However, you can create an anonymous function of an anonymous function. Just add the this line right after your matlabFunction line:
derivFun = #(evalPoints)derivFun(evalPoints)+zeros(size(evalPoints));
This only evaluates the original derivFun once. However, I do like you symvar solution (just remember that adding zeros is always better than multiplying ones).
Not 100% sure I got the problem correctly.
Would this solve your issue?:
if isscalar(derivVals)
derivVals = repmat(derivVals, size(evalPoints));
end
when I am doing a function in Matlab. Sometimes I have equations and every one of these have constants. Then, I have to declare these constants inside my function. I wonder if there is a way to call the values of that constants from outside of the function, if I have their values on the workspace.
I don't want to write this values as inputs of my function in the function declaration.
In addition to the solutions provided by Iterator, which are all great, I think you have some other options.
First of all, I would like to warn you about global variables (as Iterator also did): these introduce hidden dependencies and make it much more cumbersome to reuse and debug your code. If your only concern is ease of use when calling the functions, I would suggest you pass along a struct containing those constants. That has the advantage that you can easily save those constants together. Unless you know what you're doing, do yourself a favor and stay away from global variables (and functions such as eval, evalin and assignin).
Next to global, evalin and passing structs, there is another mechanism for global state: preferences. These are to be used when it concerns a nearly immutable setting of your code. These are unfit for passing around actual raw data.
If all you want is a more or less clean syntax for calling a certain function, this can be achieved in a few different ways:
You could use a variable number of parameters. This is the best option when your constants have a default value. I will explain by means of an example, e.g. a regular sine wave y = A*sin(2*pi*t/T) (A is the amplitude, T the period). In MATLAB one would implement this as:
function y = sinewave(t,A,T)
y = A*sin(2*pi*t/T);
When calling this function, we need to provide all parameters. If we extend this function to something like the following, we can omit the A and T parameters:
function y = sinewave(t,A,T)
if nargin < 3
T = 1; % default period is 1
if nargin < 2
A = 1; % default amplitude 1
end
end
y = A*sin(2*pi*t/T);
This uses the construct nargin, if you want to know more, it is worthwhile to consult the MATLAB help for nargin, varargin, varargout and nargout. However, do note that you have to provide a value for A when you want to provide the value of T. There is a more convenient way to get even better behavior:
function y = sinewave(t,A,T)
if ~exists('T','var') || isempty(T)
T = 1; % default period is 1
end
if ~exists('A','var') || isempty(A)
A = 1; % default amplitude 1
end
y = A*sin(2*pi*t/T);
This has the benefits that it is more clear what is happening and you could omit A but still specify T (the same can be done for the previous example, but that gets complicated quite easily when you have a lot of parameters). You can do such things by calling sinewave(1:10,[],4) where A will retain it's default value. If an empty input should be valid, you should use another invalid input (e.g. NaN, inf or a negative value for a parameter that is known to be positive, ...).
Using the function above, all the following calls are equivalent:
t = rand(1,10);
y1 = sinewave(t,1,1);
y2 = sinewave(t,1);
y3 = sinewave(t);
If the parameters don't have default values, you could wrap the function into a function handle which fills in those parameters. This is something you might need to do when you are using some toolboxes that impose constraints onto the functions that are to be used. This is the case in the Optimization Toolbox.
I will consider the sinewave function again, but this time I use the first definition (i.e. without a variable number of parameters). Then you could work with a function handle:
f = #(x)(sinewave(x,1,1));
You can work with f as you would with an other function:
e.g. f(10) will evaluate sinewave(10,1,1).
That way you can write a general function (i.e. sinewave that is as general and simple as possible) but you create a function (handle) on the fly with the constants substituted. This allows you to work with that function, but also prevents global storage of data.
You can of course combine different solutions: e.g. create function handle to a function with a variable number of parameters that sets a certain global variable.
The easiest way to address this is via global variable:
http://www.mathworks.com/help/techdoc/ref/global.html
You can also get the values in other workspaces, including the base or parent workspace, but this is ill-advised, as you do not necessarily know what wraps a given function.
If you want to go that route, take a look at the evalin function:
http://www.mathworks.com/help/techdoc/ref/evalin.html
Still, the standard method is to pass all of the variables you need. You can put these into a struct, if you wish, and only pass the one struct.