I am trying to use the linear programming function from pythons scipy library however I am unable to remove a non negativity constrain placed on the variable. To demonstrate this consider the following code.
from scipy.optimize import linprog
c = [-1]
A = [[1]]
b = [-3]
print(linprog(c, A_ub=A, b_ub=b, bounds=None))
it gives the following output:
fun: 3.0
message: 'Optimization failed. Unable to find a feasible starting point.'
nit: 0
status: 2
success: False
x: nan
This should be a formulation of the following problem: minimize c*x such that Ax≤b or equivalently minimize -1*x st. 1x≤-3. Hopefully I have done so correctly. Based on the current output I suspect that there is an additional constraint that x≥0. I don't know how to remove this constraint.
I have set the bounds to None with the understanding that this means no additional bounds are placed on the problem other than Ax≤b however there is clearly some other bound being placed on the problem. How can I remove this bound? Thanks
Your not the first one to be confused by this--the docstring should explain this better.
When you use bounds=None, you are telling linprog to use the default behavior, which is to assume the nonnegative constraint. It is only by setting bounds to something else that the default behavior is changed. In this case bounds=(None, None) will remove the bound on each variable.
In [40]: from scipy.optimize import linprog
In [41]: c = [-1]
In [42]: A = [[1]]
In [43]: b = [-3]
In [44]: print(linprog(c, A_ub=A, b_ub=b, bounds=(None, None)))
con: array([], dtype=float64)
fun: 3.0
message: 'The solution was determined in presolve as there are no non-trivial constraints.'
nit: 0
slack: array([0.])
status: 0
success: True
x: array([-3.])
Related
I have an SGPR model:
import numpy as np
import gpflow
X, Y = np.random.randn(50, 2), np.random.randn(50, 1)
Z1 = np.random.randn(13, 2)
k = gpflow.kernels.SquaredExponential()
m = gpflow.models.SGPR(data=(X, Y), kernel=k, inducing_variable=Z1)
And I would like to assign inducing variable but with different shape, like:
Z2 = np.random.randn(29, 2)
m.inducing_variable.Z.assign(Z2)
But if I do it, I got:
ValueError: Shapes (13, 2) and (29, 2) are incompatible
is there a way to reassign the inducing variables without redefining the model?
Context: Instead of optimizing the model with the inducing variables, I would like to optimize the model without optimizing the inducing variables, manually reassigning the inducing variables at each step of the optimization.
UPDATE: This issue is resolved by https://github.com/GPflow/GPflow/pull/1594, which will become part of the next GPflow patch release (2.1.4).
With that fix, you don't need a custom class. All you need to do is explicitly set the static shape with None along the first dimension:
inducing_variable = gpflow.inducing_variables.InducingPoints(
tf.Variable(
Z1, # initial value
trainable=False, # True does not work - see Note below
shape=(None, Z1.shape[1]), # or even tf.TensorShape(None)
dtype=gpflow.default_float(), # required due to tf's 32bit default
)
)
m = gpflow.models.SGPR(data=(X, Y), kernel=k, inducing_variable=inducing_variable)
Then m.inducing_variable.Z.assign(Z2) should work just fine.
Note that in this case Z cannot be trainable, as the TensorFlow optimizers need to know the shape at construction time and don't support dynamic shapes.
Right now (as of GPflow 2.1.2) there is no built-in way to change the shape of inducing variables for SGPR, though it is in principle possible. You can get what you want with your own inducing variable class though:
class VariableInducingPoints(gpflow.inducing_variables.InducingPoints):
def __init__(self, Z, name=None):
super().__init__(Z, name=name)
# overwrite with Variable with None as first element in shape so
# we can assign arrays with arbitrary length along this dimension:
self.Z = tf.Variable(Z, dtype=gpflow.default_float(),
shape=(None, Z.shape[1])
)
def __len__(self):
return tf.shape(self.Z)[0] # dynamic shape
# instead of the static shape returned by the InducingPoints parent class
and then do
m = gpflow.models.SGPR(
data=(X, Y), kernel=k, inducing_variable=VariableInducingPoints(Z1)
)
instead. Then your m.inducing_variable.Z.assign() should work as you like it.
(For SVGP, the size of the inducing variable and the distribution defined by q_mu and q_sqrt has to match, as well as be known at construction time, so in this case changing the number of inducing variables is less trivial.)
I have the following cython function.
01:
+02: cdef int count_char_in_x(unicode x,Py_UCS4 c):
03: cdef:
+04: int count = 0
05: Py_UCS4 x_k
06:
+07: for x_k in x: ## Yellow
+08: if x_k == c:
+09: count+=1
10:
+11: return count
Line 07 is not properly optimized.
The annotated HTML code is expanded as:
+07: for x_k in x: ## Yellow
if (unlikely(__pyx_v_x == Py_None)) {
PyErr_SetString(PyExc_TypeError, "'NoneType' is not iterable");
__PYX_ERR(0, 8, __pyx_L1_error)
}
__Pyx_INCREF(__pyx_v_x);
__pyx_t_1 = __pyx_v_x;
__pyx_t_6 = __Pyx_init_unicode_iteration(__pyx_t_1, (&__pyx_t_3), (&__pyx_t_4), (&__pyx_t_5)); if (unlikely(__pyx_t_6 == ((int)-1))) __PYX_ERR(0, 8, __pyx_L1_error)
for (__pyx_t_7 = 0; __pyx_t_7 < __pyx_t_3; __pyx_t_7++) {
__pyx_t_2 = __pyx_t_7;
__pyx_v_x_k = __Pyx_PyUnicode_READ(__pyx_t_5, __pyx_t_4, __pyx_t_2);
Any tips on how could this be improved?
I think it is possible to write a cdef/cpdef function that at runtime completly avoids Python None type checks. Any idea on how this could be done?
The generated C code looks pretty good to me. The loop overall is a int-iterated for loop (i.e. it's not relying on calling the Python methods __iter__ and __next__).
__Pyx_PyUnicode_READ is translated pretty directly to PyUnicode_READ (depending slightly on the Python version you're using). PyUnicode_READ is a C macro which is as close to a direct array access as you can get.
This is probably as good as it's getting. You might get a small improvement by using bytes rather than unicode (provided you're dealing with ASCII characters). You might just consider whether it's really worth reimplementing unicode.count.
If it were a regular def function you could declare x as unicode not None to remove the None check before the loop. That might make a small difference. However, as #ead points out that isn't supported for cdef functions. It's likely the cost of a def function call will be slightly larger than the cost of a None-check, but you should time it if you care.
I am intentionally casting an array of boolean values to integers but I get this warning:
Warning: Extension: Conversion from LOGICAL(4) to INTEGER(4) at (1)
which I don't want. Can I either
(1) Turn off that warning in the Makefile?
or (more favorably)
(2) Explicitly make this cast in the code so that the compiler doesn't need to worry?
The code will looking something like this:
A = (B.eq.0)
where A and B are both size (n,1) integer arrays. B will be filled with integers ranging from 0 to 3. I need to use this type of command again later with something like A = (B.eq.1) and I need A to be an integer array where it is 1 if and only if B is the requested integer, otherwise it should be 0. These should act as boolean values (1 for .true., 0 for .false.), but I am going to be using them in matrix operations and summations where they will be converted to floating point values (when necessary) for division, so logical values are not optimal in this circumstance.
Specifically, I am looking for the fastest, most vectorized version of this command. It is easy to write a wrapper for testing elements, but I want this to be a vectorized operation for efficiency.
I am currently compiling with gfortran, but would like whatever methods are used to also work in ifort as I will be compiling with intel compilers down the road.
update:
Both merge and where work perfectly for the example in question. I will look into performance metrics on these and select the best for vectorization. I am also interested in how this will work with matrices, not just arrays, but that was not my original question so I will post a new one unless someone wants to expand their answer to how this might be adapted for matrices.
I have not found a compiler option to solve (1).
However, the type conversion is pretty simple. The documentation for gfortran specifies that .true. is mapped to 1, and false to 0.
Note that the conversion is not specified by the standard, and different values could be used by other compilers. Specifically, you should not depend on the exact values.
A simple merge will do the trick for scalars and arrays:
program test
integer :: int_sca, int_vec(3)
logical :: log_sca, log_vec(3)
log_sca = .true.
log_vec = [ .true., .false., .true. ]
int_sca = merge( 1, 0, log_sca )
int_vec = merge( 1, 0, log_vec )
print *, int_sca
print *, int_vec
end program
To address your updated question, this is trivial to do with merge:
A = merge(1, 0, B == 0)
This can be performed on scalars and arrays of arbitrary dimensions. For the latter, this can easily be vectorized be the compiler. You should consult the manual of your compiler for that, though.
The where statement in Casey's answer can be extended in the same way.
Since you convert them to floats later on, why not assign them as floats right away? Assuming that A is real, this could look like:
A = merge(1., 0., B == 0)
Another method to compliment #AlexanderVogt is to use the where construct.
program test
implicit none
integer :: int_vec(5)
logical :: log_vec(5)
log_vec = [ .true., .true., .false., .true., .false. ]
where (log_vec)
int_vec = 1
elsewhere
int_vec = 0
end where
print *, log_vec
print *, int_vec
end program test
This will assign 1 to the elements of int_vec that correspond to true elements of log_vec and 0 to the others.
The where construct will work for any rank array.
For this particular example you could avoid the logical all together:
A=1-(3-B)/3
Of course not so good for readability, but it might be ok performance-wise.
Edit, running performance tests this is 2-3 x faster than the where construct, and of course absolutely standards conforming. In fact you can throw in an absolute value and generalize as:
integer,parameter :: h=huge(1)
A=1-(h-abs(B))/h
and still beat the where loop.
I have been using MATLAB fminunc function to solve my optimization problem. I want to try the minFunc package :
http://www.di.ens.fr/~mschmidt/Software/minFunc.html
When using fminunc, I defined a function funObj.m which gives me the objective value and the gradient at any point 'x'. It also takes in several external inputs say, {a,b,c} which are matrices. So the function prototype looks like :
function [objVal,G] = funObj(x,a,b,c)
I want to use the same setup in the minFunc package. From the examples, I figured this should work :
options.Method='lbfgs';
f = #(x)funObj(x,a,b,c);
x = minFunc(f,x_init,options);
But when I call this way, I get an error as:
Error using funObj
Too many output arguments.
What is the correct way to call minFunc for my case?
**EDIT : Alright, here is a sample function that I want to use with minFunc. Lets say I want to find the minimum of a*(b-x)^2, where a,b are scalar parameters and x being a scalar too. The MATLAB objective function will then look like :
function obj = testFunc(x,a,b)
obj = a*(b-x)^2;
The function call to minimize this using fminunc (in MATLAB ) is simply:
f = #(x)testFunc(x,a,b);
x = fminunc(f,x_init);
This gives me the minimum of x = 10. Now, How do I do the same using minFunc ?
"Note that by default minFunc assumes that the gradient is supplied, unless the 'numDiff' option is set to 1 (for forward-differencing) or 2 (for central-differencing)."
The error is because only one argument is returned by the function. You can either return the gradient as a second argument or turn on numerical differencing.
Agree with Mark. I think the simplest way to solve it is
minFunc(#testFunc, x_init, a, b, c)
In MATLAB temporary function can only have one return value. So f = #(x)testFunc(x,a,b); let your method drop gradient part every time. Because minFunc can accept extra paramters, you can pass a, b and c after x_init. I think this would work.
I am working on a project and have many functions to create and they do need lots of debugging so instead of just hitting the run button i have to go to command window and give a function call.
does MATLAB support assignment of default values to input arguments like python does?
In python
def some_fcn(arg1 = a, arg2 = b)
% THE CODE
if you now call it without passing the arguments it doesn't give errors but if you try the same in MATLAB it gives an error.
For assigning default values, one might find it easier to manage if you use exist function instead of nargin.
function f(arg1, arg2, arg3)
if ~exist('arg2', 'var')
arg2 = arg2Default;
end
The advantage is that if you change the order of arguments, you don't need to update this part of the code, but when you use nargin you have to start counting and updating numbers.
If you are writing a complex function that requires validation of inputs, default argument values, key-value pairs, passing options as structs etc., you could use the inputParser object. This solution is probably overkill for simple functions, but you might keep it in mind for your monster-function that solves equations, plots results and brings you coffee. It resembles a bit the things you can do with python's argparse module.
You configure an inputParser like so:
>> p = inputParser();
>> p.addRequired('x', #isfinite) % validation function
>> p.addOptional('y', 123) % default value
>> p.addParamValue('label', 'default') % default value
Inside a function, you would typically call it with p.parse(varargin{:}) and look for your parameters in p.Results. Some quick demonstration on the command line:
>> p.parse(44); disp(p.Results)
label: 'default'
x: 44
y: 123
>> p.parse()
Not enough input arguments.
>> p.parse(Inf)
Argument 'x' failed validation isfinite.
>> p.parse(44, 55); disp(p.Results)
label: 'default'
x: 44
y: 55
>> p.parse(13, 'label', 'hello'); disp(p.Results)
label: 'hello'
x: 13
y: 123
>> p.parse(88, 13, 'option', 12)
Argument 'option' did not match any valid parameter of the parser.
You can kind of do this with nargin
function out = some_fcn(arg1, arg2)
switch nargin
case 0
arg1 = a;
arg2 = b;
%//etc
end
but where are a and b coming from? Are they dynamically assigned? Because that effects the validity of this solution
After a few seconds of googling I found that as is often the case, Loren Shure has already solved this problem for us. In this article she outlines exactly my method above, why it is ugly and bad and how to do better.
You can use nargin in your function code to detect when no arguments are passed, and assign default values or do whatever you want in that case.
MathWorks has a new solution for this in R2019b, namely, the arguments block. There are a few rules for the arguments block, naturally, so I would encourage you to learn more by viewing the Function Argument Validation help page. Here is a quick example:
function ret = someFunction( x, y )
%SOMEFUNCTION Calculates some stuff.
arguments
x (1, :) double {mustBePositive}
y (2, 3) logical = true(2, 3)
end
% ...stuff is done, ret is defined, etc.
end
Wrapped into this is narginchk, inputParser, validateattributes, varargin, etc. It can be very convenient. Regarding default values, they are very simply defined as those arguments that equal something. In the example above, x isn't given an assignment, whereas y = true(2, 3) if no value is given when the function is called. If you wanted x to also have a default value, you could change it to, say, x (1, :) double {mustBePositive} = 0.5 * ones(1, 4).
There is a more in-depth answer at How to deal with name/value pairs of function arguments in MATLAB
that hopefully can spare you some headache in getting acquainted with the new functionality.