I want to integrate a matrix such that each element of the output matrix is integral of the corresponding element of the integrand matrix. Code snippets are as below:
import numpy as np
from scipy.integrate import quad
N=3
A = np.random.rand(N,N)
evs = np.linalg.eigvals(A)
evs = -np.sort(-evs)
Anew = A/(evs[0]+1) - np.eye(N)
B = np.eye(N)
def integrand(t,A,B):
prod = np.multiply(sp.linalg.expm(A*t),B)
return np.multiply(prod,prod.T)
This gives a square matrix with each element a function of t. I use the following to integrate:
np.vectorize(quad)(integrand,0,1,args=(Anew,B))
However, I receive the following error message:
integrand() missing 1 required positional argument: 'B'
Although this states that 'B' is missing, I don't understand it as I am providing B as an argument. I am also not sure if I am implementing vectorization correctly.
Try scipy.integrate.quad_vec. It's not yet released, so you'll need the in-development version of scipy, which is available from github. Or, wait until scipy 1.4 is released.
Related
I'd like to build a GP with marginalized hyperparameters.
I have seen that this is possible with the HMC sampler provided in gpflow from this notebook
However, when I tried to run the following code as a first step of this (NOTE this is on gpflow 0.5, an older version), the returned samples are negative, even though the lengthscale and variance need to be positive (negative values would be meaningless).
import numpy as np
from matplotlib import pyplot as plt
import gpflow
from gpflow import hmc
X = np.linspace(-3, 3, 20)
Y = np.random.exponential(np.sin(X) ** 2)
Y = (Y - np.mean(Y)) / np.std(Y)
k = gpflow.kernels.Matern32(1, lengthscales=.2, ARD=False)
m = gpflow.gpr.GPR(X[:, None], Y[:, None], k)
m.kern.lengthscales.prior = gpflow.priors.Gamma(1., 1.)
m.kern.variance.prior = gpflow.priors.Gamma(1., 1.)
# dont want likelihood be a hyperparam now so fixed
m.likelihood.variance = 1e-6
m.likelihood.variance.fixed = True
m.optimize(maxiter=1000)
samples = m.sample(500)
print(samples)
Output:
[[-0.43764571 -0.22753325]
[-0.50418501 -0.11070128]
[-0.5932655 0.00821438]
[-0.70217714 0.05077999]
[-0.77745654 0.09362291]
[-0.79404456 0.13649446]
[-0.83989415 0.27118385]
[-0.90355789 0.29589641]
...
I don't know too much in detail about HMC sampling but I would expect that the sampled posterior hyperparameters are positive, I've checked the code and it seems maybe related to the Log1pe transform, though I failed to figure it out myself.
Any hint on this?
It would be helpful if you specified which GPflow version you are using - especially given that from the output you posted it looks like you are using a really old version of GPflow (pre-1.0), and this is actually something that got improved since. What is happening here (in old GPflow) is that the sample() method returns a single array S x P, where S is the number of samples, and P is the number of free parameters [e.g. for a M x M matrix parameter with lower-triangular transform (such as the Cholesky of the covariance of the approximate posterior, q_sqrt), only M * (M - 1)/2 parameters are actually stored and optimised!]. These are the values in the unconstrained space, i.e. they can take any value whatsoever. Transforms (see gpflow.transforms module) provide the mapping between this value (between plus/minus infinity) and the constrained value (e.g. gpflow.transforms.positive for lengthscales and variances). In old GPflow, the model provides a get_samples_df() method that takes the S x P array returned by sample() and returns a pandas DataFrame with columns for all the trainable parameters which would be what you want. Or, ideally, you would just use a recent version of GPflow, in which the HMC sampler directly returns the DataFrame!
I am trying to have a lambda layer in keras that performs a vector matrix multiplication, before passing it to another layer. The matrix is fixed (I don't want to learn it). Code below:
model.add(Dropout(0.1))
model.add(Lambda(lambda x: x.dot(A)))
model.add(Dense(output_shape, activation='softmax'))
model.compile(<stuff here>)}
A is the fixed matrix, and I want to do x.dot(A)
WHen I run this, I get the following error:
'Tensor' object has no attribute 'dot'
Same Error when I replace dot with matmul (I am using tensorflow backend)
Finally, when I replace the lambda layer by
model.add(Lambda(lambda x: x*A))
I get the error below:
model.add(Lambda(lambda x: x*G))
model.add(Dense(output_shape, activation='softmax'))
AttributeError: 'tuple' object has no attribute '_dims'
I'm new to Keras so any help will be appreciated. Thanks
I think you can add a Dense layer with the initial weight being the matrix A, and set the arguments trainable=False and use_bias=False. This layer will be equivalent to a fixed matrix multiplication.
model.add(Dense(A.shape[1], trainable=False, weights=[A], use_bias=False))
Create a function for the lambda:
import keras.backend as K
import numpy as np
numpyA = np.array(define A correctly here, with 2 dimensions)
def multA(x):
A = K.variable(numpyA)
return K.dot(x,A)
model.add(Lambda(multA))
I want to run a regression analysis on below data, here x1 and x2 produce y value. But in that case, y value is fixed in all time. So regression will not happen. But why? Need explanation.
Your training set shows that the coefficients are all ~0 and the constant is 5. There's no more information in that dataset, you don't need regression to show that.
You did not specify what kind of regression you are running. Depending on the type of regression you are using, you will need the matrices to be invertible and not be related linearly.
It seems to work using normal equation (with expected results):
import numpy as np
import matplotlib.pyplot as plt
input = np.array([
[2,3,5],
[1,2,5],
[4,2,5],
[1,7,5],
[1,9,5]
])
m = len(input)
X = np.array([np.ones(m), input[:, 0],input[:, 1]]).T # Add Constant to X
y = np.array(input[:, 2]).reshape(-1, 1) # Get the dependant values
betaHat = np.linalg.solve(X.T.dot(X), X.T.dot(y)) # Calculate coefficients
print(betaHat) # Show Constant and coefficients (in that order)
[[ 5.00000000e+00]
[ 5.29208238e-16]
[ 4.32685981e-17]]
Matlab fit is no doubt useful but it is not clear how to use it as a function
apart from trivial integration and differentiation given on the official website:
http://uk.mathworks.com/help/curvefit/example-differentiating-and-integrating-a-fit.html
For example given a fit stored in the object 'curve' one can evaluate
curve(x) to get a number. But how one would, e.g. integrate |curve(x)|^2 (apart from clumsily creating a new fit)? Trying naively
curve = fit(x_vals,y_vals,'smoothingspline');
integral(curve(x)*curve(x), 0, 1)
gives an error:
Output of the function must be the same size as the input. If FUN is an array-valued integrand, set the 'ArrayValued' option to true.
I have also tried a work around by definining a normal function and an implicit function for the integrand (below) but both give the same error.
func=#(x)(curve(x))...; % trial solution 1
function func_val=func(curve, x)...; % trial solution 2
Defining function for the integrand followed by integration with option 'ArrayValued' set to 'true' works:
func=#(x)(curve(x)*curve(x));
integral(func,0,1,'ArrayValued',true)
You need to have the function vectorized, i.e use element-wise operations like curve(x).*curve(x) or curve(x).^2.
Also make sure that the shape of the output matches the input, i.e a row input gives a row output, similarly a column comes out as a column. It seems that evaluating the fit object always returns a column vector (e.g. f(1:10) returns a 10x1 vector not 1x10).
With that said, here is an example:
x = linspace(0,4*pi,100)';
y = sin(x);
y = y + 0.5*y.*randn(size(y));
f = fit(x, y, 'smoothingspline');
now you can integrate as:
integral(#(x) reshape(f(x).^2,size(x)), 0, 1)
in this case, it can be simplified as a simple transpose:
integral(#(x) (f(x).^2)', 0, 1)
here is my code:
C=#(k) [k,k,2.*k;3,2.*k,5;1,k,k];
AV=#(k,t) [3*t, 6, 9]*C(k)*[3*t ;6 ;9];
avaint=#(k,a,b) quadgk(#(k) AV(k,t),a,b);
AVAR=#(t) avaint(t,0,87600);
Is shows:
Error using vertcat
Dimensions of matrices being concatenated are not consistent.
when I want to print AVAR(3)
Source of Error
The quadgk function passes a vector of integration points to the function handle given to it.
From the documentation:
The function y = fun(x) should accept a vector argument x and return a vector result y, where y is the integrand evaluated at each element of x.
This creates the dimension mismatch causing the error.
Solutions
To get around this implementation, you can perform the numerical integration using the integral function with the ('ArrayValued',true) option pair:
avaint = #(t,a,b) integral(#(k) AV(k,t),a,b,'ArrayValued',true);
Or, you can use arrayfun within AV to abide by the requirement of quadgk:
AV = #(k,t) arrayfun(#(k_el) [3*t, 6, 9]*(C(k_el)*[3*t ;6 ;9]),k);