I am trying to do value function iteration for a capital structure problem, where the decision variables are capital(k) and debt(d) and the shock(a) is stochastic following an AR(1) process. The dynamics are because of the capital variable k, the current period capital depends on the previous period capital. I am using MATLAB but fminsearch is very slow, so I used a grid search approach where I made grids for a(the shock), previous period capital(k), this periods capital(kprime) and debt(d). The code looks are follows:
%%Do the policy iteration
disp('Doing the policy iteration')
%setting a few parameters
polmaxit = 50; %max number of policy iterations
accelmaxit = 250; %number of policy pushforwards
distmaxit = 500; %number of distributional pushforwards
disterrtol = 1e-8; %distributional error tolerance
%Initialize the arrays
kpolindold = zeros(na,nk,nd);
kpolind = kpolindold;
Vold = kpolindold;
V = kpolindold;
%Guess the inital policy
kpolindold(:) = floor(nk/2);
%loop over policy iterations
for polct = 1:polmaxit
%push forward the capital policy accelmaxit times
for accelct = 1:accelmaxit
for act = 1:na
for kct = 1:nk
for dct = 1:nd
%extract guessed policy
kprimect = kpolindold(act,kct,dct);
V(act,kct,dct) = Rmat(act,kct,kprimect,dct);
for aprimect = 1:na
V(act,kct,dct) = V(act,kct,dct) + disc * pi(act,aprimect)*Vold(aprimect,kprimect,dct);
end
end
end
end
Vold = V;
end
%update the policy
for act = 1:na
for kct = 1:nk
RHSvec = zeros(nk,1);
for kprimect = 1:nk
RHSvec(kprimect) = Rmat(act,kct,kprimect,dct);
for dct = 1:nd
for aprimect = 1:na
RHSvec(kprimect) = RHSvec(kprimect) + disc * pi(act,aprimect)*Vold(aprimect,kprimect,dct);
end
end
end
[RHSval,kprimeind] = max(RHSvec);
kpolind(act,kct,dct) = kprimeind;
V(act,kct,dct) = RHSval;
end
end
%{
if max(max(abs(imag(V)))) > 0
display 'Imag. g'
V = zeros(na,nk);
kpolind = zeros(na,nk);
errcode = 1;
return;
end
%}
%error calculations
kerrval = max(abs(kpolind(:)-kpolindold(:)));
Verrval = norm(abs(V(:)-Vold(:)));
%%output and check error values
disp(['Policy iteration ', num2str(polct) ',Pol Err = ' ...
num2str(kerrval) ',VF Err = ' num2str(Verrval)])
%output if the policy iteration has converged
if (Verrval < 1e-4)
break
end
%update policy if you haven't coonverged yet
kpolindold = kpolind;
end
I don't think I am properly optimizing over both the variables k and d. Can anyone help me optimize over both the variables?
Related
I'm using a code that calculates expectation value of probabilities. This code contains a while-loop that finds all possible combinations and adds up products of probability combinations. However, when the number of elements becomes large(over 40) it takes too much time, and I want to make the code faster.
The code is as follow-
function pcs = combsum(N,K,prbv)
nprbv=1-prbv; %prbv: probability vector
WV = 1:K; % Working vector.
lim = K; % Sets the limit for working index.
inc = 0; % Controls which element of WV is being worked on.
pcs = 0;
stopp=0;
while stopp==0
if logical((inc+lim)-N)
stp = inc; % This is where the for loop below stops.
flg = 0; % Used for resetting inc.
else
stp = 1;
flg = 1;
end
for jj = 1:stp
WV(K + jj - inc) = lim + jj; % Faster than a vector assignment.
end
PV=nprbv;
PV(WV)=prbv(WV);
pcs=prod(PV)+pcs;
inc = inc*flg + 1; % Increment the counter.
lim = WV(K - inc + 1 ); % lim for next run.
if (inc==K)&&(lim==N-K)
stopp=1;
WV = (N-K+1):N;
PV=nprbv;
PV(WV)=prbv(WV);
pcs=prod(PV)+pcs;
end
end
Is there a way to reduce calculation time? I wonder if parallel computing using GPU would help.
I tried to remove dependent variables in a loop for parallel computing, and I made a matrix of possible combinations using 'combnk' function. This worked faster.
nprbv=1-prbv; %prbv : a probability vector
N = 40;
K = 4;
n_combnk = size(combnk(1:N,K),1);
PV_mat = repmat(nprbv,n_combnk,1);
cnt = 0;
tic;
for i = 1:N-K+1
for j = i+1:N-K+2
for k = j+1:N-K+3
for l = k+1:N-K+4
cnt = cnt+1;
PV_mat(cnt,i) = prbv(i);
PV_mat(cnt,j) = prbv(j);
PV_mat(cnt,k) = prbv(k);
PV_mat(cnt,l) = prbv(l);
end
end
end
end
toc;
tic;
pcs_rr = sum(prod(PV_mat,2));
toc;
However, when K value gets larger, an out-of-memory problem happens in building a combination matrix(PV_mat). How can I break up the big matrix into small ones to avoid memory problem?
I am programming a simple perceptron in matlab but it is not converging and I can't figure out why.
The goal is to binary classify 2D points.
%P1 Generate a dataset of two-dimensional points, and choose a random line in
%the plane as your target function f, where one side of the line maps to +1 and
%the other side to -1. Let the inputs xn 2 R2 be random points in the plane,
%and evaluate the target function f on each xn to get the corresponding output
%yn = f(xn).
clear all;
clc
clear
n = 20;
inputSize = 2; %number of inputs
dataset = generateRandom2DPointsDataset(n)';
[f , m , b] = targetFunction();
signs = classify(dataset,m,b);
weights=ones(1,2)*0.1;
threshold = 0;
fprintf('weights before:%d,%d\n',weights);
mistakes = 1;
numIterations = 0;
figure;
plotpv(dataset',(signs+1)/2);%mapping signs from -1:1 to 0:1 in order to use plotpv
hold on;
line(f(1,:),f(2,:));
pause(1)
while true
mistakes = 0;
for i = 1:n
if dataset(i,:)*weights' > threshold
result = 1;
else
result = -1;
end
error = signs(i) - result;
if error ~= 0
mistakes = mistakes + 1;
for j = 1:inputSize
weights(j) = weights(j) + error*dataset(i,j);
end
end
numIterations = numIterations + 1
end
if mistakes == 0
break
end
end
fprintf('weights after:%d,%d\n',weights);
random points and signs are fine since plotpv is working well
The code is based on that http://es.mathworks.com/matlabcentral/fileexchange/32949-a-perceptron-learns-to-perform-a-binary-nand-function?focused=5200056&tab=function.
When I pause the infinite loop, this is the status of my vairables:
I am not able to see why it is not converging.
Additional code( it is fine, just to avoid answers asking for that )
function [f,m,b] = targetFunction()
f = rand(2,2);
f(1,1) = 0;
f(1,2) = 1;
m = (f(2,2) - f(2,1));
b = f(2,1);
end
function dataset = generateRandom2DPointsDataset(n)
dataset = rand(2,n);
end
function values = classify(dataset,m,b)
for i=1:size(dataset,1)
y = m*dataset(i,1) + b;
if dataset(i,2) >= y, values(i) = 1;
else values(i) = -1;
end
end
end
I am computing a list of error statistics of a variable u. Whenever I put the structures in the loop, Matlab becomes superslow, while it is pretty fast with standard variables. I have heard that with Matlab 2016b one can either use loops or use vectorized notation, the speed should be the same. It looks like it does not work for structures. Do you know why? Best suggestion here? This is a minimal example ( I just used trivial "if" clauses, mine are more complex and that's the main reason why I don't vectorize):
n = 1000,
m = 1000;
uMODEL = rand(n,m);
uOBS = rand(n,m);
%
%
%
tic
ERRORS = struct('Emedio',0,'MAE',0,'sigma',0,'Emax',0,'Emin',0);
Nerr(1:max(n,1),1) = 0; %Nerr at most it contains n
fields = fieldnames(ERRORS);
for i = 1:numel(fields)
ERRORS.(fields{i}) = zeros(max(n,1),1);
end
for i=1:m
for j=1:n
if (1000>2 && 2000<343532)
diff = uMODEL(j,i)-uOBS(j,i);
ERRORS.Emedio(j) = ERRORS.Emedio(j) + diff;
ERRORS.MAE(j) = ERRORS.MAE(j) + abs(diff);
ERRORS.sigma(j) = ERRORS.sigma(j) + diff^2;
ERRORS.Emax(j) = max(ERRORS.Emax(j),diff);
ERRORS.Emin(j) = min(ERRORS.Emin(j),diff);
Nerr(n) = Nerr(n) + 1;
end
end
end
ERRORS.Emedio(:) = ERRORS.Emedio(:)./Nerr(:);
ERRORS.MAE(:) = ERRORS.MAE(:)./Nerr(:);
ERRORS.sigma(:) = sqrt(ERRORS.sigma(:)./(Nerr(:)-1));
toc
clear ERRORS
tic
%
% here instead I define variables, I fill them up and then I throw them in structures, cause loops with structures are strangely slow...
Emedio = zeros(max(n,1),1);
MAE = zeros(max(n,1),1);
sigma = zeros(max(n,1),1);
Nerr = zeros(max(n,1),1);
Emax = zeros(max(n,1),1);
Emin = zeros(max(n,1),1);
for i=1:m
for j=1:n
if (1000>2 && 2000<343532)
diff = uMODEL(j,i)-uOBS(j,i);
Emedio(j) = Emedio(j) + diff;
MAE(j) = MAE(j) + abs(diff);
sigma(j) = sigma(j) + diff^2;
Emax(j) = max(Emax(j),diff);
Emin(j) = min(Emin(j),diff);
Nerr(n) = Nerr(n) + 1;
end
end
end
Emedio(:) = Emedio(:)./Nerr(:);
MAE(:) = MAE(:)./Nerr(:);
sigma(:) = sqrt(sigma(:)./(Nerr(:)-1));
ERRORS.Emedio(:) = Emedio(:);
ERRORS.MAE(:) = MAE(:);
ERRORS.sigma(:) = sigma(:);
ERRORS.Emax(:) = Emax(:);
ERRORS.Emin(:) = Emin(:);
toc
the output is:
Elapsed time is 2.372765 seconds.
Elapsed time is 0.057719 seconds.
`sol = pdepe(m,#ParticleDiffusionpde,#ParticleDiffusionic,#ParticleDiffusionbc,x,t);
% Extract the first solution component as u.
u = sol(:,:,:);
function [c,f,s] = ParticleDiffusionpde(x,t,u,DuDx)
global Ds
c = 1/Ds;
f = DuDx;
s = 0;
function u0 = ParticleDiffusionic(x)
global qo
u0 = qo;
function [pl,ql,pr,qr] = ParticleDiffusionbc(xl,ul,xr,ur,t,x)
global Ds K n
global Amo Gc kf rhop
global uavg
global dr R nr
sum = 0;
for i = 1:1:nr-1
r1 = (i-1)*dr; % radius at i
r2 = i * dr; % radius at i+1
r1 = double(r1); % convert to double precision
r2 = double(r2);
sum = sum + (dr / 2 * (r1*ul+ r2*ur));
end;
uavg = 3/R^3 * sum;
ql = 1;
pl = 0;
qr = 1;
pr = -((kf/(Ds.*rhop)).*(Amo - Gc.*uavg - ((double(ur/K)).^2).^(n/2) ));`
dq(r,t)/dt = Ds( d2q(r,t)/dr2 + (2/r)*dq(r,t)/dr )
q(r, t=0) = 0
dq(r=0, t)/dr = 0
dq(r=dp/2, t)/dr = (kf/Ds*rhop) [C(t) - Cp(at r = dp/2)]
q = solid phase concentration of trace compound in a particle with radius dp/2
C = bulk liquid concentration of trace compound
Cp = trace compound concentration at particle surface
I want to solve the above pde with initial and boundary conditions given. Tried Matlab's pdepe, but does not work satisfactorily. Maybe the boundary conditions is creating problem for me. I also used this isotherm equation for equilibrium: q = K*Cp^(1/n). This is convection-diffusion equation but i could not find any write ups that addresses solving this type of equation properly.
There are two problems with the current implementation.
Incorrect Source Term
The PDE you are attempting to solve has the form
which has the equivalent form
where the last term arises due to the factor of 2 in the original PDE.
The last term needs to be incorporated into pdepe via a source term.
Calculation of q average
The current implementation attempts to calculate the average value of q using the left and right values of q passed to the boundary condition function.
This is incorrect.
The average value of q needs to be calculated from a vector of up-to-date values of the quantity.
However, we have the complication that the only function to receive all mesh values is ParticleDiffusionpde; however, the mesh values passed to that function are not guaranteed to be from the mesh we provided.
Solution: use events (as described in the pdepe documentation).
This is a hack since the event function is meant to detect zero-crossings, but it has the advantage that the function is given all values of q on the mesh we provide.
So, the working example below (you'll notice I set all of the parameters to 1 since I didn't know better) uses the events function to update a variable qStore that can be accessed by the boundary condition function (see here for an explanation), and the boundary condition function performs a vectorized trapezoidal integration for the average calculation.
Working Example
function [] = ParticleDiffusion()
% Parameters
Ds = 1;
q0 = 0;
K = 1;
n = 1;
Amo = 1;
Gc = 1;
kf = 1;
rhop = 1;
% Space
rMesh = linspace(0,1,10);
rMesh = rMesh(:) ;
dr = rMesh(2) - rMesh(1) ;
% Time
tSpan = linspace(0,1,10);
% Vector to store current u-value
qStore = zeros(size(rMesh));
options.Events = #(m,t,x,y) events(m,t,x,y);
% Solve
[sol,~,~,~,~] = pdepe(1,#ParticleDiffusionpde,#ParticleDiffusionic,#ParticleDiffusionbc,rMesh,tSpan,options);
% Use the events function to update qStore
function [value,isterminal,direction] = events(m,~,~,y)
qStore = y; % Value of q on rMesh
value = m; % Since m is constant, it will never be zero (no event detection)
isterminal = 0; % Continue integration
direction = 0; % Detect all zero crossings (not important)
end
function [c,f,s] = ParticleDiffusionpde(r,~,~,DqDr)
% Define the capacity, flux, and source
c = 1/Ds;
f = DqDr;
s = DqDr./r;
end
function u0 = ParticleDiffusionic(~)
u0 = q0;
end
function [pl,ql,pr,qr] = ParticleDiffusionbc(~,~,R,ur,~)
% Calculate average value of current solution
qL = qStore(1:end-1);
qR = qStore(2: end );
total = sum((qL.*rMesh(1:end-1) + qR.*rMesh(2:end))) * dr/2;
qavg = 3/R^3 * total;
% Left boundary
pl = 0;
ql = 1;
% Right boundary
qr = 1;
pr = -(kf/(Ds.*rhop)).*(Amo - Gc.*qavg - (ur/K).^n);
end
end
I tried to use the modified version of NN back propagation code by Phil Brierley
(www.philbrierley.com). When i try to solve the XOR problem it works perfectly. but when i try to solve a problem of the form output = x1^2 + x2^2 (ouput = sum of squares of input), the results are not accurate. i have scaled the input and ouput between -1 and 1. I get different results every time i run the same program (i understand its due to random wts initialization), but results are very different. i tried changing learning rate but still results converge.
have given the code below
%---------------------------------------------------------
% MATLAB neural network backprop code
% by Phil Brierley
%--------------------------------------------------------
clear; clc; close all;
%user specified values
hidden_neurons = 4;
epochs = 20000;
input = [];
for i =-10:2.5:10
for j = -10:2.5:10
input = [input;i j];
end
end
output = (input(:,1).^2 + input(:,2).^2);
output1 = output;
% Maximum input and output limit and scaling factors
m1 = -10; m2 = 10;
m3 = 0; m4 = 250;
c = -1; d = 1;
%Scale input and output
for i =1:size(input,2)
I = input(:,i);
scaledI = ((d-c)*(I-m1) ./ (m2-m1)) + c;
input(:,i) = scaledI;
end
for i =1:size(output,2)
I = output(:,i);
scaledI = ((d-c)*(I-m3) ./ (m4-m3)) + c;
output(:,i) = scaledI;
end
train_inp = input;
train_out = output;
%read how many patterns and add bias
patterns = size(train_inp,1);
train_inp = [train_inp ones(patterns,1)];
%read how many inputs and initialize learning rate
inputs = size(train_inp,2);
hlr = 0.1;
%set initial random weights
weight_input_hidden = (randn(inputs,hidden_neurons) - 0.5)/10;
weight_hidden_output = (randn(1,hidden_neurons) - 0.5)/10;
%Training
err = zeros(1,epochs);
for iter = 1:epochs
alr = hlr;
blr = alr / 10;
%loop through the patterns, selecting randomly
for j = 1:patterns
%select a random pattern
patnum = round((rand * patterns) + 0.5);
if patnum > patterns
patnum = patterns;
elseif patnum < 1
patnum = 1;
end
%set the current pattern
this_pat = train_inp(patnum,:);
act = train_out(patnum,1);
%calculate the current error for this pattern
hval = (tanh(this_pat*weight_input_hidden))';
pred = hval'*weight_hidden_output';
error = pred - act;
% adjust weight hidden - output
delta_HO = error.*blr .*hval;
weight_hidden_output = weight_hidden_output - delta_HO';
% adjust the weights input - hidden
delta_IH= alr.*error.*weight_hidden_output'.*(1-(hval.^2))*this_pat;
weight_input_hidden = weight_input_hidden - delta_IH';
end
% -- another epoch finished
%compute overall network error at end of each epoch
pred = weight_hidden_output*tanh(train_inp*weight_input_hidden)';
error = pred' - train_out;
err(iter) = ((sum(error.^2))^0.5);
%stop if error is small
if err(iter) < 0.001
fprintf('converged at epoch: %d\n',iter);
break
end
end
%Output after training
pred = weight_hidden_output*tanh(train_inp*weight_input_hidden)';
Y = m3 + (m4-m3)*(pred-c)./(d-c);
% Testing for a new set of input
input_test = [6 -3.1; 0.5 1; -2 3; 3 -2; -4 5; 0.5 4; 6 1.5];
output_test = (input_test(:,1).^2 + input_test(:,2).^2);
input1 = input_test;
%Scale input
for i =1:size(input1,2)
I = input1(:,i);
scaledI = ((d-c)*(I-m1) ./ (m2-m1)) + c;
input1(:,i) = scaledI;
end
%Predict output
train_inp1 = input1;
patterns = size(train_inp1,1);
bias = ones(patterns,1);
train_inp1 = [train_inp1 bias];
pred1 = weight_hidden_output*tanh(train_inp1*weight_input_hidden)';
%Rescale
Y1 = m3 + (m4-m3)*(pred1-c)./(d-c);
analy_numer = [output_test Y1']
plot(err)
This is the sample output i get for problem
state after 20000 epochs
analy_numer =
45.6100 46.3174
1.2500 -2.9457
13.0000 11.9958
13.0000 9.7097
41.0000 44.9447
16.2500 17.1100
38.2500 43.9815
if i run once more i get different results. as can be observed for small values of input i get totally wrong ans (negative ans not possible). for other values accuracy is still poor.
can someone tell what i am doing wrong and how to correct.
thanks
raman