How to obtain transition probability matrix in MATLAB? - matlab

Question

Suppose I have a sequence x= 1,3,3,1,2,1,4,2,3,1,4,2,4,4,4,3,1,2,5,1 and it has five states 1 3 2 4 5. I have to obtain transition probability matrix in MATLAB by this equation, probability= (Number of observation pairs x(t) & x(t+1), with x(t) in state i and x(t+1) in state j)/(Number of observation pairs x(t) & x(t+1), with x(t) in state i and x(t+1) in any one of the states 1......s).
I tried by this code but it giving error
x=[1 3 3 1 2 1 4 2 3 1 4 2 4 4 4 3 1 2 5 1]
n = length(x)-1
p = zeros(5,5)
for t = 1:n
if x(t)=x(t+1);
a(t)=count (x(t)=x(t+1)) % Here i am trying to count how many number of times pair of that states occur in sequence.
q(t)=sum(x==x(t)) % (Here i am trying to count Number of observation pairs x(t) & x(t+1), with x(t) in state i and x(t+1) in any one of the states 1......s)
end
for i=1:5
p(i, :) = a(t)/q(t)
end
Transition probability matrix calculated manually by me as follows
1 3 2 4 5
1 0 1/5 2/5 2/5 0
3 3/4 1/4 0 0 0
2 1/4 1/4 0 1/4 1/4
4 0 1/5 2/5 2/5 0
5 1 0 0 0 0

Answer 1

Since it has been a while, I think it is safe to provide an answer to this now. There is no toolbox required for either approach below. The approach assumes basic knowledge of a transition probability matrix of a Discrete Time Markov Chain (DTMC).
Both approaches use the unique() function to find the statespace. Note that the order is different, e.g. your [1 3 2 4 5] vs. my [1 2 3 4 5] but that isn't a limiting issue. I've separated getting the transition counts from the transition probabilities to illustrate some techniques.
Approach 1: Vectorized Approach
This approach uses the unique() and accumarray() functions.
% MATLAB 2018b
X =[1 3 3 1 2 1 4 2 3 1 4 2 4 4 4 3 1 2 5 1];
[u,~,n] = unique(X);
NumU = length(u); % Number of Unique Observations
Counts = accumarray([n(1:end-1),n(2:end)],1,[NumU,NumU]);
P = Counts./sum(Counts,2); % Probability transition matrix
Verification: You can verify that sum(sum(Counts)) == length(X)-1 and the rows of P sum to one (sum(P,2)).
Notice that the counts matrix uses a 1-step offset to count the transitions. The output is a NumU x NumU array of the number of transitions in terms of indices as given in the n-output from unique().
Approach 2: Single for loop
This is a direct approach that can use any ordering of the statespace (see below).
States = unique(X); % <--- can be anything
% e.g. try: States = [1 3 2 4 5];
Counts = zeros(length(States));
for k = 2:length(X)
Counts(find(X(k-1) == States),find(X(k) == States)) = ...
Counts(find(X(k-1) == States),find(X(k) == States)) + 1;
end
P = Counts./sum(Counts,2); % Probability transition matrix
Using your statespace ordering: If you use Approach 2 with States = [1 3 2 4 5];, the resulting probability transition matrix, P, matches the one you manually calculated.