In a certain manufacturing system, there are 2 machines M 1 and M 2.
M 1 is a fast and high precision machine whereas M 2 is a slow and low
precision machine. M 2 is employed only when M 1 is down, and it is
assumed that M 2 does not fail. Assume that the processing time of
parts on M 1, the processing time of parts on M 2, the time to failure
of M 1, and the repair time of M 1 are independent geometric random
variables with parameters p 1, p 2, f, and r, respectively. Identify a
suitable state space for the DTMC model of the above system and
compute the TPM. Investigate the steady-state behavior of the DTMC.
How to model this into. DTMC? What is the state space? I have tried to use state-space like this:
0: M1 is working, M1 does not fail
1: M1 failed, M2 is working, M1 is repairing, M1 does not finish repairing
But there are still some problems, like what will happen after M1 finishes 1 part? Immediately process the next one or will change decide whether to fail or not? What will happen if M1 fails during process one part? What is the probability transfer matrix?
Thank you very much for your help!!!!
Related
I'm trying to do an operation that goes like: (exp((a-b').^2)*(diag(c)*d)), where a, b, c, and d are column vectors. The sizes of a and b are large so a-b' exceeds the maximum limit for matrix size in MATLAB. I then converted this to a loop that goes over each value of a:
d=zeros(length(a),1);
for i=1:length(a)
d(i)=(exp((a(i)-b').^2)*(c.*d));
end
This makes the code take a long time to run because this operation is called a lot. Is there any way to reduce the time taken for this?
I have n servers and I want to know the number of servers I need to make the probability that at least 10 servers are active to be 0.99. The probability that a server fails is equal to 0.01.
So what I have so far is that I know I need at least 10 servers to be active. So the probability would be:
sum (from k = 10 to n) of (n choose k)*(0.99 ^ k)*(0.01^(n-k)) = 0.99
and I would have to do this for every n from 10 to n. I want to know is there any shorter way? Like what if I did the probability that exactly 9 servers failed and I did one minus that probability like this:
1 - (n choose 9)*(0.01^9)*(0.99^(n-9)) = 0.99
Would this give me the right answer? Please help :)
Update, I used an online calculator to solve for the latter equation (1 - probability that exactly 9 failed) and I got the maximum number of servers that could be used to make the probability of at least 10 servers to be active to be greater than 0.99 would be 380 servers, any more than that would result in a probability of at least 10 servers to be active to be less than 0.99.
I'm not sure if this is correct though. :)
Since you want at least X=10 successes in n trials, each trial with success p=0.99,
you could consider the conjugate and figure out n in (P(X<=9|n=N,p=0.99) < 0.01).
You can use the binomial cdf for this part
In your case it becomes
Now we want to figure out how many trials n we need to make the evaluation of the above cdf less than 0.01.
You could for example use python to search numerically for this:
from scipy.stats import binom
for n in range(1000):
p = binom.cdf(9, n, 0.99)
if p < 0.01:
print(n)
break
And you would see that no more than 11 servers are needed to ensure a probability of 0.99 that at least 10 servers are active :-)
consider the Deep Q-Learning algorithm
1 initialize replay memory D
2 initialize action-value function Q with random weights
3 observe initial state s
4 repeat
5 select an action a
6 with probability ε select a random action
7 otherwise select a = argmaxa’Q(s,a’)
8 carry out action a
9 observe reward r and new state s’
10 store experience <s, a, r, s’> in replay memory D
11
12 sample random transitions <ss, aa, rr, ss’> from replay memory D
13 calculate target for each minibatch transition
14 if ss’ is terminal state then tt = rr
15 otherwise tt = rr + γmaxa’Q(ss’, aa’)
16 train the Q network using (tt - Q(ss, aa))^2 as loss
17
18 s = s'
19 until terminated
In step 16 the value of Q(ss, aa) is used to calculate the loss. When is this Q value calculated? At the time the action was taken or during the training itself?
Since replay memory only stores < s,a,r,s' > and not the q-value, is it safe to assume the q value will be calculated during the time of training?
Yes, in step 16, when training the network, you are using the the loss function (tt - Q(ss, aa))^2 because you want to update network weights in order to approximate the most recent Q-values, computed as rr + γmaxa’Q(ss’, aa’) and used as target. Therefore, Q(ss, aa) is the current estimation, which is typically computed during training time.
Here you can find a Jupyter Notebook with a simply Deep Q-learning implementation that maybe is helpful.
For an experiment I need to pseudo randomize a vector of 100 trials of stimulus categories, 80% of which are category A, 10% B, and 10% C. The B trials have at least two non-B trials between each other, and the C trials must come after two A trials and have two A trials following them.
At first I tried building a script that randomized a vector and sort of "popped" out the trials that were not where they should be, and put them in a space in the vector where there was a long series of A trials. I'm worried though that this is overcomplicated and will create an endless series of unforeseen errors that will need to be debugged, as well as it not being random enough.
After that I tried building a script which simply shuffles the vector until it reaches the criteria, which seems to require less code. However now that I have spent several hours on it, I am wondering if these criteria aren't too strict for this to make sense, meaning that it would take forever for the vector to shuffle before it actually met the criteria.
What do you think is the simplest way to handle this problem? Additionally, which would be the best shuffle function to use, since Shuffle in psychtoolbox seems to not be working correctly?
The scope of this question moves much beyond language-specific constructs, and involves a good understanding of probability and permutation/combinations.
An approach to solving this question is:
Create blocks of vectors, such that each block is independent to be placed anywhere.
Randomly allocate these blocks to get a final random vector satisfying all constraints.
Part 0: Category A
Since category A has no constraints imposed on it, we will go to the next category.
Part 1: Make category C independent
The only constraint on category C is that it must have two A's before and after. Hence, we first create random groups of 5 vectors, of the pattern A A C A A.
At this point, we have an array of A vectors (excluding blocks), blocks of A A C A A vectors, and B vectors.
Part 2: Resolving placement of B
The constraint on B is that two consecutive Bs must have at-least 2 non-B vectors between them.
Visualize as follows: Let's pool A and A A C A A in one array, X. Let's place all Bs in a row (suppose there are 3 Bs):
s0 B s1 B s2 B s3
Where s is the number of vectors between each B. Hence, we require that s1, s2 be at least 2, and overall s0 + s1 + s2 + s3 equal to number of vectors in X.
The task is then to choose random vectors from X and assign them to each s. At the end, we finally have a random vector with all categories shuffled, satisfying the constraints.
P.S. This can be mapped to the classic problem of finding a set of random numbers that add up to a certain sum, with constraints.
It is easier to reduce the constrained sum problem to one with no constraints. This can be done as:
s0 B s1 t1 B s2 t2 B s3
Where t1 and t2 are chosen from X just enough to satisfy constraints on B, and s0 + s1 + s2 + s3 equal to number of vectors in X not in t.
Implementation
Implementing the same in MATLAB could benefit from using cell arrays, and this algorithm for the random numbers of constant sum.
You would also need to maintain separate pools for each category, and keep building blocks and piece them together.
Really, this is not trivial but also not impossible. This is the approach you could try, if you want to step aside from brute-force search like you have tried before.
I'm trying to simulate an optical network algorithm in MATLAB for a homework project. Most of it is already done, but I have an issue with the diagrams I'm getting.
In the simulation I'm generating exponential traffic, however, for low lambda values (0.1) I'm getting very high packet drop rates (99%). I wrote a sample here which is very close to the testbench I'm running on my simulator.
% Run the simulation 10 times, with different lambda values
l = [1 2 3 4 5 6 7 8 9 10];
for i=l(1):l(end)
X = rand();
% In the 'real' simulation the following line defines the time
% when the next packet generation event will occur. Suppose that
% i is the current time
t_poiss = i + ceil((-log(X)/(i/10)));
distr(i)=t_poiss;
end
figure, plot(distr)
axis square
grid on;
title('Exponential test:')
The resulting image is
The diagram I'm getting in this sample is IDENTICAL to the diagram I'm getting for the drop rate/λ. So I would like to ask if I'm doing something wrong or if I miss something? Is this the right thing to expect?
So the problem is coming from might be a numerical problem. Since you are generating a random number for X, the number might be incredibly small - say, close to zero. If you have a number close to zero numerically, log(X) is going to be HUGE. So your calculation of t_poiss will be huge. I would suggest doing something like X = rand() + 1 to make sure that X is never close to zero.