I have a function that has 11 input parameters.
MyFunction(40, 40, 1, 1, 1, 5, 0, 1, 0, 1500, 'MyFile');
The input parameter 'MyFile' when passed through the MyFunction saves a text file using the save command that is 6 columns by the 10th input parameter of rows (e.g. 1500). I usually then load this files back into MATLAB when I am ready to analyze different runs.
I'd like to run MyFunction m times and ultimately have the 'MyFile' be a measure of central tendency (e.g. mean or median) of those m runs.
m=10
for i = 1:m;
MyFunction(40, 40, 1, 1, 1, 5, 0, 1, 0, 1500, 'MyFile');
end;
I could use the for-loop to generate a new 'MyFile' name for each iteration (e.g. MyFile1, MyFile2,...,MyFileM) with something like MyFile = sprintf('MyFile%m'); and then load all of the MyFiles back into MATLAB and then take their average and save it as a UltimateMyFile, but this seems cumbersome. Is their a better method to average these output files more directly? Should I store the files as an object, use dlmwrite, or -append?
Thanks.
since you are trying to find median, you need access to all the data.
you can define a 3 dimension array say
data = zeros(1500,6,m);
and then at each step of for loop update it:
data(:,:,i) = MyFunction(40, 40, 1, 1, 1, 5, 0, 1, 0, 1500);
of course you will need to redefine your function to get the right output.
However if you need to access the data at some other time, then you are better of writing it to a file and reading it from there.
in case you are only interested in the average, you can keep a running total as each case is analyzed and then then just divide it by number of cases (m).
Related
In a parameter variation experiment I am plotting data from a dataset located in Main. Like this:
I use the following code to display data in the plot:
if(root.p_X == 7) {
plot.addDataSet(root.ds_waitTime,
"X = " + root.p_X,
transparent(red, 0.5), true, Chart.INTERPOLATION_LINEAR,
1, Chart.POINT_NONE);
else if(root.p_X == 14) {
plot.addDataSet(root.ds_waitTime,
"X = " + root.p_X,
transparent(red, 0.5), true, Chart.INTERPOLATION_LINEAR,
1, Chart.POINT_NONE);
My question is related to exporting the underlying data. I am not sure what the best way is to export the data. For each simulation run, I want to export dataset root.ds_waitTime to a csv or excel file, annotated with the value of root.p_X == 14. I know you can add datasets to a 2d histogram dataset, but this also transforms the data, so that doesn't seem a good option. Also, I do not have set up an external database, and I'd prefer not to.
Is it possible to save root.ds_waitTime to column 1 and 2 for simulation run 1, to column 3 and 4 for simulation run 2, and so on?
In the Experiment window do the following:
Add a variable called i and assign 0 as its initial value
Add an Excel File element and link it to the file you want on your PC in its properties. Name the block "excelFile" which is the default name.
Then, in the Experiment window properties under "Java Actions" in the field "After Simulation Run", use the following code (where dataSet is the name of your dataset):
excelFile.writeDataSet(dataSet, 1, 2, 1 + i*2);
i++;
This way, after each simulation run, the dataset is written to the next 2 rows.
The syntax I used refers to the:
Sheet Number i.e. 1
Row Number i.e. 2; I use 2 to leave space for a top row in which you can add in your headers outside AnyLogic, to the file directly
Column Number i.e. 1 + i*2
For the header row, you can alternatively add the following code:
excelFile.setCellValue(dataset name, 1, 1, 1 + i*2);
I want to make a for loop that goes from 0 to 180, and then back again to -180. I tried the following:
for a=0:1:180 && 179:-1:-180
but this is not possible in MATLAB.
I have tried to use the && and || statements, but both don't work. I don't know any other ways to combine the two arrays. Any ideas?
You misunderstand the && and || operators. What you want is the following:
Go from 0 to 180 in steps of 1 AND then go from 180 to -180 in steps of -1.
However for any two statements A and B (both A and B need to be scalar values!), the command A && B does the following:
Return True, if both A and B are True, return False otherwise.
This is a logical AND, while you want to go through your first array AND through your second array after that. Though both is some kind of AND, you can't use && for your purpose.
Now, when you call for a=0:180, MATLAB does the following:
Create the vector 0:180, that is [0, 1, 2, ..., 180].
Run all the content inside the loop for each element in the vector created in 1).
So, what you want to do is create an array that contains the numbers [0, 1, 2, ..., 179, 180, 179, 178, ..., -179, -180]. You can do that by concatenating the arrays [0:180] and [179:-1:-180]. You should read about concatenation in MATLAB in their documentation. So, long story short, you for loop should be
for a=[0:180, 179:-1:-180]
Given the following beginning of a network
local net = nn.Sequential()
net:add(SpatialConvolution(3, 64, 4, 4, 2, 2, 1, 1))
with an input tensor input
local input = torch.Tensor(batchSize, 3, 64, 64)
// during training
local output = net:forward(input)
I want to modify the network to accept a second tensor cond as input
local cond = torch.Tensor(batchSize, 1000, 1, 1)
// during training
local output = net:forward({input, cond})
I modified the network by adding a JoinTable before the SpatialConvolution is added, like so:
local net = nn.Sequential()
net:add(nn.JoinTable(2, 4))
net:add(SpatialConvolution(3, 64, 4, 4, 2, 2, 1, 1))
This is not working because both tensors have different sizes in dimensions 2, 3, and 4. Giving the cond tensor as size of (batchSize, 1000, 64, 64) is not an option since its a waste of memory.
Is there any best practise for merging two different tensors at the beginning of a network to be feed into the first layer.
There is no such thing as "merging" tensors which do not have compatible shapes. You should simply pass a table of tensors and start your network with SelectTable operation and work with nngraph, not simple Sequential. In particular - how would you expect Spatial Convolution to work on such odd "tensor" which "narrows down" to your cond? There is no well defined operation in mathematics for such use case, thus you have to be more specific (which you will achieve with nngraph and SelectTable).
I have an OpenBUGS model that uses observed data (y.values) over time (x.values) to simulate many runs (~100000) with new estimates of y-values (y.est) for each run. The observed data exhibit a pronounced decline from a maximum value.
I want to keep track of the length of time it takes for each run to decline from the maximum abundance (T.max) to 10% of the maximum abundance (T.10%). Because the maximum abundance value changes from run to run, 10% of that maximum will also vary from run to run, and thus T.10% will vary from run to run.
Setting a parameter to store T.max is easy enough, that doesn't vary from run to run because the maximum value is sufficiently greater than any other value.
What I can't figure out, is how to store the intersection of the y-est values and T.10%.
My first attempt was to determine whether each y-est value is above or below T.10% using the step() function:
above.below[i] <- step(T.10% - y.est[i])
This generates a string of ones and zeros for each y.est value (e.g., 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, etc.) If each run simply declined continuously from a maximum to a minimum, I could use the rank() function to determine how many above.below[i] values occur above T.10%:
decline.length <- rank(above.below[1:N], 0)
In this example, decline.length would be equal to the number of '0's in the string above, which is 9. Unfortunately, the y-est values occasionally display periods of growth following their decline below T.10%. So, the vector of above.below values can look like this: 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, etc. Thus, decline.length would equal 14 rather than 9, given the subsequent 0s in the vector.
What I want to do, is figure out how to store only the number of '0's in above.below prior to the first '1'; above.below[1:10] rather than above.below[1:N]. Unfortunately, it's not always the 10th time step in which the first '1' occurs, so I need to make the maximum extent of the range of above.below vary from run to run during the simulation.
I'm struggling to accomplish this in OpenBUGS since it's a non-procedural language, but I think it can be done, I just don't know the trick to do it. I'm hoping someone more familiar with the step() and rank() functions can lend some expert advice.
Any guidance is much appreciated!
Two solutions offered to me:
1) Calculate the cumulative sum up to each time step:
for (i in 1:N){
above.below[i] <- step(T.10% - y.est[i])
cum.above.below[i] <- sum(above.below[1:i])
}
decline.length <- rank(cum.above.below[1:N], 0)
2) Calculate whether each year is above or below the threshold directly, without 1s and 0s:
for(i in 1:N){
above.below[i] <- step(T.10% - y.est[i])
dummy[i] <- above.below[i] * i + (1 - above.below[i]) * (N+1)
}
decline.length <- ranked(dummy[], 1)
So dummy is i when above.below is 1, and dummy is N+1 when above.below is 0.
In Python we have lru_cache as a function wrapper. Add it to your function and the function will only be evaluated once per different input argument.
Example (from Python docs):
#lru_cache(maxsize=None)
def fib(n):
if n < 2:
return n
return fib(n-1) + fib(n-2)
>>> [fib(n) for n in range(16)]
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]
>>> fib.cache_info()
CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)
I wonder whether a similar thing exists in Matlab? At the moment I am using cache files, like so:
function result = fib(n):
% FIB example like the Python example. Don't implement it like that!
cachefile = ['fib_', n, '.mat'];
try
load(cachefile);
catch e
if n < 2
result = n;
else
result = fib(n-1) + fib(n-2);
end
save(cachefile, 'result');
end
end
The problem I have with doing it this way, is that if I change my function, I need to delete the cachefile.
Is there a way to do this with Matlab realising when I changed the function and the cache has become invalidated?
Since matlab 2017 this is available:
https://nl.mathworks.com/help/matlab/ref/memoizedfunction.html
a = memoized(#sin)
I've created something like this for my own personal use: a CACHE class. (I haven't documented the code yet though.) It appears to be more flexible than Python's lru_cache (I wasn't aware of that, thanks) in that it has several methods for adjusting exactly what gets cached (to save memory) and how the comparisons are made. It could still use some refinement (#Daniel's suggestion to use the containers.Map class is a good one – though it would limit compatibility with old Matlab versions). The code is on GitHub so you're welcome to fork and improve it.
Here is a basic example of how it can be used:
function Output1 = CacheDemo(Input1,Input2)
persistent DEMO_CACHE
if isempty(DEMO_CACHE)
% Initialize cache object on first run
CACHE_SIZE = 10; % Number of input/output patterns to cache
DEMO_CACHE = CACHE(CACHE_SIZE,Input1,Input2);
CACHE_IDX = 1;
else
% Check if input pattern corresponds something stored in cache
% If not, return next available CACHE_IDX
CACHE_IDX = DEMO_CACHE.IN([],Input1,Input2);
if ~isempty(CACHE_IDX) && DEMO_CACHE.OUT(CACHE_IDX) > 0
[~,Output1] = DEMO_CACHE.OUT(CACHE_IDX);
return;
end
end
% Perform computation
Output1 = rand(Input1,Input2);
% Save output to cache CACHE_IDX
DEMO_CACHE.OUT(CACHE_IDX,Output1);
I created this class to cache the results from time-consuming stochastic simulations and have since used it to good effect in a few other places. If there is interest, I might be willing to spend some time documenting the code sooner as opposed to later. It would be nice if there was a way to limit memory use as well (a big consideration in my own applications), but getting the size of arbitrary Matlab datatypes is not trivial. I like your idea of caching to a file, which might be a good idea for larger data. Also, it might be nice to create a "lite" version that does what Python's lru_cache does.