How can I sum the 100 most recent values in the second column of a .csv file?
The .csv file is constantly updated with a new row every minute, containing a new value. i.e the row dimension is forever increasing every minute. Assume I may accumulate 200 values, i.e 200 rows of data after 200 minutes (from external processes) but I only want to sum the 100 most recently added values (i.e from the last row of the file backwards up 99 rows)).
I have 3 sets of values, 1 value in each column. Col2Value is the value in Column 2 of the .csv file. The file i'm appending to is called '200 mins.csv'
dlmwrite('200 mins.csv', [Col1Value, Col2Value,Col3Value], '-append');
Here's a snippet of the csv file after 8 minutes of run-time i.e 8 values in column 2:
I know that MATLAB has to read the whole file first, but that's fine. I can deal with that, my application is not resource critical.
When you have an array in Matlab, you can sum the last 100 elements with
last100sum = sum(A(end-99:end));
So if reading in your .csv file gives you a 2D array (which I will call wholeFile), and you want the sum of the last 100 elements in column 2, you do it like this:
last100sum2 = sum(wholeFile(end-99:end, 2));
If the routine you use for reading the file returns a cell array, you need to convert the cell array to a regular array first - perhaps using cell2mat if it contains only values (as opposed to strings, for example).
Finally - if you want to read "at most" 100 values, but there might not be 100 rows, then do the following:
sz = size(wholeFile);
firstRow = max(sz(1) - 99, 1); % will return 1 if fewer than 100 rows are found
atMost100sum = sum(wholeFile(firstRow:end, 2));
Any questions - please ask.
Related
I have a table (L-arrival) of 279 rows and 252 columns. Only the first column has values while others are just NaN. The cells in the first column have multiple values (i.e. some have 1, some have 4 number of values). First of all, I am trying to select a single maximum value from each cell of the first column so that I can have a column of a single value for each cell only. Then I want to do this in a loop so that for every new value that I get, they are sorted and only the maximum values are chosen. Finally, I want to make a collection of these values obtained from multiple runs for each cell. Can anyone suggest to me how it can be approached in MatLab?I tried using the following code but didn't work well.
for b=1:279
m = numel(cell2mat(L_arrival(b,1)));
g(b)=mat2cell([cell2mat(g(b)); cell(L_arrival(b,1))]',[1 2]);
end
Even if many subjects have the same topics, I don't find my answer.
I have something like 30 .txt files in a folder (with a different names), I would like to import all of them in MATLAB. Take a column from each file and make a vector with all those columns.
Bombo30m1.txt
Bombo30m2.txt
Bovolon30m2.txt
Rigutti30m4.txt
Each .txt file have 45 columns of number separated by comma.
All the numbers inside my text files are integers.
All files have the same number of columns, but not the same numbers of row (more or less 4000 for each).
For all of the files, I would like to take the column 40 and make one vector with it. So I will get a big vector.
First, create a struct with all the file information:
dir('*.txt') gives you a struct with all information about the files:
4x1 struct array with fields:
name
date
bytes
isdir
datenum
Define a variable where you'll place the values of the first columns
Use number_of_files = numel(filenames) to get the number of files. col_values is the vector you will place the values in. Note, you should try to pre-allocate memory.
Load the files
filenames(1).name gives you the name of the first file, "Bombo30m1.txt".
Loop through all files, and get the values out: load(filenames(ii).name).
And finally place the new values after the previous ones in col_values.
To sum it all up:
filenames = dir('*.txt');
number_of_files = numel(filenames);
col_values = [];
for ii = 1:number_of_files
all_values = load(filenames(ii).name);
col_values = [col_values; all_values(:,1)];
end
NOTE!! This code contain some sub-optimal code, as I've created a growing vector inside the loop. If this is a procedure that will be performed many times, then you should rewrite it a bit. Relevant
I need to ingest a CDF (common data format) file into MATLAB. I have used the [cdfread][1] command for this purpose. An image of my output is attached below:
When I open data_import, columns 4 and 5 are in a particular 3 x 1 format (as shown in data_import(1,4)).
My question is: Is there a simple way to extract the data for each cell in column 4, such that for the 2nd row in data_import(1,4), it gets inserted as a new column (i.e. column 5) in the original data (data_import)? Similarly, 3rd row in data_import(1,4) should be inserted as a new column (column 6) in the original data (data_import). This procedure should also be repeated in the original Column 5 data which also has a similar 3 x 1 structure within each cell.
I hope I'm not being too vague in what I am describing, but I'm really not sure what I'm supposed to do regarding the commands to call for the operation. Thank you in advance.
Your desired final output has columns which are made up of these cells converted from 3 x 1 arrays to 1 x 3 cell arrays and then concatenated for each row. It's easier to do the concatenation first with the elements the "wrong way round" and then transpose the final result:
data_import = [data_import(:,1:3) num2cell([data_import{:,4}; data_import{:,5}]') data_import(:,6:end)];
I have 60 values in a cell matrix from a .csv file which are stored in column 15 from rows 38556-38616
I want to copy that range of values to a regular numerical 60x1 matrix in a variable Value
Here is what I tried:
Values(60,1) = data2{38556:38616,15};
It only copies 1 value from data2 and the rest of values from rows 1-59 are 0. How can I copy
those values to Value in the same order they are in the file? data2 is the cell matrix storing .csv file values.
Image of data2:
Either
Values(1:61,1) = cell2mat(data2(38556:38616,15));
or just
Values = [data2{38556:38616,15}]';
The problem was that you tried to store 61 values into just one matrix element.
Incidentally, if you don't need the full contents of the csv file, just a subset, and the contents you are reading in are numeric, you can take this directly from the file using dlmread. For example:
values = dlmread('data.csv',',',[38555 14 38615 14]);
The numeric values are starting and ending positions: [R1 C1 R2 C2]. dlmread starts counting at zero for R/C values. You can also use spreadsheet style instead, i.e. 'A4..G6' or in your case I think it would be 'O38556..O38616'.
How do I write a text file in the same format that it is read in MATLAB?
I looked and my question is almost the same as above question.
I want to read in a file which is 84641 x 175 long.
and i want a make a new .txt file with 84641 x 40 , deleteling rest of the columns.
I have 2 rewrite the dates n times. date is on first column in format 6/26/2010 and time on 2nd column in format ' 00:00:04'
when i use the code put in above question i keep getting the error
??? Error using ==> reshape
Product of known dimensions, 181,
not divisible into total number
of elements, 14812175.
Error in ==>
write at
data = reshape(data{1},N+6,[])';
when i comment this it has error in printf statements for date and data write.
Any ideas??
thanks
As the author of the accepted answer in the question you link to, I'll try to explain what I think is going wrong.
The code in my answer is designed to read data from a file which has a date XX/XX/XXXX in the first column, a time XX:XX:XX in the second column, and N additional columns of data.
You list the number of elements in data as 14812175, which is evenly divisible by 175. This implies that your input data file has 2 columns for the date and time, then 169 additional columns of data. This value of 169 is what you have to use for N. When the date and time columns are read from the input file they are broken up into 3 columns each in data (for a total of 6 columns), which when added to the 169 additional columns of data gives you 175.
After reshaping, the size of data should be 84641-by-175. The first 6 columns contain the date and time values. If you want to write the date, the time, and the first 40 columns of the additional data to a new file, you would only have to change one line of the code in my answer. This line:
fprintf(fid,', %.1f',data(i,7:end)); %# Output all columns of data
Should be changed to this:
fprintf(fid,', %.1f',data(i,7:46)); %# Output first 40 columns of data