Here i am sharing one of my data which are in .dat file. I have 16162 different files. I merged all into one file and want to read it in matlab and need to extract three parameter's values from a single file and arrange them in either row wise or column wise. I can do it by using C sharp codes but i want to do it by using matlab. can anybody help me out for writing the codes please?
Here is the one sample file data:
DISTRIBUTION: monomodal log-normal
n : 1.000
r_mod: .010
sigma: 1.400
number conc., surface. conc., volume conc.
.1087E+01 .1866E-02 .7878E-05
part. ave. radius, surf. ave. radius, vol. ave. radius :
.1149E-01 .1169E-01 .1201E-01
surface mean radius, volume mean radius :
.1267E-01 .1392E-01
eff. variance :
.9939E-01
Let's say: I want to extract or read three parameters (r_mod, sigma, Surface means radius). The corresponding values for these three parameters from the file I put in this page is .010 , 1.400 , .1267E-01
The output should be (Which i want):
r_mod sigma surface mean radius .01 1.4 1.27E-02 .02 1.4 2.67E-02 .03 1.4 3.98E-02 ... .. .. .. .. .. .. .. ..
I have more than thousands similar files in a same directory. I want to read all those files in matlab and the output should show in this way in a single file.
Given that all files have the exact same structure, the following will do the job (just make sure to ream the comments in the code, you will need to adapt your file names and number of files to read):
n = 2; % Number of files you want to go through
vals = zeros(1,3*n);
str = 'r_mod sigma surface mean radius ';
k = 1;
for i = 1:n
path = ['myFile',num2str(i),'.dat']; % change this to fit your file names
fid = fopen(path, 'rb');
data = textscan(fid,'%s');
fclose(fid);
data = data{1};
vals(k) = str2double(data{8});
vals(k+1) = str2double(data{10});
vals(k+2) = str2double(data{40});
k = k+3;
end
out = [str, num2str(vals)];
fid = fopen('output.txt', 'w');
fprintf(fid,out);
The file output.txt now contains your desired line. You may change the format if you want the output file to be .dat as well.
Related
I am currently working with accelerometer (ActivPAL micro 3) data collected at 8 bits, and I am converting this data to a higher resolution (10 bits). After applying the conversion I need to leave the file as the original to process the data in other software, so to do that I have to add an extra header and then save the data (Time, Uncompressed sample index, X, Y, and Z axis) in each column and row. So far I have managed to write a code (see below) that works well, but I think there is a simpler and faster way to do the same thing.
Note: my files are large, they all have more than a million lines. Also, I cannot lose the configuration of the first and second column of the file.
% Loading data
[filename, ~] = uigetfile('*.csv');
data = table2array(readtable(filename));
% X axis
data(:,3) = 2.02183 * data(:,3) + 256.728;
% Y axis
data(:,4) = 2.02183 * data(:,4) + 256.728;
% Z axis
data(:,5) = 2.02183 * data(:,5) + 256.728;
newfilename = replace(filename, "2a", "4at");
fid = fopen(newfilename, 'w');
% First header
fprintf(fid, 'sep=;\n');
% Second header (variable names)
fprintf(fid, '%s;%s;%s;%s;%s\n','Time','Uncompressed sample index','X','Y','Z');
% Saving data
for k = 1:length(data)
fprintf(fid, '%.10f;%.0f;%f;%f;%f\n', data(k,1), data(k,2), data(k,3), data(k,4), data(k,5));
end
fclose(fid);
As the file is big, I could not attach it here, so I uploaded it to my Google Drive (in this folder you can find a compressed file and an uncompressed file that is bigger than the other).
Thank you in advance,
Luiz Augusto
Can someone help me to understand how I can save in matlab a group of .csv files, select only the columns in which I am interested and get as output a final file in which I have the average value of the y columns and standard deviation of y axes? I am not so good in matlab and so I kindly ask if someone to help me to solve this question.
Here what I tried to do till now:
clear all;
clc;
which_column = 5;
dirstats = dir('*.csv');
col3Complete=0;
col4Complete=0;
for K = 1:length(dirstats)
[num,txt,raw] = xlsread(dirstats(K).name);
col3=num(:,3);
col4=num(:,4);
col3Complete=[col3Complete;col3];
col4Complete=[col4Complete;col4];
avgVal(K)=mean(col4(:));
end
col3Complete(1)=[];
col4Complete(1)=[];
%columnavg = mean(col4Complete);
%columnstd = std(col4Complete);
% xvals = 1 : size(columnavg,1);
% plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');
B = reshape(col4Complete,[5000,K]);
m=mean(B,2);
C = reshape (col4Complete,[5000,K]);
S=std(C,0,2);
Now I know that I should compute mean and stdeviation inside for loop, using mean()function, but I am not sure how I can use it.
which_column = 5;
dirstats = dir('*.csv');
col3Complete=[]; % Initialise as empty matrix
col4Complete=[];
avgVal = zeros(length(dirstats),2); % initialise as columnvector
for K = 1:length(dirstats)
[num,txt,raw] = xlsread(dirstats(K).name);
col3=num(:,3);
col4=num(:,4);
col3Complete=[col3Complete;col3];
col4Complete=[col4Complete;col4];
avgVal(K,1)=mean(col4(:)); % 1st column contains mean
avgVal(K,2)=std(col4(:)); % 2nd column contains standard deviation
end
%columnavg = mean(col4Complete);
%columnstd = std(col4Complete);
% xvals = 1 : size(columnavg,1);
% plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');
B = reshape(col4Complete,[5000,K]);
meanVals=mean(B,2);
I didn't change much, just initialised your arrays as empty arrays so you do not have to delete the first entry later on and made avgVal a column vector with the mean in column 1 and the standard deviation in column 1. You can of course add two columns if you want to collect those statistics for your 3rd column in the csv as well.
As a side note: xlsread is rather heavy for reading files, since Excel is horribly inefficient. If you want to read a structured file such as a csv, it's faster to use importdata.
Create some random matrix to store in a file with header:
A = rand(1e3,5);
out = fopen('output.csv','w');
fprintf(out,['ColumnA', '\t', 'ColumnB', '\t', 'ColumnC', '\t', 'ColumnD', '\t', 'ColumnE','\n']);
fclose(out);
dlmwrite('output.csv', A, 'delimiter','\t','-append');
Load it using csvread:
data = csvread('output.csv',1);
data now contains your five columns, without any headers.
I'm using importdata to strip off a header and import a file into my workspace.
The file contains rotation matrices plus translation vectors.
In my matlab script I transform the rotation matrices into quaternions.
In my output file I would like to have quaternions plus translation vectors, i.e. I want to replace the old values from the rotation matrix with the new values for the quaternions.
However, I am only able to output the quaternions to the command window, but don't know how to replace the values.
Here is my code:
path = '\filepath';
[head DELIM NHEADERLINES] = importdata([path],' ',9);
Rotation = head.data(:,1:9);
Translation = head.data(:,10:12);
RotationMatrix= zeros([3 3 size(Rotation,1)]);
for i=1:size(Rotation,1)
RotationMatrix(:,:,i) = [Rotation(i,1:3); Rotation(i,4:6); Rotation(i,7:9)];
end
Quaternion = SpinCalc('DCMtoQ',RotationMatrix,0.1,0);
How can I get the quaternion into the original file and overwrite the rotation matrix?
Thanks in advance for your help!
The use of dlmread and dlmwrite is more recommended in your case.
M = dlmread(filename,' ',9,0); % start from row 10, column 1
Rotation = M(:,1:9);
Translation = M(:,10:12);
.
.
.
dlmwrite(Quaternion,'delimiter',' ','newline','pc');
If you prefer to modify the original file instead of replacing it, use the following
fileID = fopen(filename,'r+'); % with read and write permission
fseek(fileID, pos, 'bof'); % where pos is the starting position of the bytes you want to read
M = fscanf(fileID, repmat('%g ',1,12), nlines); % read 12 columns of numbers for at most n lines.
.
.
.
fseek(fileID, pos, 'bof'); % go back to the position
fprintf(fileID, repmat('%g ',1,12), Quaternion);
This is just a suggestion. The above codes may not run as desired, so I recommend you to read the documentation for more info. Cheers.
i want to read .txt file in matlab with both data and words
the contents of .txt file are
(title "Particle Tracks")
(labels "Time" "Particle Velocity Magnitude")
((xy/key/label "particle-1")
1e-06 45.4551
2e-06 40.3895
2e-06 44.0437
3e-06 34.9606
4e-06 33.1695
4e-06 35.3499
5e-06 29.9504
6e-06 28.0226
6e-06 35.1794
7e-06 41.2255
....
((xy/key/label "particle-2")
1e-06 43.7789
1e-06 45.0513
2e-06 44.1221
3e-06 37.8328
3e-06 43.6451
4e-06 29.1166
5e-06 41.3342
6e-06 28.7241
6e-06 36.3779
7e-06 31.9631
8e-06 29.2826
9e-06 24.7755
9e-06 24.9516
1e-05 22.7528
1e-05 26.6802
1.1e-05 34.4668
the file extends for 100 particles ,1st column is time and 2nd column is velocity
I intend to find the mean velocity of all the particles at various times of column 1,so basically i want to add corresponding column 2 values and divide them by hundred and display against the the column 1 values which is same for all the hundred particles![enter image description here][2]
thanks
The best way to read text data with a complex structure like this is to use the fscanf function in MATLAB. Follow the documentation and you should be able to read the data into an array that you can used to compute the statistics you wish to find.
Another option might be to read the data in line-by-line and use regular expressions with the regexpi function to extract the data you need.
Suppose your input file is input.txt, then use textscan as follows:
fid = fopen('input.txt');
C = textscan(fid, '%n %n', 'commentStyle', '(');
a = C{1};
b = C{2};,
%# do your computations on vectors a and b
%# for example:
ma = mean(a)
mb = mean(b)
You can use the vectors as you wish, e.g. you can process them 100 by 100 elements. That's up to you.
I have a ton of data that needs to be processed from lab work. I have a ton of .mat files that contain a signal matrix of dimensions 7 x w. I need to resize the matrix to 7 x N and w is larger and smaller than N to make the rest of the analysis easier (don't care about data past N). I have the psuedocode of how I want this to work but don’t know how to implement it. Any help would be great thanks!
Folder structure of all my data:
Main folder
Alpha 1
1111.mat
1321.mat
Alpha 2
1010.mat
1234.mat
1109.mat
933.mat
Alpha 3
1223.mat
etc.
Psudeocode:
Master_matrix = []
For all n *.mat
Load n'th *.mat from alpha 1
If w > N
Resize matrix down to N
Else
Zero pad to N
End if
Master_matrix = master_matrix .+ new resized matrix
End for
rest of my code...
First you need to generate the file list. I have my own function for that, but there is, for example, GETFILELIST or the excellent interactive UIPICKFILES to generate the list of files.
Once you have the file list (I'll assume it's a cell array containing the filenames), you can do the following:
nFiles = length(fileList);
Master_matrix = zeros(7,N);
for iFile = 1:nFiles
%# if all files contain a variable of the same name,
%# you can simplify the loading by not assigning an output
%# in the load command, and call the file by
%# its variable name (i.e. replace 'loadedData')
tmp = load(fileList{iFile});
fn = fieldnames(tmp);
loadedData = tmp.(fn{1});
%# find size
w = size(loadedData,2);
if w>=N
Master_matrix = Master_matrix + loadedData(:,1:N);
else
%# only adding to the first few columns is the same as zero-padding
Master_matrix(:,1:w) = Master_matrix(:,1:w) = loadedData;
end
end
Note: In case you don't actually want to add up the data, but simply store it in the master array, you can make Master_matrix into a 7-by-N-by-nFiles array, where the nth plane of Master_matrix is the content of the nth file. In this case, you'd initialize Master_matrix as
Master_matrix = zeros(7,N,nFiles);
and you'd write the if-clause as
if w>=N
Master_matrix(:,:,iFile) = Master_matrix(:,:,iFile) + loadedData(:,1:N);
else
%# only adding to the first few columns is the same as zero-padding
Master_matrix(:,1:w,iFile) = Master_matrix(:,1:w,iFile) = loadedData;
end
Also note that you might want to initialize Master_matrix as NaN instead of zeros, so that the zeros don't affect subsequent statistics (if that's what you want to do with the data).