I want to switch columns order in a bunch of text files and also remove some of the columns after switching. I'm trying to use textscan function in matlab. But I'm not sure how to do it.
This is a sample line in one of the text files (space is delimiter):
580.697942 1009.223279 3.012318 2 1 0 554 605 607 558 1004 996 1016 1021
For example, I want to bring the forth-to-sixth column to the first of the line:
2 1 0 580.697942 1009.223279 3.012318 554 605 607 558 1004 996 1016 1021
And maybe add some columns (with value='0') afterwards.
I would appreciate some advice on how to do this.
Thanks
Instead of textscan, I would recommend you to use the dlmread function.
out = dlmread( 'mytextfile.txt' );
The output is an array, by using MATLAB basic commands you will be able to change columns as you wish.
Related
I have a nxn .csv file in which I am finding the cumulative sum of one column. I need to append this column with a header cumsum to the end of the existing .csv file to make it nx(n+1). How could this be done? I am attaching a samaple:
filename A B
aa 23 34
aa 56 98
aa 8 90
aa 7 89
I am finding the cumsum of column A
23
79
87
94
I need this column appended to the end of .csv as
filename A B cumsum
aa 23 34 23
aa 56 98 79
aa 8 90 87
aa 7 89 94
I have 2 problems here:
1. I am extracting the column A everytime to perform the cumsum operation. How do I find it directly from the table for a single column without extraction?
How do I create a new column at the end of the existing table to add the cumsum column with a header 'cumsum'?
For point 1: You can use csvread to read a specific column directly from a .csv file without loading the whole thing. For your example, you would do this:
A = csvread('your_file.csv', 1, 1, [1 1 nan 1]);
The nan allows it to read all rows until the end (although I'm not sure this is documented anywhere).
The use of csvread is applicable to files containing numeric data, although it works fine for the above example even with character entries in the first row and first column of the .csv file. However, it appears to fail if the part of your file that you want to read is followed by columns containing character entries. A more general solution using xlsread is as follows:
A = xlsread('your_file.csv', 'B:B');
For point 2: Built-in functions like csvwrite or dlmwrite don't appear able to append new columns, just new rows. You can however use xlswrite, even though it is a .csv file. Here's how it would work for your example:
xlswrite('your_file.csv', [{'cumsum'}; num2cell(cumsum(A))], 1, 'D1');
And here's what the contents of your_file.csv would look like:
filename,A,B,cumsum
aa,23,34,23
aa,56,98,79
aa,8,90,87
aa,7,89,94
I have to read different numbers in the same line in a text file. How can I pass them to an Array (for each line), if I don't know how many numbers I have to read?
I thought about reading each number and passing it to an array, until I find the New Line character. But I have a lot of files, so doing this takes a lot of time.
With this arrays from each file I have to build plots. Is there any other way?
12 43 54 667 1 2 3 1 545 434 6 476
14 32 45 344 54 54 10 32 43 5 6 66
Thanks
You can open each file and read it line by line, then use textscan(str,'%d') to convert each line into an array.
Example for one file:
fid = fopen('file.txt');
tline = fgetl(fid);
while ischar(tline)
C = textscan(str,'%d');
celldisp(C);
tline = fgetl(fid);
end
fclose(fid);
You would have to run the code for each file, and do something with the array C.
You can read the additional details on the function textscan.
The way to read ASCII-delimited, numerical data in MATLAB is to use dlmread, as already suggested by #BillBokeey in a comment. This is as simple as
C = dlmread('file.txt');
I am using dec2hex up to 100 times in MATLAB. Because of this, the speed of code decreases. for one point I am using dec2hex 100 times. It will take 1 minute or more than it. I have do the same for 5000 points. But because of dec2hex it will take hours of time to run. So how can I do hexadecimal to decimal conversion optimally? Is there any other alternative that can be used instead of dec2hex?
As example:
%%Data[1..256]: can be any data from
for i=1:1:256
Table=dec2hex(Data);
%%Some permutation applied on Data
end;
Here I am using dec2hex more than 100 times for one point. And I have to use it for 5000 points.
Data =
Columns 1 through 16
105 232 98 250 234 216 98 199 172 226 250 215 188 11 52 174
Columns 17 through 32
111 181 71 254 133 171 94 91 194 136 249 168 177 202 109 187
Columns 33 through 48
232 249 191 60 230 67 183 122 164 163 91 24 145 124 200 142
This kind of data My code will use.
Function calls are (still) expensive in MATLAB. This is one of the reasons why vectorization and pseudo-vectorization is strongly recommended: processing an entire array of N values in one function call is way better than calling the processing function N times for each element, thus saving the N-1 supplemental calls overhead.
So, what you can do? Here are some non-mutually-exclusive choices:
Profile your code first. Just because something looks like the main culprit for execution time disasters, it isn't necessarily it. Type profview in your command window, chose the script that you want to run, and see where are the hotspots of your code. Choose to optimize those hotspots rather than your initial guesses.
Try faster functions. sprintf is usually fast and flexible:
Table = sprintf('%04X\n', Data);
(and — if you dive into the function code with edit dec2hex — you'll see that in some cases dec2hex actually calls sprintf).
Reduce the number of function calls. Suppose you have to build the table for the 100 datasets of different lengths, that are stored in a cell array:
DataSet = cell(1,100);
for k = 1:100
DataSet{k} = fix(1000*rand(k,1));
end;
The idea is to assemble all the numbers in a single array that you convert at once:
Table = dec2hex(vertcat(DataSet{:}));
Mind you, this is done at the expense of using supplemental memory for assembling the partial inputs in a single one — it's not always convenient to do that.
All the variants above. Okay, this point is not actually a point. :-)
I have a problem with ISO encoding in MATLAB.
I have a logging file, with all possible values between 0..255 stored in binary format.
When I open this file in matlab and read one line, MATLAB shows me the correct representation in ISO-8859-1. So far, so good.
For example the value 155 (0x9B) shows the character ">". (Any small character values like this work). Matlab shows this correctly, but when I want process an integer value with double(>) the return value is 8250, which is not an ASCII-Value.
What can I change in the encoding of the file?
edit: the logfile was written with python, in case that matters.
I find the problem. I missed to set the encoding in the fopen command. Working Solution:
%creating testfile
ascii=char([191 210 191 212 191 228 192 215 192 144 198 175 155 236 254 201 10]); %problem value here the 155
logID=fopen('testdatei.log','w','n','ISO-8859-1');
fwrite(logID,ascii);
fclose(logID);
% wrong filehandling
logID=fopen('testdatei.log');
line=fgetl(logID);
decode=double(line);
disp('wrong encoding')
decode(13)
fclose(logID);
%right filehandling
logID=fopen('testdatei.log','r','n','ISO-8859-1');
line=fgetl(logID);
decode=double(line);
disp('right encoding')
decode(13)
fclose(logID);
I am newly disgnosed with high blood pressure and need to a) check my bp regularly now and keep a log of it and b) take medicine. You can't help me with b), but I'd need your kind help with a).
I need to keep a log of my blood pressure, therefore I need to take a few readings each day. My log looks like this:
20121001 0801 175 101
20121001 0802 Pill
20121001 1017 125 91
20121001 1537 121 101
20121001 1907 117 89
20121002 0758 191 111
20121002 0759 Pill
20121002 1003 117 98
(...)
Could you help me out with a gnuplot config file that reads the above two datapoints, plots them against the time (%Y%m%d %H%M).
The caveat here is the lines "... Pill" which I'd like to show up on the plot as a dot, a vertical line or something similar (they simply show when I have taken my pill).
Also, and I'm not quite sure if this is possible, I'd like to visualize the difference between datapoint 1 and datapoint 2, ie. 175-101=74 in the first line.
This would tremendously help me in visualizing if (and if yes, when) my pills start working.
Thank you!
I created the following bash script. Run it with your log file name as the argument:
#! /bin/bash -e
input=$1
start=$(head -n1 "$input")
start=${start:0:9}0000
end=$(tail -n1 "$input")
end=${end:0:9}2359
gnuplot <<EOF
set terminal png
set output 'bp.png'
set xdata time
set timefmt '%Y%m%d %H%M'
set xrange ['$start':'$end']
plot '< grep -v Pill "$input"' using 1:(\$3-\$4):3:4 with errorbars pt 2 title 'Pressure', \
'< grep Pill "$input"' using 1:(100) with points lw 2 pt 7 title 'Pill'
EOF
(($?)) && exit 1
display bp.png
I will answer the second part of your question. Therefor, I updated your file to have something to log.
1 20121001 0801 175 101
2 20121001 0802 Pill
3 20121001 1017 125 91
4 20121001 1537 121 101
5 20121001 1907 117 89
6 20121002 0758 191 111
7 20121002 0759 Pill
8 20121002 1003 117 98
The gnuplot script looks like this
plot "< grep -v Pill meas2" using 1:($4-$5) with line, \
"< grep Pill meas2" using 1:($2*0) ps 10 pt 8
Where the inputlines not containing Pill are selected. You can use $4-$5 to perform the calculation. The other plot plots a symbol at the horizontal axis on the times where pill is in the line.
For the first part, I found some hints: http://t16web.lanl.gov/Kawano/gnuplot/datetime-e.html