Convert nonuniform cell array to numeric array - matlab

I am using xlsread in MATLAB to read in sheets from an excel file. My goal is to have each column of the excel sheet read as a numeric array. One of the columns has a mix of numbers and numbers+char. For example, the values could be 200, 300A, 450, 500A, 200A, 100. here is what I have so far:
[num, txt, raw] = xlsread(fileIn, sheets{ii}); % Reading in each sheet from a for loop
myCol = raw(:, 4) % I want all rows of column 4
for kk=1:numel(myCol)
if iscellstr(myCol(kk))
myCol(kk) = (cellfun(#(x)strrep(x, 'A', ''), myCol(kk), 'UniformOutput', false));
end
end
myCol = cell2mat(myCol);
This is able to strip off the char from the number but then I am left with:
myCol =
[200]
'300'
[450]
'500'
'200'
[100]
which errors out on cell2mat with:
cell2mat(myCol)
??? Error using ==> cell2mat at 46
All contents of the input cell array must be of the same data type.
I feel like I am probably mixing up () and {} somewhere. Can someone help me out with this?

Let me start from reading the file
[num, txt, raw] = xlsread('test.xlsx');
myCol = raw(:, 4);
idx = cellfun(#ischar,myCol ); %# find strings
data = zeros(size(myCol)); %# preallocate matrix for numeric data
data(~idx) = cell2mat(myCol(~idx)); %# convert numeric data
data(idx) = str2double(regexprep(myCol(idx),'\D','')); %# remove non-digits and convert to numeric

The variable myCol is initially a cell array containing both numbers and strings, something like this in your example:
myCol = {200; '300A'; 450; '500A'; '200A'; 100};
The steps you have to follow to convert the string entries into numeric values is:
Identify the cell entries in myCol that are strings. You can use a loop to do this, as in your example, or you can use the function CELLFUN to get a logical index like so:
index = cellfun(#ischar,myCol);
Remove the letters. If you know the letters to remove will always be 'A', as in your example, you can use a simple function like STRREP on all of your indexed cells like so:
strrep(myCol(index),'A','')
If you can have all sorts of other characters and letters in the string, then a function like REGEXPREP may work better for you. For your example, you could do this:
regexprep(myCol(index),'\D','')
Convert the strings of numbers to numeric values. You can do this for all of your indexed cells using the function STR2DOUBLE:
str2double(regexprep(myCol(index),'\D',''))
The final result of the above can then be combined with the original numeric values in myCol. Putting it all together, you get the following:
>> index = cellfun(#ischar,myCol);
>> result(index,1) = str2double(regexprep(myCol(index),'\D',''));
>> result(~index) = [myCol{~index}]
result =
200
300
450
500
200
100

Related

Matlab cell array parsing

I have a large cell array and I'm trying to vectorize some string parsing. The cell is 100000 x 1 and looks like this:
data =
'"2016-07-27T14:18:08.519Z"'
'"2016-07-27T14:18:16.549Z"'
'"2016-07-27T14:18:21.544Z"'
'"2016-07-27T14:18:27.517Z"'
I want to parse this into two cell arrays that look like this:
date_str, which would look like this:
'2016-07-27'
'2016-07-27'
'2016-07-27'
'2016-07-27'
time_str, which would look like this:
'14:18:08.519'
'14:18:16.549'
'14:18:21.544'
'14:18:27.517'
I have looked at using cellfun(#strsplit,data), but it doesn't let me specify a delimiter for the "strsplit" function.
You can use regexprep to remove the double quotes, and then regexp (with the 'split' option) to split at the desired character. I'm assuming the splitting criterion is simply the occurrence of 'T'.
data = regexprep(data, '^"|"$',''); % remove double quotes
result = regexp(data, 'T', 'split'); % split at 'T'
result = vertcat(result{:}); % un-nest cell array
date_str = result(:,1);
time_str = result(:,2);

MatLab find column number of text cell array

I have a cell data type matrix containing a header and a large number of rows.
sample data:
set press dp
32.7045 17.805965 123.75047
32.690094 17.80584 123.74992
32.6232 17.815094 123.790115
I am trying to find the index of a specific column using the strcmp command to search through the all the data.
dpCol = strcmp([data{:}], 'dp')
This always returns
dpCol =
0
Am I using the data cell type wrong or something? Thank you!
Try using cell notation to yield just the 1st row, EG:
data(1,:) = {'set','press','dp'}
instead of unpacking* the entire cell array since strcmp can operate on cell arrays.
>>> data = {'set' 'press' 'dp'
32.7045 17.805965 123.75047
32.690094 17.80584 123.74992
32.6232 17.815094 123.790115}
data =
'set' 'press' 'dp'
[32.7045] [17.8060] [123.7505]
[32.6901] [17.8058] [123.7499]
[32.6232] [17.8151] [123.7901]
>>> col_idx = strcmp(data(1,:),'dp')
col_idx=
0 0 1
Then return the dp using the logical indices and cell2mat...
>>> dp = cell2mat(data(2:end,col_idx))
dp =
123.7505
123.7499
123.7901
or unpack* and concatenate the comma separated list
>>> dp = [data{2:end,col_idx}]
dp =
123.7505 123.7499 123.7901
As an alternative try cell2struct.
>>> datastruct = cell2struct(data(2:end,:),data(1,:),2)
datastruct =
3x1 struct array with fields:
set
press
dp
Then dp is ...
>>> dp = [datastruct.dp]
dp =
123.7505 123.7499 123.7901
* Using the colon operator inside curly braces unpacks an cell array into a comma separated list. Using square brackets horizontally concatenates the comma separated list which returns a character array set pressdp{{{ since the first item in the list is a character array. The garbage characters between and after 'set', 'press' and 'dp' are caused by reading the doubles as char. IE: char(32.7045) is the ASCII equivalent of whitespace. The arrays always get unpacked as column.

read complicated format .txt file into Matlab

I have a txt file that I want to read into Matlab. Data format is like below:
term2 2015-07-31-15_58_25_612 [0.9934343, 0.3423043, 0.2343433, 0.2342323]
term0 2015-07-31-15_58_25_620 [12]
term3 2015-07-31-15_58_25_625 [2.3333, 3.4444, 4.5555]
...
How can I read these data in the following way?
name = [term2 term0 term3] or namenum = [2 0 3]
time = [2015-07-31-15_58_25_612 2015-07-31-15_58_25_620 2015-07-31-15_58_25_625]
data = {[0.9934343, 0.3423043, 0.2343433, 0.2342323], [12], [2.3333, 3.4444, 4.5555]}
I tried to use textscan in this way 'term%d %s [%f, %f...]', but for the last data part I cannot specify the length because they are different. Then how can I read it? My Matlab version is R2012b.
Thanks a lot in advance if anyone could help!
There may be a way to do that in one single pass, but for me these kind of problems are easier to sort with a 2 pass approach.
Pass 1: Read all the columns with a constant format according to their type (string, integer, etc ...) and read the non constant part in a separate column which will be processed in second pass.
Pass 2: Process your irregular column according to its specificities.
In a case with your sample data, it looks like this:
%% // read file
fid = fopen('Test.txt','r') ;
M = textscan( fid , 'term%d %s %*c %[^]] %*[^\n]' ) ;
fclose(fid) ;
%% // dispatch data into variables
name = M{1,1} ;
time = M{1,2} ;
data = cellfun( #(s) textscan(s,'%f',Inf,'Delimiter',',') , M{1,3} ) ;
What happened:
The first textscan instruction reads the full file. In the format specifier:
term%d read the integer after the literal expression 'term'.
%s read a string representing the date.
%*c ignore one character (to ignore the character '[').
%[^]] read everything (as a string) until it finds the character ']'.
%*[^\n] ignore everything until the next newline ('\n') character. (to not capture the last ']'.
After that, the first 2 columns are easily dispatched into their own variable. The 3rd column of the result cell array M contains strings of different lengths containing different number of floating point number. We use cellfun in combination with another textscan to read the numbers in each cell and return a cell array containing double:
Bonus:
If you want your time to be a numeric value as well (instead of a string), use the following extension of the code:
%% // read file
fid = fopen('Test.txt','r') ;
M = textscan( fid , 'term%d %f-%f-%f-%f_%f_%f_%f %*c %[^]] %*[^\n]' ) ;
fclose(fid) ;
%% // dispatch data
name = M{1,1} ;
time_vec = cell2mat( M(1,2:7) ) ;
time_ms = M{1,8} ./ (24*3600*1000) ; %// take care of the millisecond separatly as they are not handled by "datenum"
time = datenum( time_vec ) + time_ms ;
data = cellfun( #(s) textscan(s,'%f',Inf,'Delimiter',',') , M{1,end} ) ;
This will give you an array time with a Matlab time serial number (often easier to use than strings). To show you the serial number still represent the right time:
>> datestr(time,'yyyy-mm-dd HH:MM:SS.FFF')
ans =
2015-07-31 15:58:25.612
2015-07-31 15:58:25.620
2015-07-31 15:58:25.625
For comlicated string parsing situations like such it is best to use regexp. In this case assuming you have the data in file data.txt the following code should do what you are looking for:
txt = fileread('data.txt')
tokens = regexp(txt,'term(\d+)\s(\S*)\s\[(.*)\]','tokens','dotexceptnewline')
% Convert namenum to numeric type
namenum = cellfun(#(x)str2double(x{1}),tokens)
% Get time stamps from the second row of all the tokens
time = cellfun(#(x)x{2},tokens,'UniformOutput',false);
% Split the numbers in the third column
data = cellfun(#(x)str2double(strsplit(x{3},',')),tokens,'UniformOutput',false)

Creating an array from an str concatenation Matlab

Hello I have these two vectors
Q = [1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4]
and
Year = [2000,2000,2000,2000,2001,2001,2001,2001,2002.....]
and I would like to concatenate them into one single array Time
Time = [20001,20002,20003,20004,20010....]
Or
Time= {'2000Q1', '2000Q2', '2000Q3', '2000Q4', '2001Q1'....}
So far I tried with this code
m = zeros(136,1)
for i=1:136
m(i,1)= strcat(Q(i),Year(i));
end
And Matlab outputed me this:
Subscripted assignment dimension mismatch.
Help pls ?
If your vectors Year and Q have the same number of elements, you do not need a loop, just transpose them (or just make sure they are in column), then concatenate with the [] operator:
Time = [ num2str(Year.') num2str(Q.') ] ;
will give you:
20001
20002
20003
20004
20011
...
And if you want the 'Q' character, insert it in the expression:
Time = [ num2str(Year.') repmat('Q',length(Q),1) num2str(Q.') ]
Will give you:
2000Q1
2000Q2
2000Q3
2000Q4
2001Q1
...
This will be a char array, if you want a cell array, use cellstr on the same expression:
time = cellstr( [num2str(Year.') repmat('Q',length(Q),1) num2str(Q.')] ) ;
To obtain strings:
strtrim(mat2cell(num2str([Year(:) Q(:) ],'%i%i'), ones(1,numel(Q))));
Explanation:
Concat both numeric vectors as two columns (using [...])
Convert to char array, where each row is the concatenation of two numbers (using num2str with sprintf-like format specifiers). It is assumed that all numbers are integers (if not, change the format specifiers). This may introduce unwanted spaces if not all the concatenated numbers have the same number of digits.
Convert to a cell array, putting each row in a different cell (using mat2cell).
Remove whitespaces in each cell (using strtrim)
To obtain numbers: apply str2double to the above:
str2double(strtrim(mat2cell(num2str([Year(:) Q(:) ],'%i%i'), ones(1,numel(Q)))));
Or compute directly
10.^ceil(max(log10(Q)))*Year + Q;
You can use arrayfun
If you want your output in string format (with a 'Q' in the middle) then use sprintf to format the string
Time = arrayfun( #(y,q) sprintf('%dQ%d', y, q ), Year, Q, 'uni', 0 );
Resulting with a cellarray
Time =
'2000Q1' '2000Q2' '2000Q3' '2000Q4' '2001Q1' '2001Q2' '2001Q3'...
Alternatively, if you skip the 'Q' you can save each number in an array
Time = arrayfun( #(y,q) y*10+q, Year, Q )
Resulting with a regular array
Time =
20001 20002 20003 20004 20011 20012 20013 ...
Thats because you are initializing m to zeros(136,1) and then trying to save a full string into the first value. and obviously a double cannot hold a string.
I give you 2 options, but I favor the first one.
1.- you can just use cell arrays, so your code converts into:
m = cell(136,1)
for ii=1:136
m{ii}= strcat(Q(ii),Year(ii));
end
and then m will be: m{1}='2000Q1';
2.- Or if you know that your strings will ALWAYS be the same size (in your case it lokos like they are always 6) you can:
m = zeros(136,strsize)
for ii=1:136
m(ii,:)= strcat(Q(ii),Year(ii));
end
and then m will be: m(1,:)= [ 50 48 48 48 81 49 ] wich translated do ASCII will be 2000Q1

How to read data from a string

I have a string in the following format :
fileName.jpg,10,20,10,10,...,12,14,True
Basically, I have a string with comma separated values. The first value is a string, then it follows an array of 100 values and lastly another string being true or false.
Is there a way or directly reading these values into 3 variable? Two strings and an array?
The array of values might contain n\a values which I want to treat as -1 or something similar or by using a cell array and having an empty cell for those? Can you recommend me something for this type of problem?
You can use textscan:
n = 100; % number of integers between filename and logical values
M = textscan(str, ['%s' repmat('%d',1, n) '%s'], 'delimiter', ',',...
'TreatAsEmpty', 'n\a', 'EmptyValue', -1, 'CollectOutput', true);
The result M is a cell array with the file name in the first cell, the 100 integer values in the second, and a string containing the logical value in the last cell.
You can use strsplit and extract the values from your String and store them in separate variables
Code Sample:
a = strsplit("fileName.jpg,10,20,10,10,...,12,14,True",",")
fileName = a(1)
flag = a(end)
data = a(2:end-1)