I'm working with a really large number of matrix, just different in a single number (meanT0601, meanT0602,meanT0603,... and so on).
I know how to process a sequence of files by using a for loop and %d
but I'm unable to do something similar with matrices.
Any idea about that?? Although it will be a great lost of time I'm considering type every different matrix name to process them.
Thanks in advance
I dont know the command importdata (maybe my matlab version is too old), but if I'm using a combination of dlmread and dlmwrite it works fine.
A = [1 2 3 4; 5 6 7 8];
dlmwrite('file.txt',A);
B{1} = dlmread('file.txt');
Dlmread and dkmwrite save in a csv format. In the End everything is in the cell array B. If you have to use the importdata function, please provide the source or documentation
Related
I have a 17G netcdf file that I am trying to use for analysis. Each variable in the netcdf file is set up like: variable(x,y,z,time). I would like to read in and analyze the variables one 'time' at a time for analysis in Matlab. In other words, I want to use all x, y, and z points at one time. In the past I have had smaller files so reading in a variable has been set up like
fid=netcdf.open('filename/location','NC_NOWRITE');
var_id=netcdf.inqVarID(fid,'varname');
var=netcdf.getVar(fid,var_id);
Is it possible to read in the variables using one time step when the variable is read in? (Incorrect syntax) It'd essentially look like
var=netcdf.getVar(fid,var_id,[:,:,:,time_index]);
Yes, the matlab netcdf command supports this, almost the way you wrote it:
data = netcdf.getVar(fid,var_id,var_index,var_length)
See the matlab documentation for more information. You can also use high-level matlab commands instead of the netCDF library functions.
For example, if varname is a 100x4 array, you could get row 7 by using:
% read 4 columns from 1 row of data starting at row 7, column 1
v = ncread('filename/location','varname',[7 1],[1 4]);
or a four-dimensional array, as in the question:
% read all data from dim. 1-3 at dim 4 = 27
v = ncread('filename/location','varname',[1 1 1 27],[Inf Inf Inf 1]);
I simply want to import a matrix from a .csv into Matlab and find that Matlab is acting differently wrt length of a row in my csv. :
First, I read a file of 2 rows with 50000 columns and Matlab correctly shows a 2*50000 matrix in my workspace.
Now, if the file consists of 2 rows with 100000 columns, Matlab identifies it as a 200000*1 matrix.
What has gone wrong there?
What command are you using? csvread('filename.csv') ?
I would personally prefer to use
Data = importdata( 'filename.csv','\t');
If I perform s.device_macs then I get back a <1x3503 cell> so I would expect this as the output of my concatenate but I have 2 things I'm unsure on when I use: a = cat(2,s.device_macs)
To concatenate previously I used cat(1,x) but this doesn't work however the number 2 lets it run and the second thing is that it returns a <1x603326 cell>, obviously much larger than when I don't try and use cat().
Thanks alot, from a MATLAB newbie!
s.device_macs is a 1 row (first dimension) x 3503 column (second dimension) vector. That is why you have to specify 2 in cat(2,s.device_macs), so that it concatenates along the second dimension -- the columns. My guess is that the 1x603326 result is a string with 603326 characters (in columns), but not entirely sure... hopefully someone else can help here.
I have used a cheating method of solving this. As I mentioned the ans was coming out correct and so I decided to just use this:
s.device_macs; % This gives the answer of <1x3503 cell>
macId = ans; % I now make macId copy answer
clear ans; % Now I wipe ans leaving me with just macId
I know this isn't an efficient method of coding compared to just knowing the language but duct-tape-esque fixes are fun to find :P.
I have a huge CSV file that has a mix of numerical and text datatypes. I want to read this into a single matrix in Matlab. I'll use a simpler example here to illustrate my problem. Let's say I have this CSV file:
1,foo
2,bar
I am trying to read this into MatLab using:
A=fopen('filename.csv');
B=textscan(A,'%d %d', 'delimiter',',');
C=cell2mat(B);
The first two lines work fine, but the problem is that texscan doesn't create a 2x2 matrix; instead it creates a 1x2 matrix with each value being an array. So I try to use the last line to combine the arrays into one big matrix, but it generates an error because the arrays have different datatypes.
Is there a way to get around this problem? Or a better way to combine the arrays?
I am note sure if combining them is a good idea. It is likely that you would be better off with them separate.
I changed your code, so that it works better:
clear
clc
A=fopen('filename.csv');
B=textscan(A,'%d %s', 'delimiter',',')
fclose(A)
Looking at the results
K>> B{1}
ans =
1
2
K>> B{2}
ans =
'foo'
'bar'
Really, I think this is the format that is most useful. If anything, most people would want to break this cell array into smaller chunks
num = B{1}
txt = B{2}
Why are your trying to combine them? They are already together in a cell array, and that is the most combined you are going to get.
There is a natural solution to this, but it requires the Statistics toolbox (version 6.0 or higher). Mixed data types can be read into a dataset array. See the Mathworks help page here.
I believe you can't use textscan for this purpose. I'd use fscanf which always gives you a matrix as specified. If you don't know the layout of the data it gets kind of tricky however.
fscanf works as follows:
fscanf(fid, format, size)
where fid is the fid generated by the fopen
format is the file format & how you are reading the data (['%d' ',' '%s'] would work for your example file)
size is the matrix dimensions ([2 2] would work on your example file).
I've written a script that saves its output to a CSV file for later reference, but the second script for importing the data takes an ungainly amount of time to read it back in.
The data is in the following format:
Item1,val1,val2,val3
Item2,val4,val5,val6,val7
Item3,val8,val9
where the headers are on the left-most column, and the data values take up the remainder of the row. One major difficulty is that the arrays of data values can be different lengths for each test item. I'd save it as a structure, but I need to be able to edit it outside the MATLAB environment, since sometimes I have to delete rows of bad data on a computer that doesn't have MATLAB installed. So really, part one of my question is: Should I save the data in a different format?
Second part of the question:
I've tried importdata, csvread, and dlmread, but I'm not sure which is best, or if there's a better solution. Right now I'm using my own script using a loop and fgetl, which is horribly slow for large files. Any suggestions?
function [data,headers]=csvreader(filename); %V1_1
fid=fopen(filename,'r');
data={};
headers={};
count=1;
while 1
textline=fgetl(fid);
if ~ischar(textline), break, end
nextchar=textline(1);
idx=1;
while nextchar~=','
headers{count}(idx)=textline(1);
idx=idx+1;
textline(1)=[];
nextchar=textline(1);
end
textline(1)=[];
data{count}=str2num(textline);
count=count+1;
end
fclose(fid);
(I know this is probably terribly written code - I'm an engineer, not a programmer, please don't yell at me - any suggestions for improvement would be welcome, though.)
It would probably make the data easier to read if you could pad the file with NaN values when your first script creates it:
Item1,1,2,3,NaN
Item2,4,5,6,7
Item3,8,9,NaN,NaN
or you could even just print empty fields:
Item1,1,2,3,
Item2,4,5,6,7
Item3,8,9,,
Of course, in order to pad properly you would need to know what the maximum number of values across all the items is before hand. With either format above, you could then use one of the standard file reading functions, like TEXTSCAN for example:
>> fid = fopen('uneven_data.txt','rt');
>> C = textscan(fid,'%s %f %f %f %f','Delimiter',',','CollectOutput',1);
>> fclose(fid);
>> C{1}
ans =
'Item1'
'Item2'
'Item3'
>> C{2}
ans =
1 2 3 NaN %# TEXTSCAN sets empty fields to NaN anyway
4 5 6 7
8 9 NaN NaN
Instead of parsing the string textline one character at a time. You could use strtok to break the string up for example
stringParts = {};
tline = fgetl(fid);
if ~ischar(tline), break, end
i=1;
while 1
[stringParts{i},r]=strtok(tline,',');
tline=r;
i=i+1;
if isempty(r), break; end
end
% store the header
headers{count} = stringParts{1};
% convert the data into numbers
for j=2:length(stringParts)
data{count}(j-1) = str2double(stringParts{j});
end
count=count+1;
I've had the same problem with reading csv data in Matlab, and I was surprised by how little support there is for this, but then I just found the import data tool. I'm in r2015b.
On the top bar in the "Home" tab, click on "Import Data" and choose the file you'd like to read. An app window will come up like this:
Import Data tool screenshot
Under "Import Selection" you have the option to "generate function", which gives you quite a bit of customization options, including how to fill empty cells, and what you'd like the output data structure to be. Plus it's written by MathWorks, so it's probably utilizing the fastest available method to read csv files. It was almost instantaneous on my file.
Q1) If you know the max number of columns you can fill empty entries with NaN
Also, if all values are numerical, do you really need "Item#" column? If yes, you can use only "#", so all data is numerical.
Q2) The fastest way to read num. data from a file without mex-files is csvread.
I try to avoid using strings in csv files, but if I have to, I use my csv2cell function:
http://www.mathworks.com/matlabcentral/fileexchange/20135-csv2cell