I need to save a huge matrix A in Matlab in .txt format (without comma and semicolon). The matrix A is generated at each iteration of a loop. At the moment I'm using dlmwrite to save but it is very very slow, e.g.
for h=1:100
A=randn(1000000,1000);
dlmwrite(sprintf('B%d.txt',h), A, ' ')
end
Could you suggest a faster way that gives exactly the same output?
Related
Im Working on Simulation Dehumidification Process, i should save .Mat file in every loop, my program flowchart is:
enter code here
for m=9:2:21
for kk=1:ll
for jj=1:mm
for ii=1:nn
...
...
...
end
end
end
A=min(X-Y)
end
for example
mm=9 then A=1
mm=11 then A=2
..., How i can Plot A with mm?
and How i can Save .Mat file in every mm iteration? Thanks.
Blockquote
%A=(7*1)Matrix %9:2:21=7(Number)
If you want to save a .mat-file for each iteration the only thing you need to do is to generate a unique filename for each iteration within the loop. This can be done using format strings, for instance in your case something like
filename = sprintf('output_kk=%d_jj=%d_ii=%d.mat', [kk jj ii]);
save(filename);
You have the option to save specific variables by adding them as options to the save command. For more about string formatting I'd suggest you check out the sprintf documentation.
I'm not sure whether this is the most efficient way to do it. Depending on the number and size of output variables you're interested in you can also create a cell structure and save your data into a new cell for each iteration.
I have the m-file on generating all the graphs in .fig I need with a for loop, but I don't know how to extract the array data to csv, txt or ascii format (16-bit).
imagesc(x,y,C); %C is the data I want to extract
%m and n are variables created inside the for loop
I have tried dlmwrite and save, but I failed and could not fix the problem.
So I want to convert fig into csv, txt or ascii format instead.
filename_B=strcat(MM,'_profile'); %MM is a variable created inside the for loop
dlmwrite(filename_B.txt,squeeze(Data_time(:,m,n,:)),''); %Data_time is C
save(filename_B, squeeze(Data_time(:,m,n,:)),'-ascii','-double');
I have also tried csvwrite
filename_B=strcat(MM,'_profile');
csvwrite(filename_B.txt, squeeze(Data_time(:,m,n,:)));
but there is an error message, "??? Attempt to reference field of non-structure array."
It would be better if I could just extract the data directly to the desirable format without first producing fig then convert.
For confidential issue, I can't provide the whole script, but I will try my best to explain my problem.
Many thanks!
I have created a script to create ~300 variable names in my workspace. I need to save these .mat files to .txt files - is there any way to call these variables in some sort of loop or just save them all at once to .txt files?
They are all cells of varying size, about 1000x5 on average.
The first row of the data contains strings, the other elements are all non integer numbers.
Here is an example of one of the .mat files-
http://www.yourfilelink.com/get.php?fid=1065782
I don't know a whole lot about Matlab so any help would be really appreciated! Thank you.
A possible solution could be:
to load the .matfiles through a loop (the name of the file can be either dynamically generated or read from a list)
to extract the first row of the cellarray (the string) and write it in a .txt file by using the dlmwrite function
to remove the first row and write (again with dlmwrite) the numerical values in the .txt having converted them using the cell2mat function.In the writing of the numerical values, the precision has to carefully set.
file_name_root='x2413';
% dummy loop (one iteration) just to test dynamic filename generation
for i=3:3
% generate the name of the input file
input_file_name=[file_name_root int2str(i)];
% generate the name of the output file
output_file_name=[file_name_root int2str(i) '.txt'];
% extract the name of the cellarray and assign it to "the_cell" variable
tmp=load(input_file_name);
var_name=char(fieldnames(tmp));
eval(['the_cell=tmp.' var_name ';'])
[r,c]=size(the_cell);
str='';
% get the first row string (probably a cleaver way to do it exists)
for i=1:c
str=[str ' ' char(the_cell(1,i))];
end
% print the first row in a ".txt" file
dlmwrite(output_file_name,sprintf('%s',str),'delimiter','')
% remove the first row
the_cell(1,:)=[];
% write the numerical values in the ".txt" file
% the "precision" shall be carefully set
dlmwrite(output_file_name,cell2mat(the_cell),'precision','%13.9f','delimiter',' ','-append')
end
kinda new to matlab here, searching the csvwrite tutorial and some of the existing webportals regarding my question couldn't find a way to pass my variables by value to the output file names while exporting in csv; providing my bellow scripts, i would like to have the output files something like output_$aa_$dd.csv which aa and dd are respectively the first and second for counters of the scripts.
for aa=1:27
for dd=1:5
M_Normal=bench(aa,dd).Y;
for j=1:300
randRand=M_Normal(randperm(12000,12000));
for jj = 1:numel(randMin(:,1)); % loops over the rand numbers
vv= randMin(jj,1); % gets the value
randMin(jj,j+1)=min(randRand(1:vv)); % get and store the min of the selction in the matix
end
end
csvwrite('/home/amir/amir_matlab/sprintf(''%d%d',aa, bb).csv',randMin);
end
end
String concatenation in MATLAB is done like a matrix concatenation. For example
a='app';
b='le';
c=[a,b] % returns 'apple'
Hence, in your problem, the full path can be formed this way.
['/home/amir/amir_matlab/',sprintf('%d_%d',aa,bb),'.csv']
Furthermore, it is usually best not to specify the file separator explicitly, so that your code can be implemented in other operating systems. I suggest you write the full path as
fullfile('home','amir','amir_matlab',sprintf('%d_%d.csv',aa,bb))
Cheers.
I have a 30000x14000 sparse matrix in MATLAB (version 7), which I need to use in another program. Calling save won't write this as ASCII (not supported). Calling full() on this monster results in an Out of Memory error.
How do I export it?
You can use find to get index & value vectors:
[i,j,val] = find(data)
data_dump = [i,j,val]
You can recreate data from data_dump with spconvert, which is meant to "Import from sparse matrix external format" (so I guess it's a good export format):
data = spconvert( data_dump )
You can save to ascii with:
save -ascii data.txt data_dump
But this dumps indices as double, you can write it out more nicely with fopen/fprintf/fclose:
fid = fopen('data.txt','w')
fprintf( fid,'%d %d %f\n', transpose(data_dump) )
fclose(fid)
Hope this helps.
Save the sparse matrix as a .mat file. Then, in the other program, use a suitable library to read the .mat file.
For instance, if the other program is written in Python, you can use the scipy.io.mio.loadmat function, which supports sparse arrays and gives you a sparse numpy matrix.
I saved it as text using Java within MATLAB.
MATLAB Code:
pw=java.io.PrintWriter(java.io.FileWriter('c:\\retail.txt'));
line=num2str(0:size(data,2)-1);
pw.println(line);
for index=1:length(data)
disp(index);
line=num2str(full(data(index,:)));
pw.println(line);
end
pw.flush();
pw.close();
Here data is an extremely large sparse matrix.
Did you try partitioning it ?
I mean try calling full() on the 1000 first rows (or 5000) and then repeat the process if it works.
Use the find function to get the indices of non-zero elements...
idcs = find(data);
vals = data(idcs);
...save the index vector and value vector in whatever format you want...
If you want, you can use ind2sub to convert the linear indices to row, column subscripts.
If you need to recreate a sparse matrix in matlab from subscripts + values, use spconvert.
dlmwrite - Write matrix to ASCII-delimited file
Syntax
dlmwrite(filename, M)
dlmwrite(filename, M, 'D')
dlmwrite(filename, M, 'D', R, C)
dlmwrite(filename, M, 'attrib1', value1, 'attrib2', value2, ...)
dlmwrite(filename, M, '-append')
dlmwrite(filename, M, '-append', attribute-value list)
If this is pretty much a one time deal, then I would just iterate through the matrix and write the matrix to an ASCII file by brute force, or else use #Veynom's suggestion and call full() on a subset of rows. It may take a while, but it will probably be done faster than it might take to learn how to read in a .mat file outside of the MATLAB environment.
If this is something you need to do on a recurring basis, then I would take #Vebjorn's advice and use a library to read the .mat file.
Use this script:
msm_to_mm.m, writes an MATLAB sparse matrix to an MatrixMarket file.
And This thread may also be useful.