Is there an option in matlab or a plugin/app or a trick such that if you are in an interactive command session, every time it would print out a matrix way too big for a human to look through, it redacts the output to either a warning of how big the matrix is or a summary (only a few rows and columns) of the matrix?
There are many times where I want to examine a matrix in the command window, but I didn't realize how big it was, so I accidentally printed the whole thing out. Or some place inside a function I did not code myself, someone missed a semicolon and I handed it a big matrix, and it dumps the whole thing in my command window.
It make sense that in 99.99% of the time, people do not intend to print a million row matrix in their interactive command window, right? It completely spams their scroll buffer and removes all useful information that you had on screen before.
So it makes much more sense for matlab to automatically assume that the user in interactive sessions want to output a summary of a big matrix, instead of dumping the whole thing into the command window. There should at least be such an option in the settings.
One possibility is to overload the display function, which is called automatically when you enter an expression that is not terminated by ;. For example, if you put the following function in a folder called "#double" anywhere on your MATLAB path, the default display behavior will be overridden for double arrays (this is based on Mohsen Nosratinia's display.m for displaying matrix dimensions):
% #double/display.m
function display(v)
% DISPLAY Display a variable, limiting the number of elements shown.
name = inputname(1);
if isempty(name)
name = 'ans';
end
maxElementsShown = 500;
newlines = repmat('\n',1,~strcmp(get(0,'FormatSpacing'),'compact'));
if numel(v)>maxElementsShown,
warning('display:varTooLong','Data not displayed because of length.');
% OR show the first N=maxElementsShown elements
% builtin('disp', v(1:maxElementsShown));
elseif numel(v)>0,
fprintf([newlines '%s = \n' newlines], name);
builtin('disp', v);
end
end
For example,
>> xx=1:10
xx =
1 2 3 4 5 6 7 8 9 10
>> xx=1:1e4
Warning: Data not displayed because of length.
> In double.display at 17
EDIT: Updated to respect 'compact' and 'loose' output format preference.
EDIT 2: Prevent displaying an empty array. This makes whos and other commands avoid an unnecessary display.
Related
I am very new to MATLAB and i am currently trying to learn how to import files in matlab and work on it. I am importing a "*.dat" file which contains a single column of floating point numbers[they are just filter coefficients I got from a c++ code] into an array in MATLAB. When I am displaying the output in command window the first line is always " 1.0e-03 * " followed by the contents of my file. I want to know what it means? When I check my workspace the array connects the correct number of inputs. My sample code and first few lines of output are below:
Code:-
clear; clc;
coeff = fopen('filterCoeff.dat');
A = fscanf(coeff, '%f');
A
fclose(coeff);
Output:-
A =
**1.0e-03 *** <===== What does this mean?
-0.170194000000000
0
0.404879000000000
0
-0.410347000000000
P.S: I found many options to read file eg. textscan, fscanf etc. Which one is the best to use?
It is a multiplier that applies to all the numbers displayed after that. It means that, for example, the last entry of A is not -0.410347 but -0.410347e-3, that is, -0.000410347.
I think it is is just Matlab's display number type. It means each of your results are scaled by that amount.
format longg
A
And see what it displays. Look at the docs for format for other options.
I have made a script which contains a for loop selecting columns from 533 different excel files and places them into matrices so that they can be compared, however the process is taking too long (it ran for 3 hours yesterday and wasn't even halfway through!!).
I know xlsread is naturally slow, but does anyone know how I can make my script run faster? The script is below, thanks!!
%Split the data into g's and h's
CRNum = 533; %Number of Carrington Rotation files
A(:,1) = xlsread('CR1643.xlsx','A:A'); % Set harmonic coefficient columns
A(:,2) = xlsread('CR1643.xlsx','B:B');
B(:,1) = xlsread('CR1643.xlsx','A:A');
B(:,2) = xlsread('CR1643.xlsx','B:B');
for k = 1:CRNum
textFileName = ['CR' num2str(k+1642) '.xlsx'];
A(:,k+2) = xlsread(textFileName,'C:C'); %for g
B(:,k+2) = xlsread(textFileName,'D:D'); %for h
end
Don't use xlsread if you want to go through a loop. because it opens excel and then closes excel server each time you call it, which is time consuming. instead before the loop use actxserver to open excel, do what you want and finally close actxserver after your loop. For a good example of using actxserver, search for "Read Spreadsheet Data Using Excel as Automation Server" in MATLAB help.
And also take a look at readtable which works faster than xlsread, but generates a table instead.
The most obvious improvement seems to be to load the files only partially if possible. However, if that is not an option, try whether it helps to only open each file once (read everything you need, and then assign it).
M(:,k+2) = xlsread(textFileName,'C:D');
Also check how much you are reading in each time, if you read in many rows in the first file, you may make the first dimension of A big, and then you will fill it each time you read a file?
As an extra: a small but simple improvment can be found at the start. Don't use 4 load statements, but use 1 and then assign variables based on the result.
As mentioned in this post, the easiest thing to change would be to set 'Basic' to true. This disables things like formulas and macros in Excel and allows you to read a simple table more quickly. For example, you can use:
xlsread('CR1643.xlsx','A:A', 'Basic', true)
This resulted in a decrease in load time from about 22 seconds to about 1 second for me when I tested it on a 11,000 by 7 Excel sheet.
I want to know if it is possible to drag pattern values in matlab uitable. In a spreadsheet, to enter values from 1 to 50, you need to enter 1,2,3 and select the cells and drag. Please can this be done in matlab uitable? Regards.
It can be done. But not as far as comfortable as with excel.
Play around a bit with the following code, you can try to improve it or change it to your needs. I think it is a good starting point for you.
function fancyTable
defaultData = randi(99,25,2);
h = figure('Position',[300 100 402 455],'numbertitle','off','MenuBar','none');
uitable(h,'Units','normalized','Position',[0 0 1 1],...
'Data', defaultData,...
'Tag','myTable',...
'ColumnName', [],'RowName',[],...
'ColumnWidth', {200 200},...
'CellSelectionCallback',#cellSelect);
end
function cellSelect(src,evt)
try
index = evt.Indices;
data = get(src,'Data');
L = size(index,1);
rows = index(:,1);
column = index(1,2);
start = data(rows(1),column);
newdata = start:(start+L-1);
data(rows,column) = newdata';
set(src,'Data',data);
end
end
It creates a table with two columns:
You can select data and your desired drag pattern is applied immediately according to the first value.
The code is just to insert an increasing series of values at the first point of selection based on the according value. The hardest part will be to detect the pattern! I just evaluated the first data value start = data(rows(1),column); you could also require a minimal selection of 3: start = data(rows(1:3),column);. You probably need to work with a lot of try/catch structures to skip all unexplained cases. Or you use switch/case structures from the beginning to evaluate the length of the selection and evaluate the pattern.
All in all it is a heavy task, I'm not sure if it's worth it. But it can be done.
In uitable you insert data (usually a matrix) to be displayed in a table. So unlike Excel the uitable function is merely a form of displaying your data instead of a tool to manipulate it. See for more information here. However, if you want to set up a row for instance running from 1 until 10 you could use the following steps:
So say one would like to display a matrix of size 10x10, e.g.
A=magic(10);
You can now set up a table t to display this matrix by
t=uitable('Data',A);
In your case if you want a row to be, e.g., 1 till 10, just change the matrix A containing your data to hold this row using
A(1,1:10)=1:10;
And re-execute the former command to bring up your table t.
The HITRAN database is a listing of molecular rotational-vibrational transitions. It is given in a text file where each line is 160 characters, with fixed width fields defining molecule, isotope, etc. The format is well documented, and there is even a program on the MathWorks File Exchange that will read in the database and simulate a portion of the spectrum. However, I need to read in a specific portion of the spectrum and then use it to do some fitting to a measured spectrum, so I need something much more custom.
As given in the comment section of that function, as well as elsewhere, the following line should read each line in properly:
database = which('HITRAN2012.par');
fid = fopen(database);
hitran = textscan(fid,'%2u%1u%12f%10f%10f%5f%5f%10f%4f%8f%15c%15c%15c%15c%6c%12c%1c%7f%7f','delimiter','','whitespace','');
fclose(fid);
The first two fields denote the molecule code, which runs from 1-47, and the isotope code which runs from 1-9.
Unfortunately, molecules 1-9 do not have a leading zero, and no matter what I do, it seems to silently confuse MATLAB. If I load in the entire database and then type
unique(hitran{1})
I do not get the numbers 1-47, but I get 10-92 with a few numbers missing. As far as I can figure, when MATLAB encounters a leading space, it shifts the line over and then pads the end, so that ' 12' becomes '12', but I'm not exactly sure. I have also tried
hitran = textscan(fid,'%160c','delimiter','\n','whitespace','');
and then tried to parse the resulting strings, but that also sometimes gets confused by the first space.
For instance, the first water line looks like
exampleHitranLine = ' 14 0.007002 1.165E-32 2.071E-14.05870.305 818.00670.590.000000 0 0 0 0 0 0 7 5 2 7 5 3 005540 02227 5 2 0 90.0 90.0';
The first bit of code comes across this line and returns '14' instead of ' 1' and '4'. If I just read in a subset that only contains molecule 1 (as in this example), then the second method of reading works fine. If I try to read in the entire database, however, the lines with molecule 1-9 are shifted the the left, which messes up all the other fields.
I should note that I've tried reading the numerical fields both as floats and as integers, and neither gives satisfactory results. The entire database in text form is nearly 700 MB, and so I need something that works as efficiently as possible.
What am I doing wrong?
I have a new file on the FileExchange that will read in HITRAN 2004+ format data. Please try it out and let me know if there are any issues with it.
I don't have an answer as to why this is happening, but I do have a solution. If anyone has an answer as to why, I'd be happy to accept it.
It is the leading space that is screwing things up. MATLAB is being a little too clever, and when textscan encounters a leading space, it decides that it's extra and discards it and moves on to the next two characters. To get it to properly read in the file, I had to go line by line and test whether the first character is a space and then replace it with a leading zero, like this:
database = which('HITRAN2012_First100Lines.par');
fileParams = dir(database);
K = fileParams.bytes/162;
hitran = cell(K,19);
fid = fopen(database);
for k = 1:K
hitranTemp = fgetl(fid);
if abs(hitranTemp(1)) == 32;
hitranTemp(1) = '0';
end
hitran(k,:) = deal(textscan(hitranTemp,'%2u%1u%12f%10f%10f%5f%5f%10f%4f%8f%15c%15c%15c%15c%6c%12c%1c%7f%7f','delimiter','','whitespace',''));
end
fclose(fid);
I'm working in MATLAB 2013a. Should I consider this to be a bug and report it? Is there some reason that the leading space should be gobbled up like this?
Update:
My workaround above was slow, but worked. Then I had to process the HITEMP database, which is several times larger, so I finally did submit a support ticket to MathWorks. The workaround suggested by MathWorks technical support is to read everything in as text and then convert. This saves a lot of disk reads and works.
fileParams = dir(database);
fid = fopen(database);
hitran = textscan(fid,'%2c%1c%12c%10c%10c%5c%5c%10c%4c%8c%15c%15c%15c%15c%6c%12c%1c%7c%7c','delimiter','','whitespace','');
fclose(fid);
moleculeNumber = uint8(str2num(hitran{1}));
isotopologueNumber = uint8(str2num(hitran{2});
vacuumWavenumber = str2num(hitran{3});
...
etc.
Depending on the application, for larger databases one would probably want to do this in chunks rather than all at once.
He also said he would forward the behavior to the development team for consideration in a future update.
I've written a script that saves its output to a CSV file for later reference, but the second script for importing the data takes an ungainly amount of time to read it back in.
The data is in the following format:
Item1,val1,val2,val3
Item2,val4,val5,val6,val7
Item3,val8,val9
where the headers are on the left-most column, and the data values take up the remainder of the row. One major difficulty is that the arrays of data values can be different lengths for each test item. I'd save it as a structure, but I need to be able to edit it outside the MATLAB environment, since sometimes I have to delete rows of bad data on a computer that doesn't have MATLAB installed. So really, part one of my question is: Should I save the data in a different format?
Second part of the question:
I've tried importdata, csvread, and dlmread, but I'm not sure which is best, or if there's a better solution. Right now I'm using my own script using a loop and fgetl, which is horribly slow for large files. Any suggestions?
function [data,headers]=csvreader(filename); %V1_1
fid=fopen(filename,'r');
data={};
headers={};
count=1;
while 1
textline=fgetl(fid);
if ~ischar(textline), break, end
nextchar=textline(1);
idx=1;
while nextchar~=','
headers{count}(idx)=textline(1);
idx=idx+1;
textline(1)=[];
nextchar=textline(1);
end
textline(1)=[];
data{count}=str2num(textline);
count=count+1;
end
fclose(fid);
(I know this is probably terribly written code - I'm an engineer, not a programmer, please don't yell at me - any suggestions for improvement would be welcome, though.)
It would probably make the data easier to read if you could pad the file with NaN values when your first script creates it:
Item1,1,2,3,NaN
Item2,4,5,6,7
Item3,8,9,NaN,NaN
or you could even just print empty fields:
Item1,1,2,3,
Item2,4,5,6,7
Item3,8,9,,
Of course, in order to pad properly you would need to know what the maximum number of values across all the items is before hand. With either format above, you could then use one of the standard file reading functions, like TEXTSCAN for example:
>> fid = fopen('uneven_data.txt','rt');
>> C = textscan(fid,'%s %f %f %f %f','Delimiter',',','CollectOutput',1);
>> fclose(fid);
>> C{1}
ans =
'Item1'
'Item2'
'Item3'
>> C{2}
ans =
1 2 3 NaN %# TEXTSCAN sets empty fields to NaN anyway
4 5 6 7
8 9 NaN NaN
Instead of parsing the string textline one character at a time. You could use strtok to break the string up for example
stringParts = {};
tline = fgetl(fid);
if ~ischar(tline), break, end
i=1;
while 1
[stringParts{i},r]=strtok(tline,',');
tline=r;
i=i+1;
if isempty(r), break; end
end
% store the header
headers{count} = stringParts{1};
% convert the data into numbers
for j=2:length(stringParts)
data{count}(j-1) = str2double(stringParts{j});
end
count=count+1;
I've had the same problem with reading csv data in Matlab, and I was surprised by how little support there is for this, but then I just found the import data tool. I'm in r2015b.
On the top bar in the "Home" tab, click on "Import Data" and choose the file you'd like to read. An app window will come up like this:
Import Data tool screenshot
Under "Import Selection" you have the option to "generate function", which gives you quite a bit of customization options, including how to fill empty cells, and what you'd like the output data structure to be. Plus it's written by MathWorks, so it's probably utilizing the fastest available method to read csv files. It was almost instantaneous on my file.
Q1) If you know the max number of columns you can fill empty entries with NaN
Also, if all values are numerical, do you really need "Item#" column? If yes, you can use only "#", so all data is numerical.
Q2) The fastest way to read num. data from a file without mex-files is csvread.
I try to avoid using strings in csv files, but if I have to, I use my csv2cell function:
http://www.mathworks.com/matlabcentral/fileexchange/20135-csv2cell