I am using MATLAB to load a text file that I want to make a sparse matrix out of. The columns in the text file refer to the row indices and are double type. I need them to be integers to be able to use them as indices for rows and columns. I tried using uint8, int32 and int64 to convert them to integers to use them to build a sparse matrix as so:
??? Undefined function or method 'sparse' for input
arguments of type 'int64'.
Error in ==> make_network at 5
graph =sparse(int64(listedges(:,1)),int64(listedges(:,2)),ones(size(listedges,1),1));
How can I convert the text file entries loaded as double so as to be used by the sparse function?
There is no need for any conversion, keep the indices double:
r = round(listedges);
graph = sparse(r(:, 1), r(:, 2), ones(size(listedges, 1), 1));
There are two reasons why one might want to convert to int:
The first, because you have data type restrictions.
The second, your inputs may contain fractions and are un-fit to be used as integers.
If you want to convert because of the first reason - then there's no need to: Matlab works with double type by default and often treats doubles as ints (for example, when used as indices).
However, if you want to convert to integers becuase of the second reason (numbers may be fractionals), then you should use round(), ceil() or floor() - whatever suits your purpose best.
There is another very good reason ( and really the primary one..) why one may want to convert indices of any structure (array, matrix, etc.) to int.
If you ever program in any language other than Matlab, you would be familiar with wanting to save memory space, especially with large structures. Being able to address elements in such structures with indices other than double is key.
One major issue with Matlab is the inability to more finely control the size of multidimensional structures in this way. There are sparse matrix solutions, but those are not adequate for many cases. Cell arrays will preserve the data types upon access, however the storage for every element in the cell array is extremely wasteful in terms of storage (113 bytes for a single uint8 encapsulated in a cell).
Related
The following error occurs quite frequently:
Subscript indices must either be real positive integers or logicals
I have found many questions about this but not one with a really generic answer. Hence I would like to have the general solution for dealing with this problem.
Subscript indices must either be real positive integers or logicals
In nearly all cases this error is caused by one of two reasons. Fortunately there is an easy check for this.
First of all make sure you are at the line where the error occurs, this can usually be achieved by using dbstop if error before you run your function or script. Now we can check for the first problem:
1. Somewhere an invalid index is used to access a variable
Find every variable, and see how they are being indexed. A variable being indexed is typically in one of these forms:
variableName(index,index)
variableName{index,index}
variableName{indices}(indices)
Now simply look at the stuff between the brackets, and select every index. Then hit f9 to evaluate the result and check whether it is a real positive integer or logical. Visual inspection is usually sufficient (remember that acceptable values are in true,false or 1,2,3,... BUT NOT 0) , but for a large matrix you can use things like isequal(index, round(index)), or more exactly isequal(x, max(1,round(abs(x)))) to check for real positive integers. To check the class you can use class(index) which should return 'logical' if the values are all 'true' or 'false'.
Make sure to check evaluate every index, even those that look unusual as per the example below. If all indices check out, you are probably facing the second problem:
2. A function name has been overshadowed by a user defined variable
MATLAB functions often have very intuitive names. This is convenient, but sometimes results in accidentally overloading (builtin) functions, i.e. creating a variable with the same name as a function for example you could go max = 9 and for the rest of you script/function Matlab will consider max to be a variable instead of the function max so you will get this error if you try something like max([1 8 0 3 7]) because instead of return the maximum value of that vector, Matlab now assumes you are trying to index the variable max and 0 is an invalid index.
In order to check which variables you have you can look at the workspace. However if you are looking for a systematic approach here is one:
For every letter or word that is followed by brackets () and has not been confirmed to have proper indices in step 1. Check whether it is actually a variable. This can easily be done by using which.
Examples
Simple occurrence of invalid index
a = 1;
b = 2;
c = 3;
a(b/c)
Here we will evaluate b/c and find that it is not a nicely rounded number.
Complicated occurrence of invalid index
a = 1;
b = 2;
c = 3;
d = 1:10;
a(b+mean(d(cell2mat({b}):c)))
I recommend working inside out. So first evaluate the most inner variable being indexed: d. It turns out that cell2mat({b}):c, nicely evaluates to integers. Then evaluate b+mean(d(cell2mat({b}):c)) and find that we don't have an integer or logical as index to a.
Here we will evaluate b/c and find that it is not a nicely rounded number.
Overloaded a function
which mean
% some directory\filename.m
You should see something like this to actually confirm that something is a function.
a = 1:4;
b=0:0.1:1;
mean(a) = 2.5;
mean(b);
Here we see that mean has accidentally been assigned to. Now we get:
which mean
% mean is a variable.
In Matlab (and most other programming languages) the multiplication sign must always be written. While in your math class you probably learned that you can write write a(a+a) instead of a*(a+a), this is not the same in matlab. The first is an indexing or function call, while the second is a multiplication.
>> a=0
a =
0
>> a*(a+a)
ans =
0
>> a(a+a)
Subscript indices must either be real
positive integers or logicals.
Answers to this question so far focused on the sources of this error, which is great. But it is important to understand the powerful yet very intuitive feature of matrix indexing in Matlab. Hence how indexing works and what is a valid index would help avoid this error in the first place by using valid indices.
At its core, given an array A of length n, there are two ways of indexing it.
Linear indexing: with subset of integers from 1 : n (duplicates allowed). 0 is not allowed, as Matlab arrays are 1-based, unless you use the method below. For higher-dimensional arrays, multiple subscripts are internally converted into a linear index, although in an efficient and transparent manner.
Logical indexing:wherein you use a n-length array of 0s and 1s, to pick those elements where indexing is true. In this case, unique(index) must have only 0 and 1.
So a valid indexing array into another array with n number of elements ca be:
entirely logical of the same size, or
linear with subsets of integers from 1:n
Keeping this in mind, invalid indexing error occurs when you mix the two types of indexing: one or more zeros occur in your linearly indexing array, or you mix 0s and 1s with anything other than 0s and 1s :)
There is tons of material online to learn this including this one:
http://www.mathworks.com/company/newsletters/articles/matrix-indexing-in-matlab.html
I have a problem in general when Matlab is unable to work out how to logically store values of differing uint types. For example:
tempC = {uint8(5) uint16(16)}
For me, it seems logical to be able to convert this into a matrix of type integer using cell2mat(tempC), which returns
>> cell2mat(tempC)
Error using cell2mat (line 45)
All contents of the input cell array must be of the same data type.
Of course, I understand that the truncation behaviour of integers depends on the type (e.g. uint8 forces all numbers greater than 255 to be 255), however, in this case I would say it would be safe enough to output cell2mat(tempC) with uint16 type. Does anyone have any ideas on how this can be achieved in general?
cell2mat will not work if there are cells of differing types. cell2mat merges the cells together into a matrix, but matrix elements in MATLAB must all share the same type. This is fundamental to how MATLAB works with numeric matrices. If you didn't have all of the same type, then you should use cell arrays... which is what they are for.
However, one thing I can suggest is figure out the type of all of the elements in your matrix, then iterate through each cell and cast them all to be the largest precision type. You can then use cell2mat on this intermediate result to complete the conversion. However, what I have written doesn't actually require calling cell2mat in the end. You'll see later.
Something like this:
%// Get all of the possible types in the array
types = unique(cellfun(#class, tempC, 'uni', 0));
%// Figure out the largest type
vals = cellfun(#(x) double(intmax(x)), types);
[~,ind_max] = max(vals);
%// Cast all values to this type
class_max = types{ind_max};
tempC = cellfun(#(x) cast(x, class_max), tempC);
We first determine all of the possible classes that your cell array contains. We then figure out which of the types is the largest of them all. This can be done by using intmax on each of the types. intmax tells you the largest possible integer that is available for that type, so we basically choose the type that generates the largest possible integer. Take note that I had to cast to double as the output of intmax certainly does output the maximum associated for an integer type, but the output is also cast to that type. This is required so that I can combine all of these elements into an array of the same type - double.
Once we get the type producing the maximum possible integer, we then go through the cell array and cast all of the values to this type. Take note that I used cellfun for the final call which outputs a numeric array - no need to use cell2mat here. In the last line of code, I use cast to cast all of the numbers in the cell array to this type, thus achieving "coercion".
Using your example array, this is what I get, as well as what class the final array is in:
>> tempC
tempC =
5 16
>> class(tempC)
ans =
uint16
I have a real 2D double precision array. I want to perform a FFT on it, some operations on the result, and an inverse FFT. I am using IBM ESSL library on Blue Gene Q.
The function DRCFT2 is doing the real to complex transform (http://www-01.ibm.com/support/knowledgecenter/SSFHY8_5.3.0/com.ibm.cluster.essl.v5r3.essl100.doc/am5gr_hsrcft2.htm?lang=en). The function DCRFT2 is doing the complex to real transform (http://www-01.ibm.com/support/knowledgecenter/SSFHY8_5.3.0/com.ibm.cluster.essl.v5r3.essl100.doc/am5gr_hscrft2.htm?lang=en).
Beginning real array size is (nx,nz). After DRCFT2, the complex array size is (nx/2+1,nz). After DCRFT2, the final real array size is (nx+2,nz).
Beginning and final real arrays have a different size, how can I compare them?
ps: If I put the first real array in a complex one and perform complex to complex DFTs (DCFT2), then the final result and the first one will have the same size and I can compare them. Anyway to do something similar with DRCFT2 and DCRFT2?
According to the DCRFT2 documentation you link to:
x
is the array X, containing n2 columns of data to be transformed. Due to complex conjugate symmetry, the input consists of only the first ((n1)/2)+1 rows of the array
[...]
On Return
y
[...]
is the array Y, containing n1 rows and n2 columns of results of the real discrete Fourier transform of X.
Where in you case n1=nx and n2=nz. In other words, if you put in a complex array of size (nx/2+1,nz) as input argument to DCRFT2 you should get a real array output of size (nx,nz), so you can readily compare your beginning and final real arrays.
I am fairly new to matlab and I am trying to figure out when it is best to use cells, tables, or matrixes to store sets of data and then work with the data.
What I want is to store data that has multiple lines that include strings and numbers and then want to work with the numbers.
For example a line would look like
'string 1' , time, number1, number 2
. I know a matrix works best if al elements are numbers, but when I use a cell I keep having to convert the numbers or strings to a matrix in order to work with them. I am running matlab 2012 so maybe that is a part of the problem. Any help is appreciated. Thanks!
Use a matrix when :
the tabular data has a uniform type (all are floating points like double, or integers like int32);
& either the amount of data is small, or is big and has static (predefined) size;
& you care about the speed of accessing data, or you need matrix operations performed on data, or some function requires the data organized as such.
Use a cell array when:
the tabular data has heterogeneous type (mixed element types, "jagged" arrays etc.);
| there's a lot of data and has dynamic size;
| you need only indexing the data numerically (no algebraic operations);
| a function requires the data as such.
Same argument for structs, only the indexing is by name, not by number.
Not sure about tables, I don't think is offered by the language itself; might be an UDT that I don't know of...
Later edit
These three types may be combined, in the sense that cell arrays and structs may have matrices and cell arrays and structs as elements (because thy're heterogeneous containers). In your case, you might have 2 approaches, depending on how you need to access the data:
if you access the data mostly by row, then an array of N structs (one struct per row) with 4 fields (one field per column) would be the most effective in terms of performance;
if you access the data mostly by column, then a single struct with 4 fields (one field per column) would do; first field would be a cell array of strings for the first column, second field would be a cell array of strings or a 1D matrix of doubles depending on how you want to store you dates, the rest of the fields are 1D matrices of doubles.
Concerning tables: I always used matrices or cell arrays until I
had to do database related things such as joining datasets by a unique key; the only way I found to do this in was by using tables. It takes a while to get used to them and it's a bit annoying that some functions that work on cell arrays don't work on tables vice versa. MATLAB could have done a better job explaining when to use one or the other because it's not super clear from the documentation.
The situation that you describe, seems to be as follows:
You have several columns. Entire columns consist of 1 datatype each, and all columns have an equal number of rows.
This seems to match exactly with the recommended situation for using a [table][1]
T = table(var1,...,varN) creates a table from the input variables,
var1,...,varN . Variables can be of different sizes and data types,
but all variables must have the same number of rows.
Actually I don't have much experience with tables, but if you can't figure it out you can always switch to using 1 cell array for the first column, and a matrix for all others (in your example).
I'm working on a model to use matlab as graphical representation for other model. Therefore I'd like to have a matrix that can be updated with both letters and numbers. Numbers will represent a speed while for example '-' may represent a empty section. In the matlab documentation and on internet I found a lot of interesting tips, but not what I need.
Thanks in advance!
You cannot represent data of numeric type (integers/floating points) and data of char type in a matrix. However, you can, use cells, which are similar to matrices, and can hold different data types in each cell. Here's an example.
A={[1 2 3],'hello';'world',[4,5,6]'}
A =
[1x3 double] 'hello'
'world' [3x1 double]
Here the first cell contains a row vector, the second and third cells contain strings and the fourth cell contains a column vector. Indexing into a cell is similar to that of arrays, with one minor difference: use {} to group the indices. e.g., to access the element in the second row, first column, do
A{2,1}
ans =
world
You can also access an element of an array inside a cell like
A{2,2}(2)
ans =
5
If you're wanting to store mixtures of numeric and character type data, yoda has the correct suggestion: use cell arrays.
However, based on the example you described you may have another option. If the character entries in your matrix are there for the purpose of identifying "missing data", it may make more sense to use a purely numeric matrix containing unique values like NaN or Inf to identify data points that are empty or where data is not available.
When performing operations on your matrix, you would then have to index only elements that are finite (using, for example, ISFINITE) and perform your calculations on them. There are even some functions in the Statistics Toolbox that will perform operations ignoring NaN values. This may be a cleaner way to go since you can keep your matrix as a numeric type ('single' or 'double' precision) instead of having to mess with cell arrays.