How can I resolve the "need struct type but got struct" - scala

As you can see on my picture, I have a column named probability and I want to create a new column from the probability column. I want to extract values from the probability column which is an array. But while trying to do so, I receive an error:
"Can't extract value from probability#52427: need struct type but got struct<type:tinyint,size:int,indices:array<int>,values:array<double>>"
Here is my extraction code:
preds_test = preds.withColumn("newCol", col("probability").getItem(3))
Can someone please tell me what I did wrong?

I figured it out. I used a lambda function. This is my code:
preds_subset = preds.select('CustomerID','prediction', probs_churn('probability')).orderBy(asc("probability"))```

Related

Matlab Array of structures: string not working

I am reading an input from a file and importing it into my data to run in Matlab:
parts = strread(tline,'%s','delimiter',';')
employee(i).name = parts(1);
employee(i).salary= str2double(parts(2));
Then I try to print it out:
for i = 1:3
fprintf('salary: %.2f\n',employee(i).salary);
fprintf('employee name: %s\n',employee(i).name);
end
The salary prints with no problem. But the for the variable "name" it gives an error:
Error using fprintf
Function is not defined for 'cell' inputs.
fprintf('employee name: %s\n',employee(i).name);
I looked for some other examples:
access struct data (matlab)
How do I access structure fields dynamically?
Matlab Error: Function is not defined for 'cell' inputs
How do i define a structure in Matlab
But there is nothing to address this case, where only string is not working.
I have not explicitly declared the data as struct, i.e. inside the code there is nowhere the "struct" word is included, but Matlab apparently automatically understand it as an "Array of structures".
Any hints what might be missing here?
All comments are highly appreciated!
The issue is that employee(k).name is a cell (check with iscell(employee(1).name)) and the format string %s doesn't know how to handle that.
The reason that it is a cell is because strread returns a cell array. To grab an element from the result (parts), you want to use {} indexing which returns a string rather than () which will return a cell.
employee(i).name = parts{1};

Check if Matlab struct, dynamic Field Value is empty

I have a struct x, with dynamic Fields, respectively dynamic Field names. But basically, only the first Field is relevant for me.
So I want to check if the Value of the first Field is empty, speak a 1x1cell or a 0x1cell..
or
I'm experimenting e.g. with:
isempty(fieldnames(x))
isempty(x(1))
if isempty(x(1))
msgbox('empty')
else
msgbox('result')
end
but got to no solution. Does anybody have a clue?
Speak, check if the Value of the first Field of the struct is empty or not..
If only the first field is relevant to you, then you can proceed as follows :
Get the fieldnames list of your struct
names=fieldnames(x);
Get the size of the first field
SizeOfFirstField=size(x.(names{1}));
Then you can just check if the first value in SizeOfFirstField is 0 or 1 in your if condition :
if SizeOfFirstField(1)==0
msgbox('empty')
else
msgbox('result')
end
Maybe you can also try this shorter form:
isempty(fieldnames(x))
where x is your struct variable.

Matlab bad cell reference operation when if statement

I have a <850x1> cell called x. Each of the individual structures has a 'Tag' name and 'Data' cell with <7168x1 double> data values.
(i.e.
x{1,1}.Tag = 'Channel1', x{1,1}.Data= <7168x1 double>)
So, I want to go through the x cell, identify the structures with 'Channel1' Tag names and pull out that structure's data. Then, combine the data into a cell called Ch1. Here is my approach so far:
n=1:850
if x{n,1}.Tag == 'Channel1'
Ch1{:,n} = x{n,1}.Data;
end
However, this gives an error: Bad cell reference operation.
Any ideas what may be going wrong?
There are 2 issues here. First, your if statement will compare each entry in the string x{n,1}.Tag to each entry in the string 'Channel1'. If the dimensions are not the same, you will get an error. To fix this, you could use the string compare function, strcmp. The other issue is that you are assigning n to all values between 1 and 850 at once. This is the issue that is producing the actual error you are seeing. Instead, you want to step through each of these values one at a time with a for loop. I would suggest trying the following code:
for n=1:850
if strcmp(x{n,1}.Tag, 'Channel1')
Ch1{:,n} = x{n,1}.Data;
end
end

search a name in dataset error :Undefined function 'eq' for input arguments of type 'cell'

I load a file which has some columns with data.The first line contains ,CITY,YEAR2000 .
The first column has name of cities.The other columns contain number data.
I am trying to search for a specific city using:
data(data.CITY=='Athens',3:end)
where
data = dataset('File','cities.txt','Delimiter',',')
but I receive an error
Undefined function 'eq' for input arguments of type 'cell'.
--------UPDATE-----------------------------
Ok, use :
data(find(strncmp(data.CITY,'Athens',length('Athens'))),3:end)
Have you tried with using strncmp tangled with find?
I would use it this way
find(strncmp(data.CITY,'ATHENS',length('ATHENS')))
EDIT
Other opportunities to exploit would encompass strfind
strfind(data.CITY,'ATHENS')
EDIT 2
You could also try with
data(ismember(data.CITY,'ATHENS'),3:end)
This should lead you to the results you expect (at least I guess so).
EDIT 3
Given your last request I would go for this solution:
inp = input('Name of the CITY: ','s')
Name of the City: ATHENS
data(find(strncmp(data.CITY,inp,length(inp))),3:end)

Set dataset observation names to first column

I am trying to set the first column of my dataset as the observation name. Each observation is a separate stock within dataset 'portfolio'.
I tried this command:
portfolio.Properties.ObsNames = portfolio.ticker
I got the error message
Error using dataset/subsasgnDot (line 31)
NEWNAMES must be a nonempty string or a cell array of nonempty strings.
Error in dataset/subsasgn (line 83)
a = subsasgnDot(a,s,b);
I looked at the code but didn't see where I went wrong.
I tried casting to a cell array but it didn't work. I used this code:
portfolio.Properties.ObsNames = dataset2cell(portfolio.ticker)
and got this error message
Undefined function 'dataset2cell' for input arguments of type 'cell'.
What should I try next?
I managed to cast it by doing the cast on a separate line.