Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 6 years ago.
Improve this question
I want to do k-means clustering to classify Testing data based on Training data both of which have 3 classes (1,2 and 3).
How would I classify the Testing data set using a cluster size of e.g. k=10 in kmeans (e.g. using Matlab)? I know that I can have k=3 and then use nearest neighbour to identify the data based on its nearest cluster size... but not sure what I would use for values other that k=3? How would you label each of those 10 clusters?
Thanks
The classification of 10 clusters would be no different than the classification of 3 clusters. The number of clusters given by k-means is independent of the number of "classes" in the data. k-means is an unsupervised learning algorithm, meaning that it gives no consideration to the class of the training data during training.
The algorithm would look something like this:
distances = dist(test_point, cluster_centers)
cluster = clusters[ min(distances) ]
class = mode(cluster.class)
where we find the cluster with minimum distance between the cluster center and our test point, then we find the most common class label among the elements contained in that minimally-distant cluster.
It is a little bit unclear what exactly you want to do, although here is an outline from what I understand.
When you are clustering data, the labels are ideally not present, as either you use the clustering to get insights from the data or use it for pre-processing.
Although, if you want to perform a clustering and then assign class id to a new datapoint based on the nearness of the cluster centers, then you can do the following.
First, you select the k by bootstrapping or other methods, maybe use Silhouette coefficients. Once you get the cluster centers, check which center is closest to the new datapoint and assign the class id accordingly.
In such cases you might be interested to use the Rand Index or the Adjusted Rand Index, to get the cluster quality.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I collected data from different sources FB, Twitter, Linkedin, then made them in a structured format. As a result now: I'm having a csv file with 10000 rows (10000 person) and the data associated is about their names, age,their interests and buying habits.
I'm really stuck on this step: CLASSIFICATION or CLUSTERING. For the classification I don't really have predefined classes or a model for my users to classify them.
For clustering: I started calculating similarities and KMeans, but still can't get the result I wanted. How can I decide what to choose before moving on to the next step of Collaborative filtering?
Foremost, you have to understand that clustering is a pre-processing activity/task. The idea in clustering is to identify objects with similar properties and group them. The clustering process can be understood in terms of cattle-herding. Wherein the jockey herds loose cattle (read data points) into groups.
Note: If you are looking at the partitioning clustering algorithm family includes K-means, k-modes, k-prototype etc. The algorithm k-means will work only for numerical data. K-modes will work only for categorical data and k-prototype will work for both numerical and categorical data.
Question: Is the data preprocessed? If the answer is no, then you may try the following steps;
Is the data (column values) all categorical (=text) format or numerical or mixed?
a. If all categorical then discretize or bin or interval scale them.
b. if mixed, then discretize or bin or interval scale the categorical values only
c. Perform missing value and outlier treatment for both numerical and categorical data. This will help in retaining maximum variance as well as reduce dimensionality.
d. Normalize the numerical values to a median of zero.
Now apply a suitable clustering algorithm (based on your problem) to determine patterns. Once you have found the patterns, then you may label them. Once the identified patterns are labelled, thereafter or subsequently a classification algorithm can be used to classify any new incoming data points into an appropriate class.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I have the following theoretical questions regarding the conv layer in a CNN. Imagine a conv layer with 6 filters (conv1 layer and its 6 filters in the figure).
1) what guarantees the diversity of learned filters within a conv layer? (I mean, how the learning (optimization process) makes sure that it does not learned the same (similar) filters?
2) diversity of filters within a conv layer is a good thing or not? Is there any research on this?
3) during the learning (optimization process), is there any interaction between the filters of the same layer? if yes, how?
1.
Assuming you are training your net with SGD (or a similar backprop variant) the fact that the weights are initialized at random encourage them to be diverse, since the gradient w.r.t loss for each different random filter is usually different the gradient will "pull" the weights in different directions resulting with diverse filters.
However, there is nothing that guarantees diversity. In fact, sometimes filters become tied to each other (see GrOWL and references therein) or drop to zero.
2.
Of course you want your filters to be as diverse as possible to capture all sorts of different aspects of your data. Suppose your first layer will only have filters responding to vertical edges, how is your net going to cope with classes containing horizontal edges (or other types of textures)?
Moreover, if you have several filters that are the same, why computing the same responses twice? This is highly inefficient.
3.
Using "out-of-the-box" optimizers, the learned filters of each layer are independent of each other (linearity of gradient). However, one can use more sophisticated loss functions/regularization methods to make them dependent.
For instance, using group Lasso regularization, can force some of the filters to zero while keeping the others informative.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
When building a convolutional neural network, how do you determine the number of filters used in each convolutional layer. I know that there is no hard rule about the number of filters, but from your experience/ papers you have read, etc. is there an intuition/observation about number of filters used?
For instance (I'm just making this up as example):
use more/less filters as the network gets deeper.
use larger/smaller filter with large/small kernel size
If the object of interest in the image is large/small, use ...
As you said, there are no hard rules for this.
But you can get inspiration from VGG16 for example.
It double the number of filters between each conv layers.
For the kernel size, I usually keep 3x3 or 5x5.
But, you can also take a look at Inception by Google.
They use varying kernel size, then concat them. Very interesting.
As far as I am concerned there is no foxed depth for the convolutional layers. Just several suggestions:
In CS231 they mention using 3 x 3 or 5 x 5 filters with stride of 1 or 2 is a widely used practice.
How many of them: Depends on the dataset. Also, consider using fine-tuning if the data is suitable.
How the dataset will reflect the choice? A matter of experiment.
What are the alternatives? Have a look at the Inception and ResNet papers for approaches which are close to the state of the art.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I would like to implement bag of words representation for my project. I computed the codebook of visual words of images by using their features and descriptors.Then, I obtained cluster centers using k-means. For the bag of words representation part, it is asked that you should use manually labeled segments provided as part of the dataset. In dataset, there are three different binary masks for each image. Are those labeled segments that binary masks? If so, how will I use that computed visual words?
The bag of words approach provides a concise representation of an image or a part of an image. That representation is typically used as an input to a classification algorithm which is used to estimate the class to which the image data belongs. Typically, the classifier is a supervised learning method which will require pairs (descriptor, label) from some training set during the training process. In your case, the descriptor is the BOW representation of the image data from your training set. Then, during testing you will feed the BOW descriptor of new image data to the classifier to infer the class.
From what I understand, the fact that you have three different masks for the images, means that you also have three classes. Then, each mask will tell you which part of an image should be considered image data belonging to a particular class. This is your training data.
Under that assumption, you should extract the parts of the images that correspond to each mask, compute the BOW representation for those image parts (separately for each mask) and use those with the mask number as a label to train the classifier.
This will allow you to later to e.g. use the sliding window approach to classify parts of a test image as belonging to one of the 3 classes used during training. That would be a simple case of a detection problem.
I am not sure I understood your problem correctly, but I hope that this will help you move forward a bit.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I need help about the weight option of libSVM. I'm confused at some point; should we also use the -wi option while doing cross-validation? If so, should we use the calculated weights of the whole data or the calculated weights according to the v-1 subsets (for v-fold cross-validation)? And my second question is should we use -wi option during predict? If so, should we use the calculated weights during training or should we calculate the weights according to the distribution of negative and positive instances in the test data?
For example; we have 50 + data and 200 - data. So after calculating the best c and gamma parameter values we will use -w1 4 -w-1 1 options while training. But what about training during grid search and cross validation? Let's say we are performing 5-fold cross-validation. While training on each remaining 4 subsets, the distribution of negative and possitive instances will probably change. So should we recalculate the weights during this 5-fold cross validation?
Besides shoud we use -w1 4 -w-1 1 options while testing?
Thanks
To answer your first question, if you are applying non-trivial weights to a subset of classes during model training, then you should do the same throughout your training/tuning which includes cross-validation based tuning of C and gamma (otherwise you would be tuning the model based on the cost-sensitive objective/risk/loss function which is different from the one you are actualy specifying by enabling non-trivial class weights)
The class weights are external to libSVM in the sense that they are not calculated by libSVM - that command-line option allows a user to set his/her own class weights to emphasize/reduce importance of a subset of classes. Some people tune the class weights as well but that is a different story.
As for the prediction, the class weights are not used there explicitly (since they come in as a "tweak" to the objective/risk/loss function during the model training/tuning stage so the resulting model is already "aware" of the weights)