I am using a GloVe based pre-trained embedded vectors for words in my I/P sentences to a NMT-like model. The model then generates a series of word embeddings as its output for each sentence.
How can I convert these output word embeddings to respective words? One way I tried is using cosine similarity between each output embedding vector and all the i/p embedding vectors. Is there a better way than this?
Also, is there a better way to approach this than using embedding vectors?
First of all the question is lacking a lot of details like the library used for word embedding, the nature of the model, and the training data, etc ...
But I will try to give you an idea what you can do in those situations, assuming you are using some word embedding library like Gensim.
How to get the word from the vector:
We are dealing with predicted word vectors here, so our word vector may not be the exact vector of the original word, we have to use similarity, in Gensim you can use similar_by_vector, something like
target_word_candidates = similar_by_vector(target_word_vector,top=3)
That's would solve the reverse lookup problem, as highlighted here, given all the word vectors how to get the most similar word, but we need to find the best single word according to the context.
You can use some sort of post-processing on the output target word vectors, this would be beneficial for trying to solve some problems like:
1.How to guide the translation of out-of-vocabulary
terms?
2.How to enforce the presence of a
given translation recommendation in the decoder’s
output?
3.How to place these word(s) in the right
position?
One of the ideas is to use an external resource for the target language, i.e. language model, to predict which combination of words are gonna be used. Some other techniques incorporate the external knowledge inside the translation network itself
Related
I am working on a small project in Matlab just because of my interest in image processing and I have not studied a degree or a course related to image processing.
I want to understand a small concept about feature extraction and feature vectors. I have read some articles about that and in general I can understand that, but my question is:
For example, I want to extract some information from different objects of a binary image, the information is about length, width and distance between the objects. In one application I want to extract the features on which I want to apply some algorithms to compute width of all the objects and ignore the length and distance. Can we name this as feature extraction regarding the width? And storing them in different vectors as Feature Vectors?
It makes me think that, I might be complicating the simple things. Should I use some other terminologies for this instead of feature extraction and feature vectors?
Please suggest me if I am going in the right direction or not?
Thank you!
Feature extraction is the process of computing numerical values on regions/objects/shapes/blobs detected in an image. [Sometimes the detection itself can be called extraction and the features need not be numbers.]
The feature values can indeed be stored in vectors, usually they fill a table. Sometimes they are structured in a more complicated way (such as a graph f.i.). Most of the time they are used for classification/recognition purposes or they can just be the output of the process on hand.
I'm beginning my journey into Neural Networks and trying to understand both character and word embeddings as ways to input text data into a NN. Specifically, I am trying to embed HTML tag information. I tried googling some different combinations of my problem and came up empty.
My current understanding is that embeddings "embed" words or characters into an N-Dimensional space, which allows NNs to be able to understand them as inputs. So in this case, something like word2vec would not necessarily help me because it is not meant to understand the "meaning" of HTML elements? So thus a character embedding would be better?
If anyone could point me in a direction that would be awesome, as I am having trouble finding this on my own.
Thanks in advance.
How could one, in an unsupervised environment, get a feature vectors for a sentence. I believe for an image data one could build a conv auto encoder and take the hidden layers outputs. What would be the best way to do this for RNN type models (LSTM, GRU, etc.).
Would this (https://papers.nips.cc/paper/5271-pre-training-of-recurrent-neural-networks-via-linear-autoencoders.pdf) be on the right track?
The easiest thing to do would be doing something with the word2vec representations of all the words (like summing them?).
See this post:
How to get vector for a sentence from the word2vec of tokens in sentence
I am doing my project based on health care.I am going to train my autoencoders with the symptoms and the diseases i.e my input is in textual form. Will that work? (I am using Rstudio).Please anyone help me with this
You have to convert the text to vectors/numbers. To do this traditional approaches like Bag of words, Tf-Idf will help but the latest Neural Word Embedding like Word2Vec, RNN Language model etc are the best techniques to obtain numeric representation of text.
Please use any Neural Word Embedding technique and convert the text(word level[word2vec], document level[doc2vec]) into numbers/vectors.
Now these vectors come with some dimension and to compress this representation to even smaller dimension u can use AutoEncoder.
Feel Free to ask any other information required.
Try using Python for these tasks as it has the latest packages.
You can use Autoencoder on Textual data as explained here.
Autoencoder usually worked better on image data but recent approaches changed the autoencoder in a way it is also good on the text data.
have a look at this.
the code is also available in GitHub.
I've been reading about neural networks and how CBOW and Skip-Gram works but I can't figure out one thing: How do I generate the word vectors itself?
It always seems to me that I use those methods to calculate the weight matrix, and I use the word vector to do adjust it, and I'm struggling to understand how I got the word vectors in the first place.
When I found Rumelhart paper I thought I would find the answer there, but all I got was the same thing: calculate the error comparing the output expected with the one I found and adjust the model. But who is my expected output? How did I get it?
For example, Omer Levy and Yoav Goldberg explained in a perfect clear way (in Linguistic Regularities in Sparse and Explicit Word Representations) how the Explicit Vector Space Representation works, but I couldn't find an explanation on how Distributed representations for words works.