It is a mapping of words to the vector space. Words will be represented as vectors to be used in machine learning domain.

We can utilize this transformation and treat the words as mathematical objects to extract the required information.

Sample Word Embedding

Granted we have the embeddings, we can do operations easily. Now it comes how to find the correct word embeddings. We have CBOW and Skip-Gram methods for that.

CBOW

This is also called continuous bag of words. We predict a specific missing word in a sentence considering the words around it. We backprop using a neural network structure similar to encoder/decoder model.

Skip-Gram

This is another way to build the vectors. We predict a context words given the center word. We again backprop using a neural network structure similar to encoder/decoder model.


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *