[Python] How to get cosine similarity of word embedding from BERT model

I was interesting in how to get the similarity of word embedding in different sentences from BERT model (actually, that means words have different meanings in different scenarios).

For example:

sent1 = 'I like living in New York.'
sent2 = 'New York is a prosperous city.'


I want to get the cos(New York, New York)'s value from sent1 and sent2, even if the phrase 'New York' is same, but it appears in different sentence. I got some intuition from https://discuss.huggingface.co/t/ge...odels-like-bert-for-downstream-process/2958/2

But I still do not know which layer's embedding I need to extract and how to caculate the cos similarity for my above example.

Thanks in advance for any suggestions!

Continue reading...