Rik van Noord Profile picture
May 27, 2019 7 tweets 3 min read Read on X
1/7 Do word embeddings really say that man is to doctor as woman is to nurse? Apparently not. Check out this thread for a description of a short paper I co-wrote with Malvina Nissim and Rob van der Goot, available here: arxiv.org/abs/1905.09866 #NLProc #bias
2/7 The original analogy codes, both w2v and gensim, implement a constraint whereby no input vector can be returned. In any query A:B :: C:D, one could never get an answer D such that D==A or D==B or D==C simply because the code does not allow it.
3/7 Therefore, for the query "man is to doctor as woman is to X", doctor cannot be returned as answer!
4/7 We modified the code to make it unrestricted and tested several classic (biased) examples as well as the original analogy set. Specific outcomes are shown and discussed in the paper. We saw that: (i) if you let it, B is almost always returned.
5/7 (ii) then, the analogy task doesn't work that well (no "queen" for "man is to king as woman is to X", but "king") and (iii) analogy-based biases have often been overemphasized in the literature.
6/7 This is not beneficial for dealing with the problem of actual biases in word embeddings. Moreover, these are usually not captured by the analogy task anyway (we agree that they are "party tricks" as stated by @yoavgo and @hila_gonen in this fine work: arxiv.org/abs/1903.03862)
7/7 If you're curious, you can try analogies using the unrestricted code through this simple demo: let.rug.nl/rob/embs/. We'd love to get comments on the paper, so if you have any, let us know! Paper link again: arxiv.org/abs/1905.09866 #NLProc #bias

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Rik van Noord

Rik van Noord Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(