🚨 Our #ICLR2021 paper shows that KG-augmented models are surprisingly robust to KG perturbation! 🧐
arXiv: arxiv.org/abs/2010.12872
Code: github.com/INK-USC/deceiv…
To learn more, come find us at Poster Session 9 (May 5, 5-7PM PDT): iclr.cc/virtual/2021/p….
🧵[1/n]
KGs have helped neural models perform better on knowledge-intensive tasks and even “explain” their predictions, but are KG-augmented models really using KGs in a way that makes sense to humans?
[2/n]
We primarily investigate this question by measuring how the performance of KG-augmented models changes when the KG’s semantics and/or structure are perturbed, such that the KG becomes less human-comprehensible.
[3/n]
If the KG has been greatly perturbed in this manner, then a “human-like” KG-augmented model should achieve much worse performance with the perturbed KG than with the original KG.
[4/n]
We propose four heuristics (RS, RR, ER, ED) and one RL algorithm (RL-RR) for perturbing the semantics and/or structure of the KG. Unlike the heuristics, RL-RR aims to maximize the downstream performance of models using the perturbed KG.
[5/n]
Interestingly, for both commonsense QA and item recommendation, the KG can be extensively perturbed with little to no effect on KG-augmented models’ performance! Here, we show results for KGs perturbed using RL-RR.
[6/n]
Plus, we find that original KG and perturbed KG (RL-RR) paths successfully utilized by KG-augmented models are hard for humans to read or use. This suggests that models and humans process KG info differently.
[7/n]
These findings raise doubts about the role of KGs in KG-augmented models and the plausibility of KG-based explanations. We hope that our paper can help guide future work in designing KG-augmented models that perform and explain better.
[8/n]
This work was led by @MrigankRaman, an undergrad intern at INK Lab (inklab.usc.edu). Many thanks to all of our co-authors: Siddhant Agarwal, Peifeng Wang, Hansen Wang, Sungchul Kim, Ryan Rossi, Handong Zhao, Nedim Lipka, and @xiangrenNLP.
[9/n]
Oops, adding Siddhant's Twitter handle here too: @agsidd10.
[10/n]
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.