Aaron Chan Profile picture
PhD Student @CSatUSC / @nlp_usc / @USC_ISI. Designing #NLProc systems to be more knowledgeable and trustworthy. Prev: @Google @Penn @UofMaryland.

3 May, 10 tweets

🚨 Our #ICLR2021 paper shows that KG-augmented models are surprisingly robust to KG perturbation! 🧐

arXiv: arxiv.org/abs/2010.12872
Code: github.com/INK-USC/deceiv…

To learn more, come find us at Poster Session 9 (May 5, 5-7PM PDT): iclr.cc/virtual/2021/p….


KGs have helped neural models perform better on knowledge-intensive tasks and even “explain” their predictions, but are KG-augmented models really using KGs in a way that makes sense to humans?


We primarily investigate this question by measuring how the performance of KG-augmented models changes when the KG’s semantics and/or structure are perturbed, such that the KG becomes less human-comprehensible.


If the KG has been greatly perturbed in this manner, then a “human-like” KG-augmented model should achieve much worse performance with the perturbed KG than with the original KG.


We propose four heuristics (RS, RR, ER, ED) and one RL algorithm (RL-RR) for perturbing the semantics and/or structure of the KG. Unlike the heuristics, RL-RR aims to maximize the downstream performance of models using the perturbed KG.


Interestingly, for both commonsense QA and item recommendation, the KG can be extensively perturbed with little to no effect on KG-augmented models’ performance! Here, we show results for KGs perturbed using RL-RR.


Plus, we find that original KG and perturbed KG (RL-RR) paths successfully utilized by KG-augmented models are hard for humans to read or use. This suggests that models and humans process KG info differently.


These findings raise doubts about the role of KGs in KG-augmented models and the plausibility of KG-based explanations. We hope that our paper can help guide future work in designing KG-augmented models that perform and explain better.


This work was led by @MrigankRaman, an undergrad intern at INK Lab (inklab.usc.edu). Many thanks to all of our co-authors: Siddhant Agarwal, Peifeng Wang, Hansen Wang, Sungchul Kim, Ryan Rossi, Handong Zhao, Nedim Lipka, and @xiangrenNLP.


Oops, adding Siddhant's Twitter handle here too: @agsidd10.


Share this Scrolly Tale with your friends.

A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.

Keep scrolling