Discover and read the best of Twitter Threads about #ACL2023NLP
Most recents (2)
Excited to share our #NLProc #ACL2023NLP paper:
Controlled Text Generation with Hidden Representation Transformations
Work done during my applied scientist internship at @AmazonScience @alexa99 team.
Paper: arxiv.org/abs/2305.19230
Code: github.com/amazon-science… (Coming soon)
Controlled Text Generation with Hidden Representation Transformations
Work done during my applied scientist internship at @AmazonScience @alexa99 team.
Paper: arxiv.org/abs/2305.19230
Code: github.com/amazon-science… (Coming soon)
@AmazonScience @alexa99 LLMs are notoriously difficult to control. This work is an effort to fix that.
We create CHRT : a novel framework to attribute control LLMs using learned transformation blocks.
It can be used to minimize toxicity, maximize positive sentiment and more.
We create CHRT : a novel framework to attribute control LLMs using learned transformation blocks.
It can be used to minimize toxicity, maximize positive sentiment and more.
The approach has minimal loss in linguistic quality while achieving high attribute control.
Also has the least latency delta as compared to all other included baselines, something that makes it ideal for production environments.
Also has the least latency delta as compared to all other included baselines, something that makes it ideal for production environments.
#ACL2023NLP
Do causal language models (CLMs) yield good representations with good isotropy and discrimination?
The answer is not always! To address the issue, our ACL2023 paper (arxiv.org/pdf/2210.01185…) proposes ContraCLM.
Joint work with @DejiaoZhang @nihalj_
Do causal language models (CLMs) yield good representations with good isotropy and discrimination?
The answer is not always! To address the issue, our ACL2023 paper (arxiv.org/pdf/2210.01185…) proposes ContraCLM.
Joint work with @DejiaoZhang @nihalj_
We show that CodeGen (350M to 16B) pretrained on source code, and text-based CLMs (smaller than GPT2-Large) generated representations suffer from anisotropy and poor discrimination.
We show that ContraCLM enhances both isotropy and discrimination, regardless of whether the original CLMs suffer from the degenerated representations.
