Post

How to get URL link on X (Twitter) App

On the Twitter thread, click on or icon on the bottom
Click again on or Share Via icon
Click on Copy Link to Tweet
Paste it above and click "Unroll Thread"!
More info at Twitter Help

Kyunghyun Cho

@kchonyc

Sep 16, 2020 • 8 tweets • 4 min read • Read on X

@hugo_larochelle

denoising in a discrete input has always fascinated me ever since i read jmlr.org/papers/volume1… by Vincent & @hugo_larochelle et al., and yoshua has always motivated me to look into denoising for sequence modeling ever since 2013.

https://twitter.com/jasonleeinf/status/1306036621301682178

@jasondeanlee

it took me 5 years to look at refinement in the discrete space with @jasondeanlee & @elmanmansimov arxiv.org/abs/1802.06901. it took another 2 years to look at refinement in the hybrid space with jason and @raphaelshu aaai.org/Papers/AAAI/20…

@jasondeanlee

now, we have finally moved on to refinement in the continuous space for sequence modeling with discrete tokens, again with awesome @jasondeanlee & @raphaelshu arxiv.org/abs/2009.07177.

and, ofc, the journey continues!

it's a small step overall that took 7 years, but i like it!

@jasonleeinf

tagged a wrong jason lee. it should’vebeen @jasonleeinf

@jasonleeinf

same here. it should’ve been @jasonleeinf

here's the new link to the latent variable approach: arxiv.org/abs/1908.07181

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @kchonyc

Kyunghyun Cho

@kchonyc

Jul 23

enjoying #ICML2024 ? already finished with llama-3.1 tech report? if so, you must be concerned about the emptiness you'll feel on your flight back home in a couple of days.

do not worry! Wanmo and i have a new textbook on linear algebra for you to read, enjoy and cry on your long flight.

(1/5)

have you ever wondered why SVD comes so late in your linear algebra course?

both wanmo (math prof) and i (cs prof) began to question this a couple of years ago. after all, svd is one of the most widely used concepts from linear algebra in engineering, data science and AI. why wait until the end of the course?

(2/5)

we began to wonder further whether SVD can be introduced as early as possible. i mean ... even before introducing positive definite matrices, matrix determinants and even ... eigenvalues (gasp!) without compromising on mathematical rigors.

(3/5)

Read 5 tweets

Kyunghyun Cho

@kchonyc

Jul 23

very cool to see a pretty exhaustive and extensive technical report on llama-3.1!

a few fun snippets 🧵

PLEASE release this custom html parse PLEASE 🙏

lesson 1: AGI won’t happen due to the degrading QC of NVIDIA.

lesson 2: even Meta couldn’t figure out NCCL watchdog timeout error 😂

Read 9 tweets

Kyunghyun Cho

@kchonyc

Jul 10

we all want to and need to be prepared to train our own large-scale language models from scratch.

why?

1. transparency or lack thereof
2. maintainability or lack thereof
3. compliance or lack thereof

and because we can, thanks to amazing open-source and open-platform ecosystem.

(1/12)

we have essentially lost any transparency into pretraining data.

(2/12)

we are being force-fed so-called values of silicon valley tech co's, ignoring the diversity in values across multiple geographies, multiple sectors and multiple groups.

(3/12)

Read 13 tweets

Kyunghyun Cho

@kchonyc

May 15

this semester (spring 2024), i created and taught a new introductory course on causal inference in machine learning, aimed at msc and phd students in cs and ds. the whole material was created from scratch, including the lecture note and lab materials;

1/4docs.google.com/document/d/1qN…

now that the course is finally over, i've put all the lab materials, prepared by amazing @taromakino, @Daniel_J_Im and @dmadaan_, into one @LightningAI studio, so that you can try them out yourselves without any hassle;

2/4lightning.ai/kc119/studios/…

i'm also making the lecture note i used to teach lectures throughout the semester publicly as well at .

3/4arxiv.org/abs/2405.08793

Read 4 tweets

Kyunghyun Cho

@kchonyc

Aug 23, 2021

@genentech

good morning!

as i tweeted last week, Prescient Design Team at gRED within @genentech is hiring awesome people. in particular, we have the following positions already open and ready:

[Engineering Lead] we want you to work with us to build a team for creating an ML infrastructure that seamlessly integrate between ML and bio: gene.com/careers/detail…

@stephenrra

[Machine Learning Scientist] we have a ton of challenging problems inspired & motivated by biology, chemistry & medicine that are waiting for your creativity, knowledge and ingenuity in ML/AI: gene.com/careers/detail…

cc: @stephenrra

Read 6 tweets