More from @kchonyc

Kyunghyun Cho

@kchonyc

Jul 23

enjoying #ICML2024 ? already finished with llama-3.1 tech report? if so, you must be concerned about the emptiness you'll feel on your flight back home in a couple of days.

do not worry! Wanmo and i have a new textbook on linear algebra for you to read, enjoy and cry on your long flight.

(1/5)

have you ever wondered why SVD comes so late in your linear algebra course?

both wanmo (math prof) and i (cs prof) began to question this a couple of years ago. after all, svd is one of the most widely used concepts from linear algebra in engineering, data science and AI. why wait until the end of the course?

(2/5)

we began to wonder further whether SVD can be introduced as early as possible. i mean ... even before introducing positive definite matrices, matrix determinants and even ... eigenvalues (gasp!) without compromising on mathematical rigors.

(3/5)

Read 5 tweets

Kyunghyun Cho

@kchonyc

Jul 23

very cool to see a pretty exhaustive and extensive technical report on llama-3.1!

a few fun snippets 🧵

PLEASE release this custom html parse PLEASE 🙏

lesson 1: AGI won’t happen due to the degrading QC of NVIDIA.

lesson 2: even Meta couldn’t figure out NCCL watchdog timeout error 😂

Read 9 tweets

Kyunghyun Cho

@kchonyc

Jul 10

we all want to and need to be prepared to train our own large-scale language models from scratch.

why?

1. transparency or lack thereof
2. maintainability or lack thereof
3. compliance or lack thereof

and because we can, thanks to amazing open-source and open-platform ecosystem.

(1/12)

we have essentially lost any transparency into pretraining data.

(2/12)

we are being force-fed so-called values of silicon valley tech co's, ignoring the diversity in values across multiple geographies, multiple sectors and multiple groups.

(3/12)

Read 13 tweets

Kyunghyun Cho

@kchonyc

May 15

this semester (spring 2024), i created and taught a new introductory course on causal inference in machine learning, aimed at msc and phd students in cs and ds. the whole material was created from scratch, including the lecture note and lab materials;

1/4docs.google.com/document/d/1qN…

now that the course is finally over, i've put all the lab materials, prepared by amazing @taromakino, @Daniel_J_Im and @dmadaan_, into one @LightningAI studio, so that you can try them out yourselves without any hassle;

2/4lightning.ai/kc119/studios/…

i'm also making the lecture note i used to teach lectures throughout the semester publicly as well at .

3/4arxiv.org/abs/2405.08793

Read 4 tweets

Kyunghyun Cho

@kchonyc

Aug 23, 2021

@genentech

good morning!

as i tweeted last week, Prescient Design Team at gRED within @genentech is hiring awesome people. in particular, we have the following positions already open and ready:

[Engineering Lead] we want you to work with us to build a team for creating an ML infrastructure that seamlessly integrate between ML and bio: gene.com/careers/detail…

@stephenrra

[Machine Learning Scientist] we have a ton of challenging problems inspired & motivated by biology, chemistry & medicine that are waiting for your creativity, knowledge and ingenuity in ML/AI: gene.com/careers/detail…

cc: @stephenrra

Read 6 tweets