Tweet

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @karlhigley

Karl Higley

@karlhigley

7 Apr

Counter-intuitively, if your recommender is trained on data collected from past interactions with recs, retraining new models regularly doesn’t resolve the issue of performance decaying over time and can actually make it worse. Here’s why...

The usual concern is that the performance of a model will decline as the distribution of data encountered during serving deviates from the distribution of the training data.

This is an accurate concern about a real phenomenon, and is a good reason to retrain on a regular basis.

However, if the training data is collected from interactions with past recommendations, then which data you collected was determined by which recommendations were made, which was determined by the last model served in production. See the issue?

Read 15 tweets

Karl Higley

@karlhigley

5 Apr

The typical explanation of skip-connections in neural nets depicts them as a detour around parts of the network and suggests that they allow higher level information to flow backward to the lower levels during back-propagation of the gradients. I find this somewhat backward.

The “output” of a skip connection seems to usually be combined with the output of some other blocks of the network with an element wide operation like a sum, so...

To me, it makes a heck of a lot more sense to consider the skip connection to be the default path in the forward pass, onto which something extra is being added or combined.

Read 8 tweets

Karl Higley

@karlhigley

9 Oct 20

“Why Are Deep Learning Models Not Consistently Winning Recommender Systems Competitions Yet?“

dl.acm.org/doi/abs/10.114…

My take is that we haven’t had the right model architectures. Here’s why I think that...

Going way back to the Netflix prize, multiplicative interactions have been a key component of successful modeling strategies. Matrix factorization did well on the Netflix data and became a classic approach to making recommendations.

Many further iterations on the key concept of factorizing matrices into low-rank approximations with vector embeddings per user/item/attribute have also been successful.

Read 17 tweets

Karl Higley

@karlhigley

6 Oct 20

https://twitter.com/aspergersgirls/status/1313543348607623168

This has been exactly my experience in the workplace as an autistic ML engineer. I’m still doing machine learning, building recommender systems, attending conferences, contributing to open source projects, publishing my work.

I’m just not getting paid, because I don’t fit in.

https://twitter.com/aspergersgirls/status/1313543348607623168

When I mention that I’m not currently employed, people sometimes reach out to see if I’d be interested in a position with their company—which I appreciate but often don’t know how to respond to.

I might be, but the truth is that I have absolutely been through the wringer with work these past five years or so, and I have some lingering workplace-related trauma that makes me very hesitant to pursue the next thing.

Read 11 tweets

Karl Higley

@karlhigley

9 Aug 20

It’s actually not that difficult to understand why people whose foundational worldview is “I feel safe and comfortable when I’m part of a homogeneous in-group that defines the terms of public life” would feel like they’re under siege by the (reasonable) demands from out-groups.

When people say that they feel like their freedoms are being taken away when freedoms are granted to others, this is what they’re referring to: the freedom to live in a social bubble where they don’t have to grapple with differences they find unsettling or threatening.

Here’s the thing: we all feel more comfortable in homogeneous groups that dictate the terms of interaction. Some of us grow beyond it to become capable of more engaging outside our bubbles, and some of us don’t. This mindset is in all of us to some extent and is never going away.

Read 4 tweets

Karl Higley

@karlhigley

27 Jul 20

This story and picture of paint-covered magazines really doesn't add up, so I did some digging and found some interesting stuff. 1/many

First of all, the paint goes entirely unremarked on and makes no sense. However, reporting from various sources indicates that federal agents claim that they are being hit with balloons full of paint. 🤔

Oh really, you don't say? Well, here's a Reuters photo from yesterday of a DHS police officer covered in red paint.

Source (slide 2): mobile.reuters.com/news/picture/t…

Read 54 tweets

Share this page!

Karl Higley

Try unrolling a thread yourself!

More from @karlhigley

Karl Higley

Karl Higley

Karl Higley

Karl Higley

Karl Higley

Karl Higley

Did Thread Reader help you today?

Like this author's thread?