Tweet

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @tunguz

Bojan Tunguz

@tunguz

Sep 20

https://twitter.com/javisantana/status/1572296064211861507

All right, here is one trick for using XGBoost for *data analysis*.

1/5

https://twitter.com/javisantana/status/1572296064211861507

First, you create a simple model with XGBoost. It doesn't have to be fancy, or even too accurate, it's just for reference purposes. Use that model to calculate the Shapley values for your training set. Here is an example:

kaggle.com/code/tunguz/tp…

2/5

Next, use those Shapley values for some simple clustering, dimensionality reduction and visualization:

kaggle.com/code/tunguz/tp…

3/5

Read 5 tweets

Bojan Tunguz

@tunguz

Aug 5

@DeepMind

A very good paper I came across this morning by the @DeepMind researchers. For the past five years Transformers have been one of the most dominant approaches to Deep Learning problems, especially in the #NLP domain.

1/5

However, despite many interesting papers on the topic, and lots of good open code, there has been a noticeable lack of *formal* definition of what transformed are, especially on the level of pseudocode.

2/5

This paper aims to rectify that. It provides pseudocode for almost all major Transformer architectures, including training algorithms.

3/5

Read 5 tweets

Bojan Tunguz

@tunguz

Jul 22

The longer you work with ML algorithms, the more you appreciate what an outsize effect your *data* has on the quality of your models. I've seen that shift on Kaggle over the years, where more and more time is spent on some kind of dataset augmentation.

1/5

There is still only so much you can do there, and unless you are "enterprising" and decide to scrape the competition host's website for their dat (yes, this has happened) your legitimate options are rather limited.

2/5

Outside of the Kaggle world, however, things are different. Large computational resources and advanced algorithms still dominate the ML discourse, but those who are paying attention know that neither of them would be worth much without the huge datasets that are bing used.

3/5

Read 5 tweets

Bojan Tunguz

@tunguz

Jul 22

It's actually scary how ignorant academics who try to do research on NNs for tabular data are of tabular data. I think that part of the problem is that almost all of the interesting and relevant tabular data problems are in the industry, 1/3

and academics tend to be completely inured from any kind of practical application of ML/DS.

If you are an academic who is interested in doing research on tabular data,

2/3

I would BEG YOU, FOR THE LOVE OF EVERYTHING THAT IS DECENT, PLEASE, PLEASE PLEASE GET OUT OF YOUR IVORY TOWER AND TRY TO LEARN WHAT KINDS OF PROBLEMS ACTUAL DATA SCIENTISTS DEAL WITH IN THEIR PROFESSIONAL LIVES!!!

3/3

Read 4 tweets

Bojan Tunguz

@tunguz

Jul 2

@Google

This week @Google researchers announced Minerva, an internally developed project that can answer mathematical questions and tackle other complex topics such as physics.

1/5

This project makes some really impressive gains with automatic NLP approach to tackling the challenging quantitative reasoning problems. Minerva is a large language model pretrained on general natural language data and further trained on technical content.

2/5

The model achieves state-of-the-art performance on technical benchmarks without the use of external tools.

3/5

Read 5 tweets

Bojan Tunguz

@tunguz

Jul 1

Neural Networks and Deep Learnig have been an incredible Machine Learning breakthrough(s), both in terms of extending the scope of what we can do with Machine Learning, as well as their practical utility. They have more or less become synonymous with Artificial Intelligence.

1/5

In the fields of Computer Vision and Natural Language Processing in particular they have only gone from strength to strength. I for one am really excited about these developments, and am really bullish about what else we may achieve in the upcoming years.

2/5

I don’t see us hitting any walls there, either currently, or in the near future. All the SOTA work so far has indicated that there are no diminishing returns on how much we can get out of large models.

3/5

Read 5 tweets

Share this page!

Bojan Tunguz

People who liked this thread also liked...

Try unrolling a thread yourself!

More from @tunguz

Bojan Tunguz

Bojan Tunguz

Bojan Tunguz

Bojan Tunguz

Bojan Tunguz

Bojan Tunguz

Did Thread Reader help you today?

Don't want to be a Premium member but still want to support us?

Send Email!