Today @SemiAnalysis published their extremely comprehensive, detailed, and honest report on performance comparison between @NVIDIA's H100/H200 GPUs and @AMD's MI300X.

1/9 On paper, MI300X has many advantages compared to the H100/H200, but in practice AMD's hardware is effectively nerfed by their catastrophically weak software. TL;DR: out of the box you will not be able to use MI300X for ML/AI training.

2/9

I've been holding off saying more about Google's purported quantum computing breakthrough until I read a bit more about it. (You should try doing something like that too!) It turns out, as I had suspected, it was waaaaay overhyped.

1/4

https://twitter.com/skdh/status/1866352680899104960

Yes, it's good science, but in terms of any kind of practical applications we are probably at least a decade away. Even then it will most likely be very specialized areas of application, like molecular dynamics.

2/4

At the end of the last week @DarioAmodei, co-founder and the CEO of @AnthropicAI, one of the top AI labs, published a long essay on his vision of what an advanced AI in the upcoming years could potentially accomplish. There have been several other similar essays over the past few months from other top AI voices, but in my opinion this one is the most thoughtful and most detailed so far. It stirs away from many ideological squabbles that have become all too common these days, and provides ample citations that bolster his points, and help with further reading and self-guided research. Dario's own scientific background is in Biophysics and Neuroscience, and his takes on potential in those fields are particularly insightful.

1/7 Some key takeaways:

Biology and health

AI could drastically accelerate biological and medical progress, compressing 50-100 years of advancements into just 5-10 years. This could lead to the elimination of most diseases, significant extensions of the human lifespan, and greater control over biological functions like reproduction and aging. Key breakthroughs such as CRISPR, mRNA vaccines, and genome sequencing are examples of innovations AI could multiply, transforming human health.

2/7

.@NVIDIA has just announced TensorRT-LLM, open source software designed to accelerate Large Language Model inference on H100s.

1/7 This software has been developed though close collaboration with many leading AI companies, such as @Meta, @anyscalecompute , @cohere , Deci AI, @Grammarly, @databricks and many others.

2/7

By now most of us well aware of transformer-based large language model capabilities and, in many instances, failures. The failures in particular can seem extremely head-scratching, as they often involve the kind of mental reasoning that even a young schoolboy cold do.

1/6 A new paper tries to investigate the nature of these failures, and understand the limits of LLM-based reasoning. It seems that the failures primarily arise from the tasks with low in-domain knowledge and high compositional complexity.

2/6

Very exciting news - Python is now available in the official version of Excel! Excel is the most widely used analytics tool in the World, and Python has become the most popular programming language for Data Science and Machine Learning tasks.

1/4 It is a very intuitive and easy to learn programming language. The merger of these two tools will open new opportunities and use cases.

This merger is the culmination of years-long effort and collaboration between Microsoft and the open source Python community.

2/4

Large Language Models (LLMs) have emerged as the cornerstone of the current Generative AI revolution. The big problem with LLMs is that they are, well, large. Really, really, large.

1/7 They require an enormous amount of high-quality data to train and even more unfathomably large amount of computational power.

2/7

Would you like to win an RTX 4080? You are in luck, because at @nvidia we are giving away one (1) for GTC 2023. All you have to do is:

1. Like and share this tweet

2. Register for GTC: nvda.ws/3j6gw41

3. Post a screenshot of you in a session as a response below

1/7 A few points:

1. I am working with the NVIDIA marketing team to promote one giveaway; there are other influencers who are giving away more GPUs in their own giveaways.

2. GTC registration is completely free and open to the general public. All sessions are online.

2/7

Things seem to be moving at a breakneck speed in the world of generative AI and large language models. In a surprise press event yesterday, @Microsoft announced a wide integration of @OpenAI tools into a couple of their major products,

1/4 Bing search engine and Edge web browser. In particular, this seems to be the first time that we'll see anywhere a public use of OpenAI's next generation LLM, GPT4. Most of the new features are still relatively limited, and you'll need to join the waitlist for the full access. 2/4

In a highly anticipated move, @Google yesterday announced that they are launching Bard, a conversational AI app that is based on their LaMDA model.

1/5 LaMDA - Language Model for Dialogue Applications - has been around for at least a year, but due to variety of considerations it has never been accessible to to the public.

2/5

Deep Learning and Neural Networks have become the default approaches to Machine Learning in recent years. However, despite their spectacular success in certain domains (vision and NLP in particular),

1/5 their use across the board for all ML problems and with all datasets is problematic, to say the least. Oftentimes better and more robust results can be obtained with simpler, easier to train and deploy, classical ML algorithms.

2/5

There was nothing that shocked me more when I entered the industry from academia than this kind of attitude. I came from an environment where teaching and learning were the norm, to the one where giving help to “underperformers” was viewed with disdain as a liability.

1/5 Fortunately not all organizations and managers are this cutthroat, but this kind of mindset is pervasive, especially at startups. There is a widespread attitude that *it’s someone else’s responsibility to do the educating*: yours, your previous job’s, your college’s etc.

2/5

Last week @DeepMind’s research on AlphaCode - a competative programming system - has been published in Science. AlphaCode has been able to beat 54% of humans on a competative coding challenges, putting it on par with many junior-level developers.

1/4 The original announcement from DeepMind came out in February, which in the fast-paced world of AI is already ancient history.

2/4

Last week @OpenAI released ChatGPT - a Large Language AI Model that interacts with users in a natural conversational way. The chatbot is able to answer complex questions, even in highly technically demanding categories.

1/7 It is also able to answer the follow up question, backtrack on wrong assumptions, and provide other detailed resources, including code fragments.

2/7

PyTorch 2.0 is out! This major release upgrade brings about many new features, but the main improvements are under the hood.

1/6 The three main principles behind PyTorch

1. High-Performance eager execution
2. Pythonic internals
3. Good abstractions for Distributed, Autodiff, Data loading, Accelerators, etc.

PyTorch 2.0 is fully backward compatible with the previous versions of PyTorch.

2/6

Decision trees based Machine Learning models are some of the best performant algorithms in eras of predictive capability, especially on small and heterogenous datasets.

1/4 They also provide an unparalleled level of interpretability compared to all other non-linear algorithms. However, they are very hard to optimize on Von Neumann architecture machines due to their non-uniform memory access patterns.

2/4

This past week I came across another paper that purports to get the SOTA for NNs for tabular data. Due to the extreme penchant for exaggeration in this community, I have given up on checking most of these claims, but decided to take a look at this particular work.

1/6 I decided to check how does XGBoost *really* perform on the datasets used in the paper, and the results were not pretty.

2/6

This week @NVIDIA open sourced the 3D object generation AI model, GET3D. GET3D is a generative model of high quality 3D textured shapes learned from images.

1/4 Trained using only 2D images, GET3D generates 3D shapes with high-fidelity textures and complex geometric details.

2/4

I have just done something really cool - I've managed to *train* XGBoost in browser completely within an HTML file! This has been possible thanks to the PyScript project that allows running Python inside of HTML, similar to how JavaScript works.

trainxgb.com

1/5 The example below is very simple - the script loads the small Iris dataset from sklearn. With a slider you are able to adjust the number of XGBoost trees, and the script will train different XGBoost models accordingly and print out accuracy.

2/5

All right, here is one trick for using XGBoost for *data analysis*.

1/5

https://twitter.com/javisantana/status/1572296064211861507

First, you create a simple model with XGBoost. It doesn't have to be fancy, or even too accurate, it's just for reference purposes. Use that model to calculate the Shapley values for your training set. Here is an example:

kaggle.com/code/tunguz/tp…

2/5

NVIDIA GTC starts today! There are tons of exciting topics and webinars covered. This year again the whole conference is online and free, so go and register if you have not done so already.

Here are a few special highlight sessions:

1/4 GTC 2022 Keynote - September: lnkd.in/gYNqxsnr

How CUDA Programming Works: lnkd.in/gKmdjZub

Building the Future of Work with AI-powered Digital Humans: lnkd.in/gXJWk6vz

Building Future-Ready Intelligence for Cars: lnkd.in/gJ9BJMGM

2/4