When evaluating research like GPT-2 it is important to remember that humans will draw wildly different conclusions from identical results based on how they are narrated. Let me give you some examples 1/n Narrative 1: GPT-2 is an advance in datasets. OpenAI collected, preprocessed and cleaned a new dataset called WebText. This dataset is so good that language models trained on it generate impressive samples and zero-shot other tasks 2/n

I have been playing around with porting some of my TensorFlow code to PyTorch. Here are my initial thoughts 👇 Important caveat: This is from a research perspective, i.e. implementing non-standard models and low level layers. If all you need is a library to run a relatively standard architectures on some new dataset, the choice of framework is probably irrelevant. Either will do fine.

1/ I think people worry too much *what* to study instead of *how* to study. This applies to topics like programming languages, machine learning, javascript frameworks, etc. Let's take machine learning as an example. 2/ When reading a paper/book you can go breadth-first or depth-first. You can skim it and skip the parts you don't understand and get the high level idea (breadth). Or you can analyze every sentence and formula, follow the references, and re-implement it in code (depth).