Thread by @TheZachMueller

Thanks everyone for joining me on this much more brief stream, in this thread I'll try and summarize all we discussed including:

- x is not implemented for y
- LossMetrics
- TensorBase and tensor metadata

And here is the stream:

1/

.@fastdotai had utilized their own tensor subclassing system in v2, however originally there weren't many issues as fastai just "let" you do things with these classes. Then @PyTorch came along and introduced them in 1.7. Suddenly, people were getting this! Why?

Pytorch became much more explicit in what subclasses can interact with oneanother. As a result @fastdotai had to make the TensorBase class which allows for any tensor-types to interact with each other. A quick function to convert and an example are below 3/

But what if I have metadata? Won't that all get destroyed?

In the cast, the metadata itself is maintained inside the tensor, as seen below, BUT 4/

When interacting two tensors of the same type, the metadata is only kept in the first tensor of the operation. Such as below 5/

Finally we looked at the LossMetrics class. What this allows you to do is take a complex loss function that perhaps has multiple losses at once, and allows you to report them as a part of your metrics. A basic example taken from the @fastdotai is shown below 6/:

To use simply make sure that each loss is stored away as an *attribute* in your overall loss function, and then pass in a comma-separated string of each attribute into LossMetrics 7/

Finally, to return back to the TensorBase stuff, why is it some loss functions will just *work* when using say cnn_learner? And you don't need to do that big conversion?

fastai has a `Flatten` layer which is added on top, which internally casts to a TensorBase 8/

As an added bonus, if you write a loss function inheriting `BaseLoss`, it will perform this for you, just pass in the original loss function to it

Hope this has helped alleviate some more @fastdotai mysteries and solved a bug here or there!

@threadreaderapp unroll