Syntax: What do you write down to make your program "textually wellformed". This include symbols, formatting, whitespace, operators, and keywords.
New languages are hard because syntax is unfamiliar
For example APL vs Python showcases the vast difference in syntax 2/
Semantics: The rules that define program behavior. This defines the behavior implied by the syntax and is often the real challenge in learning a new language.
Run time behavior and compile time behavior ( type checking) are two examples of semantics
I'm super excited to share our work on self-supervised learning for audio. We extend the permutation pre-text task by using differentiable ranking and show improved performance on low-resource tasks (it also works great on images and video)
1/
When using permutations in pretraining, a subset of permutations are used to train a classifier which predicts permutations as classes.
However, since there are n! different permutations of length n, it's not feasible to use any reasonable fraction of them for classes.
2/
We fix this problem by using a differentiable ranking objective which allows arbitrary permutations to be used.
By increasing the number of usable permutations, we find improved representations are learned which can be used on downstream tasks.
3/