Many people in engineering believe that to understand something, it is necessary and sufficient to have a low-level mathematical description of that thing. That you need to "know the math behind it". In nearly all cases, it is neither sufficient nor at all necessary - far from it
My go-to example of this is PCA. If you know how to diagonalize a 5x5 matrix by hand, then you "know the math" behind PCA. But this gives you absolutely no understanding of what PCA is, what it does, and why it works. You need higher-level mental models.
This is almost universally true: to understand something, you need the *right* mental models, that capture what *actually matters* about that thing, not just the lowest-level mathematical description you can find. In most cases, the two are completely orthogonal
The same is true of backprop in deep learning -- knowing how to code up backprop by hand gives you no useful knowledge wrt deep learning, and inversely, developing powerful mental models for deep learning does not in any way require knowing the algorithmic details of backprop
(coming from someone who had to implement backprop a lot in the past, first in C, then in Matlab, then in Numpy)
In addition, if you have the right mental model for something, it is generally easy to work out the algorithmic details on your own when you need them, at least down to a level where you can roll out a working implementation (& it becomes trivial if you can just look up details)
Similar to how, say, you can always reinvent the Pythagorean theorem on the fly if you think about geometry through the lens of vector products, or how you don't need to memorize the quadratic formula if you understand what an equation is and the general process for solving them
