When I let myself spend a day doing CUDA work, I'm usually quite happy afterwards. Optimization is a grand puzzle game with an objective score function! I have to make myself stop at a wise point, despite all the intriguing rabbit holes left to pursue. \
CUDA is not my go-to go-fast tool -- with a modern, big CPU, it is pretty great how much performance you can trivially get with #pragma omp parallel for. The last couple times I have done it, my first cut at a CUDA kernel was slower than the C++ running on 128 threads. \
After dialing everything in for occupancy and memory traffic it usually winds up 6x+ faster, unless it really leans on GPU features like tensor ops or texture filtering, which can go much higher. Of course, I only bother with functions I know will turn out well. Many won't. \
Little thing I just learned: I have had problems in the past where a kernel tightly optimized for release builds would fail to launch in debug mode. Using the __launch_bounds__ annotation fixed it for me today.
The secret to optimization is changing the problem to make it easier to optimize!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
