After complaining that numpy took many hours to solve a 64k x 64k matrix, I broke out cuSolver, Nvidia's GPU linear algebra library. A 32k matrix gets solved (LU decomp) over 1000x faster than base numpy (with MKL not loving my AMD CPU), but a 64k matrix of floats is too big \
to solve directly on my 24 GB Titan RTX card. The nice thing about working with a low level library is that you have to explicitly allocate the temporary working buffers, so when it doesn't fit on the device, I can put it in pinned host memory or on my other card connected \
by NVLink. The 64k matrix gets solved in 109 s with nvlink memory, which is still 200x faster. At 32k, the comparison is:
Local mem: 2.2
Nvlink mem: 21.7
Host mem: 80.8
Clearly very bandwidth bound! There is probably a super-linear speedup for explicit multi-gpu computation. \
Local mem: 2.2
Nvlink mem: 21.7
Host mem: 80.8
Clearly very bandwidth bound! There is probably a super-linear speedup for explicit multi-gpu computation. \
The beauty of numpy is that one line of python equals a couple pages of ugly C code to use cuSolver. Pytorch would go fast, but still hit the out of memory wall before 64k.
One hiccup in the process: The cusolverDnSgetrf_bufferSize() function fills in an int for the buffer primitive count, and fails when it is over 2 billion, so I had to extrapolate myself. It should be a size_t! @NVIDIAHPCDev
• • •
Missing some Tweet in this thread? You can try to
force a refresh
