Thread by @VikParuchuri

Announcing Surya OCR 2! It uses a new architecture and improves on v1 in every way:

- OCR with automatic language detection for 93 languages (no more specifying languages!)
- More accurate on old/noisy documents
- 20% faster
- Basic English handwriting support

Find Surya here - .

Surya OCR 2 is more accurate across all document types. It also compares favorably to Tesseract and Google Cloud OCR. The benchmarking script is in the repo.

Language is not hinted to Surya 2 for these benchmarks. github.com/VikParuchuri/s…

My earlier benchmark compared mainly clean documents, so I made a new noisy document benchmark to compare v2 and v1. This was created from tapuscorpus by @Alix_Tz. Again, language is not hinted.

v2 is 20% faster than v1. I tested using an A10 GPU, with batch size 525 for v2, and 300 for v1, which use about the same VRAM (20GB).

This is despite having twice as many active inference parameters!

There's now very basic support for English handwriting, with improvements to come soon.

As you can see, there are mistakes, but this is a good step forward. It's hard to find handwriting data, please let me know if you have any you can share!

Surya OCR 2 uses a custom architecture with a swin transformer encoder. I really wanted to use a non-autoregressive model, but it's hard to beat the kv cache baseline.

Instead, I used a shallow decoder to get the benefits of autoregression, but with much faster processing.

Unlike the original model, which used an MoE, you no longer need to specify the language of a document. Languages are determined automatically.

You can still give language hints to improve accuracy - up to 4 languages can be hinted.

See the hosted version and on-prem options at .

The model is trained from scratch, so it's okay for commercial usage. There are some restrictions if your company is over $5M in revenue or funding raised (see link for details).datalab.to

I'm hiring people to work with me on Surya and other models! We'll be training and open sourcing models with task-specific architectures.

More details here - .

Join the Discord - - if you want to discuss Surya generally.discord.com/channels/11820…
discord.gg/h9JC3hFY85

I'm still testing some approaches to OCR, and will hopefully have more updates soon, including:

- Improved speed, especially on CPU
- Better handwriting support
- Math support

This model is built on amazing open source work. Thank you to everyone whose open contributions made it possible, especially the @huggingface transformers team and the swin transformer authors.