BERT, from scratch
A 7.5M-parameter Bengali BERT, pre-trained from nothing on a laptop, that scores above published mBERT and IndicBERT on Bengali news classification.
বিশ্বমানব হবি যদিশাশ্বত বাঙ্গালী হ’॥
Writing
Attention Is All You Need, from scratch
The original transformer rebuilt component by component, then trained to translate English into Bengali on one laptop.
Vision Transformer, from scratch
A ViT trained on 1,935 photographs of Bengali terracotta temples — a dataset I had to build first — to test the paper's claim about pre-training. It failed, exactly as predicted.