Back to All Projects
Urdu Conversational Chatbot
NLP Research

Urdu Conversational Chatbot

Low-resource NLP Transformer with Multi-Head Attention

Multi-Head AttentionTransformerPyTorchGradioHugging Face

Problem Statement

Low-resource languages like Urdu lack high-quality pre-trained conversational models and right-to-left UI interfaces.

Proposed Solution

Designed a ground-up multi-head attention Transformer trained on curated conversational Urdu datasets.

Key Features

  • Custom BPE tokenization tuned for Urdu script dynamics
  • Multi-head self-attention encoder-decoder PyTorch architecture
  • RTL (Right-To-Left) responsive Gradio web interface
  • Published engineering writeup on Medium

System Architecture Flow

Urdu Text -> BPE Tokenizer -> Transformer Encoder/Decoder -> Gradio Output

Engineering Challenges

Handling script normalization, diacritics removal, and vocabulary sparseness.

Lessons Learned

Building custom multi-head attention layers and publishing technical AI research articles.