Back to All Projects
Problem Statement
Generating natural, grammatically coherent English captions for high-complexity unlabelled photographs.
Proposed Solution
Combined a pre-trained ResNet50 CNN encoder with a attention-based LSTM text decoder.
Key Features
- ResNet50 visual feature vector extraction layer
- LSTM decoder with word embedding lookup tables
- Interactive Gradio web demo deployed on HuggingFace Spaces
- Achieved competitive BLEU-4 validation score of 0.64
System Architecture Flow
Input Image -> ResNet50 Encoder -> Feature Vector -> LSTM Decoder -> Output Caption
Engineering Challenges
Managing sequence padding and vocabulary coverage across large text corpora.
Lessons Learned
Deep understanding of encoder-decoder neural network architectures and BLEU evaluation metrics.