Transformer-based Sentiment Classifier

🎬 Sentiment Analysis with Transformers 🎬
Classify movie reviews as positive or negative using a Transformer model built with PyTorch!

📚 Overview

This project uses a custom Transformer model for binary sentiment classification on the IMDb dataset. With PyTorch, torch.nn, datasets, and transformers, this model learns to identify patterns in text data to predict sentiment.

🚀 Getting Started

Prerequisites

Make sure you have the following libraries installed:

pip install torch transformers datasets numpy scikit-learn tqdm

Project Structure

1. Data Preparation - Load and preprocess IMDb dataset.
2. Model Definition - Build a custom Transformer model for text classification.
3. Training Loop - Train the model with a validation phase.
4. Inference - Test the model on new data for sentiment prediction.

🛠️ Code Walkthrough

PART 1: Data Preparation

• IMDBDataset - Custom PyTorch Dataset class to preprocess and tokenize text data.
• prepare_data - Function to load IMDb data, split it into training and validation sets, and create data loaders.

PART 2: Model Definition

• PositionalEncoding - Adds position information to tokens.
• TransformerClassifier - Custom Transformer model with an embedding layer, Transformer encoder, and a fully connected classifier.

PART 3: Training the Model

• train_model - Training function that includes memory management and gradient clipping for stability.

🏃‍♂️ Running the Code

1. Clone the Repository:

   git clone https://github.com/yourusername/transformer-sentiment-classifier.git
   cd transformer-sentiment-classifier
   
   

2. Train the Model:

   python train.py
   

   The model will automatically save as transformer_classifier.pth.

3. Run Inference: To classify a sample text, use:

   from predict import predict
   text = "The movie had an excellent plot and fantastic acting!"
   print("Prediction:", predict(text))
   

🎨 Sample Output

Training Loop Animation
The training progress and loss are updated dynamically with tqdm:

Epoch 1/20
Training |████████████████████| 100% - loss: 0.2453 - acc: 92.30%
Validation |████████████████████| 100% - loss: 0.3501 - acc: 87.50%

Prediction
Text: "The movie was a masterpiece!"
Prediction: Positive

📊 Model Evaluation

The final model accuracy on validation data is printed after training. Modify hyperparameters for different results and performance.

🎈 Example Animation

Training Loop Animation
Epoch 1 |█████               | 30% - loss: 0.255
Epoch 2 |█████████          | 55% - loss: 0.195
Epoch 3 |██████████████     | 80% - loss: 0.145
Epoch 4 |█████████████████  | 95% - loss: 0.110
...

📑 License

This project is licensed under the MIT License.

Happy Training! 🎉