Model Details
- Model Name: DistilBERT for Educational Query Classification
- Model Architecture: DistilBERT (base model:
distilbert-base-uncased) - Language: English
- Model Type: Transformer-based text classification model
- License: Apache License 2.0
Overview
This model is a fine-tuned version of DistilBERT specifically designed for classifying queries as either educational or non-educational. It was trained on a dataset containing a variety of questions and statements, with each entry labeled as either "educational" or "non-educational."
Intended Use
- Primary Use Case: This model is intended to classify text inputs into two categories: "educational" or "non-educational." It is useful for applications that need to filter out or prioritize educational content.
- Potential Applications:
- Educational chatbots or virtual assistants
- Content moderation for educational platforms
- Automated tagging of educational content
- Filtering non-educational queries from educational websites or apps
Training Data
- Dataset: The model was fine-tuned on a custom educational dataset. This dataset includes various types of queries that are labeled based on their content as either "educational" or "non-educational."
- Dataset Source: The dataset was manually curated to include a balanced mix of educational questions (covering various academic subjects) and non-educational questions (general queries that do not pertain to educational content).
Training Procedure
- Framework: The model was trained using the Hugging Face Transformers library with PyTorch.
- Fine-Tuning Parameters:
- Batch Size: 16
- Learning Rate: 5e-5
- Epochs: 3
- Optimizer: AdamW with weight decay
- Hardware: Fine-tuning was performed on a single NVIDIA V100 GPU.
Limitations and Bias
While this model has been fine-tuned for classifying queries as educational or non-educational, there are some limitations and potential biases:
- Bias in Data: The model may reflect any biases present in the training data, particularly if certain topics or types of educational content are overrepresented or underrepresented.
- Binary Classification: The model categorizes inputs strictly as "educational" or "non-educational." It may not handle nuanced or ambiguous queries effectively.
- Not Suitable for Other Classifications: This model is specifically designed for educational vs. non-educational classification. It may not perform well on other types of classification tasks without further fine-tuning.
How to Use
You can load the model using the Hugging Face Transformers library:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("haider0941/distilbert-base-educationl")
model = AutoModelForSequenceClassification.from_pretrained("haider0941/distilbert-base-educationl")
input_text = "What is the capital of France?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model(**inputs)
Citation
If you use this model, please cite it as follows:
@misc{Haider0941_2024,
title={Fine-Tuned DistilBERT for Educational Query Classification},
author={Haider},
year={2024},
howpublished={\url{https://huggingface.co./haider0941/distilbert-base-educationl}},
}
- Downloads last month
- 20