Gilper: Fine-Tuned BERT Model for Question Answering

Gilper is a fine-tuned BERT-based model, designed specifically for question-answering tasks. It excels at answering questions given a relevant context, making it ideal for applications like customer support, knowledge base queries, and more.

Features

• Model Architecture: Based on BERT (bert-large-uncased-whole-word-masking-finetuned-squad).
• Fine-Tuned Tasks: Specialized in question answering with input in the form of question + context.
• Performance: Optimized for accuracy and relevance in providing answers.
• Use Cases: Customer support bots, knowledge retrieval systems, educational tools, and more.

How to Use

Installation

To use Gilper, you need the Hugging Face Transformers library. Install it using:

pip install transformers

Quick Start

Here’s an example of how to use Gilper with the Transformers library:

from transformers import BertTokenizerFast, BertForQuestionAnswering, pipeline
import torch

# Load the tokenizer and model
tokenizer = BertTokenizerFast.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
model = BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')

# Define question-answering pipeline
question_answerer = pipeline(
    "question-answering",
    model="fawez9/gilper",
    tokenizer="fawez9/gilper",
    device=0 if torch.cuda.is_available() else -1
)

# Example input
question = "How many parameters does BLOOM have?"
context = "BLOOM has 176 billion parameters and can generate text in 46 natural languages and 13 programming languages."

# Get response
response = question_answerer(question=question, context=context)
print(response)

Input Format

• Question: A string representing the query.
• Context: A string containing the information from which the model will extract the answer.

Output Format

• The model returns the most relevant span of text from the context as the answer.

Training Details

• Base Model: bert-large-uncased-whole-word-masking-finetuned-squad.
• Dataset: Fine-tuned on SQuAD.
• Optimization: Trained using the AdamW optimizer with a learning rate of 2e-6.
• Epochs: 3.
• Batch Size: 4.
• Training Metrics:
  • Global Steps: 750
  • Training Loss: 0.2977
  • Training Runtime: 3884.56 seconds
  • Samples per Second: 3.089
  • Steps per Second: 0.193
  • Total FLOPs: 8358362929152000.0

Model Card

• Model Name: Gilper
• Architecture: BERT-based
• Fine-Tuned Task: Question Answering
• Languages: English

Limitations

While Gilper is highly effective at question answering, it has the following limitations:

• It relies heavily on the relevance and quality of the provided context.
• It may not perform as well on questions requiring external knowledge not present in the context.
• Biases from the training dataset may influence its responses.

Citation

If you use Gilper in your research or applications, please cite it as:

@misc{gilper2024,
  title={Gilper: Fine-Tuned BERT Model for Question Answering},
  author={Fawez},
  year={2024},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co./fawez9/gilper}}
}

Feedback and Contributions

We welcome feedback and contributions! If you encounter issues or have suggestions for improvements, please open an issue or submit a pull request.

Acknowledgments

• Hugging Face for their excellent Transformers library.
• The creators of the base BERT model.
• The SQuAD dataset used for fine-tuning.