Gilper: Fine-Tuned BERT Model for Question Answering

Gilper is a fine-tuned BERT-based model, designed specifically for question-answering tasks. It excels at answering questions given a relevant context, making it ideal for applications like customer support, knowledge base queries, and more.

Features

  • Model Architecture: Based on BERT (bert-large-uncased-whole-word-masking-finetuned-squad).
  • Fine-Tuned Tasks: Specialized in question answering with input in the form of question + context.
  • Performance: Optimized for accuracy and relevance in providing answers.
  • Use Cases: Customer support bots, knowledge retrieval systems, educational tools, and more.

How to Use

Installation

To use Gilper, you need the Hugging Face Transformers library. Install it using:

pip install transformers

Quick Start

Here鈥檚 an example of how to use Gilper with the Transformers library:

from transformers import BertTokenizerFast, BertForQuestionAnswering, pipeline
import torch

# Load the tokenizer and model
tokenizer = BertTokenizerFast.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
model = BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')

# Define question-answering pipeline
question_answerer = pipeline(
    "question-answering",
    model="fawez9/gilper",
    tokenizer="fawez9/gilper",
    device=0 if torch.cuda.is_available() else -1
)

# Example input
question = "How many parameters does BLOOM have?"
context = "BLOOM has 176 billion parameters and can generate text in 46 natural languages and 13 programming languages."

# Get response
response = question_answerer(question=question, context=context)
print(response)

Input Format

  • Question: A string representing the query.
  • Context: A string containing the information from which the model will extract the answer.

Output Format

  • The model returns the most relevant span of text from the context as the answer.

Training Details

  • Base Model: bert-large-uncased-whole-word-masking-finetuned-squad.
  • Dataset: Fine-tuned on SQuAD.
  • Optimization: Trained using the AdamW optimizer with a learning rate of 2e-6.
  • Epochs: 3.
  • Batch Size: 4.
  • Training Metrics:
    • Global Steps: 750
    • Training Loss: 0.2977
    • Training Runtime: 3884.56 seconds
    • Samples per Second: 3.089
    • Steps per Second: 0.193
    • Total FLOPs: 8358362929152000.0

Model Card

  • Model Name: Gilper
  • Architecture: BERT-based
  • Fine-Tuned Task: Question Answering
  • Languages: English

Limitations

While Gilper is highly effective at question answering, it has the following limitations:

  • It relies heavily on the relevance and quality of the provided context.
  • It may not perform as well on questions requiring external knowledge not present in the context.
  • Biases from the training dataset may influence its responses.

Citation

If you use Gilper in your research or applications, please cite it as:

@misc{gilper2024,
  title={Gilper: Fine-Tuned BERT Model for Question Answering},
  author={Fawez},
  year={2024},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co./fawez9/gilper}}
}

Feedback and Contributions

We welcome feedback and contributions! If you encounter issues or have suggestions for improvements, please open an issue or submit a pull request.

Acknowledgments

  • Hugging Face for their excellent Transformers library.
  • The creators of the base BERT model.
  • The SQuAD dataset used for fine-tuning.
Downloads last month
15
Safetensors
Model size
334M params
Tensor type
F32
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Model tree for fawez9/gilper