sims2k
/

Saul-Instruct-v1-gdpr-finetuned-v3

@@ -1,199 +1,113 @@
 ---
 library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
 library_name: transformers
+tags:
+- GDPR
+- Law
+- English
+- Data Protection
+license: mit
+datasets:
+- sims2k/GDPR_QA_instruct_dataset
+language:
+- en
+metrics:
+- bleu
+- rouge
+- meteor
+- bertscore
+base_model: Equall/Saul-7B-Instruct-v1
 ---
+## **Model Overview**
+**Model Name**: Equall/Saul-7B-Instruct-GDPR-v1
+**Base Model**: Equall/Saul-7B-Instruct-v1
+**Finetuning Method**: QLoRA (Quantized Low-Rank Adaptation)
+Equall/Saul-7B-Instruct-GDPR-v1 is a fine-tuned version of the Equall/Saul-7B-Instruct-v1 model, specifically optimized for handling **GDPR compliance tasks**. The model is designed to generate accurate, legally grounded responses aligned with the General Data Protection Regulation (GDPR). It supports various compliance-related tasks such as data audits, legal advisory, and ensuring adherence to GDPR requirements for data controllers and processors.
+---
+## **GDPR Task Focus**
+The model is optimized to address detailed queries related to GDPR compliance, providing step-by-step instructions aligned with relevant GDPR articles. Below is an example illustrating how the model approaches a specific GDPR task related to conducting a Data Protection Impact Assessment (DPIA):
+### Example Task: Conducting a Data Protection Impact Assessment (DPIA)
+**Input**: What are the benefits of conducting a DPIA?
+**Response**:
+Conducting a **Data Protection Impact Assessment (DPIA)** provides several benefits in ensuring GDPR compliance:
+1. **Identifying Risks**: DPIAs help identify and mitigate risks to individuals’ rights and freedoms by evaluating potential impacts on personal data processing activities.
+2. **Accountability**: DPIAs demonstrate compliance with **GDPR Article 24(1)**, which requires controllers to implement appropriate technical and organizational measures to ensure and maintain data protection.
+3. **Building Compliance**: Through a structured approach, DPIAs facilitate building compliance by identifying necessary measures for processing activities.
+4. **Legal Basis**: A DPIA can serve as a legal basis for processing personal data, especially when dealing with special categories of data or large-scale monitoring (referencing **Article 6(1)(c)** and **Article 9(2)(i)**).
+5. **Regulatory Compliance**: Meeting DPIA requirements helps prevent fines imposed by supervisory authorities for non-compliance with GDPR articles, including **Articles 35(1), (3)-(4)**, **Article 35(2)**, **Article 35(7)-(9)**, and **Article 36(3)(e)**.
+**Relevant GDPR Articles**:
+- **Article 35** (DPIA requirements)
+- **Article 24** (Accountability of controllers)
+- **Article 6(1)(c)** (Legal basis for processing)
+- **Article 9(2)(i)** (Processing of special categories of data)
+- **Article 36(3)(e)** (Consultation with supervisory authorities)
+This demonstrates the model's capacity to generate structured, article-specific responses that assist organizations in navigating GDPR compliance tasks.
+---
+## **Fine-Tuning Methodology**
+The fine-tuning of this model was conducted using **QLoRA** (Quantized Low-Rank Adaptation) to optimize model efficiency and accuracy, particularly when handling legal texts. QLoRA enabled the fine-tuning process to maintain a high level of performance while significantly reducing the computational load by quantizing the model weights to 4-bit precision.
+Training was conducted using the **bwUniCluster 2.0 computing facility**, utilizing **Tesla V100 GPUs** for efficient training over multiple iterations. Each iteration aimed to improve the model’s capacity to understand and generate responses to GDPR-specific inquiries by referencing the appropriate articles of the regulation.
+---
+## **Datasets**
+### **1. Training Dataset**
+**Dataset Name**: sims2k/GDPR_QA_instruct_dataset
+- **Number of Entries**: 316 Question-Answer pairs
+- **Creation Method**: This dataset was synthetically generated using **ChatGPT-4** to create specialized Q&A pairs focused on GDPR compliance tasks. The dataset was carefully crafted by synthesizing information from trusted sources, including **GDPR articles**, **Legal FAQs**, and **Guidelines, Recommendations, and Best Practices from the European Data Protection Board (EDPB)**.
+  - **Advanced Prompt Engineering** techniques were employed, including **one-shot** and **chain-of-thought prompting**, to create precise, contextually relevant responses. The output generation was controlled using a **temperature setting of zero**, ensuring determinism and reliability in the responses.
+  - Each dataset entry was fact-checked for accuracy and cross-referenced with the related GDPR articles, ensuring legal validity and practical utility in real-world settings.
+### **2. Evaluation Dataset**
+**Dataset Name**: sims2k/GDPR_QA_instruct_eval_dataset
+- **Number of Entries**: 63 Question-Answer pairs
+- **Description**: This evaluation dataset was designed to rigorously test the model's ability to generalize its learning. Each entry focuses on unseen GDPR queries, ensuring the model’s ability to respond accurately to new contexts. The dataset was evaluated using advanced NLP metrics like **ROUGE**, **BLEU**, **METEOR**, and **BERTScore**, which help measure the structural and semantic quality of the responses.
+---
+## **Performance Metrics**
+The model’s performance was assessed using advanced NLP metrics to evaluate both the quality of generated text and the adherence to legal standards in GDPR queries.
+### **Metrics Used**:
+1. **BLEU**: Measures precision by calculating n-gram overlap between the generated response and the reference text.
+2. **ROUGE**: Focuses on recall, assessing how much of the reference content is captured in the generated response.
+3. **METEOR**: Combines both precision and recall, weighting recall more heavily and evaluating the quality of text alignment.
+4. **BERTScore**: Uses contextual embeddings to compare the generated and reference texts, focusing on semantic coherence.
+The results are presented in the **Composite Scores for All Evaluated Models** graph below, showcasing the model’s performance across these metrics.
+<p align="center">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/653e07af6d28265c85c84f6b/O011sXNOkCMOVnT-QtQ8F.png" alt="image/png">
+</p>
+### **Understanding the Graph**:
+- **Higher Composite Scores** represent a stronger performance in generating accurate, legally valid, and contextually appropriate responses.
+- **Normalization** was applied to all metrics using **Min-Max scaling**, ensuring an equal contribution of each metric to the final score.
+- **Equal Weighting** was used across metrics to provide a balanced assessment of the model’s capabilities.
+---
+## **Limitations and Future Work**
+Despite its strong performance in GDPR compliance tasks, the model may face challenges in handling **edge cases** or **complex legal nuances**. The model's accuracy could further be improved by expanding the dataset to include additional legal scenarios and by incorporating domain-specific datasets from other regulatory frameworks.
+Future improvements will focus on:
+- Expanding the dataset size and diversity.
+- Conducting more fine-tuning iterations to address subtle legal interpretations.
+- Potentially integrating legal reasoning from other regulatory domains beyond GDPR.