DrishtiSharma/bge-small-en-obliqa-5-epochs

SentenceTransformer based on BAAI/bge-small-en

This is a sentence-transformers model finetuned from BAAI/bge-small-en. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-small-en
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("DrishtiSharma/bge-small-en-obliqa-5-epochs")
# Run inference
sentences = [
    'How should an Authorised Person document and justify any deviations when mapping activities into business lines for Operational Risk capital purposes, particularly when they differ from regulatory capital calculations in other risk categories?',
    'Principles for business line mapping. The mapping of activities into business lines for Operational Risk capital purposes should be consistent with the definitions of business lines used for regulatory capital calculations in other risk categories, i.e. credit and Market Risk. Any deviations from this principle should be clearly motivated and documented.',
    'REGULATORY REQUIREMENTS FOR AUTHORISED PERSONS ENGAGED IN REGULATED ACTIVITIES IN RELATION TO VIRTUAL ASSETS\nPlanned and Unplanned system outages\nAuthorised Persons should have a programme of planned systems outages to provide for adequate opportunities to perform updates and testing. Authorised Persons should also have multiple communication channels to ensure that its Clients are informed, ahead of time, of any outages which may affect them.\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.6255
cosine_accuracy@3	0.7518
cosine_accuracy@5	0.7934
cosine_accuracy@10	0.8368
cosine_precision@1	0.6255
cosine_precision@3	0.2641
cosine_precision@5	0.1703
cosine_precision@10	0.0931
cosine_recall@1	0.5488
cosine_recall@3	0.6705
cosine_recall@5	0.7117
cosine_recall@10	0.7628
cosine_ndcg@10	0.68
cosine_mrr@10	0.6976
cosine_map@100	0.6393
dot_accuracy@1	0.6255
dot_accuracy@3	0.7518
dot_accuracy@5	0.7934
dot_accuracy@10	0.8368
dot_precision@1	0.6255
dot_precision@3	0.2641
dot_precision@5	0.1703
dot_precision@10	0.0931
dot_recall@1	0.5488
dot_recall@3	0.6705
dot_recall@5	0.7117
dot_recall@10	0.7628
dot_ndcg@10	0.68
dot_mrr@10	0.6976
dot_map@100	0.6393

Training Details

Training Dataset

Unnamed Dataset

• Size: 22,291 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 14 tokens mean: 34.77 tokens max: 68 tokens	min: 12 tokens mean: 113.08 tokens max: 369 tokens

• Samples:

  sentence_0	sentence_1
  Could you outline the process for obtaining an ADGM Green Fund Designation or an ADGM Climate Transition Fund Designation, and what are the ongoing compliance obligations associated with these designations?	AUTHORISED PERSONS CONDUCTING A REGULATED ACTIVITY IN RELATION TO VIRTUAL ASSETS – EXTENSION INTO TO DIGITAL SECURITIES ACTIVITIES MTFs using Virtual Assets – Becoming a Digital Securities RIE Migration of a Virtual Assets MTF to a RIE is more complex than the extension of a Virtual Assets MTF into Digital Securities (as dealt with in paragraphs 63 - 67 above). This is due to a number of factors, including that a RIE is required to meet the full suite of requirements in Chapters 2 and 3 of MIR, and the primary market considerations associated with operating a RIE (e.g., requirement for Approved Prospectuses, admission to the Official List of Securities, and the ongoing technical/operational and regulatory requirements related to Digital Securities being admitted to trading and admitted to the Official List).
  Can the ADGM provide examples of effective internal risk control and reporting mechanisms that ensure an accurate assessment of a Reporting Entity's financial position and prospects, as per Rule 9.2.8?	Risk control. Authorised Persons should recognise and control the Credit Risk arising from their new products and services. Well in advance of entering into business transactions involving new types of products and activities, they should ensure that they understand the risks fully and have established appropriate Credit Risk policies, procedures and controls, which should be approved by the Governing Body or its appropriate delegated committee. A formal risk assessment of new products and activities should also be performed and documented.
  What are the specific criteria used by the FSRA to determine whether a company's ESG disclosures align with a globally recognized standard?	The Regulator may refuse to grant an application for an ADGM Green Bond Designation or an ADGM Sustainability-Linked Bond Designation if it is not satisfied that the requirements of this section have been met or will be met on an ongoing basis.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• num_train_epochs: 5
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss	cosine_map@100
0.0897	200	-	0.5994
0.1794	400	-	0.6027
0.2242	500	0.925	-
0.2691	600	-	0.6053
0.3587	800	-	0.6123
0.4484	1000	0.5995	0.5981
0.5381	1200	-	0.6230
0.6278	1400	-	0.6236
0.6726	1500	0.5963	-
0.7175	1600	-	0.6082
0.8072	1800	-	0.6192
0.8969	2000	0.5078	0.6128
0.9865	2200	-	0.6159
1.0	2230	-	0.6235
1.0762	2400	-	0.6232
1.1211	2500	0.4599	-
1.1659	2600	-	0.6122
1.2556	2800	-	0.6242
1.3453	3000	0.4054	0.6246
1.4350	3200	-	0.6364
1.5247	3400	-	0.6260
1.5695	3500	0.4197	-
1.6143	3600	-	0.6230
1.7040	3800	-	0.6324
1.7937	4000	0.3896	0.6384
1.8834	4200	-	0.6346
1.9731	4400	-	0.6279
2.0	4460	-	0.6296
2.0179	4500	0.3875	-
2.0628	4600	-	0.6263
2.1525	4800	-	0.6326
2.2422	5000	0.3117	0.6306
2.3318	5200	-	0.6351
2.4215	5400	-	0.6330
2.4664	5500	0.3327	-
2.5112	5600	-	0.6355
2.6009	5800	-	0.6323
2.6906	6000	0.3017	0.6249
2.7803	6200	-	0.6324
2.8700	6400	-	0.6326
2.9148	6500	0.2971	-
2.9596	6600	-	0.6306
3.0	6690	-	0.6368
3.0493	6800	-	0.6351
3.1390	7000	0.2755	0.6308
3.2287	7200	-	0.6372
3.3184	7400	-	0.6390
3.3632	7500	0.2639	-
3.4081	7600	-	0.6326
3.4978	7800	-	0.6351
3.5874	8000	0.2474	0.6377
3.6771	8200	-	0.6375
3.7668	8400	-	0.6380
3.8117	8500	0.2402	-
3.8565	8600	-	0.6407
3.9462	8800	-	0.6401
4.0	8920	-	0.6433
4.0359	9000	0.2628	0.6452
4.1256	9200	-	0.6432
4.2152	9400	-	0.6426
4.2601	9500	0.2318	-
4.3049	9600	-	0.6404
4.3946	9800	-	0.6390
4.4843	10000	0.2246	0.6389
4.5740	10200	-	0.6394
4.6637	10400	-	0.6388
4.7085	10500	0.2054	-
4.7534	10600	-	0.6396
4.8430	10800	-	0.6389
4.9327	11000	0.2194	0.6394
5.0	11150	-	0.6393

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.1.1
• Transformers: 4.45.2
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.0.dev0
• Datasets: 3.1.0
• Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}