metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:3012496
  - loss:MatryoshkaLoss
  - loss:CachedMultipleNegativesRankingLoss
base_model: google-bert/bert-base-uncased
widget:
  - source_sentence: are the sequels better than the prequels?
    sentences:
      - '[''Automatically.'', ''When connected to car Bluetooth and,'', ''Manually.'']'
      - >-
        The prequels are also not scared to take risks, making movies which are
        very different from the original trilogy. The sequel saga, on the other
        hand, are technically better made films, the acting is more consistent,
        the CGI is better and the writing is stronger, however it falls down in
        many other places.
      - >-
        While both public and private sectors use budgets as a key planning
        tool, public bodies balance budgets, while private sector firms use
        budgets to predict operating results. The public sector budget matches
        expenditures on mandated assets and services with receipts of public
        money such as taxes and fees.
  - source_sentence: are there bbqs at lake leschenaultia?
    sentences:
      - >-
        Vestavia Hills. The hummingbird, or, el zunzún as they are often called
        in the Caribbean, have such a nickname because of their quick movements.
        The ruby-throated hummingbird, the most commonly seen hummingbird in
        Alabama, is the inspiration for this restaurant.
      - >-
        Common causes of abdominal tenderness Abdominal tenderness is generally
        a sign of inflammation or other acute processes in one or more organs.
        The organs are located around the tender area. Acute processes mean
        sudden pressure caused by something. For example, twisted or blocked
        organs can cause point tenderness.
      - >-
        Located on 168 hectares of nature reserve, Lake Leschenaultia is the
        perfect spot for a family day out in the Perth Hills. The Lake offers
        canoeing, swimming, walk and cycle trails, as well as picnic, BBQ and
        camping facilities. ... There are picnic tables set amongst lovely
        Wandoo trees.
  - source_sentence: how much folic acid should you take prenatal?
    sentences:
      - >-
        Folic acid is a pregnancy superhero! Taking a prenatal vitamin with the
        recommended 400 micrograms (mcg) of folic acid before and during
        pregnancy can help prevent birth defects of your baby's brain and spinal
        cord. Take it every day and go ahead and have a bowl of fortified
        cereal, too.
      - >-
        ['You must be unemployed through no fault of your own, as defined by
        Virginia law.', 'You must have earned at least a minimum amount in wages
        before you were unemployed.', 'You must be able and available to work,
        and you must be actively seeking employment.']
      - >-
        Wallpaper is printed in batches of rolls. It is important to have the
        same batch number, to ensure colours match exactly. The batch number is
        usually located on the wallpaper label close to the pattern number.
        Remember batch numbers also apply to white wallpapers, as different
        batches can be different shades of white.
  - source_sentence: what is the difference between minerals and electrolytes?
    sentences:
      - >-
        North: Just head north of Junk Junction like so. South: Head below Lucky
        Landing. East: You're basically landing between Lonely Lodge and the
        Racetrack. West: The sign is west of Snobby Shores.
      - >-
        The fasting glucose tolerance test is the simplest and fastest way to
        measure blood glucose and diagnose diabetes. Fasting means that you have
        had nothing to eat or drink (except water) for 8 to 12 hours before the
        test.
      - >-
        In other words, the term “electrolyte” typically implies ionized
        minerals dissolved within water and beverages. Electrolytes are
        typically minerals, whereas minerals may or may not be electrolytes.
  - source_sentence: how can i download youtube videos with internet download manager?
    sentences:
      - >-
        ['Go to settings and then click on extensions (top left side in
        chrome).', 'Minimise your browser and open the location (folder) where
        IDM is installed. ... ', 'Find the file “IDMGCExt. ... ', 'Drag this
        file to your chrome browser and drop to install the IDM extension.']
      - >-
        Coca-Cola might rot your teeth and load your body with sugar and
        calories, but it's actually an effective and safe first line of
        treatment for some stomach blockages, researchers say.
      - >-
        To fix a disabled iPhone or iPad without iTunes, you have to erase your
        device. Click on the "Erase iPhone" option and confirm your selection.
        Wait for a while as the "Find My iPhone" feature will remotely erase
        your iOS device. Needless to say, it will also disable its lock.
datasets:
  - sentence-transformers/gooaq
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
co2_eq_emissions:
  emissions: 242.52371141034885
  energy_consumed: 0.623932244779674
  source: codecarbon
  training_type: fine-tuning
  on_cloud: false
  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
  ram_total_size: 31.777088165283203
  hours_used: 1.619
  hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
  - name: bert-base-uncased adapter finetuned on GooAQ pairs
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoClimateFEVER
          type: NanoClimateFEVER
        metrics:
          - type: cosine_accuracy@1
            value: 0.24
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.42
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.46
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.56
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.24
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.15999999999999998
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.10800000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.13166666666666665
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.20833333333333337
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.24166666666666664
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.29666666666666663
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.25516520961338873
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3378809523809523
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.20756281994556017
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoDBPedia
          type: NanoDBPedia
        metrics:
          - type: cosine_accuracy@1
            value: 0.54
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.84
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.92
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.54
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.4866666666666667
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.4440000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.3899999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.046781664425339056
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.11117774881295754
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.15829952609979633
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.2554819210350403
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4644109757573673
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6797460317460318
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3253011706807197
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoFEVER
          type: NanoFEVER
        metrics:
          - type: cosine_accuracy@1
            value: 0.54
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.82
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.92
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.54
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2733333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.184
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09599999999999997
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.53
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.7766666666666666
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8566666666666666
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.8866666666666667
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7348538316509182
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6961904761904762
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6788071339639872
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoFiQA2018
          type: NanoFiQA2018
        metrics:
          - type: cosine_accuracy@1
            value: 0.24
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.4
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.24
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08800000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.11474603174603175
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.22874603174603172
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3166031746031746
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.3986031746031745
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.2925721974861802
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3385
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2372091627126374
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoHotpotQA
          type: NanoHotpotQA
        metrics:
          - type: cosine_accuracy@1
            value: 0.6
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.68
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.74
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.88
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.6
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2866666666666667
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.192
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.118
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.43
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.48
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.59
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5291588954628265
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6639365079365079
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.45230644038161627
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoMSMARCO
          type: NanoMSMARCO
        metrics:
          - type: cosine_accuracy@1
            value: 0.28
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.48
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.54
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.66
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.28
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.10800000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.066
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.28
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.48
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.54
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.66
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.46795689507567784
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4079126984126984
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.42763462709531985
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNFCorpus
          type: NanoNFCorpus
        metrics:
          - type: cosine_accuracy@1
            value: 0.32
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.48
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.56
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.32
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.30666666666666664
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.244
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.184
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.02092621665706462
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.053426190783308986
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.06393651269284006
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.08045448545888809
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.23067635403503162
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.39788888888888885
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.09661097314535905
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNQ
          type: NanoNQ
        metrics:
          - type: cosine_accuracy@1
            value: 0.38
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.54
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.62
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.74
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.38
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.128
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07600000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.38
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.51
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.71
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5386606354769653
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.490547619047619
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.48961052316839493
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoQuoraRetrieval
          type: NanoQuoraRetrieval
        metrics:
          - type: cosine_accuracy@1
            value: 0.84
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.94
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.98
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 1
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.84
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.38666666666666655
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.24799999999999997
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.12999999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7573333333333332
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.912
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.946
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9793333333333334
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9157663307482551
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.9009999999999999
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8893741502029173
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoSCIDOCS
          type: NanoSCIDOCS
        metrics:
          - type: cosine_accuracy@1
            value: 0.26
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.46
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.68
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.26
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.20666666666666664
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.184
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.126
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.054000000000000006
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.12866666666666668
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.18966666666666668
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.25866666666666666
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.24181947685643387
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3803571428571429
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.18652061021747493
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoArguAna
          type: NanoArguAna
        metrics:
          - type: cosine_accuracy@1
            value: 0.16
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.58
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.74
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.84
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.16
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.19333333333333336
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14800000000000002
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08399999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.16
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.58
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.74
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.84
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5045313323048141
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3963333333333333
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.40074428294573644
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoSciFact
          type: NanoSciFact
        metrics:
          - type: cosine_accuracy@1
            value: 0.42
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.58
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.62
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.64
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.42
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.20666666666666667
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07600000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.4
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.56
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.605
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.64
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5380316349319392
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5056666666666666
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.5079821472790408
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoTouche2020
          type: NanoTouche2020
        metrics:
          - type: cosine_accuracy@1
            value: 0.4489795918367347
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8979591836734694
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9183673469387755
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9795918367346939
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.4489795918367347
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.4965986394557823
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.45714285714285713
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.38979591836734706
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.03475887574057735
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.11109807516506923
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.1656210426064535
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.2684807614936963
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.43233093716838594
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6532555879494653
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.33493945959592186
            name: Cosine Map@100
      - task:
          type: nano-beir
          name: Nano BEIR
        dataset:
          name: NanoBEIR mean
          type: NanoBEIR_mean
        metrics:
          - type: cosine_accuracy@1
            value: 0.4053061224489796
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.6213814756671899
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6891051805337519
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7676609105180533
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.4053061224489796
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2694819466248038
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.20962637362637365
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.14567660910518054
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.24693944527453943
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3915472856287718
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.4541123273847895
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.5280272058403178
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4727642081975526
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5268627619545987
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.40266180779497585
            name: Cosine Map@100

bert-base-uncased adapter finetuned on GooAQ pairs

This is a sentence-transformers model finetuned from google-bert/bert-base-uncased on the gooaq dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google-bert/bert-base-uncased
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- gooaq
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tomaarsen/bert-base-uncased-gooaq")
# Run inference
sentences = [
    'how can i download youtube videos with internet download manager?',
    "['Go to settings and then click on extensions (top left side in chrome).', 'Minimise your browser and open the location (folder) where IDM is installed. ... ', 'Find the file “IDMGCExt. ... ', 'Drag this file to your chrome browser and drop to install the IDM extension.']",
    "Coca-Cola might rot your teeth and load your body with sugar and calories, but it's actually an effective and safe first line of treatment for some stomach blockages, researchers say.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Datasets: NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact and NanoTouche2020
Evaluated with InformationRetrievalEvaluator

Metric	NanoClimateFEVER	NanoDBPedia	NanoFEVER	NanoFiQA2018	NanoHotpotQA	NanoMSMARCO	NanoNFCorpus	NanoNQ	NanoQuoraRetrieval	NanoSCIDOCS	NanoArguAna	NanoSciFact	NanoTouche2020
cosine_accuracy@1	0.24	0.54	0.54	0.24	0.6	0.28	0.32	0.38	0.84	0.26	0.16	0.42	0.449
cosine_accuracy@3	0.42	0.8	0.82	0.4	0.68	0.48	0.48	0.54	0.94	0.46	0.58	0.58	0.898
cosine_accuracy@5	0.46	0.84	0.9	0.5	0.74	0.54	0.5	0.62	0.98	0.6	0.74	0.62	0.9184
cosine_accuracy@10	0.56	0.92	0.92	0.6	0.88	0.66	0.56	0.74	1.0	0.68	0.84	0.64	0.9796
cosine_precision@1	0.24	0.54	0.54	0.24	0.6	0.28	0.32	0.38	0.84	0.26	0.16	0.42	0.449
cosine_precision@3	0.16	0.4867	0.2733	0.16	0.2867	0.16	0.3067	0.18	0.3867	0.2067	0.1933	0.2067	0.4966
cosine_precision@5	0.108	0.444	0.184	0.14	0.192	0.108	0.244	0.128	0.248	0.184	0.148	0.14	0.4571
cosine_precision@10	0.07	0.39	0.096	0.088	0.118	0.066	0.184	0.076	0.13	0.126	0.084	0.076	0.3898
cosine_recall@1	0.1317	0.0468	0.53	0.1147	0.3	0.28	0.0209	0.38	0.7573	0.054	0.16	0.4	0.0348
cosine_recall@3	0.2083	0.1112	0.7767	0.2287	0.43	0.48	0.0534	0.51	0.912	0.1287	0.58	0.56	0.1111
cosine_recall@5	0.2417	0.1583	0.8567	0.3166	0.48	0.54	0.0639	0.6	0.946	0.1897	0.74	0.605	0.1656
cosine_recall@10	0.2967	0.2555	0.8867	0.3986	0.59	0.66	0.0805	0.71	0.9793	0.2587	0.84	0.64	0.2685
cosine_ndcg@10	0.2552	0.4644	0.7349	0.2926	0.5292	0.468	0.2307	0.5387	0.9158	0.2418	0.5045	0.538	0.4323
cosine_mrr@10	0.3379	0.6797	0.6962	0.3385	0.6639	0.4079	0.3979	0.4905	0.901	0.3804	0.3963	0.5057	0.6533
cosine_map@100	0.2076	0.3253	0.6788	0.2372	0.4523	0.4276	0.0966	0.4896	0.8894	0.1865	0.4007	0.508	0.3349

Nano BEIR

Dataset: NanoBEIR_mean
Evaluated with NanoBEIREvaluator

Metric	Value
cosine_accuracy@1	0.4053
cosine_accuracy@3	0.6214
cosine_accuracy@5	0.6891
cosine_accuracy@10	0.7677
cosine_precision@1	0.4053
cosine_precision@3	0.2695
cosine_precision@5	0.2096
cosine_precision@10	0.1457
cosine_recall@1	0.2469
cosine_recall@3	0.3915
cosine_recall@5	0.4541
cosine_recall@10	0.528
cosine_ndcg@10	0.4728
cosine_mrr@10	0.5269
cosine_map@100	0.4027

Training Details

Training Dataset

gooaq

Dataset: gooaq at b089f72
Size: 3,012,496 training samples
Columns: question and answer
Approximate statistics based on the first 1000 samples:
question answer
type string string
details
min: 8 tokens
mean: 11.86 tokens
max: 21 tokens

min: 14 tokens
mean: 60.48 tokens
max: 138 tokens

	question	answer
type	string	string
details	min: 8 tokens mean: 11.86 tokens max: 21 tokens	min: 14 tokens mean: 60.48 tokens max: 138 tokens

Samples:

question	answer
`what is the difference between broilers and layers?`	`An egg laying poultry is called egger or layer whereas broilers are reared for obtaining meat. So a layer should be able to produce more number of large sized eggs, without growing too much. On the other hand, a broiler should yield more meat and hence should be able to grow well.`
`what is the difference between chronological order and spatial order?`	`As a writer, you should always remember that unlike chronological order and the other organizational methods for data, spatial order does not take into account the time. Spatial order is primarily focused on the location. All it does is take into account the location of objects and not the time.`
`is kamagra same as viagra?`	`Kamagra is thought to contain the same active ingredient as Viagra, sildenafil citrate. In theory, it should work in much the same way as Viagra, taking about 45 minutes to take effect, and lasting for around 4-6 hours. However, this will vary from person to person.`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "CachedMultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64,
        32
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Evaluation Dataset

gooaq

Dataset: gooaq at b089f72
Size: 3,012,496 evaluation samples
Columns: question and answer
Approximate statistics based on the first 1000 samples:
question answer
type string string
details
min: 8 tokens
mean: 11.88 tokens
max: 22 tokens

min: 14 tokens
mean: 61.03 tokens
max: 127 tokens

	question	answer
type	string	string
details	min: 8 tokens mean: 11.88 tokens max: 22 tokens	min: 14 tokens mean: 61.03 tokens max: 127 tokens

Samples:

question	answer
`how do i program my directv remote with my tv?`	`['Press MENU on your remote.', 'Select Settings & Help > Settings > Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you wish to program. ... ', 'Follow the on-screen prompts to complete programming.']`
`are rodrigues fruit bats nocturnal?`	`Before its numbers were threatened by habitat destruction, storms, and hunting, some of those groups could number 500 or more members. Sunrise, sunset. Rodrigues fruit bats are most active at dawn, at dusk, and at night.`
`why does your heart rate increase during exercise bbc bitesize?`	`During exercise there is an increase in physical activity and muscle cells respire more than they do when the body is at rest. The heart rate increases during exercise. The rate and depth of breathing increases - this makes sure that more oxygen is absorbed into the blood, and more carbon dioxide is removed from it.`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "CachedMultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64,
        32
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 1024
per_device_eval_batch_size: 1024
learning_rate: 2e-05
num_train_epochs: 1
warmup_ratio: 0.1
seed: 12
bf16: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 1024
per_device_eval_batch_size: 1024
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 12
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	NanoClimateFEVER_cosine_ndcg@10	NanoDBPedia_cosine_ndcg@10	NanoFEVER_cosine_ndcg@10	NanoFiQA2018_cosine_ndcg@10	NanoHotpotQA_cosine_ndcg@10	NanoMSMARCO_cosine_ndcg@10	NanoNFCorpus_cosine_ndcg@10	NanoNQ_cosine_ndcg@10	NanoQuoraRetrieval_cosine_ndcg@10	NanoSCIDOCS_cosine_ndcg@10	NanoArguAna_cosine_ndcg@10	NanoSciFact_cosine_ndcg@10	NanoTouche2020_cosine_ndcg@10	NanoBEIR_mean_cosine_ndcg@10
0	0	-	-	0.1046	0.2182	0.1573	0.0575	0.2597	0.1602	0.0521	0.0493	0.7310	0.1320	0.2309	0.1240	0.0970	0.1826
0.0010	1	28.4268	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0256	25	24.7252	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0512	50	13.3628	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0768	75	7.843	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1024	100	5.7393	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1279	125	4.6576	2.3368	0.2890	0.4610	0.7408	0.2882	0.5446	0.4091	0.2179	0.4664	0.9079	0.2394	0.5433	0.5003	0.4318	0.4646
0.1535	150	4.0846	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1791	175	3.7129	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2047	200	3.4899	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2303	225	3.3263	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2559	250	3.2013	1.6545	0.2622	0.4744	0.7456	0.2934	0.5371	0.4326	0.2290	0.5157	0.9130	0.2577	0.5189	0.5155	0.4302	0.4712
0.2815	275	2.9109	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3071	300	2.9064	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3327	325	2.8215	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3582	350	2.7893	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3838	375	2.6663	1.4146	0.2629	0.4657	0.7330	0.2853	0.5299	0.4346	0.2311	0.5216	0.9172	0.2513	0.5133	0.5429	0.4287	0.4706
0.4094	400	2.6672	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4350	425	2.5587	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4606	450	2.5001	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4862	475	2.4476	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5118	500	2.4127	1.2843	0.2565	0.4668	0.7289	0.2838	0.5392	0.4599	0.2284	0.5238	0.9021	0.2416	0.4971	0.5349	0.4320	0.4688
0.5374	525	2.414	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5629	550	2.3723	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5885	575	2.3418	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6141	600	2.2862	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6397	625	2.207	1.2078	0.2613	0.4542	0.7382	0.2817	0.5230	0.4664	0.2282	0.5266	0.9095	0.2453	0.5127	0.5414	0.4239	0.4702
0.6653	650	2.2305	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6909	675	2.2409	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7165	700	2.2001	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7421	725	2.1923	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7677	750	2.195	1.1538	0.2549	0.4671	0.7333	0.2804	0.5265	0.4659	0.2321	0.5331	0.9086	0.2429	0.5070	0.5430	0.4369	0.4717
0.7932	775	2.1826	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8188	800	2.1754	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8444	825	2.1141	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8700	850	2.1572	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8956	875	2.1126	1.1256	0.2505	0.4622	0.7293	0.2857	0.5286	0.4823	0.2308	0.5397	0.9158	0.2412	0.5050	0.5365	0.4387	0.4728
0.9212	900	2.0755	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9468	925	2.1032	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9724	950	2.1211	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9980	975	2.0826	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
1.0	977	-	-	0.2552	0.4644	0.7349	0.2926	0.5292	0.4680	0.2307	0.5387	0.9158	0.2418	0.5045	0.5380	0.4323	0.4728

Environmental Impact

Carbon emissions were measured using CodeCarbon.

Energy Consumed: 0.624 kWh
Carbon Emitted: 0.243 kg of CO2
Hours Used: 1.619 hours

Training Hardware

On Cloud: No
GPU Model: 1 x NVIDIA GeForce RTX 3090
CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
RAM Size: 31.78 GB

Framework Versions

Python: 3.11.6
Sentence Transformers: 3.4.0.dev0
Transformers: 4.46.2
PyTorch: 2.5.0+cu121
Accelerate: 0.35.0.dev0
Datasets: 2.20.0
Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}