Update README.md
Browse files
README.md
CHANGED
|
@@ -14,6 +14,13 @@ license: cc-by-4.0
|
|
| 14 |
|
| 15 |
# bespin-global/klue-sroberta-base-continue-learning-by-mnr
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
|
| 18 |
|
| 19 |
<!--- Describe your model here -->
|
|
|
|
| 14 |
|
| 15 |
# bespin-global/klue-sroberta-base-continue-learning-by-mnr
|
| 16 |
|
| 17 |
+
해당 모델은 KLUE/NLI, KLUE/STS 데이터셋을 활용하였으며, sentence-transformers의 공식 문서 내 소개된 [continue-learning](https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/sts/training_stsbenchmark_continue_training.py) 방법을 통해 아래와 같이 학습되었습니다.
|
| 18 |
+
1. NLI 데이터셋을 통해 nagative sampling 후, MultipleNegativeRankingLoss를 활용하여 1차 NLI training 수행
|
| 19 |
+
2. 1에서 학습완료 된 모델에 STS 데이터셋을 통해, CosineSimilarityLoss를 활용하여 2차 STS training 수행
|
| 20 |
+
|
| 21 |
+
학습에 관한 자세한 내용은 [Blog](https://velog.io/@jaehyeong/Basic-NLP-sentence-transformers-%EB%9D%BC%EC%9D%B4%EB%B8%8C%EB%9F%AC%EB%A6%AC%EB%A5%BC-%ED%99%9C%EC%9A%A9%ED%95%9C-SBERT-%ED%95%99%EC%8A%B5-%EB%B0%A9%EB%B2%95#225-continue-learning-by-sts)와 [Colab 실습 코드](https://colab.research.google.com/drive/1uDt3o_Nv2cTiVbIAIUkst_eOSD37Wkmf)를 참고해주세요.
|
| 22 |
+
|
| 23 |
+
---
|
| 24 |
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
|
| 25 |
|
| 26 |
<!--- Describe your model here -->
|