twenkid commited on
Commit
13a05ff
·
verified ·
1 Parent(s): b574915

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -9,7 +9,7 @@ Updates 13.1.2025: The dataset was discovered, adding more details about it.
9
  * GPT2-Medium 345M for Bulgarian
10
  * The model was created and trained from scratch, using tensorflow in free Google Colab T4. The research experiment started in June 2021, the date of the last the video explanation is uploaded on 17.9.2024.
11
  * "Last modified" dates of the lastly added data is between 24.7.2021-28.7.2021, about 11 MiB texts.
12
- * The dataset was quite small with a maximum of about 141 MiB UTF8 (148.585 M bytes), it includes some words and texts in other languages within the text, ~82,48M characters), but IMO the results were decent (subjectively for the size, no systematic formal test).
13
  * It is supposed to be run with the provided code here and in the notebook. Read the comments in gen_comments-1-2023-clean.py
14
  * That was the biggest, as far as I knew, GPT/Transformer model in Bulgarian at the time, except one with unknown size, which was demoed for a few seconds in a video in Linkedin* (more in a footnote)
15
  * A method for unlimited length multi-step generation with hidden injections of tokens for directed topic change (but it needed more smoothing etc.). The methods are explained in videos on Youtube.
 
9
  * GPT2-Medium 345M for Bulgarian
10
  * The model was created and trained from scratch, using tensorflow in free Google Colab T4. The research experiment started in June 2021, the date of the last the video explanation is uploaded on 17.9.2024.
11
  * "Last modified" dates of the lastly added data is between 24.7.2021-28.7.2021, about 11 MiB texts.
12
+ * The dataset was quite small with a maximum of about 141 MiB UTF8 (148.585 M bytes), it includes some words and texts in other languages (English, Latin?), ~82,48M characters), but IMO the results were decent (subjectively for the size, no systematic formal test).
13
  * It is supposed to be run with the provided code here and in the notebook. Read the comments in gen_comments-1-2023-clean.py
14
  * That was the biggest, as far as I knew, GPT/Transformer model in Bulgarian at the time, except one with unknown size, which was demoed for a few seconds in a video in Linkedin* (more in a footnote)
15
  * A method for unlimited length multi-step generation with hidden injections of tokens for directed topic change (but it needed more smoothing etc.). The methods are explained in videos on Youtube.