8 contributors

History: 195 commits

versae

Add v2 files

376bcea over 2 years ago

configs
Changed and added vocab and tokenizer about 3 years ago
evaluation
Adding stepwise 512 and fixing some markdown warnings about 3 years ago
images
image style update about 3 years ago
mc4
Add eval scripts about 3 years ago
utils
Fixes and eval configs about 3 years ago
.gitattributes

737 Bytes

initial commit about 3 years ago
.gitignore

1.84 kB

Initial test with BETO's corpus about 3 years ago
README.md

42.8 kB

Automatic correction of README.md metadata. Contact [email protected] for any question almost 3 years ago
config.json

674 Bytes

Add v2 files over 2 years ago
config.py

256 Bytes

Preparing code for final runs about 3 years ago
convert.py

876 Bytes

Improved version of conversion script Flax → PyTorch about 3 years ago
events.out.tfevents.1625704081.t1v-n-a4d97d44-w-0.212075.3.v2

40 Bytes
LFS

Adding tfevents about 3 years ago
events.out.tfevents.1625704245.t1v-n-a4d97d44-w-0.216676.3.v2

40 Bytes
LFS

Adding tfevents about 3 years ago
events.out.tfevents.1625705283.t1v-n-a4d97d44-w-0.234462.3.v2

40 Bytes
LFS

Adding tfevents about 3 years ago
flax_model.msgpack

499 MB
LFS

Add v2 files over 2 years ago
get_embeddings_and_perplexity.py

1.53 kB

Add script to generate dataset of embeddings and perplexities. Add script to generate t-SNE plot for embedding and perplexity visualization. about 3 years ago
merges.txt

509 kB

Add v2 files over 2 years ago
perplexity.py

751 Bytes

Adding checkpointing, wandb, and new mlm script about 3 years ago
pytorch_model.bin
Detected Pickle imports (4)
- "torch.LongStorage",
- "torch.FloatStorage",
- "collections.OrderedDict",
- "torch._utils._rebuild_tensor_v2"
What is a pickle import?
499 MB
LFS

Add v2 files over 2 years ago
run.sh

883 Bytes

Adding base config and organizing configs about 3 years ago
run_mlm_flax.py

30 kB

Adding sampling to mc4 about 3 years ago
run_mlm_flax_stream.py

35.2 kB

Fixes and eval configs about 3 years ago
run_stream.sh

932 Bytes

Preparing code for final runs about 3 years ago
special_tokens_map.json

772 Bytes

Add v2 files over 2 years ago
tokenizer.json

2.21 MB

Add v2 files over 2 years ago
tokenizer_config.json

1.12 kB

Add v2 files over 2 years ago
tokens.py

649 Bytes

Scripts for perplexity sampling and fixes about 3 years ago
tokens.py.orig

899 Bytes

Adjust batch size for extrating tokens about 3 years ago
tsne_plot.py

3.02 kB

Remove unused imports about 3 years ago
vocab.json

851 kB

Add v2 files over 2 years ago

Detected Pickle imports (4)