2 2 5

Samuel Lima Braz PRO

samuellimabraz

AI & ML interests

None yet

Recent Activity

upvoted an article about 11 hours ago

PEFT: Parameter-Efficient Fine-Tuning Methods for LLMs

upvoted an article about 13 hours ago

FineWeb2-C: Help Build Better Language Models in Your Language

new activity 3 days ago

tech4humans/signature-detection:[bot] Conversion to Parquet

View all activity

Articles

PEFT: Parameter-Efficient Fine-Tuning Methods for LLMs

3 days ago

• 6

Organizations

Posts 1

Post

312

I wrote a article on Parameter-Efficient Fine-Tuning (PEFT), exploring techniques for efficient fine-tuning in LLMs, their implementations, and variations.

The study is based on the article "Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning" and the PEFT library integrated with Hugging Face's Transformers.

Article: https://huggingface.co./blog/samuellimabraz/peft-methods
Notebook: https://colab.research.google.com/drive/1B9RsKLMa8SwTxLsxRT8g9OedK10zfBEP?usp=sharing
Collection: samuellimabraz/service-summary-6793ccfe774073328ea9f8df

Analyzed methods:
- Adapters: Soft Prompts (Prompt Tuning, Prefix Tuning, P-tuning), IA³.
- Reparameterization: LoRA, QLoRA, LoHa, LoKr, X-LoRA, Intrinsic SAID, and variations of initializations (PiSSA, OLoRA, rsLoRA, DoRA).
- Selective Tuning: BitFit, DiffPruning, FAR, FishMask.

I'm starting out in generative AI, I have more experience with computer vision and robotics. Just sharing here 🤗