Post
612
Fine tuning on the edge. Pushing the MI100 to it's limits.
QWQ-32B 4bit QLORA fine tuning
VRAM usage 31.498G/31.984G :D
QWQ-32B 4bit QLORA fine tuning
VRAM usage 31.498G/31.984G :D
Csaba Kecskemeti PRO
csabakecskemeti
AI & ML interests
None yet
Recent Activity
updated
a model
about 3 hours ago
DevQuasar/prithivMLmods.Sombrero-QwQ-32B-Elite10-GGUF
published
a model
about 7 hours ago
DevQuasar/prithivMLmods.Sombrero-QwQ-32B-Elite10-GGUF
updated
a model
about 7 hours ago
DevQuasar/prithivMLmods.Bellatrix-Tiny-1B-v2-GGUF
Organizations
Posts
20
Post
1906
-UPDATED-
4bit inference is working! The blogpost is updated with code snippet and requirements.txt
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
-UPDATED-
I've played around with an MI100 and ROCm and collected my experience in a blogpost:
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
Unfortunately I've could not make inference or training work with model loaded in 8bit or use BnB, but did everything else and documented my findings.
4bit inference is working! The blogpost is updated with code snippet and requirements.txt
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
-UPDATED-
I've played around with an MI100 and ROCm and collected my experience in a blogpost:
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
Unfortunately I've could not make inference or training work with model loaded in 8bit or use BnB, but did everything else and documented my findings.
datasets
None public yet