Training Code
#1
by
ahandleman
- opened
Any chance your team would be willing to release your training code? I'm interested in your approach to knowledge distillation and afaik right now there aren't any open source knowledge distillation projects.
@ahandleman Heya, I'm the dev behind the pipeline, opensourcing it is in my plans, no promises on when exactly as this has been one hell of a project for me, for which it earned its loving nickname of "purgatory", but it'll get released. Currently aiming to opensource at the same time as we release a nice distilled model.
Hey
@ahandleman
, the training code had been released!
https://github.com/golololologol/LLM-Distillery