Post
3314
š® GPT-3 implemented in pure Free Pascal!
https://github.com/joaopauloschuler/gpt-3-for-pascal
This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
Clean Pascal Implementation
https://github.com/joaopauloschuler/gpt-3-for-pascal
This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
āāāāāāāāāāāāāāāāāāāāāāāāāāā
ā Input Layer ā
āāāāāāāāāāāāāāāāāāāāāāāāāāā¤
ā Token & Positional ā
ā Embedding ā
āāāāāāāāāāāāāāāāāāāāāāāāāāā¤
ā 12x Transformer ā
ā Blocks ā
ā - 12 heads ā
ā - 768 hidden dims ā
ā - 3072 intermediate ā
āāāāāāāāāāāāāāāāāāāāāāāāāāā¤
ā Output Layer ā
āāāāāāāāāāāāāāāāāāāāāāāāāāāClean Pascal Implementation
for CntLayer := 1 to {Layers=}12 do
begin
Result.AddTransformerBlockCAI(
{Heads=}12,
{intermediate dimensions=}4*768,
{NoForward=}true,
{HasNorm=}true,
false
);
end;