add llama 3 support to llm.c#754

Draft

karpathy wants to merge 75 commits intomasterfrom

Commits on Sep 13, 2024

llama3 starting point is at gpt-2 exact copy paste for both train/test files
karpathy
committed
first set of changes to match up the .py and the .cu version. default hyperparameters, introduce int+float section of header, read the header and EXIT for now
karpathy
committed
change the export code of Llama 3 to be very GPT-2 friendly, using a combination of 3 hacks. this will make it so that we have to change very little code on the C side
karpathy
committed

Commits on Sep 17, 2024

Commits on Sep 21, 2024

add repkv kernel to replicate K,V heads after the QKV projection
karpathy
committed

Commits on Sep 24, 2024

Commits on Sep 25, 2024

Commits on Sep 26, 2024

Commits on Sep 27, 2024

Commits on Oct 1, 2024

Commits on Apr 13, 2025

define llama3.2 1B and 3B for export from python (will untie embeddings and lm-head for now)
ngc92
committed

Commits on Apr 14, 2025

Commits on May 1, 2025

Merge pull request #802 from ngc92/ngc92/llama3-dev
karpathy
authored

Commits on May 4, 2025

Commits on May 10, 2025

Merge pull request #811 from ngc92/llama-fixes
karpathy
authored