查看“︁NanoGPT Tutorial”︁的源代码

=环境准备=
本文所有操作均在MacBook Air（2020,M1芯片）上测试验证。

== Miniconda 和Python==

在MacOS下，可以通过以下脚本安装<ref>https://docs.conda.io/projects/miniconda/en/latest/</ref>：

<syntaxhighlight lang="bash">
$ mkdir -p ~/miniconda3
$ curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh -o ~/miniconda3/miniconda.sh
$ bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
$ rm -rf ~/miniconda3/miniconda.sh
</syntaxhighlight>

安装完成后，可以使用conda命令来管理机器学习的Python环境了。默认系统会自动创建一个python环境：

<syntaxhighlight lang="bash">
$ python --version
Python 3.11.5
$ whereis python
python: /Users/riguz/miniconda3/bin/python
</syntaxhighlight>

== 下载 ==


<syntaxhighlight lang="bash">
(base) ➜  nanoGPT git:(master) python data/shakespeare_char/prepare.py
length of dataset in characters: 1,115,394
all the unique characters:
 !$&',-.3:;?ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
vocab size: 65
train has 1,003,854 tokens
val has 111,540 tokens
</syntaxhighlight>

<syntaxhighlight lang="bash">
python train.py config/train_shakespeare_char.py

# Run it without GPU (mac air), pytorch nightly
# raise AssertionError("Torch not compiled with CUDA enabled")
</syntaxhighlight>

<syntaxhighlight lang="bash">
python train.py config/train_shakespeare_char.py --device=cpu --compile=False --eval_iters=20 --log_interval=1 --block_size=64 --batch_size=12 --n_layer=4 --n_head=4 --n_embd=128 --max_iters=2000 --lr_decay_iters=2000 --dropout=0.0

step 2000: train loss 1.7640, val loss 1.8925
saving checkpoint to out-shakespeare-char
iter 2000: loss 1.6982, time 306.45ms, mfu 0.05%

# total cost： 48s

python sample.py --out_dir=out-shakespeare-char --device=cpu
Overriding: out_dir = out-shakespeare-char
Overriding: device = cpu
number of parameters: 0.80M
Loading meta from data/shakespeare_char/meta.pkl...

I by doth what letterd fain flowarrman,
Lotheefuly daught shouss blate thou his though'd that opt--
Hammine than you, not neme your down way.

ELANUS:
I would and murser wormen that more?
...
</syntaxhighlight>


<syntaxhighlight lang="bash">
python train.py config/train_shakespeare_char.py --device=mps --compile=False --eval_iters=20 --log_interval=1 --block_size=64 --batch_size=12 --n_layer=4 --n_head=4 --n_embd=128 --max_iters=2000 --lr_decay_iters=2000 --dropout=0.0

...
iter 1998: loss 1.8794, time 22.56ms, mfu 0.06%
iter 1999: loss 1.9167, time 22.62ms, mfu 0.06%
step 2000: train loss 1.7640, val loss 1.8925
saving checkpoint to out-shakespeare-char
iter 2000: loss 1.6982, time 352.67ms, mfu 0.05%
# total cost: 51s
</syntaxhighlight>
[[Category:Deep Learning]]
[[Category:PyTorch]]