{"id":8007,"date":"2026-03-03T02:08:13","date_gmt":"2026-03-02T18:08:13","guid":{"rendered":"https:\/\/kyle.ai\/blog\/?p=8007"},"modified":"2026-03-03T02:12:50","modified_gmt":"2026-03-02T18:12:50","slug":"microgpt%ef%bc%8c%e5%85%a8%e7%bd%91%e6%9c%80%e5%a5%bd%e6%87%82%e7%9a%84%e5%a4%a7%e6%a8%a1%e5%9e%8b%e5%b7%a5%e4%bd%9c%e5%8e%9f%e7%90%86%e8%a7%a3%e9%87%8a","status":"publish","type":"post","link":"https:\/\/kyle.ai\/blog\/8007.html","title":{"rendered":"MicroGPT\uff0c\u5168\u7f51\u6700\u597d\u61c2\u7684\u5927\u6a21\u578b\u5de5\u4f5c\u539f\u7406\u89e3\u91ca"},"content":{"rendered":"\n<p>\u4e4b\u524d\u770b\u8fc7\u4e00\u672c\u8bb2 GPT \u539f\u7406\u7684\u7535\u5b50\u4e66<a data-darkify_alpha_bg=\"rgba(0, 0, 0, 0)\" data-darkify_preserved_classes=\"darkify_style_link darkify_style_txt_border darkify_processed\" href=\"https:\/\/book.douban.com\/subject\/36808317\/\" class=\"darkify_style_link darkify_style_txt_border darkify_processed\">\u300aBuild a Large Language Model (From Scratch)\u300b<\/a> ,\u611f\u89c9\u8fd8\u662f\u975e\u5e38\u590d\u6742\uff0c\u8981\u6c42\u6709\u4e00\u4e9b\u673a\u5668\u5b66\u4e60\u7684\u57fa\u672c\u529f\u624d\u80fd\u61c2\u3002<\/p>\n\n\n\n<p>\u7136\u540e\u6700\u8fd1\uff0c\u6709\u5927\u795e Andrej Karpathy \u60c5\u4eba\u8282\u524d\uff082.12\uff09\u5199\u4e86\u7bc7\u535a\u5ba2\uff1a<a data-darkify_alpha_bg=\"rgba(0, 0, 0, 0)\" data-darkify_preserved_classes=\"darkify_style_link darkify_style_txt_border darkify_processed\" href=\"https:\/\/karpathy.github.io\/2026\/02\/12\/microgpt\/\" class=\"darkify_style_link darkify_style_txt_border darkify_processed\">https:\/\/karpathy.github.io\/2026\/02\/12\/microgpt\/<\/a> \uff0c\u5206\u4eab\u5982\u4f55\u7528 200 \u884c Python \u4ee3\u7801\u4ece\u96f6\u5199\u4e00\u4e2a\u7c7b ChatGPT \u7b97\u6cd5\uff0c\u7528\u4e8e\u751f\u6210\u4eba\u540d\u3002<\/p>\n\n\n\n<p>\u9879\u76ee\u53eb MicroGPT\uff0c\u5df2\u5f00\u6e90\u5230 Github\uff1a<a href=\"https:\/\/gist.github.com\/karpathy\/8627fe009c40f57531cb18360106ce95\">https:\/\/gist.github.com\/karpathy\/8627fe009c40f57531cb18360106ce95<\/a><\/p>\n\n\n\n<p>\u6700\u8fd1\u6709\u597d\u5fc3\u4eba\u7528\u66f4\u901a\u4fd7\u7684\u8bed\u8a00\u548c\u56fe\u8868\u5199\u4e86\u7bc7\u89e3\u8bfb\uff1a<a data-darkify_alpha_bg=\"rgba(0, 0, 0, 0)\" href=\"https:\/\/growingswe.com\/blog\/microgpt\" class=\"darkify_style_txt_border darkify_style_link darkify_processed\">https:\/\/growingswe.com\/blog\/microgpt<\/a> \uff0c\u53d1\u73b0\u8fd8\u662f\u6709\u70b9\u96be\u61c2\uff0c\u8ba9 AI \u91cd\u5199\u4e00\u904d\uff0c\u53bb\u516c\u5f0f\uff0c\u52a0\u540d\u8bcd\u89e3\u91ca\u3002\u73b0\u5728\u7ec8\u4e8e\u80fd\u770b\u61c2\u90e8\u5206\u4e86\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u8bad\u7ec3\u6570\u636e\uff1a32000 \u4e2a\u540d\u5b57<\/h2>\n\n\n\n<p>\u6a21\u578b\u8981\u5b66\u4ec0\u4e48\uff1f<\/p>\n\n\n\n<p>32000 \u4e2a\u4eba\u540d\uff0c\u4e00\u884c\u4e00\u4e2a\uff1aemma\u3001olivia\u3001ava\u3001isabella\u3001sophia&#8230;<\/p>\n\n\n\n<p>\u4efb\u52a1\u5f88\u7b80\u5355\uff1a\u5b66\u4f1a\u8fd9\u4e9b\u540d\u5b57\u7684\u7edf\u8ba1\u89c4\u5f8b\uff0c\u7136\u540e\u751f\u6210\u65b0\u7684\u3001\u542c\u8d77\u6765\u50cf\u771f\u540d\u5b57\u7684\u8bcd\u3002<\/p>\n\n\n\n<p>\u8bad\u7ec3\u5b8c\u4e4b\u540e\uff0c\u6a21\u578b\u80fd\u8f93\u51fa &#8220;kamon&#8221;\u3001&#8221;karai&#8221;\u3001&#8221;anna&#8221;\u3001&#8221;anton&#8221; \u8fd9\u6837\u7684\u540d\u5b57\u3002<\/p>\n\n\n\n<p>\u5b83\u5b66\u4f1a\u4e86\u54ea\u4e9b\u5b57\u6bcd\u5bb9\u6613\u8ddf\u5728\u54ea\u4e9b\u540e\u9762\uff0c\u54ea\u4e9b\u97f3\u8282\u5e38\u51fa\u73b0\u5728\u5f00\u5934\u6216\u7ed3\u5c3e\uff0c\u4e00\u4e2a\u540d\u5b57\u901a\u5e38\u591a\u957f\u3002<\/p>\n\n\n\n<p>\u4ece ChatGPT \u7684\u89c6\u89d2\u770b\uff0c\u4f60\u8ddf\u5b83\u7684\u5bf9\u8bdd\u5c31\u662f\u4e00\u4e2a\u6587\u6863\u3002<\/p>\n\n\n\n<p>\u4f60\u8f93\u5165\u63d0\u793a\u8bcd\uff0c\u5b83\u7684\u56de\u5e94\u5c31\u662f\u7edf\u8ba1\u610f\u4e49\u4e0a\u7684\u6587\u6863\u7eed\u5199\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u7b2c\u4e00\u6b65\uff1a\u628a\u6587\u5b57\u53d8\u6210\u6570\u5b57<\/h2>\n\n\n\n<p>\u795e\u7ecf\u7f51\u7edc\u53ea\u8ba4\u6570\u5b57\uff0c\u4e0d\u8ba4\u5b57\u6bcd\u3002<\/p>\n\n\n\n<p>\u6240\u4ee5\u5f97\u6709\u4e2a\u8f6c\u6362\u65b9\u5f0f\uff0c\u6700\u7b80\u5355\u7684\u505a\u6cd5\u662f\u8fd9\u6837\u7684\uff1a<\/p>\n\n\n\n<p>\u7ed9\u6bcf\u4e2a\u72ec\u7279\u5b57\u7b26\u5206\u914d\u4e00\u4e2a\u6574\u6570\u3002<\/p>\n\n\n\n<p>26 \u4e2a\u5c0f\u5199\u5b57\u6bcd\u5f97\u5230 0 \u5230 25 \u7684\u7f16\u53f7\uff0c\u518d\u52a0\u4e00\u4e2a\u7279\u6b8a\u6807\u8bb0 BOS\uff08Beginning of Sequence\uff0c\u5e8f\u5217\u5f00\u59cb\uff09\uff0c\u7f16\u53f7 26\uff0c\u7528\u6765\u6807\u8bb0\u540d\u5b57\u7684\u8d77\u6b62\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Tokenizer\uff08\u5206\u8bcd\u5668\uff09\uff1a\u628a\u6587\u672c\u8f6c\u6362\u6210\u6570\u5b57\u5e8f\u5217\u7684\u5de5\u5177\u3002\u5c31\u50cf\u7ed9\u6bcf\u4e2a\u5b57\u7b26\u53d1\u4e00\u5f20\u8eab\u4efd\u8bc1\uff0c\u4e0a\u9762\u5199\u7740\u4e13\u5c5e\u7f16\u53f7\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u6bd4\u5982 &#8220;emma&#8221; \u53d8\u6210\uff1a<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>BOS(26) \u2192 e(4) \u2192 m(12) \u2192 m(12) \u2192 a(0) \u2192 BOS(26)<\/p>\n<\/blockquote>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"2066\" height=\"828\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYk-BQaoAAWuS6.jpeg\" alt=\"\" class=\"wp-image-8008\"\/><\/figure>\n\n\n\n<p>\u8fd9\u4e9b\u6574\u6570\u672c\u8eab\u6ca1\u6709\u5927\u5c0f\u5173\u7cfb\u3002<\/p>\n\n\n\n<p>4 \u4e0d\u6bd4 2 &#8220;\u66f4\u5927&#8221;\uff0c\u5b83\u4eec\u53ea\u662f\u4e0d\u540c\u7684\u7b26\u53f7\uff0c\u5c31\u50cf\u7ed9\u6bcf\u4e2a\u5b57\u6bcd\u6d82\u4e0a\u4e0d\u540c\u989c\u8272\u3002<\/p>\n\n\n\n<p>\u751f\u4ea7\u73af\u5883\u7684 Tokenizer\uff08\u6bd4\u5982 GPT-4 \u7528\u7684 tiktoken\uff09\u4f1a\u6309\u5b57\u7b26\u5757\u5207\u5206\uff0c\u8bcd\u6c47\u8868\u6709 10 \u4e07\u4e2a Token\uff0c\u4f46\u539f\u7406\u4e00\u6837\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u9884\u6d4b\u6e38\u620f\uff1a\u731c\u4e0b\u4e00\u4e2a\u5b57<\/h2>\n\n\n\n<p>\u6838\u5fc3\u4efb\u52a1\u6765\u4e86\uff1a\u7ed9\u5b9a\u76ee\u524d\u770b\u5230\u7684 Token\uff0c\u9884\u6d4b\u4e0b\u4e00\u4e2a\u662f\u4ec0\u4e48\u3002<\/p>\n\n\n\n<p>\u6211\u4eec\u4e00\u4e2a\u4f4d\u7f6e\u4e00\u4e2a\u4f4d\u7f6e\u5f80\u524d\u6ed1\u3002<\/p>\n\n\n\n<p>\u5728\u4f4d\u7f6e 0\uff0c\u6a21\u578b\u53ea\u770b\u5230 BOS\uff0c\u8981\u9884\u6d4b\u7b2c\u4e00\u4e2a\u5b57\u6bcd\u3002<\/p>\n\n\n\n<p>\u5728\u4f4d\u7f6e 1\uff0c\u5b83\u770b\u5230 BOS \u548c\u7b2c\u4e00\u4e2a\u5b57\u6bcd\uff0c\u8981\u9884\u6d4b\u7b2c\u4e8c\u4e2a\u5b57\u6bcd\u3002\u4ee5\u6b64\u7c7b\u63a8\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1642\" height=\"572\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlCKCbgAAG41r.jpeg\" alt=\"\" class=\"wp-image-8009\"\/><\/figure>\n\n\n\n<p>\u5bf9\u4e8e &#8220;emma&#8221; \u8fd9\u4e2a\u540d\u5b57\uff0c\u4f1a\u4ea7\u751f 5 \u4e2a\u8bad\u7ec3\u6837\u672c\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u770b\u5230 [BOS] \u2192 \u9884\u6d4b&#8221;e&#8221;<\/li>\n\n\n\n<li>\u770b\u5230 [BOS, e] \u2192 \u9884\u6d4b&#8221;m&#8221;<\/li>\n\n\n\n<li>\u770b\u5230 [BOS, e, m] \u2192 \u9884\u6d4b&#8221;m&#8221;<\/li>\n\n\n\n<li>\u770b\u5230 [BOS, e, m, m] \u2192 \u9884\u6d4b&#8221;a&#8221;<\/li>\n\n\n\n<li>\u770b\u5230 [BOS, e, m, m, a] \u2192 \u9884\u6d4bBOS\uff08\u7ed3\u675f\uff09<\/li>\n<\/ul>\n\n\n\n<p>\u8fd9\u4e2a\u6ed1\u52a8\u7a97\u53e3\uff0c\u5c31\u662f\u6240\u6709\u8bed\u8a00\u6a21\u578b\u7684\u8bad\u7ec3\u65b9\u5f0f\uff0cChatGPT \u4e5f\u4e00\u6837\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u4ece\u5206\u6570\u5230\u6982\u7387\uff1a\u8ba9\u6a21\u578b\u7ed9\u51fa\u786e\u5b9a\u7b54\u6848<\/h2>\n\n\n\n<p>\u6bcf\u4e2a\u4f4d\u7f6e\uff0c\u6a21\u578b\u8f93\u51fa 27 \u4e2a\u539f\u59cb\u5206\u6570\uff0c\u5bf9\u5e94 27 \u4e2a\u53ef\u80fd\u7684\u4e0b\u4e00\u4e2a Token\u3002<\/p>\n\n\n\n<p>\u8fd9\u4e9b\u5206\u6570\u53ef\u4ee5\u662f\u4efb\u4f55\u503c\uff1a\u6b63\u7684\u3001\u8d1f\u7684\u3001\u5927\u7684\u3001\u5c0f\u7684\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Logits\uff08\u539f\u59cb\u5206\u6570\uff09\uff1a\u6a21\u578b\u6700\u521d\u8f93\u51fa\u7684\u672a\u7ecf\u5904\u7406\u7684\u6570\u5b57\uff0c\u5c31\u50cf\u8003\u8bd5\u7684\u539f\u59cb\u5377\u9762\u5206\uff0c\u8fd8\u6ca1\u6709\u8f6c\u6362\u6210\u7b49\u7ea7\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u6211\u4eec\u9700\u8981\u628a\u5b83\u4eec\u8f6c\u6210\u6982\u7387\uff1a\u6b63\u6570\uff0c\u52a0\u8d77\u6765\u7b49\u4e8e 1\uff0cSoftmax \u5c31\u5e72\u8fd9\u4e2a\u4e8b\u3002<\/p>\n\n\n\n<p>\u4f60\u53ef\u4ee5\u8fd9\u6837\u7406\u89e3\uff1a\u5047\u8bbe\u6a21\u578b\u7ed9 5 \u4e2a\u5019\u9009\u5b57\u6bcd\u6253\u5206\u662f [1.2, 2.8, 0.5, 1.8, -0.3]\u3002<\/p>\n\n\n\n<p>Softmax \u505a\u4e24\u4ef6\u4e8b\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u628a\u6bcf\u4e2a\u5206\u6570\u53d8\u6210\u6b63\u6570\uff08\u901a\u8fc7\u6307\u6570\u8fd0\u7b97\uff09<\/li>\n\n\n\n<li>\u8ba9\u5b83\u4eec\u52a0\u8d77\u6765\u7b49\u4e8e 100%<\/li>\n<\/ol>\n\n\n\n<p>\u7ed3\u679c\u5c31\u662f\u4e00\u4e2a\u6982\u7387\u5206\u5e03\uff0c\u6bd4\u5982 &#8220;e&#8221; \u6709 60% \u7684\u53ef\u80fd\uff0c&#8221;a&#8221; \u6709 22% \u7684\u53ef\u80fd\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1199\" height=\"597\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlGCzaUAAP_qW.jpeg\" alt=\"\" class=\"wp-image-8010\"\/><\/figure>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Softmax\uff1a\u628a\u4efb\u610f\u6570\u5b57\u8f6c\u6362\u6210\u6982\u7387\u5206\u5e03\u7684\u51fd\u6570\u3002\u5c31\u50cf\u628a\u8003\u8bd5\u5206\u6570\u8f6c\u6362\u6210\u6392\u540d\u767e\u5206\u6bd4\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u6ce8\u610f\u4e00\u4e2a\u5927\u7684\u5206\u6570\u4f1a\u4e3b\u5bfc\u6574\u4e2a\u5206\u5e03\uff0c\u56e0\u4e3a\u6307\u6570\u8fd0\u7b97\u4f1a\u653e\u5927\u5dee\u5f02\u3002<\/p>\n\n\n\n<p>\u8fd9\u4e5f\u662f\u4e3a\u4ec0\u4e48\u6a21\u578b\u80fd&#8221;\u786e\u4fe1&#8221;\u67d0\u4e2a\u7b54\u6848\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1772\" height=\"702\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlImnaYAAOL_Y.jpeg\" alt=\"\" class=\"wp-image-8011\"\/><\/figure>\n\n\n\n<p>\u4ee3\u7801\u91cc\u6709\u4e2a\u5c0f\u6280\u5de7\uff1a\u5148\u51cf\u53bb\u6700\u5927\u503c\u518d\u505a\u6307\u6570\u3002<\/p>\n\n\n\n<p>\u6570\u5b66\u4e0a\u7ed3\u679c\u4e00\u6837\uff0c\u4f46\u80fd\u9632\u6b62\u8ba1\u7b97\u6ea2\u51fa\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u8861\u91cf\u9519\u8bef\uff1a\u6a21\u578b\u731c\u5f97\u6709\u591a\u79bb\u8c31<\/h2>\n\n\n\n<p>\u9884\u6d4b\u6709\u591a\u79bb\u8c31\uff1f<\/p>\n\n\n\n<p>\u6211\u4eec\u9700\u8981\u4e00\u4e2a\u6570\u5b57\u6765\u8868\u793a&#8221;\u6a21\u578b\u89c9\u5f97\u6b63\u786e\u7b54\u6848\u6709\u591a\u4e0d\u53ef\u80fd&#8221;\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u4ea4\u53c9\u71b5\u635f\u5931\uff08Cross-Entropy Loss\uff09\uff1a\u8861\u91cf\u9884\u6d4b\u548c\u771f\u5b9e\u7b54\u6848\u4e4b\u95f4\u5dee\u8ddd\u7684\u6307\u6807\u3002\u6a21\u578b\u8d8a\u786e\u4fe1\u9519\u8bef\u7b54\u6848\uff0c\u60e9\u7f5a\u8d8a\u91cd\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u4e3e\u4e2a\u4f8b\u5b50\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5982\u679c\u6a21\u578b\u7ed9\u6b63\u786e\u7b54\u6848 90% \u7684\u6982\u7387\uff0c\u635f\u5931\u5f88\u5c0f\uff080.1\uff09<\/li>\n\n\n\n<li>\u5982\u679c\u53ea\u7ed9 1% \u7684\u6982\u7387\uff0c\u635f\u5931\u5f88\u5927\uff084.6\uff09<\/li>\n\n\n\n<li>\u5982\u679c\u5b8c\u5168\u786e\u4fe1\u6b63\u786e\u7b54\u6848\uff08100%\uff09\uff0c\u635f\u5931\u662f 0<\/li>\n<\/ul>\n\n\n\n<p>\u6a21\u578b\u4e0d\u4ec5\u8981\u7b54\u5bf9\uff0c\u8fd8\u8981&#8221;\u6709\u4fe1\u5fc3\u5730\u7b54\u5bf9&#8221;\u3002<\/p>\n\n\n\n<p>\u5982\u679c\u5b83\u7ed9\u6b63\u786e\u7b54\u6848\u53ea\u5206\u914d\u4e86\u5f88\u5c0f\u7684\u6982\u7387\uff0c\u5373\u4f7f\u8499\u5bf9\u4e86\uff0c\u635f\u5931\u4e5f\u4f1a\u5f88\u5927\u3002<\/p>\n\n\n\n<p>\u8bad\u7ec3\u5c31\u662f\u8ba9\u8fd9\u4e2a\u6570\u5b57\u4e0d\u65ad\u53d8\u5c0f\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u53cd\u5411\u4f20\u64ad\uff1a\u627e\u5230\u95ee\u9898\u51fa\u5728\u54ea<\/h2>\n\n\n\n<p>\u6a21\u578b\u6709 4192 \u4e2a\u53c2\u6570\uff08\u53ef\u4ee5\u7406\u89e3\u4e3a 4192 \u4e2a\u65cb\u94ae\uff09\u3002<\/p>\n\n\n\n<p>\u8981\u6539\u8fdb\uff0c\u9700\u8981\u77e5\u9053\uff1a\u6bcf\u4e2a\u65cb\u94ae\u5f80\u4e0a\u8c03\u4e00\u70b9\u70b9\uff0c\u635f\u5931\u4f1a\u4e0a\u5347\u8fd8\u662f\u4e0b\u964d\uff1f<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u53cd\u5411\u4f20\u64ad\uff08Backpropagation\uff09\uff1a\u5012\u7740\u8ffd\u8e2a\u8ba1\u7b97\u8fc7\u7a0b\uff0c\u627e\u51fa\u6bcf\u4e2a\u53c2\u6570\u5bf9\u6700\u7ec8\u9519\u8bef\u7684\u8d21\u732e\u3002\u5c31\u50cf\u51fa\u4e86\u4e8b\u6545\uff0c\u5012\u63a8\u6bcf\u4e2a\u73af\u8282\u7684\u8d23\u4efb\u3002<\/p>\n<\/blockquote>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1550\" height=\"820\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlPWKbMAArpyy.jpeg\" alt=\"\" class=\"wp-image-8012\"\/><\/figure>\n\n\n\n<p>\u60f3\u8c61\u4e00\u4e0b\u591a\u7c73\u8bfa\u9aa8\u724c\u3002<\/p>\n\n\n\n<p>\u524d\u5411\u8ba1\u7b97\u662f\u63a8\u5012\u7b2c\u4e00\u5f20\u724c\uff0c\u4e00\u8def\u4f20\u5bfc\u5230\u6700\u540e\u3002<\/p>\n\n\n\n<p>\u53cd\u5411\u4f20\u64ad\u662f\u4ece\u6700\u540e\u4e00\u5f20\u5012\u63a8\u56de\u53bb\uff0c\u770b\u6bcf\u5f20\u724c\u5bf9\u6700\u7ec8\u7ed3\u679c\u7684\u5f71\u54cd\u6709\u591a\u5927\u3002<\/p>\n\n\n\n<p>\u6bcf\u4e2a\u6570\u5b66\u64cd\u4f5c\uff08\u52a0\u3001\u4e58\u3001\u6307\u6570\u3001\u5bf9\u6570\uff09\u8bb0\u4f4f\u81ea\u5df1\u7684\u8f93\u5165\u3002<\/p>\n\n\n\n<p>\u53cd\u5411\u4f20\u64ad\u4ece\u635f\u5931\u5f00\u59cb\uff0c\u6cbf\u7740\u8ba1\u7b97\u8def\u5f84\u5f80\u56de\u8d70\uff0c\u7ed9\u6bcf\u4e2a\u53c2\u6570\u7b97\u51fa\u4e00\u4e2a&#8221;\u8d23\u4efb\u5206\u6570&#8221;\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u68af\u5ea6\uff08Gradient\uff09\uff1a\u544a\u8bc9\u4f60\u53c2\u6570\u8be5\u5f80\u54ea\u4e2a\u65b9\u5411\u8c03\u6574\uff0c\u4ee5\u53ca\u8c03\u6574\u7684\u529b\u5ea6\u3002\u5c31\u50cf\u6307\u5357\u9488\uff0c\u6307\u5411&#8221;\u8ba9\u9519\u8bef\u53d8\u5c0f&#8221;\u7684\u65b9\u5411\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u4e3e\u4e2a\u4f8b\u5b50\uff0c\u5047\u8bbe\u53c2\u6570 a \u5728\u4e24\u4e2a\u5730\u65b9\u88ab\u7528\u5230\u3002<\/p>\n\n\n\n<p>\u90a3\u5b83\u7684\u603b\u8d23\u4efb\u662f\u4e24\u6761\u8def\u5f84\u7684\u8d23\u4efb\u4e4b\u548c\u3002<\/p>\n\n\n\n<p>\u8fd9\u5c31\u662f\u4e3a\u4ec0\u4e48\u6709\u4e9b\u53c2\u6570\u7684\u68af\u5ea6\u7279\u522b\u5927\uff0c\u5b83\u4eec\u5728\u591a\u4e2a\u5730\u65b9\u5f71\u54cd\u4e86\u6700\u7ec8\u7ed3\u679c\u3002<\/p>\n\n\n\n<p>PyTorch \u7684 loss.backward() \u8dd1\u7684\u5c31\u662f\u8fd9\u4e2a\u7b97\u6cd5\uff0c\u53ea\u4e0d\u8fc7\u64cd\u4f5c\u7684\u662f\u5927\u6279\u91cf\u6570\u636e\u800c\u4e0d\u662f\u5355\u4e2a\u6570\u5b57\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u4ece ID \u5230\u610f\u4e49\uff1a\u7ed9\u6bcf\u4e2a\u5b57\u7b26\u4e00\u4e2a\u6027\u683c<\/h2>\n\n\n\n<p>\u539f\u59cb Token ID\uff08\u6bd4\u5982 4\uff09\u53ea\u662f\u4e2a\u7d22\u5f15\uff0c\u6a21\u578b\u6ca1\u6cd5\u76f4\u63a5\u7528\u6574\u6570\u505a\u6570\u5b66\u8fd0\u7b97\u3002<\/p>\n\n\n\n<p>\u6240\u4ee5\u6bcf\u4e2a Token \u4f1a\u5bf9\u5e94\u4e00\u4e2a 16 \u4e2a\u6570\u5b57\u7684\u5217\u8868\u3002<\/p>\n\n\n\n<p>\u53ef\u4ee5\u7406\u89e3\u4e3a\u6bcf\u4e2a Token \u6709\u4e2a 16 \u7ef4\u7684&#8221;\u6027\u683c\u6863\u6848&#8221;\uff0c\u8bad\u7ec3\u65f6\u53ef\u4ee5\u8c03\u6574\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u5d4c\u5165\uff08Embedding\uff09\uff1a\u628a\u79bb\u6563\u7684\u7b26\u53f7\uff08\u6bd4\u5982\u5b57\u7b26\uff09\u8f6c\u6362\u6210\u8fde\u7eed\u7684\u6570\u5b57\u5411\u91cf\u3002\u5c31\u50cf\u7ed9\u6bcf\u4e2a\u4eba\u5efa\u7acb\u4e00\u4efd\u591a\u7ef4\u5ea6\u7684\u6027\u683c\u6863\u6848\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u4f4d\u7f6e\u4e5f\u91cd\u8981\uff0c\u4f4d\u7f6e 0 \u7684 &#8220;a&#8221; \u548c\u4f4d\u7f6e 4 \u7684 &#8220;a&#8221; \u4f5c\u7528\u4e0d\u540c\uff0c\u6240\u4ee5\u6709\u7b2c\u4e8c\u4efd\u6863\u6848\uff0c\u6309\u4f4d\u7f6e\u7d22\u5f15\u3002<\/p>\n\n\n\n<p>\u4e24\u4efd\u6863\u6848\u76f8\u52a0\uff0c\u5f62\u6210\u8f93\u5165\u5230\u7f51\u7edc\u7684\u5411\u91cf\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1046\" height=\"648\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlTfBaEAEWqh7.jpeg\" alt=\"\" class=\"wp-image-8013\"\/><\/figure>\n\n\n\n<p>\u8fd9\u4e9b\u6570\u5b57\u4e00\u5f00\u59cb\u662f\u968f\u673a\u7684\u5c0f\u6570\uff0c\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\uff0c\u6a21\u578b\u4f1a\u81ea\u5df1\u8c03\u6574\u3002<\/p>\n\n\n\n<p>\u8bad\u7ec3\u540e\uff0c\u884c\u4e3a\u76f8\u4f3c\u7684 Token\uff08\u6bd4\u5982\u5143\u97f3\u5b57\u6bcd\uff09\u5f80\u5f80\u6709\u76f8\u4f3c\u7684\u5411\u91cf\u3002<\/p>\n\n\n\n<p>\u6a21\u578b\u4ece\u96f6\u5b66\u4e60\u8fd9\u4e9b\u8868\u793a\uff0c\u4e0d\u9700\u8981\u9884\u5148\u77e5\u9053\u4ec0\u4e48\u662f\u5143\u97f3\u3002<\/p>\n\n\n\n<p>\u5c31\u50cf\u5c0f\u5b69\u5b66\u8bf4\u8bdd\uff0c\u6ca1\u4eba\u544a\u8bc9\u4ed6\u5143\u97f3\u548c\u8f85\u97f3\u7684\u533a\u522b\uff0c\u4f46\u4ed6\u81ea\u5df1\u80fd\u603b\u7ed3\u51fa\u89c4\u5f8b\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Token \u4e4b\u95f4\u600e\u4e48\u4ea4\u6d41\uff1a\u6ce8\u610f\u529b\u673a\u5236<\/h2>\n\n\n\n<p>\u8fd9\u662f Transformer \u7684\u6838\u5fc3\u3002<\/p>\n\n\n\n<p>\u6bcf\u4e2a\u4f4d\u7f6e\u9700\u8981\u4ece\u4e4b\u524d\u7684\u4f4d\u7f6e\u6536\u96c6\u4fe1\u606f\u3002<\/p>\n\n\n\n<p>\u60f3\u8c61\u4f60\u5728\u8bfb\u4e00\u4e2a\u53e5\u5b50\uff0c\u8bfb\u5230&#8221;\u5979&#8221;\u7684\u65f6\u5019\uff0c\u4f60\u4f1a\u56de\u5934\u770b\u770b\u524d\u9762\u63d0\u5230\u7684\u5973\u6027\u89d2\u8272\u662f\u8c01\uff0c\u8fd9\u5c31\u662f\u6ce8\u610f\u529b\u5728\u505a\u7684\u4e8b\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u6ce8\u610f\u529b\u673a\u5236\uff08Attention\uff09\uff1a\u8ba9\u6a21\u578b\u51b3\u5b9a\u5728\u5904\u7406\u5f53\u524d\u4f4d\u7f6e\u65f6\uff0c\u5e94\u8be5\u91cd\u70b9\u5173\u6ce8\u4e4b\u524d\u54ea\u4e9b\u4f4d\u7f6e\u7684\u4fe1\u606f\u3002\u5c31\u50cf\u4f60\u8bfb\u4e66\u65f6\uff0c\u89c6\u7ebf\u4f1a\u5728\u4e0d\u540c\u8bcd\u8bed\u4e4b\u95f4\u8df3\u8f6c\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u6bcf\u4e2atoken\u4ea7\u751f\u4e09\u4e2a\u4e1c\u897f\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Query\uff08&#8221;\u6211\u5728\u627e\u4ec0\u4e48\uff1f&#8221;\uff09<\/li>\n\n\n\n<li>Key\uff08&#8221;\u6211\u5305\u542b\u4ec0\u4e48\uff1f&#8221;\uff09<\/li>\n\n\n\n<li>Value\uff08&#8221;\u5982\u679c\u88ab\u9009\u4e2d\uff0c\u6211\u63d0\u4f9b\u4ec0\u4e48\u4fe1\u606f\uff1f&#8221;\uff09<\/li>\n<\/ul>\n\n\n\n<p>\u4f60\u53ef\u4ee5\u8fd9\u6837\u7406\u89e3\uff1aQuery \u662f\u641c\u7d22\u5173\u952e\u8bcd\uff0cKey \u662f\u6bcf\u4e2a\u4f4d\u7f6e\u7684\u6807\u7b7e\uff0cValue \u662f\u5b9e\u9645\u5185\u5bb9\u3002<\/p>\n\n\n\n<p>\u6a21\u578b\u7528 Query \u53bb\u5339\u914d\u6240\u6709\u4e4b\u524d\u4f4d\u7f6e\u7684 Key\uff0c\u627e\u5230\u6700\u76f8\u5173\u7684\uff0c\u7136\u540e\u628a\u5bf9\u5e94\u7684 Value \u62ff\u8fc7\u6765\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"816\" height=\"820\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlY30aoAAh1Bk.png\" alt=\"\" class=\"wp-image-8014\"\/><\/figure>\n\n\n\n<p>\u6709\u4e2a\u91cd\u8981\u9650\u5236\uff1a\u6bcf\u4e2a\u4f4d\u7f6e\u53ea\u80fd\u770b\u8fc7\u53bb\uff0c\u4e0d\u80fd\u770b\u672a\u6765\u3002<\/p>\n\n\n\n<p>\u4f4d\u7f6e 2 \u4e0d\u80fd\u5173\u6ce8\u4f4d\u7f6e 4\uff0c\u56e0\u4e3a\u4f4d\u7f6e 4 \u8fd8\u6ca1\u53d1\u751f\u3002<\/p>\n\n\n\n<p>\u8fd9\u8ba9\u6a21\u578b\u6210\u4e3a\u81ea\u56de\u5f52\u7684\uff0c\u7b26\u5408\u6211\u4eec\u751f\u6210\u6587\u672c\u7684\u65b9\u5f0f\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u591a\u5934\u6ce8\u610f\u529b\uff08Multi-Head Attention\uff09\uff1a\u540c\u65f6\u8fd0\u884c\u591a\u4e2a\u6ce8\u610f\u529b\u673a\u5236\uff0c\u6bcf\u4e2a\u5173\u6ce8\u4e0d\u540c\u7684\u6a21\u5f0f\u3002\u5c31\u50cf\u540c\u65f6\u7528\u591a\u4e2a\u89c6\u89d2\u770b\u540c\u4e00\u4e2a\u95ee\u9898\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u4e0d\u540c\u7684\u6ce8\u610f\u529b\u5934\u5b66\u4e60\u4e0d\u540c\u6a21\u5f0f\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u4e00\u4e2a\u5934\u53ef\u80fd\u5f3a\u70c8\u5173\u6ce8\u6700\u8fd1\u7684 Token<\/li>\n\n\n\n<li>\u53e6\u4e00\u4e2a\u53ef\u80fd\u805a\u7126\u5f00\u5934\u7684 BOS Token\uff08\u8bb0\u4f4f&#8221;\u6211\u4eec\u5728\u751f\u6210\u540d\u5b57&#8221;\uff09<\/li>\n\n\n\n<li>\u7b2c\u4e09\u4e2a\u53ef\u80fd\u5bfb\u627e\u5143\u97f3<\/li>\n<\/ul>\n\n\n\n<p>\u56db\u4e2a\u5934\u5e76\u884c\u5de5\u4f5c\uff0c\u6700\u540e\u628a\u7ed3\u679c\u62fc\u8d77\u6765\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u5b8c\u6574\u6d41\u7a0b\uff1a\u4fe1\u606f\u600e\u4e48\u5728\u6a21\u578b\u91cc\u6d41\u52a8<\/h2>\n\n\n\n<p>\u628a\u6bcf\u4e2a Token \u60f3\u8c61\u6210\u4e00\u4e2a\u5305\u88f9\uff0c\u5728\u6d41\u6c34\u7ebf\u4e0a\u7ecf\u8fc7\u591a\u4e2a\u5de5\u5e8f\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u5d4c\u5165\uff1a\u7ed9\u5305\u88f9\u8d34\u4e0a\u6807\u7b7e\uff08Token \u6027\u683c + \u4f4d\u7f6e\u4fe1\u606f\uff09<\/li>\n\n\n\n<li>\u5f52\u4e00\u5316\uff1a\u628a\u5305\u88f9\u6574\u7406\u6210\u6807\u51c6\u5927\u5c0f<\/li>\n\n\n\n<li>\u6ce8\u610f\u529b\uff1a\u5305\u88f9\u4e4b\u95f4\u4e92\u76f8\u4ea4\u6362\u4fe1\u606f<\/li>\n\n\n\n<li>\u52a0\u6b8b\u5dee\uff1a\u4fdd\u7559\u539f\u59cb\u5305\u88f9\u7684\u4e00\u90e8\u5206<\/li>\n\n\n\n<li>\u5f52\u4e00\u5316\uff1a\u518d\u6b21\u6574\u7406<\/li>\n\n\n\n<li>MLP\uff1a\u6bcf\u4e2a\u5305\u88f9\u72ec\u7acb\u52a0\u5de5<\/li>\n\n\n\n<li>\u52a0\u6b8b\u5dee\uff1a\u518d\u6b21\u4fdd\u7559\u539f\u59cb\u4fe1\u606f<\/li>\n\n\n\n<li>\u8f93\u51fa\uff1a\u5f97\u5230 27 \u4e2a\u5206\u6570\uff0c\u5bf9\u5e94\u4e0b\u4e00\u4e2a\u53ef\u80fd\u7684\u5b57\u7b26<\/li>\n<\/ol>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1020\" height=\"328\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlnn8acAAu-K6.jpeg\" alt=\"\" class=\"wp-image-8015\"\/><\/figure>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>MLP\uff08\u591a\u5c42\u611f\u77e5\u673a\uff09\uff1a\u4e00\u4e2a\u7b80\u5355\u7684\u795e\u7ecf\u7f51\u7edc\uff0c\u8d1f\u8d23\u5bf9\u6bcf\u4e2a\u4f4d\u7f6e\u7684\u4fe1\u606f\u505a\u72ec\u7acb\u5904\u7406\u3002\u5982\u679c\u8bf4\u6ce8\u610f\u529b\u662f&#8221;\u4ea4\u6d41&#8221;\uff0cMLP\u5c31\u662f&#8221;\u72ec\u7acb\u601d\u8003&#8221;\u3002<\/p>\n<\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u6b8b\u5dee\u8fde\u63a5\uff08Residual Connection\uff09\uff1a\u5728\u5904\u7406\u4fe1\u606f\u65f6\u4fdd\u7559\u539f\u59cb\u8f93\u5165\u7684\u4e00\u90e8\u5206\u3002\u5c31\u50cf\u505a\u83dc\u65f6\u4fdd\u7559\u4e00\u4e9b\u539f\u6750\u6599\u7684\u539f\u5473\uff0c\u4e0d\u8981\u5168\u90e8\u52a0\u5de5\u6389\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u6b8b\u5dee\u8fde\u63a5\u7279\u522b\u5173\u952e\uff0c\u60f3\u8c61\u4fe1\u606f\u5728\u7f51\u7edc\u91cc\u4f20\u9012\u5c31\u50cf\u6253\u7535\u8bdd\uff0c\u6bcf\u4f20\u4e00\u5c42\u5c31\u4f1a\u635f\u5931\u4e00\u4e9b\u3002<\/p>\n\n\n\n<p>\u6b8b\u5dee\u8fde\u63a5\u76f8\u5f53\u4e8e\u7ed9\u4fe1\u606f\u5f00\u4e86\u4e00\u6761\u9ad8\u901f\u901a\u9053\uff0c\u76f4\u63a5\u8df3\u8fc7\u67d0\u4e9b\u5c42\uff0c\u4fdd\u8bc1\u91cd\u8981\u4fe1\u606f\u4e0d\u4f1a\u4e22\u5931\u3002<\/p>\n\n\n\n<p>\u6ca1\u6709\u5b83\uff0c\u6df1\u5ea6\u7f51\u7edc\u6839\u672c\u8bad\u7ec3\u4e0d\u8d77\u6765\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>RMSNorm\uff08\u5747\u65b9\u6839\u5f52\u4e00\u5316\uff09\uff1a\u628a\u6bcf\u4e2a\u5411\u91cf\u7f29\u653e\u5230\u6807\u51c6\u5927\u5c0f\u3002\u5c31\u50cf\u628a\u4e0d\u540c\u5927\u5c0f\u7684\u7167\u7247\u7edf\u4e00\u8c03\u6574\u6210\u540c\u6837\u5c3a\u5bf8\uff0c\u65b9\u4fbf\u5904\u7406\u3002<\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\">\u5b66\u4e60\u8fc7\u7a0b\uff1a\u6a21\u578b\u600e\u4e48\u53d8\u806a\u660e<\/h2>\n\n\n\n<p>\u8bad\u7ec3\u5faa\u73af\u91cd\u590d 1000 \u6b21\uff0c\u6bcf\u6b21\u505a\u8fd9\u4e9b\u4e8b\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u968f\u673a\u6311\u4e00\u4e2a\u540d\u5b57<\/li>\n\n\n\n<li>\u628a\u5b83\u53d8\u6210\u6570\u5b57\u5e8f\u5217<\/li>\n\n\n\n<li>\u5728\u6bcf\u4e2a\u4f4d\u7f6e\u8fd0\u884c\u6a21\u578b\uff0c\u9884\u6d4b\u4e0b\u4e00\u4e2a\u5b57\u7b26<\/li>\n\n\n\n<li>\u8ba1\u7b97\u9884\u6d4b\u6709\u591a\u79bb\u8c31\uff08\u635f\u5931\uff09<\/li>\n\n\n\n<li>\u53cd\u5411\u4f20\u64ad\uff0c\u627e\u51fa\u6bcf\u4e2a\u53c2\u6570\u7684\u8d23\u4efb<\/li>\n\n\n\n<li>\u8c03\u6574\u53c2\u6570\uff0c\u8ba9\u4e0b\u6b21\u9884\u6d4b\u66f4\u51c6<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Adam\u4f18\u5316\u5668\uff1a\u4e00\u4e2a\u806a\u660e\u7684\u53c2\u6570\u66f4\u65b0\u7b56\u7565\u3002\u5b83\u4f1a\u8bb0\u4f4f\u6bcf\u4e2a\u53c2\u6570\u6700\u8fd1\u7684\u8868\u73b0\uff0c\u5bf9\u7a33\u5b9a\u7684\u53c2\u6570\u8d70\u5927\u6b65\uff0c\u5bf9\u6447\u6446\u4e0d\u5b9a\u7684\u53c2\u6570\u8d70\u5c0f\u6b65\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u635f\u5931\u4ece\u7ea6 3.3 \u5f00\u59cb\uff08\u5b8c\u5168\u968f\u673a\u731c\uff0c27 \u4e2a\u9009\u9879\u4e2d\u778e\u8499\uff09\uff0c\u6162\u6162\u964d\u5230 2.37 \u5de6\u53f3\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1996\" height=\"706\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlqvGbEAAwrPm.jpeg\" alt=\"\" class=\"wp-image-8016\"\/><\/figure>\n\n\n\n<p>\u751f\u6210\u7684\u540d\u5b57\u4e5f\u4ece\u4e71\u7801\u6f14\u53d8\u6210\u5408\u7406\u7684\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u521a\u5f00\u59cb\uff1axqbzjfmwplk\uff08\u5b8c\u5168\u4e71\u7801\uff09<\/li>\n\n\n\n<li>\u8bad\u7ec3\u4e2d\u671f\uff1akamon, karai\uff08\u6709\u70b9\u610f\u601d\u4e86\uff09<\/li>\n\n\n\n<li>\u8bad\u7ec3\u540e\u671f\uff1aanna, anton\uff08\u5b8c\u5168\u50cf\u771f\u540d\u5b57\uff09<\/li>\n<\/ul>\n\n\n\n<p>\u4f60\u80fd\u770b\u5230\u6a21\u578b\u5728&#8221;\u5f00\u7a8d&#8221;\u3002<\/p>\n\n\n\n<p>\u5b83\u9010\u6e10\u7406\u89e3\u4e86\u4ec0\u4e48\u662f\u5408\u7406\u7684\u5b57\u6bcd\u7ec4\u5408\uff0c\u4ec0\u4e48\u6837\u7684\u97f3\u8282\u542c\u8d77\u6765\u50cf\u540d\u5b57\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u751f\u6210\u65b0\u540d\u5b57\uff1a\u8ba9\u6a21\u578b\u81ea\u7531\u53d1\u6325<\/h2>\n\n\n\n<p>\u8bad\u7ec3\u5b8c\u6210\u540e\uff0c\u751f\u6210\u5f88\u76f4\u63a5\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u4ece BOS \u5f00\u59cb<\/li>\n\n\n\n<li>\u8ba9\u6a21\u578b\u9884\u6d4b\u4e0b\u4e00\u4e2a\u5b57\u7b26<\/li>\n\n\n\n<li>\u5f97\u5230 27 \u4e2a\u6982\u7387<\/li>\n\n\n\n<li>\u968f\u673a\u9009\u4e00\u4e2a\uff08\u6309\u6982\u7387\uff09<\/li>\n\n\n\n<li>\u628a\u9009\u4e2d\u7684\u5b57\u7b26\u5582\u56de\u53bb<\/li>\n\n\n\n<li>\u91cd\u590d\uff0c\u76f4\u5230\u6a21\u578b\u8bf4&#8221;\u6211\u5b8c\u6210\u4e86&#8221;\uff08\u8f93\u51fa BOS\uff09<\/li>\n<\/ol>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u6e29\u5ea6\uff08Temperature\uff09\uff1a\u63a7\u5236\u751f\u6210\u7684\u968f\u673a\u6027\u3002\u4f4e\u6e29\u5ea6\u8ba9\u6a21\u578b\u66f4\u4fdd\u5b88\uff08\u603b\u9009\u6700\u53ef\u80fd\u7684\uff09\uff0c\u9ad8\u6e29\u5ea6\u8ba9\u6a21\u578b\u66f4\u5927\u80c6\uff08\u66f4\u591a\u5c1d\u8bd5\u4e0d\u5e38\u89c1\u7684\u9009\u62e9\uff09\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u4f60\u53ef\u4ee5\u8fd9\u6837\u7406\u89e3\u6e29\u5ea6\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u6e29\u5ea6 0.5\uff1a\u6a21\u578b\u5f88\u8c28\u614e\uff0c\u503e\u5411\u4e8e\u751f\u6210\u5e38\u89c1\u7684\u3001\u5b89\u5168\u7684\u540d\u5b57<\/li>\n\n\n\n<li>\u6e29\u5ea6 1.0\uff1a\u6a21\u578b\u6309\u5b83\u5b66\u5230\u7684\u771f\u5b9e\u6982\u7387\u751f\u6210<\/li>\n\n\n\n<li>\u6e29\u5ea6 2.0\uff1a\u6a21\u578b\u5f88\u5927\u80c6\uff0c\u4f1a\u5c1d\u8bd5\u4e0d\u5bfb\u5e38\u7684\u7ec4\u5408<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1806\" height=\"700\" src=\"https:\/\/kyle.ai\/blog\/wp-content\/uploads\/2026\/03\/HCYlutNbQAAwMSq.jpeg\" alt=\"\" class=\"wp-image-8017\"\/><\/figure>\n\n\n\n<p>\u6e29\u5ea6\u592a\u4f4e\uff0c\u751f\u6210\u7684\u4e1c\u897f\u5f88\u65e0\u804a\uff0c\u603b\u662f\u90a3\u51e0\u4e2a\u6700\u5e38\u89c1\u7684\u540d\u5b57\u3002<\/p>\n\n\n\n<p>\u6e29\u5ea6\u592a\u9ad8\uff0c\u751f\u6210\u7684\u4e1c\u897f\u53ef\u80fd\u662f\u80e1\u8bdd\u3002<\/p>\n\n\n\n<p>\u5bf9\u4e8e\u540d\u5b57\uff0c\u6700\u4f73\u70b9\u5728 0.5 \u5de6\u53f3\u3002<\/p>\n\n\n\n<p>\u65e2\u6709\u521b\u610f\uff0c\u53c8\u4e0d\u4f1a\u592a\u79bb\u8c31\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u5176\u4ed6\u90fd\u662f\u6548\u7387\u95ee\u9898<\/h2>\n\n\n\n<p>\u8fd9\u4e2a 200 \u884c\u811a\u672c\u5305\u542b\u5b8c\u6574\u7b97\u6cd5\u3002<\/p>\n\n\n\n<p>\u4ece\u8fd9\u4e2a\u7b97\u6cd5\u5230 ChatGPT \u7684\u5b8c\u6574\u5b9e\u73b0\uff0c\u6838\u5fc3\u601d\u60f3\u6ca1\u53d8\u3002<\/p>\n\n\n\n<p>\u53d8\u7684\u662f\u4ec0\u4e48\uff1f\u89c4\u6a21\u548c\u6548\u7387\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u8bad\u7ec3\u6570\u636e\uff1a\u4ece 32000 \u4e2a\u540d\u5b57\u5230\u6574\u4e2a\u4e92\u8054\u7f51\u7684\u6587\u672c<\/li>\n\n\n\n<li>\u8bcd\u6c47\u8868\uff1a\u4ece 27 \u4e2a\u5b57\u7b26\u5230 10 \u4e07\u4e2a\u5b50\u8bcd<\/li>\n\n\n\n<li>\u53c2\u6570\uff1a\u4ece 4192 \u4e2a\u5230\u6570\u5343\u4ebf\u4e2a<\/li>\n\n\n\n<li>\u5c42\u6570\uff1a\u4ece 1 \u5c42\u5230\u51e0\u767e\u5c42<\/li>\n\n\n\n<li>\u786c\u4ef6\uff1a\u4ece\u4f60\u7684\u7b14\u8bb0\u672c\u5230\u6570\u5343\u4e2a GPU \u96c6\u7fa4<\/li>\n\n\n\n<li>\u8bad\u7ec3\u65f6\u95f4\uff1a\u4ece\u51e0\u5206\u949f\u5230\u51e0\u4e2a\u6708<\/li>\n<\/ul>\n\n\n\n<p>\u4f46\u5faa\u73af\u662f\u4e00\u6837\u7684\uff1a<\/p>\n\n\n\n<p>\u5206\u8bcd \u2192 \u5d4c\u5165 \u2192 \u6ce8\u610f\u529b \u2192 \u8ba1\u7b97 \u2192 \u9884\u6d4b\u4e0b\u4e00\u4e2atoken \u2192 \u8861\u91cf\u9519\u8bef \u2192 \u53cd\u5411\u4f20\u64ad \u2192 \u8c03\u6574\u53c2\u6570 \u2192 \u91cd\u590d<\/p>\n\n\n\n<p>\u5c31\u8fd9\u4e48\u7b80\u5355\uff0c\u4e5f\u5c31\u8fd9\u4e48\u590d\u6742\u3002<\/p>\n\n\n\n<p>\u4f60\u73b0\u5728\u77e5\u9053\u4e86\uff0cChatGPT \u5e76\u4e0d\u795e\u79d8\u3002<\/p>\n\n\n\n<p>\u5b83\u5c31\u662f\u4e00\u4e2a\u8d85\u5927\u89c4\u6a21\u7684&#8221;\u731c\u4e0b\u4e00\u4e2a\u8bcd&#8221;\u6e38\u620f\uff0c\u73a9\u4e86\u65e0\u6570\u6b21\u4e4b\u540e\uff0c\u53d8\u5f97\u51fa\u5947\u5730\u806a\u660e\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p><em>\u539f\u6587\uff1a<a href=\"https:\/\/x.com\/vista8\/status\/2028351145715109958\">https:\/\/x.com\/vista8\/status\/2028351145715109958<\/a><\/em><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u4e4b\u524d\u770b\u8fc7\u4e00\u672c\u8bb2 GPT \u539f\u7406\u7684\u7535\u5b50\u4e66\u300aBuild a Large Language Model (From S [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-8007","post","type-post","status-publish","format-standard","hentry","category-skill"],"_links":{"self":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/8007","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/comments?post=8007"}],"version-history":[{"count":3,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/8007\/revisions"}],"predecessor-version":[{"id":8020,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/8007\/revisions\/8020"}],"wp:attachment":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/media?parent=8007"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/categories?post=8007"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/tags?post=8007"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}