Skip to content

Commit 25f5f39

Browse files
committed
docs(examples): 给这门设备语言一个真的编译器,并把插件系统的机制写进第 31 章
例 12 的 `.toy` 原来是「每行一个整数,入口返回它们的和」,而它的编译器是一段 shell。那不是一门语言,也不是任何生态作者会遇到的形状。 **语言。** `.toy` 现在有 `let`、赋值、`if`/`else`、`while`、kernel 之间的调用, 以及算术与比较运算符,文法写在 README 与 `compile.cppm` 的开头。示例源是 Euclid 算法加一个调用者。 **编译器。** `toyc/` 是一个普通的 mcpp 包:词法、递归下降语法分析、语义检查、 C++ 发射,三个模块加一个驱动。它由 `tools = ["toyc"]` + `reexport = true` 为 **构建机**构建,规则用 `mcpp::dep_bin` 取到它 —— 这是本仓库第一个「依赖产出 host 工具」的示例,而这条能力此前只有文字。 四项判据实测:`.toy` 编译并进链接(`answer() = 42`)、语言被执行而不是被模式匹配 (`gcd(1071, 462) = 21` 由发射出的循环算出)、改 `.toy` 到达产物(42 → 63)、 以及去掉 feature 后构建停在模块 import 且 tool store 里没有 `toyc`。 **并且量到了一条边界。** 改编译器的**源码**不到达产物:tool store 的键是包身份、 版本、host 三元组、编译器身份、profile、feature 与依赖闭包的版本,里面没有源码 内容。对来自索引的工具这是精确的(已发布的版本不可变),对正在被编辑的 `path` 依赖不是 —— 实测 `mcpp run` 报 `Finished dev in 0.00s` 并打印上一次的答案。 第 30 章补上这一条与两个出路。 **第 31 章新增「扩展模型」一节**,回答「mcpp 的插件系统是什么、已经用在哪」: 五个扩展点各自加了什么、声明在哪;生态用它们建出的七样东西;模型能表达的三种 形态(新语言 / 预处理与代码生成 / 一半 C++ 一半外来语言的文件);以及边界 —— 声明不能重新归类引擎已拥有的扩展名(实测:把 `.cpp` 加进 `device_extensions` 不被诊断也不生效),模块接口扩展名是工程的轴,两张表都不含的扩展名被点名拒绝 (诊断原文引用)。 build_examples.sh 把 `toyc` 列进 BUILD 并写明它与 app 是两个信号;12 of 12。
1 parent 7d0c7cb commit 25f5f39

20 files changed

Lines changed: 1131 additions & 109 deletions

.github/tools/build_examples.sh

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,14 @@ BUILD=(
4848
# name the optional package -- which a build alone cannot show.
4949
examples/11-features/counters
5050
examples/11-features/greeter
51-
# A device language the engine does not know. No payload: its compiler is a
52-
# shell script, because the subject is the graph rather than a vendor.
51+
# A device language the engine does not know, and the compiler for it.
52+
#
53+
# `toyc` is built here as an ordinary package as well as by the app as a
54+
# host tool, and the two are not the same signal: this one fails at the
55+
# compiler, the app's fails somewhere in `tools = [...]` / `reexport` /
56+
# `dep_bin`, and a single line telling them apart is worth one build of a
57+
# three-file package.
58+
examples/12-a-new-device-language/toyc
5359
examples/12-a-new-device-language/app
5460
)
5561

docs/03-examples.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ the map; the table below is what each sub-example adds.
8181
| example | first to introduce |
8282
|---|---|
8383
| [`08-build-rules`](../examples/08-build-rules/) | two rule packages and a project using both; `host-module = true`, `mcpp::action` with `role = "check"` |
84-
| [`12-a-new-device-language`](../examples/12-a-new-device-language/) | `device_extensions` and `rule_module`: a rule package teaching mcpp a language the engine has never heard of |
84+
| [`12-a-new-device-language`](../examples/12-a-new-device-language/) | `device_extensions` and `rule_module`: a rule package teaching mcpp a language the engine has never heard of, whose compiler is a package built through `tools = [...]` for the build machine |
8585

8686
[31 — Authoring a Rule Package](31-authoring-a-rule-package.md) is the reference
8787
these two illustrate.

docs/09-commands-by-scenario.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ them. Confusing them costs a full rebuild.
3131
| Store | Scope | Growth trigger | Emptied by |
3232
|---|---|---|---|
3333
| `target/<triple>/<fingerprint>/` | one project | a configuration fingerprint changes and opens a new directory | `mcpp clean`, `mcpp clean --stale` |
34-
| the build cache (`mcpp cache dir`) | the whole machine | any project compiles a dependency or a `std` module | `mcpp cache gc`, `mcpp cache prune`, `mcpp cache clean` |
34+
| the build cache (`mcpp cache dir`) | the whole machine | any project compiles a dependency or a `std` module, or builds a host tool | `mcpp cache gc`, `mcpp cache prune`, `mcpp cache clean` |
3535

3636
`mcpp clean` removes `target/` entirely, and the next build recompiles
3737
everything. `mcpp clean --stale` removes only the fingerprint directories that

docs/30-build-mcpp.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -931,6 +931,16 @@ Four properties worth knowing:
931931
- **Cached globally**, keyed on package version × host toolchain × features ×
932932
its own dependency closure — built once per machine, not once per project.
933933

934+
**The key holds no source content, and for a `path` dependency that is visible.**
935+
A published version is immutable, so for a tool that arrives from an index the
936+
key is exact. A tool being edited next door has the same version from one build
937+
to the next, and the cached binary stays: measured on
938+
[`examples/12-a-new-device-language`](../examples/12-a-new-device-language/),
939+
a change to the tool's emitter left `mcpp run` reporting `Finished dev in 0.00s`
940+
and printing the previous answer. Bump the tool package's version, or empty the
941+
build cache with `mcpp cache clean` — the tool store lives inside it, at
942+
`<mcpp cache dir>/tool/<index>/<name>@<version>/`.
943+
934944
### `[tools.overrides]` — use an existing binary
935945

936946
```toml

docs/31-authoring-a-rule-package.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,84 @@ which belong to `mcpp:plugins`. Examples:
1616
[`12-a-new-device-language`](../examples/12-a-new-device-language/) adds a
1717
language.
1818

19+
## The extension model
20+
21+
mcpp's build surface is extended from packages rather than from releases. Five
22+
points do the extending, and the engine holds no name that comes through any of
23+
them.
24+
25+
| extension point | effect | declared on |
26+
|---|---|---|
27+
| `mcpp::action` | one edge in the build graph: a command with declared inputs and outputs | a build program, or a rule module it imports |
28+
| `device_extensions` | an extension the engine classifies as a **device source** instead of refusing it | a feature of a package |
29+
| `rule_module` | the module a consumer's build program imports to reach the rule | the same feature |
30+
| `tools = [...]` | a generator or compiler **built from source for the build machine**, reached with `mcpp::dep_bin` | a dependency edge |
31+
| `[xlings]`, `[feature-xlings]` | a prebuilt tool the rule runs, installed on demand | the package, or one of its features |
32+
33+
What the ecosystem has built out of them:
34+
35+
| surface | package | points used |
36+
|---|---|---|
37+
| CUDA, HIP, SYCL, Ascend C | `mcpp:plugins`, one feature each ([42](42-heterogeneous-builds.md)) | actions driving a vendor compiler, plus payloads gated on the accelerator |
38+
| Slang | `mcpp:plugins`' `rules-slang` | `device_extensions = [".slang"]` — the first language mcpp supports without naming it in the engine |
39+
| GLSL and HLSL to SPIR-V, and the module over the result | `mcpp:plugins`' `rules-spirv` | one action per shader, plus a generated module |
40+
| the island boundary between a device and C++ | `mcpp.tools.island` | a generator, plus `mcpp::generated` |
41+
| an asset as a linkable object | [`08-build-rules`](../examples/08-build-rules/) | `role = "object"` |
42+
| a check that can fail the build | [`08-build-rules`](../examples/08-build-rules/) | `role = "check"` |
43+
| a language the engine has never heard of | [`12-a-new-device-language`](../examples/12-a-new-device-language/) | `device_extensions`, plus a compiler built through `tools = [...]` |
44+
45+
### The shapes the model expresses
46+
47+
**A new language, whatever compiles it.** A rule claims the extension, submits
48+
one action per source, and declares the compiler among that action's inputs.
49+
The compiler may be a vendor toolkit, an LLVM front end, an interpreter that
50+
emits a device binary, or a program the rule package builds from source. Whether
51+
it produces a device binary, an object or C++ is the action's `role` and nothing
52+
else. The engine never learns the language: it learns that an extension is a
53+
device source and that an action claims it.
54+
55+
**Preprocessing and code generation.** An action with `role = "source"` produces
56+
C++ that the declaring package then compiles, and every compile edge of that
57+
package waits for it. The input can be a template, an interface definition, a
58+
table, or another action's output — chaining is ordinary, because actions are
59+
ordered and fingerprinted by their files.
60+
61+
**A file that is partly C++ and partly another language.** Classification
62+
happens before any rule runs, so a file with an extension the engine owns is
63+
compiled as C++ and never reaches a rule. A source carrying a foreign block
64+
therefore uses an extension the rule claims, and the rule splits it: the C++ it
65+
extracts goes through `role = "source"`, the foreign half through its own
66+
compiler, and the seam between the two is the `extern "C"` boundary of
67+
[42 — Heterogeneous Builds](42-heterogeneous-builds.md). No package in this
68+
repository ships that shape today.
69+
70+
### The boundary
71+
72+
**A declaration cannot reclassify what the engine already owns.** A dependency's
73+
`device_extensions` is consulted *after* the built-in roles, so a rule package
74+
cannot claim `.cpp`, `.cppm`, `.c` or `.S`. Those are the engine's own
75+
vocabulary, and a package must not be able to move a file out of it. Claiming
76+
one is not diagnosed and has no effect: measured by adding `".cpp"` to a rule's
77+
`device_extensions`, after which the consumer's `main.cpp` was still compiled as
78+
C++ and the build succeeded.
79+
80+
**Module-interface extensions are the project's axis, not a rule's.** A project
81+
that spells its interfaces `.ixx` declares `[build] module_extensions`
82+
([04 — The mcpp.toml Project File](04-mcpp-toml.md)). No rule-package key adds
83+
one, because a module interface is scanned for imports, produces a BMI and joins
84+
the link — three engine behaviours rather than a command to run.
85+
86+
**An extension in neither table is refused by name**, which is why a mistyped
87+
`device_extensions` surfaces at once instead of dropping a source:
88+
89+
```
90+
error: scanner errors:
91+
.../orphan.zzz: 'orphan.zzz' is listed in [build] sources, and mcpp has no role
92+
for the extension '.zzz'.
93+
Its object would be compiled and then linked by nothing, so this is refused rather
94+
than built.
95+
```
96+
1997
## The definition of a rule package
2098

2199
Three parts, and none of them is special to mcpp:

docs/zh/03-examples.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ mcpp build && mcpp run
7676
| 示例 | 首次引入的内容 |
7777
|---|---|
7878
| [`08-build-rules`](../../examples/08-build-rules/) | 两个规则包与同时使用它们的工程;`host-module = true``role = "check"``mcpp::action` |
79-
| [`12-a-new-device-language`](../../examples/12-a-new-device-language/) | `device_extensions``rule_module`:规则包教会 mcpp 一门引擎从未听说过的语言 |
79+
| [`12-a-new-device-language`](../../examples/12-a-new-device-language/) | `device_extensions``rule_module`:规则包教会 mcpp 一门引擎从未听说过的语言,而它的编译器是一个经 `tools = [...]` 为构建机构建出来的包 |
8080

8181
[31 —— 编写规则包](31-authoring-a-rule-package.md) 是这两个示例所演示内容的参考。
8282

docs/zh/09-commands-by-scenario.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
| 存储 | 作用域 | 增长时机 | 清空方式 |
2727
|---|---|---|---|
2828
| `target/<三元组>/<指纹>/` | 单个工程 | 配置指纹变化,开出新目录 | `mcpp clean``mcpp clean --stale` |
29-
| 构建缓存(`mcpp cache dir`) | 整台机器 | 任何工程编译依赖或 `std` 模块 | `mcpp cache gc``mcpp cache prune``mcpp cache clean` |
29+
| 构建缓存(`mcpp cache dir`) | 整台机器 | 任何工程编译依赖、`std` 模块,或构建一个 host 工具 | `mcpp cache gc``mcpp cache prune``mcpp cache clean` |
3030

3131
`mcpp clean` 整个删掉 `target/`,下次构建重编一切。`mcpp clean --stale` 只删已无构建
3232
记录使用的指纹目录,在用的配置保留:

docs/zh/30-build-mcpp.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -785,6 +785,14 @@ grpc = { version = "1.83.0", tools = ["grpc_cpp_plugin"] }
785785
- **全局缓存**,按 包版本 × host 工具链 × feature × 自身依赖闭包 键控 —— 每台机器
786786
构建一次,而不是每个工程一次。
787787

788+
**这个键里没有源码内容,而对 `path` 依赖这一点是看得见的。** 已发布的版本不可变,
789+
所以对来自索引的工具,这个键是精确的。而正在旁边被编辑的工具,两次构建之间版本相同,
790+
缓存里的二进制就留在原地:在
791+
[`examples/12-a-new-device-language`](../../examples/12-a-new-device-language/)
792+
上实测,改动工具的 emitter 之后,`mcpp run` 报告 `Finished dev in 0.00s` 并打印上一次
793+
的答案。抬工具包的版本,或用 `mcpp cache clean` 清空构建缓存 —— tool store 就住在
794+
里面,路径是 `<mcpp cache dir>/tool/<index>/<name>@<version>/`
795+
788796
### `[tools.overrides]` —— 使用已有的二进制
789797

790798
```toml

docs/zh/31-authoring-a-rule-package.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,74 @@
1212
嵌入,[`12-a-new-device-language`](../../examples/12-a-new-device-language/) 新增
1313
一门语言。
1414

15+
## 扩展模型
16+
17+
mcpp 的构建表面由包扩展,而不是由发布扩展。做扩展的有五个点,而经由它们进来的任何
18+
名字,引擎都不持有。
19+
20+
| 扩展点 | 效果 | 声明位置 |
21+
|---|---|---|
22+
| `mcpp::action` | 构建图里的一条边:一条命令,带声明的输入与输出 | 构建程序,或它 import 的规则模块 |
23+
| `device_extensions` | 一个扩展名被引擎归类为**设备源**,而不是被拒绝 | 包的某个 feature |
24+
| `rule_module` | 消费者的构建程序 import 哪个模块来够到这条规则 | 同一个 feature |
25+
| `tools = [...]` | 一个**从源码为构建机构建**的生成器或编译器,用 `mcpp::dep_bin` 取到 | 一条依赖边 |
26+
| `[xlings]``[feature-xlings]` | 规则要运行的预建工具,按需安装 | 包本身,或它的某个 feature |
27+
28+
生态用它们建出来的东西:
29+
30+
| 扩展面 | 所在包 | 使用的扩展点 |
31+
|---|---|---|
32+
| CUDA、HIP、SYCL、Ascend C | `mcpp:plugins`,各一个 feature([42](42-heterogeneous-builds.md)) | 驱动厂商编译器的 action,加上按加速器设闸的载荷 |
33+
| Slang | `mcpp:plugins``rules-slang` | `device_extensions = [".slang"]` —— 第一门无需引擎点名即被支持的语言 |
34+
| GLSL 与 HLSL 到 SPIR-V,以及其上的模块 | `mcpp:plugins``rules-spirv` | 每个着色器一条 action,加上一个生成的模块 |
35+
| 设备与 C++ 之间的岛边界 | `mcpp.tools.island` | 一个生成器,加上 `mcpp::generated` |
36+
| 作为可链接对象的资源文件 | [`08-build-rules`](../../examples/08-build-rules/) | `role = "object"` |
37+
| 能让构建失败的检查 | [`08-build-rules`](../../examples/08-build-rules/) | `role = "check"` |
38+
| 引擎从未听说过的语言 | [`12-a-new-device-language`](../../examples/12-a-new-device-language/) | `device_extensions`,加上经 `tools = [...]` 构建出来的编译器 |
39+
40+
### 这个模型能表达的形态
41+
42+
**一门新语言,不论由什么编译它。** 规则声明它认领的扩展名、为每个源提交一条
43+
action,并把编译器列进这条 action 的输入。这个编译器可以是厂商工具包、一个 LLVM
44+
前端、一个发出设备二进制的解释器,也可以是规则包自己从源码构建出来的程序。它产出
45+
的是设备二进制、目标文件还是 C++,由 action 的 `role` 决定,别无其他。引擎始终不
46+
学习这门语言:它学到的是「某个扩展名是设备源」以及「某条 action 认领它」。
47+
48+
**预处理与代码生成。** `role = "source"` 的 action 产出 C++,由声明它的包随后编译,
49+
而该包的每一条编译边都等它。输入可以是模板、接口定义、一张表,或另一条 action 的
50+
输出 —— 串联是常规做法,因为 action 之间由文件定序并计入指纹。
51+
52+
**一个一半是 C++、一半是另一种语言的文件。** 归类发生在任何规则运行之前,所以带着
53+
引擎自有扩展名的文件按 C++ 编译,永远到不了规则那里。因此一个携带外来代码块的源要
54+
用规则认领的扩展名,再由规则把它拆开:抽出来的 C++ 走 `role = "source"`,外来的那
55+
一半走规则自己的编译器,两者之间的缝就是
56+
[42 —— 异构构建](42-heterogeneous-builds.md) 里的 `extern "C"` 边界。本仓库今天没有
57+
任何包是这个形态。
58+
59+
### 边界
60+
61+
**声明不能重新归类引擎已经拥有的东西。** 依赖的 `device_extensions` 在内建角色
62+
**之后**才被查询,所以规则包认领不了 `.cpp``.cppm``.c``.S`。这些是引擎自己
63+
的词汇,一个包不得把文件从中挪走。认领它们不会被诊断,也不产生任何效果:实测把
64+
`".cpp"` 加进某条规则的 `device_extensions`,消费者的 `main.cpp` 仍按 C++ 编译,
65+
构建成功。
66+
67+
**模块接口的扩展名是工程的轴,不是规则的轴。** 把接口写成 `.ixx` 的工程声明
68+
`[build] module_extensions`([04 —— mcpp.toml 工程文件指南](04-mcpp-toml.md))。没有
69+
任何规则包的键能新增一个,因为模块接口要被扫描 import、要产出 BMI、还要进链接 ——
70+
这是三项引擎行为,而不是一条要跑的命令。
71+
72+
**两张表都不包含的扩展名会被点名拒绝**,所以写错的 `device_extensions` 会立刻显形,
73+
而不是把一个源默默丢掉:
74+
75+
```
76+
error: scanner errors:
77+
.../orphan.zzz: 'orphan.zzz' is listed in [build] sources, and mcpp has no role
78+
for the extension '.zzz'.
79+
Its object would be compiled and then linked by nothing, so this is refused rather
80+
than built.
81+
```
82+
1583
## 规则包的定义
1684

1785
三部分,没有一部分是 mcpp 特有的:

0 commit comments

Comments
 (0)