Skip to content

Commit c0225bb

Browse files
committed
A rule package declares what it compiles, so a new device language costs no engine change
Two keys on a feature, and the engine holds no package name, no feature spelling and no module name: [features.rules-slang] sources = ["rules/slang.cppm"] rule_module = "mcpp.rules.slang" device_extensions = [".slang"] `device_extensions` classifies those extensions as device sources in a consumer that activates the feature. `rule_module` is what a build program imports to reach the rule. Two things follow. A NEW DEVICE LANGUAGE NO LONGER TOUCHES THE ENGINE. Adding `.slang` to the built-in table cost an engine change, a release, and a version bump in the rule package's CI before the rule could route one file. `.slang` is removed from that table here and `rules-slang` declares it instead; `tests/slang-consumer` builds and runs unchanged, which is the only honest test of whether the mechanism carries a language. The built-in list is now what mcpp knows without being told -- a compatibility set for languages whose support shipped before the declaration existed -- rather than a registry a sixth backend joins. A CONSUMER WRITES ONE EDGE AND NO BUILD PROGRAM. `host-module = true` is implied by `rule_module`, because a feature naming one has already said that is the only way to use it. And a package with no `build.mcpp` gets the program its rules describe written into the build directory: [build-dependencies.mcpp] plugins = { version = "0.3.0", features = ["rules-spirv"] } is the whole declaration in `tests/spirv-zero-config`, which compiles a shader and reaches it through a generated module. A project that writes its own `build.mcpp` keeps it: synthesis fills an absence and never overrides. THE FEATURE IS STILL REQUESTED BY NAME. An earlier revision derived it from the extensions a project's sources carried, so a consumer could name the package alone. That was withdrawn for two reasons and neither was cost. Two packages may claim one extension -- a third-party CUDA rule is a thing someone will write -- and derivation would then guess or refuse where `features = [...]` has already said which. And a manifest's job is to describe the build: a derived feature set is information the file no longer states, which is worse for a reader and worse for anything reading the manifest as context. Which rules ran is said out loud, for the same reason the resolved toolchain is: Rules mcpp.rules.spirv (mcpp:plugins) Two ordering defects were found by running it rather than by reading it. The collection must sit between feature activation and the extension table that narrows the constrained globs; placed after, the declared extensions arrived too late to classify anything and the rule was handed an empty list. And the guard on the build-program call asked whether the FILE existed while the function asked whether a program was WANTED, which left the synthesis unreachable. Both now ask the same question, and the comments say what the failure looked like.
1 parent 90be6a9 commit c0225bb

8 files changed

Lines changed: 353 additions & 23 deletions

File tree

modules/manifest/src/toml.cppm

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -758,6 +758,40 @@ std::expected<Manifest, ManifestError> parse_string(std::string_view content,
758758
read_str_array(ft, "provides", provs);
759759
if (!reqs.empty()) m.featureRequires[fname] = std::move(reqs);
760760
if (!provs.empty()) m.featureProvides[fname] = std::move(provs);
761+
// The device extensions this feature's rule compiles. Normalised
762+
// the same way `module_extensions` is, so `comp` and `.comp` are
763+
// one entry and a consumer cannot be surprised by a missing dot.
764+
std::vector<std::string> devExts;
765+
read_str_array(ft, "device_extensions", devExts);
766+
if (!devExts.empty()) {
767+
for (auto& e : devExts) e = mcpp::normalize_extension(e);
768+
std::erase(devExts, std::string{});
769+
if (!devExts.empty())
770+
m.featureDeviceExtensions[fname] = std::move(devExts);
771+
}
772+
// The module a consumer's build program imports for this rule.
773+
if (auto it = ft.find("rule_module");
774+
it != ft.end() && it->second.is_string())
775+
m.featureRuleModule[fname] = it->second.as_string();
776+
// The two halves of "this feature is a build rule" must arrive
777+
// together. One without the other is a declaration nothing can
778+
// act on, and the failure would otherwise land in a consumer's
779+
// build rather than in the package that wrote it.
780+
{
781+
const bool hasExts = m.featureDeviceExtensions.contains(fname);
782+
const bool hasMod = m.featureRuleModule.contains(fname);
783+
if (hasExts != hasMod) {
784+
return std::unexpected(error(origin, std::format(
785+
"[features].{} declares `{}` without `{}`. A build rule states "
786+
"both:\n"
787+
" `device_extensions` is what it compiles, `rule_module` is "
788+
"how a\n"
789+
" consumer's build program reaches it.",
790+
fname,
791+
hasExts ? "device_extensions" : "rule_module",
792+
hasExts ? "rule_module" : "device_extensions")));
793+
}
794+
}
761795
// #253: per-feature per-glob compile flags — same entry grammar
762796
// as [build].flags (shared parse_glob_flags_value), gated by
763797
// this feature and folded in AFTER base globFlags at activation

modules/manifest/src/types.cppm

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -639,6 +639,18 @@ struct BuildConfig : BuildInputs {
639639
// Scoped to the declaring package — a dependency is classified by its own
640640
// manifest, never by its consumer's.
641641
std::vector<std::string> moduleExtensions;
642+
// Device extensions this package's build-dependencies declared through
643+
// `[features].<f>.device_extensions`, for the features this package
644+
// requested. NOT written in a manifest: filled by prepare from the
645+
// resolved edges, and carried here so every site that already builds an
646+
// extension table for a package gets the device axis without a second
647+
// plumbing route. A package with no rule dependency leaves it empty, which
648+
// is every package that has none today.
649+
std::vector<std::string> deviceExtensions;
650+
// The rule modules a synthesised `build.mcpp` imports, in the order the
651+
// features were collected. Filled by prepare beside `deviceExtensions`
652+
// above and read only when this package has no build program of its own.
653+
std::vector<std::string> ruleModules;
642654
// [build] accel — which accelerator backends and device architectures this
643655
// build targets, in the wire form mcpp.pack.abi_tag reads. Empty means the
644656
// build asks for none, and then every prebuilt artifact satisfies it
@@ -1478,6 +1490,56 @@ struct Manifest {
14781490
// see the member there.)
14791491
std::map<std::string, std::vector<std::string>> featureProvides; // feature → caps
14801492
std::map<std::string, std::vector<std::string>> featureRequires; // feature → caps
1493+
1494+
// `[features].<f>.device_extensions` — the device source extensions this feature's
1495+
// rule compiles (mcpp 2026.9.7.1+).
1496+
//
1497+
// THE ENGINE EXPOSES THE CAPABILITY; THE PACKAGE SUPPLIES THE FACT. The key
1498+
// is named after `[build] module_extensions` because it is the same shape:
1499+
// mcpp knows what it means for a file to be a device source -- never
1500+
// scanned, never a BMI, compiled by something mcpp does not drive -- and
1501+
// does not know that `.cu` is CUDA. A rule package states which extensions
1502+
// it compiles, and a consumer that activates that feature gets them
1503+
// classified as device sources. A NEW device language therefore costs no
1504+
// engine change, which is what `docs/20`'s "a sixth backend is a package
1505+
// rather than an engine change" has claimed and, until this key, was not.
1506+
//
1507+
// NOTHING IS DERIVED FROM IT. An earlier revision also used it to ACTIVATE
1508+
// the matching feature, so a consumer could name the package and nothing
1509+
// else. That was withdrawn for two reasons, and neither was cost:
1510+
//
1511+
// - Two packages may claim one extension. A third-party rule for `.cu`
1512+
// is a thing someone will write, and derivation would then have to
1513+
// guess or refuse, where `features = ["rules-cuda"]` has already said
1514+
// which one.
1515+
// - A manifest's job is to describe the build. A derived feature set is
1516+
// absent information that has to be reconstructed by running the
1517+
// build, which is worse for a reader and worse for anything reading the
1518+
// manifest as context.
1519+
//
1520+
// So the feature is requested by name, as every other feature is, and this
1521+
// key answers only "what does that feature compile".
1522+
//
1523+
// WHICH FEATURE COMPILES AN EXTENSION IS NOT STATED SEPARATELY. It is the
1524+
// feature the line is written on. A second key naming the rule would be the
1525+
// same fact twice.
1526+
std::map<std::string, std::vector<std::string>> featureDeviceExtensions;
1527+
1528+
// `[features].<f>.rule_module` — the module a consumer's build program
1529+
// imports to reach this feature's rule, and whose `compile()` it calls.
1530+
//
1531+
// DECLARED RATHER THAN DISCOVERED, which is the trade this codebase already
1532+
// makes for `mcpp::action`'s `provides`/`imports` and for
1533+
// `[modules] scan_overrides`. The name is in the feature's own interface
1534+
// unit and could be scanned out of it, but the program that imports it has
1535+
// to be written BEFORE anything is compiled, and a build that had to scan a
1536+
// dependency's sources to decide what to write would order the two the
1537+
// wrong way round.
1538+
//
1539+
// Present exactly when `device_extensions` is: together they say "this
1540+
// feature is a build rule, here is what it compiles and here is how to
1541+
// reach it". A feature with one and not the other is refused at parse time.
1542+
std::map<std::string, std::string> featureRuleModule;
14811543
// Feature System v2 Stage 2a — dependencies activated by a feature. A dep
14821544
// declared ONLY here is optional: pulled into the resolution worklist only
14831545
// when its feature is active (root --features or a dep spec's features=[...]).

modules/source-kind/src/source_kind.cppm

Lines changed: 70 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,24 @@ std::string_view to_string(SourceKind k);
9393
struct ExtensionTable {
9494
// Always contains the built-ins first, in their historical order.
9595
std::vector<std::string> moduleInterface;
96+
// Device extensions a DEPENDENCY declared it compiles, through
97+
// `[features].<f>.device_extensions`. Empty for a package with no rule
98+
// dependency, which is every package that has none today.
99+
//
100+
// WHY THE PACKAGE AND NOT THIS FILE. mcpp knows what a device source IS --
101+
// never scanned, never a BMI, compiled by something mcpp does not drive --
102+
// and does not know that `.cu` is CUDA. The built-in list below is the set
103+
// of languages the project has already shipped support for; this axis is
104+
// how a NEW one arrives without an engine release, which is what makes
105+
// "a sixth backend is a package rather than an engine change" true rather
106+
// than aspirational. Slang measured the difference: adding it cost an
107+
// engine change, a release, and a version bump in the rule package's CI
108+
// before its rule could route a single file.
109+
//
110+
// Same shape as `moduleInterface` above, and for the same reason: built-ins
111+
// are what needs no declaring, and everything else is declared by whoever
112+
// knows it.
113+
std::vector<std::string> device;
96114
};
97115

98116
// Trim, then supply a leading dot if absent. Does NOT change case — see the
@@ -117,6 +135,15 @@ ExtensionTable builtin_extension_table();
117135
// validation still classifies exactly like a default one.
118136
ExtensionTable extension_table_for(std::span<const std::string> extras);
119137

138+
// The same, plus device extensions a dependency declared through
139+
// `[features].<f>.device_extensions`. Separate parameters rather than one list
140+
// because the two axes are validated differently and must not be able to leak
141+
// into each other: a module extension the project names is checked against the
142+
// reserved roles, and a device extension a dependency names is added only where
143+
// no built-in role already claims the spelling.
144+
ExtensionTable extension_table_for(std::span<const std::string> moduleExtras,
145+
std::span<const std::string> deviceExtras);
146+
120147
// Extensions that already name a non-module role. Declaring one of these as a
121148
// module interface has no legitimate use and would route (say) a C file to the
122149
// C++ module rule, failing somewhere that names neither the file nor the key.
@@ -272,19 +299,26 @@ constexpr std::string_view kHeaderExtensions[] = { ".h", ".hpp", ".hh", ".hxx" }
272299
// own -- so the island is the shape Ascend already has, not one mcpp imposes.
273300
// `.cce` is the older spelling of the same thing and is accepted beside it.
274301
//
275-
// `.slang` is the Slang shading language, compiled by `slangc`. It is a
276-
// LANGUAGE rather than a second driver for GLSL -- its own module system,
277-
// generics, and a target set beyond SPIR-V -- which is why it has an extension
278-
// of its own here and a rule of its own outside.
302+
// `.slang` IS NOT HERE, AND ITS ABSENCE IS THE POINT.
279303
//
280-
// THIS TABLE IS WHAT DECIDES, NOT THE GLOB'S `accel` KEY. A constrained glob
281-
// carrying `accel = "vulkan1.2"` does not make a file a device source; this
282-
// list does, and a file whose extension is absent from it reaches the ordinary
283-
// source scan and is refused with "mcpp has no role for the extension". A rule
284-
// package therefore cannot introduce a device language on its own, and adding
285-
// one here is the engine half of doing so.
304+
// Slang is a language rather than a second driver for GLSL, so it needs a rule;
305+
// it reaches this build through `mcpp:plugins`' `rules-slang`, which declares
306+
// `device_extensions = [".slang"]` in its own manifest. That is the first
307+
// device language mcpp supports without naming it here.
308+
//
309+
// The list below is what mcpp knows WITHOUT being told: the languages whose
310+
// support shipped before the declaration existed. It is a compatibility set,
311+
// not a registry -- a new language does not join it, and `.slang` was removed
312+
// after the mechanism proved able to carry it, which is the only honest test of
313+
// whether the mechanism works.
314+
//
315+
// THIS TABLE AND THE DECLARED ONE ARE WHAT DECIDE, NOT THE GLOB'S `accel` KEY.
316+
// A constrained glob carrying `accel = "vulkan1.2"` does not make a file a
317+
// device source; membership here or in a dependency's declaration does, and a
318+
// file in neither reaches the ordinary source scan and is refused with "mcpp
319+
// has no role for the extension".
286320
constexpr std::string_view kDeviceExtensions[] = {
287-
".cu", ".hip", ".sycl", ".asc", ".cce", ".slang",
321+
".cu", ".hip", ".sycl", ".asc", ".cce",
288322
".comp", ".vert", ".frag", ".geom", ".tesc", ".tese", ".mesh", ".task",
289323
".rgen", ".rint", ".rahit", ".rchit", ".rmiss", ".rcall",
290324
".glsl", ".hlsl", ".cl", ".metal",
@@ -354,6 +388,26 @@ ExtensionTable extension_table_for(std::span<const std::string> extras) {
354388
return t;
355389
}
356390

391+
ExtensionTable extension_table_for(std::span<const std::string> moduleExtras,
392+
std::span<const std::string> deviceExtras) {
393+
auto t = extension_table_for(moduleExtras);
394+
for (auto const& raw : deviceExtras) {
395+
auto ext = normalize_extension(raw);
396+
if (ext.empty()) continue;
397+
// A declaration cannot move a file out of a role the engine already
398+
// owns. `.cpp` is a C++ translation unit whatever a dependency says,
399+
// and silently accepting the entry would let one package change what
400+
// every source in a consumer means.
401+
if (std::ranges::find(t.moduleInterface, ext) != t.moduleInterface.end()) continue;
402+
if (contains(kCxxExtensions, ext) || contains(kCExtensions, ext)
403+
|| contains(kGasExtensions, ext) || contains(kNasmExtensions, ext)
404+
|| contains(kHeaderExtensions, ext)) continue;
405+
if (std::ranges::find(t.device, ext) != t.device.end()) continue;
406+
t.device.push_back(std::move(ext));
407+
}
408+
return t;
409+
}
410+
357411
bool is_reserved_non_module_extension(std::string_view ext) {
358412
return contains(kCxxExtensions, ext) || contains(kCExtensions, ext)
359413
|| contains(kGasExtensions, ext) || contains(kNasmExtensions, ext)
@@ -408,6 +462,11 @@ SourceKind classify(const std::filesystem::path& p, const ExtensionTable& t) {
408462
if (contains(kGasExtensions, ext)) return SourceKind::GasAsm;
409463
if (contains(kNasmExtensions, ext)) return SourceKind::NasmAsm;
410464
if (contains(kDeviceExtensions, ext)) return SourceKind::Device;
465+
// A dependency's `device_extensions`. Checked AFTER the built-in roles so a
466+
// rule package cannot reclassify `.cpp`; the built-ins are the engine's own
467+
// vocabulary and a declaration must not be able to move a file out of it.
468+
for (auto const& d : t.device)
469+
if (ext == d) return SourceKind::Device;
411470
if (contains(kHeaderExtensions, ext)
412471
|| contains(kDeviceHeaderExtensions, ext)) return SourceKind::Header;
413472
return SourceKind::Other;

src/build/build_program.cppm

Lines changed: 67 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,13 @@ struct BuildProgramEnv {
134134
// and an engine older than this one leaves the variable absent -- which a
135135
// rule reads as "header", the behaviour every consumer had before.
136136
bool languageModules = true;
137+
// The rule modules a synthesised build program imports, from
138+
// `BuildConfig::ruleModules`. When this package has no `build.mcpp` and
139+
// this list is not empty, mcpp writes the program these entries describe.
140+
// The program is the one the project would have written by hand, which is
141+
// what makes the declaration a LAYER above `build.mcpp` rather than a
142+
// second way of doing the same thing.
143+
std::vector<std::string> ruleModules;
137144
// The device-kind sources (`.cu`, `.hip`, ...) this package's effective
138145
// source set matches, package-root-relative with `/` separators, one per
139146
// line. The engine has no compile rule for them and hands the list to the
@@ -730,6 +737,38 @@ bool cache_fresh(const fs::path& root, const fs::path& bdir, const CacheRecord&
730737
return true;
731738
}
732739

740+
741+
// The program a set of rule modules describes.
742+
//
743+
// It is the program a project writes by hand for the same rules, and that is
744+
// the whole contract: the declaration is a layer ABOVE `build.mcpp`, not a
745+
// second mechanism beside it, so a project that outgrows it copies this file
746+
// into its root and edits it. Synthesis then stops, because a project that has
747+
// its own program keeps it.
748+
//
749+
// The namespace comes from the module name with `.` exchanged for `::`, which
750+
// is the convention `mcpp.rules.<x>` already follows and the one a third-party
751+
// rule opts into by naming its module. Nothing here knows what any rule does.
752+
std::string synthesised_rule_program(const std::vector<std::string>& modules) {
753+
std::string s =
754+
"// Generated by mcpp from the build rules this package's dependencies\n"
755+
"// declare. Do not edit. To take it over, copy this file to `build.mcpp`\n"
756+
"// in the project root; mcpp synthesises nothing once a project has one.\n"
757+
"import std;\n"
758+
"import mcpp;\n";
759+
for (auto const& m : modules) s += "import " + m + ";\n";
760+
s += "\nint main() {\n bool ok = true;\n";
761+
for (auto const& m : modules) {
762+
std::string ns;
763+
for (char c : m) { if (c == '.') ns += "::"; else ns += c; }
764+
// `&&` would stop at the first refusal, and a project with two rules
765+
// wants both diagnostics rather than one and then silence.
766+
s += " ok = " + ns + "::compile() && ok;\n";
767+
}
768+
s += " return ok ? 0 : 1;\n}\n";
769+
return s;
770+
}
771+
733772
} // namespace
734773

735774
std::expected<void, std::string> run_build_program(
@@ -742,7 +781,34 @@ std::expected<void, std::string> run_build_program(
742781

743782
fs::path src = root / "build.mcpp";
744783
std::error_code ec;
745-
if (!fs::exists(src, ec)) return {}; // no build program — nothing to do
784+
if (!fs::exists(src, ec)) {
785+
// The layer above this file: when a dependency's rules claimed device
786+
// sources in this package and the package wrote no program, mcpp writes
787+
// the program those rules describe.
788+
//
789+
// SYNTHESISED ONLY IN THE ABSENCE. A project with its own `build.mcpp`
790+
// keeps it, because the two would otherwise both submit the same
791+
// actions and the second submission is one nobody asked for. That is
792+
// also what makes the descent safe: copy the generated file into the
793+
// project root, edit it, and synthesis stops.
794+
if (env.ruleModules.empty()) return {}; // nothing to do
795+
src = build_dir(root, env) / "build.mcpp";
796+
fs::create_directories(src.parent_path(), ec);
797+
const std::string text = synthesised_rule_program(env.ruleModules);
798+
// Written only when it differs, so a package whose rule set did not
799+
// change does not rebuild its build program on every configure.
800+
bool same = false;
801+
if (std::ifstream in(src, std::ios::binary); in) {
802+
std::string prev((std::istreambuf_iterator<char>(in)),
803+
std::istreambuf_iterator<char>());
804+
same = (prev == text);
805+
}
806+
if (!same) {
807+
std::ofstream out(src, std::ios::binary | std::ios::trunc);
808+
if (!out) return {};
809+
out << text;
810+
}
811+
}
746812

747813
fs::path bdir = build_dir(root, env);
748814
fs::path outDir = bdir / "out";

src/build/execute.cppm

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1139,7 +1139,8 @@ fast_path_identity(const std::filesystem::path& projectRoot,
11391139
std::string(mcpp::build::cache_mode_name(
11401140
mcpp::build::resolve_cache_mode(*m, ""))),
11411141
m->resources.files,
1142-
mcpp::extension_table_for(m->buildConfig.moduleExtensions),
1142+
mcpp::extension_table_for(m->buildConfig.moduleExtensions,
1143+
m->buildConfig.deviceExtensions),
11431144
m->buildConfig.target,
11441145
m->hooks.active(),
11451146
normalize_features(featuresRequested),

src/build/plan.cppm

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1017,7 +1017,8 @@ make_plan(const mcpp::manifest::Manifest& manifest,
10171017
// needs it — every scanned unit arrives with its kind already set by the
10181018
// scanner, using its OWN package's table.
10191019
const auto rootExtTable =
1020-
mcpp::extension_table_for(manifest.buildConfig.moduleExtensions);
1020+
mcpp::extension_table_for(manifest.buildConfig.moduleExtensions,
1021+
manifest.buildConfig.deviceExtensions);
10211022

10221023
// Artifact naming and shared-library link shape are properties of the
10231024
// TARGET. Resolved once here from tc.targetTriple (empty = host target, in

0 commit comments

Comments
 (0)