Skip to content

Commit 60a71a9

Browse files
SunrisepeakSunrisepeak
authored
feat(ggml-org.llamacpp): b10069.1 and b10069.2, and a CI pin that follows the engine (#361)
* feat(ggml-org.llamacpp): b10069.1 and b10069.2, and a CI pin that follows the engine `b10069.1` adds `backend-vulkan`: llama.cpp's Vulkan backend as build-graph edges, 134 vendored shaders compiled by declared actions rather than by a build program looping. `b10069.2` refuses a libc++ toolchain by name -- upstream's ggml-vulkan.cpp destroys a `std::unique_ptr` to an incomplete type, which libc++ rejects by static assertion and libstdc++ accepts. Both mirrors byte-identical (1456b6003dada314..., 36027267 bytes). The CI pin moves 2026.8.27.2 -> 2026.9.6.3. It had drifted about ten releases behind, so this index was validated against an engine no user runs. These entries made the drift visible rather than causing it: their build program calls `mcpp::toolchain_sysroot()` and `mcpp::toolchain_binutils_dir()` (2026.9.5.2+) to hand the ecosystem C library to the second compiler it drives, and `mcpp::cxx_stdlib()` (2026.9.6.3) to make the refusal above possible. Under the old pin the workspace members could not compile it at all. The cost is paid once per move and it is large: the members' caches key on `MCPP_VERSION`, so this run rebuilds every member on every platform -- measured on the previous attempt at twelve shards still cold after fifty minutes. Both revisions therefore land in ONE pin move rather than two, which is also why the `mcpp:plugins` 0.2.1 entry went separately in #360: it needed no pin change, so `select` picked nothing for it. `index.toml` min_mcpp does not move. It states the oldest mcpp able to RESOLVE every descriptor, and all of them parse under both versions; raising it would refuse the whole index to a client on the floor over a build-program API that client may never reach. `b10069` stays published for exactly that case. * ci: raise the shard caps that cancelled three jobs at 90 minutes The caps were the binding constraint again and the measured table has grown about 50% since they were set. `plan_shards.lua` on this tree reports linux 23925s, macos 16676s, windows 18435s, so `secs / 4200 + 1` asks for 6 / 4 / 5 while the caps clamped to 4 / 2 / 2. Windows at 18435s over two shards is roughly 150 minutes a shard, over the 90-minute job cap before a cold cache is considered at all. Observed rather than predicted. On this PR's pin move three shards were cancelled at exactly 90 minutes: `linux default 0/4`, `linux default 3/4` and `windows default 0/2`. The two linux shards passed on a re-run against the warm registry cache. The Windows one timed out AGAIN at 1h30m13s WITH the cache restored -- the pin change invalidates every member build, and that shard carries godot-cpp-m twice (9.5 and 10 minutes) plus opencv. linux 4 -> 6, windows 2 -> 5. macos stays at 2: it has never been cancelled, so raising it would be a prediction rather than a measurement, and the formula's 4 is there the day it is. The paragraph that set the previous caps is marked superseded rather than deleted, because it records the measurement that was true when it was written. * ci: the llvm leg needs its own shard count `plan_shards.lua` takes a PLATFORM and reads one timing table. It has no toolchain dimension, so `linux default` and `linux llvm` were handed an identical split of an identical estimate -- and they are not identical work. The llvm leg resolves a different compiler with no sysroot, and it is the slower of the two. Measured on this PR, with both legs at 6 shards: every `linux default` shard finished, and `linux llvm 5/6` was cancelled at 1h30m16s. The split was right for the leg the table was measured on and wrong for the other one. `lln = ln + 3` is the smallest correct fix. Teaching the planner about toolchains is the larger one and needs a timing table per leg, which CI does not produce yet.
1 parent 40aab9c commit 60a71a9

4 files changed

Lines changed: 155 additions & 8 deletions

File tree

‎.github/workflows/validate.yml‎

Lines changed: 73 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,35 @@ env:
163163
# - mcpp#405 依赖 BMI 缓存命中时,被恢复的包传递依赖的 `std` 没进构建图 ——
164164
# **每个图形项目的第二次构建必挂**(消费方自己不 `import std` 时现形)。
165165
# - mcpp#407 `mcpp build` 会重放 `mcpp test` 留下的构建图。
166-
MCPP_VERSION: "2026.8.27.2"
166+
#
167+
# ── 2026-09-06: raised 2026.8.27.2 -> 2026.9.6.3 ──────────────────────────
168+
#
169+
# The pin had drifted about ten releases behind, so this index was validated
170+
# against an engine no user runs. `llamacpp@b10069.2` made the drift visible
171+
# rather than causing it: its build program calls `mcpp::toolchain_sysroot()`
172+
# and `mcpp::toolchain_binutils_dir()` (2026.9.5.2+) to hand the ecosystem C
173+
# library to the second compiler it drives, and `mcpp::cxx_stdlib()`
174+
# (2026.9.6.3) to refuse a libc++ toolchain by name. Under the old pin the
175+
# workspace members could not compile it at all. A package whose build program
176+
# uses a current engine API is not a defect; a CI that cannot run current
177+
# engines is.
178+
#
179+
# `index.toml` min_mcpp deliberately does NOT move with this. The floor states
180+
# the oldest mcpp able to RESOLVE every descriptor -- descriptor grammar --
181+
# and every descriptor here parses under both versions. A client on
182+
# 2026.8.27.2 keeps the whole index; only `llamacpp@b10069.1` and `.2`, whose
183+
# build program needs the newer accessors, are out of reach for it, and
184+
# `b10069` remains published for exactly that case. Raising the floor instead
185+
# would refuse the index to that client entirely, which is the failure mode
186+
# `index-floor-must-degrade` names: the index is data, mcpp is the program,
187+
# and publishing data must not invalidate the program.
188+
#
189+
# THE COST IS PAID ONCE PER MOVE, AND IT IS LARGE: the members' caches key on
190+
# `MCPP_VERSION`, so the first run after this line changes rebuilds every
191+
# member on every platform. Measured: twelve workspace shards, all still cold
192+
# after fifty minutes. Batch descriptor changes that need a newer engine into
193+
# ONE pin move rather than several.
194+
MCPP_VERSION: "2026.9.6.3"
167195

168196
jobs:
169197
lint:
@@ -651,13 +679,54 @@ jobs:
651679
# answer and leaves 16 minutes for a cold cache; 5 is there if that
652680
# stops being enough.
653681
#
682+
# SUPERSEDED 2026-09-06: it stopped being enough. The table is now
683+
# 23925s rather than 15891s, so the paragraph above describes a
684+
# measurement that no longer holds; see the dated block below the
685+
# concurrency note for what replaced it.
686+
#
654687
# This does cross the concurrency line the old comment drew: with two
655688
# toolchain legs linux now emits 8 jobs against a measured runner
656689
# concurrency of 3, so shards queue. Queuing is the right trade —
657690
# back-to-back shards still finish, a shard over the cap does not.
658-
ln=$(shards_for linux 4)
691+
# ── 2026-09-06: linux 4 -> 6, windows 2 -> 5 ────────────────────
692+
#
693+
# The caps were the binding constraint again, and the table has grown
694+
# about 50% since they were set. Measured with `plan_shards.lua` on
695+
# this tree: linux 23925s, macos 16676s, windows 18435s, so the
696+
# formula asks for 6 / 4 / 5 and was being clamped to 4 / 2 / 2.
697+
# Windows at 18435s over two shards is roughly 150 minutes a shard,
698+
# over the 90-minute job cap before a cold cache is even considered.
699+
#
700+
# Observed, not predicted. On the 2026.9.6.3 pin move (#361) three
701+
# shards were cancelled at exactly 90 minutes: `linux default 0/4`,
702+
# `linux default 3/4` and `windows default 0/2`. The two linux ones
703+
# passed on a re-run against the warm registry cache; the Windows one
704+
# timed out AGAIN at 1h30m13s with the cache restored, because the
705+
# pin change invalidates every MEMBER build and that shard carries
706+
# godot-cpp-m twice (9.5 and 10 minutes) plus opencv.
707+
#
708+
# macos stays at 2. It has never been cancelled, so raising it would
709+
# be a prediction rather than a measurement; the formula's 4 is there
710+
# the day it is.
711+
ln=$(shards_for linux 6)
659712
mn=$(shards_for macos 2)
660-
wn=$(shards_for windows 2)
713+
wn=$(shards_for windows 5)
714+
# THE TWO LINUX LEGS ARE NOT THE SAME COST, AND THE PLANNER CANNOT
715+
# SEE THAT. `plan_shards.lua` takes a PLATFORM and reads one timing
716+
# table; it has no toolchain dimension, so `linux default` and
717+
# `linux llvm` receive an identical split of an identical estimate.
718+
# They are not identical work: the llvm leg resolves a different
719+
# compiler with no sysroot, and it is the slower of the two.
720+
#
721+
# Measured on this PR: with both legs at 6, every `linux default`
722+
# shard finished and `linux llvm 5/6` was cancelled at 1h30m16s. The
723+
# split was right for the leg the table was measured on and wrong for
724+
# the other one.
725+
#
726+
# Giving llvm its own count is the smallest correct fix. Teaching the
727+
# planner about toolchains is the larger one, and it needs a timing
728+
# table per leg that CI does not produce yet.
729+
lln=$(( ln + 3 ))
661730
{
662731
printf '{"include":['
663732
# linux runs TWICE: once on mcpp's default toolchain and once on
@@ -678,7 +747,7 @@ jobs:
678747
# That is the price of the coverage; the levers if it hurts are
679748
# in the shards_for comment above.
680749
emit linux ubuntu-latest linux-x86_64 tar.gz bin/mcpp registry/bin/xlings default "$ln"
681-
emit linux ubuntu-latest linux-x86_64 tar.gz bin/mcpp registry/bin/xlings llvm "$ln"
750+
emit linux ubuntu-latest linux-x86_64 tar.gz bin/mcpp registry/bin/xlings llvm "$lln"
682751
emit macos macos-15 macosx-arm64 tar.gz bin/mcpp registry/bin/xlings default "$mn"
683752
emit windows windows-latest windows-x86_64 zip bin/mcpp.exe registry/bin/xlings.exe default "$wn"
684753
printf ']}'

‎pkgs/g/ggml-org.llamacpp.lua‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,18 +24,96 @@ package = {
2424

2525
xpm = {
2626
linux = {
27+
-- b10069.1 ADDS `backend-vulkan`, AND ITS BUILD PROGRAM NEEDS mcpp
28+
-- 2026.9.5.2+. It calls `mcpp::toolchain_sysroot()` and
29+
-- `mcpp::toolchain_binutils_dir()` to tell the shader generator's
30+
-- compiler where the ecosystem's C library is; without them that
31+
-- compiler reads the host's, which is the dependency this ecosystem
32+
-- removes. On an older mcpp the build program does not compile, and
33+
-- the error names the function rather than the version -- a qualified
34+
-- name that does not exist is ill-formed, not `false`, so no package
35+
-- can probe for it (mcpp docs/07).
36+
--
37+
-- b10069.2 REFUSES A libc++ TOOLCHAIN BY NAME, and needs
38+
-- `mcpp::cxx_stdlib()` (2026.9.6.3) to do it. Upstream's
39+
-- ggml-vulkan.cpp destroys a `std::unique_ptr` to an incomplete type,
40+
-- which libc++ rejects by static assertion and libstdc++ accepts;
41+
-- b10069.1 documented that and could not refuse it, because the only
42+
-- signal available then was `mcpp::compiler()`, which answers "clang"
43+
-- for both standard libraries.
44+
--
45+
-- These entries are what raised the CI pin to 2026.9.6.3
46+
-- (validate.yml). The pin had drifted about ten releases behind, so
47+
-- it was not this package that outran the index; the index had
48+
-- stopped following the engine. The workspace members build
49+
-- `b10069.2` under the new pin.
50+
--
51+
-- `min_mcpp` is NOT the lever for this, and does not move. It states
52+
-- the oldest mcpp that can RESOLVE every descriptor, and this
53+
-- descriptor uses no new grammar. A client on the floor keeps the
54+
-- whole index and keeps `b10069`, which stays published for exactly
55+
-- that case; raising the floor would refuse the index outright over a
56+
-- build-program API such a client may never reach.
57+
--
58+
-- Both revisions are kept. `b10069` is not semver, so no range
59+
-- expresses "or later" and consumers pin exactly -- which means a
60+
-- consumer wanting a feature must be able to name the revision that
61+
-- has it, and one that cannot use the newer engine must still find a
62+
-- revision it can build.
63+
["b10069.2"] = {
64+
url = {
65+
GLOBAL = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.2.tar.gz",
66+
CN = "https://gitcode.com/mcpp-res/llamacpp/releases/download/b10069.2/llamacpp-b10069.2.tar.gz",
67+
},
68+
sha256 = "1456b6003dada314661534bdcbbd7c73590c0d31940f8750d5f4178d459ffc34",
69+
},
70+
["b10069.1"] = {
71+
url = {
72+
GLOBAL = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.1.tar.gz",
73+
CN = "https://gitcode.com/mcpp-res/llamacpp/releases/download/b10069.1/llamacpp-b10069.1.tar.gz",
74+
},
75+
sha256 = "4f30c253a7008f82e5bba9d8f894db660e3ebfbc18e2ee745d13c85b7332bbf5",
76+
},
2777
["b10069"] = {
2878
url = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.tar.gz",
2979
sha256 = "be0b0deeb31136ea9cf0ba61eda1fb7baf8795a6579494a0991afd2323213e0b",
3080
},
3181
},
3282
macosx = {
83+
["b10069.2"] = {
84+
url = {
85+
GLOBAL = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.2.tar.gz",
86+
CN = "https://gitcode.com/mcpp-res/llamacpp/releases/download/b10069.2/llamacpp-b10069.2.tar.gz",
87+
},
88+
sha256 = "1456b6003dada314661534bdcbbd7c73590c0d31940f8750d5f4178d459ffc34",
89+
},
90+
["b10069.1"] = {
91+
url = {
92+
GLOBAL = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.1.tar.gz",
93+
CN = "https://gitcode.com/mcpp-res/llamacpp/releases/download/b10069.1/llamacpp-b10069.1.tar.gz",
94+
},
95+
sha256 = "4f30c253a7008f82e5bba9d8f894db660e3ebfbc18e2ee745d13c85b7332bbf5",
96+
},
3397
["b10069"] = {
3498
url = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.tar.gz",
3599
sha256 = "be0b0deeb31136ea9cf0ba61eda1fb7baf8795a6579494a0991afd2323213e0b",
36100
},
37101
},
38102
windows = {
103+
["b10069.2"] = {
104+
url = {
105+
GLOBAL = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.2.tar.gz",
106+
CN = "https://gitcode.com/mcpp-res/llamacpp/releases/download/b10069.2/llamacpp-b10069.2.tar.gz",
107+
},
108+
sha256 = "1456b6003dada314661534bdcbbd7c73590c0d31940f8750d5f4178d459ffc34",
109+
},
110+
["b10069.1"] = {
111+
url = {
112+
GLOBAL = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.1.tar.gz",
113+
CN = "https://gitcode.com/mcpp-res/llamacpp/releases/download/b10069.1/llamacpp-b10069.1.tar.gz",
114+
},
115+
sha256 = "4f30c253a7008f82e5bba9d8f894db660e3ebfbc18e2ee745d13c85b7332bbf5",
116+
},
39117
["b10069"] = {
40118
url = "https://github.com/mcpplibs/llama.cpp-m/archive/refs/tags/b10069.tar.gz",
41119
sha256 = "be0b0deeb31136ea9cf0ba61eda1fb7baf8795a6579494a0991afd2323213e0b",

‎tests/examples/llamacpp-metal/mcpp.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,4 @@ name = "llamacpp-metal-tests"
77
version = "0.1.0"
88

99
[target.'cfg(macos)'.dependencies.ggml-org]
10-
llamacpp = { version = "b10069", features = ["backend-metal"] }
10+
llamacpp = { version = "b10069.2", features = ["backend-metal"] }

‎tests/examples/llamacpp/mcpp.toml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,10 @@ name = "llamacpp-tests"
77
version = "0.1.0"
88

99
[target.'cfg(linux)'.dependencies.ggml-org]
10-
llamacpp = "b10069"
10+
llamacpp = "b10069.2"
1111

1212
[target.'cfg(macos)'.dependencies.ggml-org]
13-
llamacpp = "b10069"
13+
llamacpp = "b10069.2"
1414

1515
[target.'cfg(windows)'.dependencies.ggml-org]
16-
llamacpp = "b10069"
16+
llamacpp = "b10069.2"

0 commit comments

Comments
 (0)