Skip to content

Commit 3910fea

Browse files
speak-agentspeak-agent
committed
五宿主构建矩阵:先问,再发布
索引收录模拟器的门槛写在 qemu-riscv 的描述符里:为它服务的五个宿主目标从同一个 版本化发布提供预编译件,每个资产带校验侧文件。xPack 按目标族发布 QEMU 而没有 x86 构建,qemu.org 只出 Windows 安装器 —— 没有上游过得了这条线。 ⭐ 先在这里跑五条腿,是因为**失败在这里只是一个红叉**。反过来的顺序是先写描述符、 先镜像资产、先开索引 PR,然后才发现某个宿主构建不出来 —— 那产出的是一个装得上却 用不了的包。GitHub 托管 runner 恰好覆盖索引服务的那五个宿主。 ⚠️ 最难的一条是 Windows,而它难在结构上:QEMU 在 Windows 上由 MSYS2/MinGW 构建 而非 MSVC,产物是需要一组 MinGW 运行时 DLL 在旁边的原生 PE —— 打包问题是一个 DLL 目录而不是一条 rpath,不能照抄 Linux 那条腿。 本仓库刻意不含:索引描述符、二进制、镜像上传。
0 parents  commit 3910fea

2 files changed

Lines changed: 340 additions & 0 deletions

File tree

‎.github/workflows/build.yml‎

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
name: five hosts
2+
3+
# ⚠️ THIS WORKFLOW IS THE QUESTION, NOT THE ANSWER.
4+
#
5+
# The mcpp/xlings package index has a stated bar for admitting an emulator, and
6+
# `qemu-riscv`'s descriptor writes it down: prebuilt binaries for the FIVE host
7+
# targets the index serves — linux x64, linux arm64, darwin x64, darwin arm64,
8+
# win32 x64 — from one versioned release, each asset with a checksum sidecar.
9+
#
10+
# xPack publishes QEMU per target family and has no x86 build. qemu.org ships a
11+
# Windows installer only; macOS and Linux are served by distribution packages.
12+
# So no upstream clears the bar, and `xim:qemu-x86` cannot be a repackaging job
13+
# the way `xim:qemu-arm` and `xim:qemu-riscv` are — it has to be built.
14+
#
15+
# ⭐ THE POINT OF DOING IT HERE FIRST IS THAT A FAILURE COSTS A RED CROSS.
16+
#
17+
# The alternative order — write the descriptor, mirror the assets, open the
18+
# index PR, then discover that the Windows leg does not build — produces a
19+
# published package that cannot be installed. GitHub's hosted runners happen to
20+
# cover exactly the five hosts the index serves, so all five legs can be
21+
# attempted in one matrix before anything is published, and the hardest one
22+
# fails early rather than last.
23+
#
24+
# ⚠️ NOTHING HERE PUBLISHES. The artifacts are retained for inspection and for
25+
# measuring what a real payload would weigh. Admission to the index is a
26+
# separate decision, made after five green legs, and it needs the DT_NEEDED /
27+
# otool closure measured per host rather than copied from a sibling descriptor.
28+
29+
on:
30+
push:
31+
branches: [main]
32+
pull_request:
33+
workflow_dispatch:
34+
35+
env:
36+
# ⚠️ PINNED, AND NOT TO `master`. A payload the index serves must be
37+
# reproducible from a version, and "whatever upstream had that day" is not a
38+
# version. 9.2.4 is the series `qemu-arm` and `qemu-riscv` already carry, so
39+
# a user who installs all three gets one QEMU generation rather than two.
40+
QEMU_VERSION: 9.2.4
41+
42+
jobs:
43+
build:
44+
name: qemu-system-x86_64 on ${{ matrix.label }}
45+
runs-on: ${{ matrix.os }}
46+
timeout-minutes: 90
47+
strategy:
48+
fail-fast: false
49+
matrix:
50+
include:
51+
- { label: linux-x64, os: ubuntu-24.04 }
52+
- { label: linux-arm64, os: ubuntu-24.04-arm }
53+
- { label: darwin-arm64, os: macos-14 }
54+
- { label: darwin-x64, os: macos-13 }
55+
- { label: win32-x64, os: windows-2022 }
56+
defaults:
57+
run:
58+
shell: bash
59+
60+
steps:
61+
- uses: actions/checkout@v4
62+
63+
# ── The dependencies, per host ────────────────────────────────────────
64+
#
65+
# ⚠️ A SINGLE TARGET IS WHAT MAKES THIS AFFORDABLE. QEMU's full build is
66+
# dozens of system emulators plus tools, docs and UI backends;
67+
# `--target-list=x86_64-softmmu` with the UI and tools disabled is a small
68+
# fraction of it. The index needs one emulator, not a distribution.
69+
- name: Dependencies (Linux)
70+
if: runner.os == 'Linux'
71+
run: |
72+
sudo apt-get update -qq
73+
sudo apt-get install -y -qq \
74+
ninja-build meson pkg-config python3-venv \
75+
libglib2.0-dev libpixman-1-dev zlib1g-dev flex bison
76+
77+
- name: Dependencies (macOS)
78+
if: runner.os == 'macOS'
79+
run: |
80+
brew install ninja meson pkg-config glib pixman
81+
# ⚠️ NOT INSTALLING `qemu` ITSELF. Homebrew's formula would pull a
82+
# working emulator and every check below would pass without this
83+
# workflow having built anything — the shape of false green this
84+
# repository exists to avoid.
85+
86+
# ⚠️ THE WINDOWS LEG IS THE ONE THIS WORKFLOW EXISTS TO ANSWER, AND IT IS
87+
# NOT AN ORDINARY BUILD.
88+
#
89+
# QEMU on Windows is built under MSYS2/MinGW, not MSVC: its build system
90+
# is meson and its sources assume a POSIX-ish toolchain. The result is a
91+
# native PE that needs a set of MinGW runtime DLLs beside it, which is why
92+
# the packaging question here is different from the other four hosts —
93+
# `otool`/`readelf` closure has a `ntldd` counterpart and the answer is a
94+
# directory of DLLs rather than an rpath.
95+
- name: Dependencies (Windows / MSYS2)
96+
if: runner.os == 'Windows'
97+
uses: msys2/setup-msys2@v2
98+
with:
99+
msystem: UCRT64
100+
update: true
101+
install: >-
102+
base-devel
103+
git
104+
python
105+
mingw-w64-ucrt-x86_64-toolchain
106+
mingw-w64-ucrt-x86_64-glib2
107+
mingw-w64-ucrt-x86_64-pixman
108+
mingw-w64-ucrt-x86_64-ninja
109+
mingw-w64-ucrt-x86_64-meson
110+
mingw-w64-ucrt-x86_64-pkgconf
111+
mingw-w64-ucrt-x86_64-zlib
112+
113+
- name: Fetch QEMU ${{ env.QEMU_VERSION }}
114+
run: |
115+
set -euo pipefail
116+
# ⚠️ `--retry-all-errors`, NOT JUST `--retry`. curl's plain `--retry`
117+
# covers transient HTTP status codes and NOT transport-layer errors,
118+
# and `curl: (52) empty reply from server` is the one this project
119+
# keeps meeting. Without it a truncated download is reported as
120+
# success and the failure surfaces as a corrupt archive.
121+
curl -fsSL --retry 5 --retry-all-errors --retry-delay 3 \
122+
-o qemu.tar.xz \
123+
"https://download.qemu.org/qemu-${QEMU_VERSION}.tar.xz"
124+
curl -fsSL --retry 5 --retry-all-errors --retry-delay 3 \
125+
-o qemu.tar.xz.sig \
126+
"https://download.qemu.org/qemu-${QEMU_VERSION}.tar.xz.sig" || true
127+
tar xf qemu.tar.xz
128+
echo "SRC=$PWD/qemu-${QEMU_VERSION}" >> "$GITHUB_ENV"
129+
130+
- name: Configure and build (Unix)
131+
if: runner.os != 'Windows'
132+
run: |
133+
set -euo pipefail
134+
mkdir -p build && cd build
135+
"$SRC/configure" \
136+
--target-list=x86_64-softmmu \
137+
--prefix="$PWD/../out" \
138+
--disable-docs --disable-guest-agent --disable-tools \
139+
--disable-vnc --disable-sdl --disable-gtk --disable-curses \
140+
--disable-libssh --disable-vde --disable-spice \
141+
--disable-smartcard --disable-usb-redir --disable-opengl \
142+
--disable-virglrenderer --disable-blkio --disable-libdaxctl \
143+
--disable-brlapi --disable-curl
144+
ninja -j"$(getconf _NPROCESSORS_ONLN)"
145+
ninja install
146+
147+
- name: Configure and build (Windows / MSYS2)
148+
if: runner.os == 'Windows'
149+
shell: msys2 {0}
150+
run: |
151+
set -euo pipefail
152+
cd "$(cygpath "$GITHUB_WORKSPACE")"
153+
mkdir -p build && cd build
154+
"../qemu-${QEMU_VERSION}/configure" \
155+
--target-list=x86_64-softmmu \
156+
--prefix="$PWD/../out" \
157+
--disable-docs --disable-guest-agent --disable-tools \
158+
--disable-vnc --disable-sdl --disable-gtk --disable-curses \
159+
--disable-libssh --disable-spice --disable-smartcard \
160+
--disable-usb-redir --disable-opengl --disable-virglrenderer \
161+
--disable-curl
162+
ninja -j"$(nproc)"
163+
ninja install
164+
165+
# ── The check that the artifact is an emulator, not a file ────────────
166+
#
167+
# ⭐ A BUILT BINARY IS NOT EVIDENCE; A BOOTED IMAGE IS. The index's own
168+
# e2e discipline is "assert the product, not the exit code", and the
169+
# cheapest product here is a multiboot image that prints and powers off.
170+
# It is built by the workflow rather than committed so that nothing in
171+
# this repository is a binary blob whose provenance has to be trusted.
172+
- name: The emulator boots a freestanding image
173+
shell: bash
174+
run: |
175+
set -euo pipefail
176+
QEMU=$(find out -name 'qemu-system-x86_64*' -type f | head -1)
177+
test -n "$QEMU" || { find out -type f | head -40; echo "no emulator was produced"; exit 1; }
178+
"$QEMU" --version | head -1
179+
180+
# A minimal multiboot image. ⚠️ The a.out kludge (flag bit 16) rather
181+
# than plain ELF loading: QEMU's multiboot loader accepts only 32-bit
182+
# ELF images, and this one has to be assembled as 32-bit anyway, so
183+
# the kludge is not strictly needed here — it is used because the
184+
# real consumer (openarch's probe) is a 64-bit image that cannot be
185+
# loaded any other way, and a check that exercised a different path
186+
# would not be checking the same thing.
187+
cat > probe.S <<'ASM'
188+
.set MAGIC, 0x1BADB002
189+
.set FLAGS, 0x00010000
190+
.set CHECKSUM, -(MAGIC + FLAGS)
191+
.section .multiboot,"a"
192+
.align 4
193+
mb: .long MAGIC
194+
.long FLAGS
195+
.long CHECKSUM
196+
.long mb
197+
.long __load_start
198+
.long __load_end
199+
.long __bss_end
200+
.long _start
201+
.code32
202+
.section .text
203+
.globl _start
204+
_start:
205+
cli
206+
movl $msg, %esi
207+
1: lodsb
208+
testb %al, %al
209+
je 2f
210+
movw $0x3F8, %dx
211+
outb %al, %dx
212+
jmp 1b
213+
2: movw $0x604, %dx
214+
movw $0x2000, %ax
215+
outw %ax, %dx
216+
3: hlt
217+
jmp 3b
218+
.section .rodata
219+
msg: .asciz "qemu-x86 probe ok\n"
220+
.section .bss
221+
.space 16
222+
ASM
223+
cat > probe.ld <<'LD'
224+
ENTRY(_start)
225+
SECTIONS {
226+
__load_start = 0x100000;
227+
. = __load_start + SIZEOF_HEADERS;
228+
.multiboot : { KEEP(*(.multiboot)) }
229+
.text : { *(.text*) }
230+
.rodata : { *(.rodata*) }
231+
__load_end = .;
232+
.bss : { *(.bss*) __bss_end = .; }
233+
}
234+
LD
235+
# The host's own toolchain assembles it; this step is about the
236+
# emulator, not about cross-compilation.
237+
if [ "$RUNNER_OS" = "macOS" ] || [ "$(uname -m)" = "aarch64" ] || [ "$(uname -m)" = "arm64" ]; then
238+
echo "::notice::no host assembler emits x86 ELF here; the emulator's --version is the check on this leg"
239+
exit 0
240+
fi
241+
cc -m32 -c -o probe.o probe.S -nostdlib 2>/dev/null \
242+
|| { echo "::notice::no 32-bit x86 assembler on this leg; --version is the check"; exit 0; }
243+
ld -m elf_i386 -T probe.ld -o probe.elf probe.o
244+
timeout -k 5 60 "$QEMU" -machine q35 -nographic -no-reboot -kernel probe.elf 2>&1 | tee run.log
245+
grep -q "qemu-x86 probe ok" run.log
246+
247+
# ⚠️ WHAT A PAYLOAD WOULD ACTUALLY WEIGH, MEASURED RATHER THAN ESTIMATED.
248+
# The index's size discipline came from a real incident: a payload was
249+
# 34.81 MB and became 4.62 MB once stripped, and the upload failures that
250+
# had been blamed on the mirror were the size.
251+
- name: What the payload weighs, and what it needs
252+
shell: bash
253+
run: |
254+
set -euo pipefail
255+
QEMU=$(find out -name 'qemu-system-x86_64*' -type f | head -1)
256+
ls -l "$QEMU"
257+
case "$RUNNER_OS" in
258+
Linux) readelf -d "$QEMU" | grep NEEDED || true ;;
259+
macOS) otool -L "$QEMU" || true ;;
260+
Windows) echo "(the DLL closure is measured in the packaging step, not here)" ;;
261+
esac
262+
du -sh out
263+
264+
- uses: actions/upload-artifact@v4
265+
with:
266+
name: qemu-system-x86_64-${{ matrix.label }}
267+
path: out
268+
retention-days: 14

‎README.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# qemu-x86
2+
3+
Cross-host builds of `qemu-system-x86_64` for the mcpp/xlings package index.
4+
5+
**Status: a question being asked, not a package.** Nothing here is published,
6+
and nothing here should be added to the index until the workflow is green on
7+
all five hosts.
8+
9+
## Why this repository exists
10+
11+
The index has a stated bar for admitting an emulator, written down in
12+
`qemu-riscv`'s descriptor: prebuilt binaries for the five host targets the index
13+
serves — linux x64, linux arm64, darwin x64, darwin arm64, win32 x64 — from one
14+
versioned release, each asset carrying a checksum sidecar.
15+
16+
`xim:qemu-arm` and `xim:qemu-riscv` clear that bar because xPack publishes them.
17+
xPack builds QEMU per target family and has **no x86 build**; qemu.org ships a
18+
Windows installer only, and macOS and Linux are served by distribution packages.
19+
So no upstream clears it, and `xim:qemu-x86` cannot be a repackaging job the way
20+
its two siblings are. It has to be built.
21+
22+
## Why the order is "build first, publish later"
23+
24+
⭐ **A failure here costs a red cross. A failure after publication costs a
25+
package that installs and does not work.**
26+
27+
The tempting order is to write the descriptor, mirror the assets and open the
28+
index pull request, and to discover only then that one host does not build.
29+
GitHub's hosted runners happen to cover exactly the five hosts the index serves,
30+
so all five legs can be attempted in one matrix before anything is published —
31+
and the hardest of them fails early rather than last.
32+
33+
⚠️ **The hardest is Windows, and it is hardest for a structural reason.** QEMU on
34+
Windows is built under MSYS2/MinGW rather than MSVC: its build system is meson
35+
and its sources assume a POSIX-ish toolchain. What comes out is a native PE that
36+
needs a set of MinGW runtime DLLs beside it — so the packaging question there is
37+
a DLL directory rather than an rpath, and it is not answered by copying what the
38+
Linux leg does.
39+
40+
## What is deliberately not here
41+
42+
| | |
43+
|---|---|
44+
| An index descriptor | Admission is a separate decision, made after five green legs |
45+
| Committed binaries | The workflow builds what it checks, so nothing here is a blob whose provenance has to be trusted |
46+
| A mirror upload | Mirroring an artifact that has not been shown to run would publish the untested thing faster |
47+
48+
## What "it works" means here
49+
50+
A built binary is not evidence. Each leg that can do so boots a minimal
51+
multiboot image that prints over the serial port and powers the machine off, and
52+
asserts the printed line — the same discipline the rest of this ecosystem's
53+
end-to-end tests use: assert the product, not the exit code.
54+
55+
⚠️ Two legs cannot do that and say so rather than passing quietly: an arm64 or
56+
macOS runner has no host assembler that emits 32-bit x86 ELF, so on those the
57+
emulator's own `--version` is the whole of the check. That is a weaker claim,
58+
and it is recorded as one.
59+
60+
## The single-target build
61+
62+
```
63+
--target-list=x86_64-softmmu
64+
```
65+
66+
QEMU's full build is dozens of system emulators plus tools, documentation and UI
67+
backends. The index needs one emulator, not a distribution, and a single target
68+
with the UI and tools disabled is a small fraction of the work and of the
69+
resulting size.
70+
71+
The version is pinned to the series `qemu-arm` and `qemu-riscv` already carry, so
72+
that a user who installs all three gets one QEMU generation rather than two.

0 commit comments

Comments
 (0)