-
Notifications
You must be signed in to change notification settings - Fork 0
685 lines (659 loc) · 34.8 KB
/
Copy pathbuild.yml
File metadata and controls
685 lines (659 loc) · 34.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
name: five hosts
# ⚠️ THIS WORKFLOW IS THE QUESTION, NOT THE ANSWER.
#
# The mcpp/xlings package index has a stated bar for admitting an emulator, and
# `qemu-riscv`'s descriptor writes it down: prebuilt binaries for the FIVE host
# targets the index serves — linux x64, linux arm64, darwin x64, darwin arm64,
# win32 x64 — from one versioned release, each asset with a checksum sidecar.
#
# xPack publishes QEMU per target family and has no x86 build. qemu.org ships a
# Windows installer only; macOS and Linux are served by distribution packages.
# So no upstream clears the bar, and `xim:qemu-x86` cannot be a repackaging job
# the way `xim:qemu-arm` and `xim:qemu-riscv` are — it has to be built.
#
# ⭐ THE POINT OF DOING IT HERE FIRST IS THAT A FAILURE COSTS A RED CROSS.
#
# The alternative order — write the descriptor, mirror the assets, open the
# index PR, then discover that the Windows leg does not build — produces a
# published package that cannot be installed. GitHub's hosted runners happen to
# cover exactly the five hosts the index serves, so all five legs can be
# attempted in one matrix before anything is published, and the hardest one
# fails early rather than last.
#
# ⚠️ NOTHING HERE PUBLISHES. The artifacts are retained for inspection and for
# measuring what a real payload would weigh. Admission to the index is a
# separate decision, made after five green legs, and it needs the DT_NEEDED /
# otool closure measured per host rather than copied from a sibling descriptor.
on:
push:
branches: [main]
# ⚠️ A TAG BUILDS THE SAME FIVE LEGS AND PUBLISHES THEM. The matrix is not
# duplicated into a separate release workflow: a release built by different
# steps from the ones that were tested is a release nobody tested.
tags: ['[0-9]+.[0-9]+.[0-9]+*']
pull_request:
workflow_dispatch:
env:
# ⚠️ PINNED, AND NOT TO `master`. A payload the index serves must be
# reproducible from a version, and "whatever upstream had that day" is not a
# version. 9.2.4 is the series `qemu-arm` and `qemu-riscv` already carry, so
# a user who installs all three gets one QEMU generation rather than two.
QEMU_VERSION: 9.2.4
jobs:
build:
name: qemu-system-x86_64 on ${{ matrix.label }}
runs-on: ${{ matrix.os }}
timeout-minutes: 90
permissions:
contents: write # the tag path attaches archives to the release
strategy:
fail-fast: false
matrix:
include:
- { label: linux-x64, os: ubuntu-24.04 }
- { label: linux-arm64, os: ubuntu-24.04-arm }
- { label: darwin-arm64, os: macos-14 }
# ⚠️ `macos-15-intel`, NOT `macos-13`. The first attempt used the
# older label and the job sat QUEUED for the whole run without ever
# being scheduled — GitHub has retired that image, and a retired label
# does not fail, it waits. A leg that never runs is indistinguishable
# from a slow one on the summary page, which is the worst of the
# possible outcomes for a matrix whose purpose is to answer questions.
- { label: darwin-x64, os: macos-15-intel }
- { label: win32-x64, os: windows-2022 }
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
# ── The dependencies, per host ────────────────────────────────────────
#
# ⚠️ A SINGLE TARGET IS WHAT MAKES THIS AFFORDABLE. QEMU's full build is
# dozens of system emulators plus tools, docs and UI backends;
# `--target-list=x86_64-softmmu` with the UI and tools disabled is a small
# fraction of it. The index needs one emulator, not a distribution.
- name: Dependencies (Linux)
if: runner.os == 'Linux'
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq \
ninja-build meson pkg-config python3-venv \
libglib2.0-dev libpixman-1-dev zlib1g-dev flex bison
- name: Dependencies (macOS)
if: runner.os == 'macOS'
run: |
brew install ninja meson pkg-config glib pixman
# ⚠️ `distlib`, AND IT IS A PROPERTY OF QEMU 9.2.4 RATHER THAN OF THIS
# RUNNER — MEASURED ON TWO HOSTS BEFORE IT WAS CALLED THAT.
#
# QEMU's `configure` builds a non-isolated virtual environment with its
# own `mkvenv` script, which needs `distlib` to populate it. Neither
# Homebrew's Python 3.14 nor MSYS2's carries it, and the failure is a
# configure-time
#
# *** Ouch! ***
# found no usable distlib, please install it
#
# twenty seconds in, which reads like a QEMU problem and is a Python
# packaging one. The first round fixed it on macOS alone, on the
# reading that Homebrew was unusual; the Windows leg then reached the
# same line and showed it was not.
#
# `--break-system-packages` because Homebrew's Python is
# externally-managed (PEP 668) and refuses otherwise. On a throwaway
# runner that is the right trade; on a developer's machine it is not,
# which is why it is spelled out rather than hidden in a helper.
python3 -m pip install --break-system-packages distlib
# ⚠️ NOT INSTALLING `qemu` ITSELF. Homebrew's formula would pull a
# working emulator and every check below would pass without this
# workflow having built anything — the shape of false green this
# repository exists to avoid.
# ⚠️ THE WINDOWS LEG IS THE ONE THIS WORKFLOW EXISTS TO ANSWER, AND IT IS
# NOT AN ORDINARY BUILD.
#
# QEMU on Windows is built under MSYS2/MinGW, not MSVC: its build system
# is meson and its sources assume a POSIX-ish toolchain. The result is a
# native PE that needs a set of MinGW runtime DLLs beside it, which is why
# the packaging question here is different from the other four hosts —
# `otool`/`readelf` closure has a `ntldd` counterpart and the answer is a
# directory of DLLs rather than an rpath.
- name: Dependencies (Windows / MSYS2)
if: runner.os == 'Windows'
uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: true
install: >-
base-devel
git
python
mingw-w64-ucrt-x86_64-toolchain
mingw-w64-ucrt-x86_64-glib2
mingw-w64-ucrt-x86_64-pixman
mingw-w64-ucrt-x86_64-ninja
mingw-w64-ucrt-x86_64-meson
mingw-w64-ucrt-x86_64-pkgconf
mingw-w64-ucrt-x86_64-zlib
mingw-w64-ucrt-x86_64-python-distlib
mingw-w64-ucrt-x86_64-python-pip
# ⚠️ WINDOWS EXTRACTS UNDER MSYS2 WITH SYMLINK EMULATION FORCED ON, AND IT
# TOOK TWO ROUNDS TO GET THAT RIGHT.
#
# QEMU's source tarball is full of symlinks — `roms/u-boot`, and also
# `tests/lcitool/libvirt-ci`. Git Bash's `tar` fails on all of them, and
# the first fix moved the extraction into MSYS2 on the belief that "MSYS2's
# tar copies instead". It does not: `msys2/setup-msys2` sets
# `MSYS=winsymlinks:nativestrict`, under which MSYS2 asks Windows for a
# REAL symlink — and Windows must know whether the target is a file or a
# directory, so a link created before its target fails with
#
# tar: …/alpine-320-prep.sh: Cannot create symlink to
# 'alpine-prep.sh': No such file or directory
#
# ⚠️ `No such file or directory` AND NOT `Permission denied`, WHICH IS THE
# WHOLE DIAGNOSIS. The runner is privileged enough to create symlinks; the
# problem is ordering, not rights, and a fix aimed at privileges would not
# have worked.
#
# `winsymlinks:sysfile` writes a plain file carrying a marker instead.
# Nothing in this build follows those links, so the emulation is enough.
#
# `roms/` stays excluded regardless: it holds firmware SOURCE for machines
# this build does not produce, and `x86_64-softmmu` takes its firmware
# from the prebuilt blobs in `pc-bios/`.
# The same `distlib` gap as macOS, on MSYS2's Python. Preferring the
# packaged form and falling back to pip: pacman's copy is what an MSYS2
# user would have, and pip is what makes the step work if that package is
# ever renamed.
# ⚠️ TWO PYTHON PACKAGES, AND THE SECOND IS THE SUBTLEST FINDING ON THIS
# LEG.
#
# `distlib` is the same gap macOS had. `pycotap` is different: QEMU
# VENDORS its Python build dependencies as wheels under `python/wheels`
# and `mkvenv` installs from there OFFLINE, never reaching PyPI. On this
# host that directory is handed to pip as
#
# file://D:/a/qemu-x86/qemu-x86/qemu-9.2.4/python/wheels
#
# and pip answers `is ignored: it is neither a file nor a directory` —
# a `file://` URL needs THREE slashes before a drive letter, and with two
# the `D:` is parsed as a HOST. So the vendored wheels are unreachable,
# and because mkvenv is offline there is no fallback:
#
# ERROR: No matching distribution found for pycotap==1.3.1
# mkvenv was configured to operate offline and did not check PyPI.
#
# ⭐ THE FIX IS NOT TO REPAIR THE URL. A non-isolated virtual environment
# sees the system's packages, and `mkvenv`'s own check is "is it
# importable" — which is why `meson` passes on this host and `pycotap`
# does not. Installing it beforehand satisfies the check the same way
# meson already does, and does not depend on how QEMU spells a path.
#
# This is the boundary MSYS2 makes easy to trip over: the shell speaks
# `/d/a/...` and the Python it ships is a NATIVE Windows build that
# speaks `D:/a/...`. This repository met the same boundary from the other
# side in openarch's CI, where `$PWD` in a manifest would have been the
# POSIX form and mcpp wanted the native one.
- name: The Python packages QEMU's mkvenv needs (Windows / MSYS2)
if: runner.os == 'Windows'
shell: msys2 {0}
run: |
set -euo pipefail
# ⚠️ `pip` IS A SEPARATE PACKAGE HERE, AND ASSUMING IT WAS PRESENT COST
# A ROUND. MSYS2's Python ships without it:
#
# python3.exe: No module named pip.__main__;
# 'pip' is a package and cannot be directly executed
#
# so it is now in the install list above rather than reached for here.
# `distlib` is there too, because pacman's copy is what an MSYS2 user
# would have and installing it declaratively is one less step that can
# fail halfway.
python3 -m pip install --no-input pycotap
python3 -c "import distlib, pycotap; print('distlib', distlib.__version__, '/ pycotap ok')"
- name: Fetch QEMU ${{ env.QEMU_VERSION }} (Windows)
if: runner.os == 'Windows'
shell: msys2 {0}
env:
MSYS: winsymlinks:sysfile
run: |
set -euo pipefail
cd "$(cygpath "$GITHUB_WORKSPACE")"
curl -fsSL --retry 5 --retry-all-errors --retry-delay 3 \
-o qemu.tar.xz \
"https://download.qemu.org/qemu-${QEMU_VERSION}.tar.xz"
tar xf qemu.tar.xz --exclude="qemu-${QEMU_VERSION}/roms/*"
test -d "qemu-${QEMU_VERSION}/pc-bios" || { echo "the firmware blobs the build needs are missing"; exit 1; }
- name: Fetch QEMU ${{ env.QEMU_VERSION }}
if: runner.os != 'Windows'
run: |
set -euo pipefail
# ⚠️ `--retry-all-errors`, NOT JUST `--retry`. curl's plain `--retry`
# covers transient HTTP status codes and NOT transport-layer errors,
# and `curl: (52) empty reply from server` is the one this project
# keeps meeting. Without it a truncated download is reported as
# success and the failure surfaces as a corrupt archive.
curl -fsSL --retry 5 --retry-all-errors --retry-delay 3 \
-o qemu.tar.xz \
"https://download.qemu.org/qemu-${QEMU_VERSION}.tar.xz"
curl -fsSL --retry 5 --retry-all-errors --retry-delay 3 \
-o qemu.tar.xz.sig \
"https://download.qemu.org/qemu-${QEMU_VERSION}.tar.xz.sig" || true
tar xf qemu.tar.xz
echo "SRC=$PWD/qemu-${QEMU_VERSION}" >> "$GITHUB_ENV"
- name: Configure and build (Unix)
if: runner.os != 'Windows'
run: |
set -euo pipefail
mkdir -p build && cd build
"$SRC/configure" \
--target-list=x86_64-softmmu \
--prefix="$PWD/../out" \
--disable-docs --disable-guest-agent --disable-tools \
--disable-vnc --disable-sdl --disable-gtk --disable-curses \
--disable-libssh --disable-vde --disable-spice \
--disable-smartcard --disable-usb-redir --disable-opengl \
--disable-virglrenderer --disable-blkio --disable-libdaxctl \
--disable-brlapi --disable-curl
ninja -j"$(getconf _NPROCESSORS_ONLN)" qemu-system-x86_64
# ⚠️ ASSEMBLED EXPLICITLY, THE SAME WAY THE WINDOWS LEG DOES, AND THE
# NUMBERS ARE WHY.
#
# `ninja install` was used here first. Measured against the explicit
# assembly on the same release:
#
# meson install (linux-x64) 392 MB → 343 MB stripped
# explicit (win32-x64) 110 MB → 58 MB stripped
#
# So the install target ships roughly 285 MB that an emulator payload
# has no use for. The index's asset budget is a per-connection upload
# rate measured at 0.012 MB/s into the CN mirror, with no multipart
# and no resume — a payload has to SELECT, and selecting here makes
# the contents explicit rather than "whatever the install target
# copied".
#
# ⚠️ NOTHING IS PRUNED FROM `pc-bios`. Which blobs a machine type
# loads at run time is a question for the descriptor, and answering it
# by deleting files until something breaks is how a payload ends up
# missing one on somebody else's machine.
mkdir -p ../out/bin ../out/share/qemu ../out/lib
cp qemu-system-x86_64 ../out/bin/
cp -r "$SRC/pc-bios/." ../out/share/qemu/
# ⚠️ THE PAYLOAD MUST CARRY WHAT IT LINKS AGAINST, AND MEASURING THAT
# IS WHAT CAUGHT THIS.
#
# An earlier build shipped only the emulator. Its measured closure:
#
# linux libpixman-1.so.0, libglib-2.0.so.0, libgio, libgobject,
# libgmodule, libz.so.1, libzstd.so.1 -- none of them in
# the payload, none of them core glibc
# darwin /opt/homebrew/opt/{pixman,glib,zstd,gnutls,...}/lib/*.dylib
# -- ABSOLUTE paths into the build machine's Homebrew
#
# So that payload installs and then fails on first run for anyone
# without those libraries, and on macOS for anyone whose Homebrew is
# not at the same prefix -- an Intel mac uses /usr/local. The index's
# sibling packages avoid this by bundling: xPack's QEMU carries 51
# shared objects and reaches them through its own relative rpath.
#
# ⚠️ SYSTEM LIBRARIES ARE DELIBERATELY NOT BUNDLED. Core libc, libm
# and the macOS frameworks come from the running system by design;
# carrying a second libc into a process is the two-glibcs shape this
# index's own contract text names as a SIGSEGV source.
if [ "$RUNNER_OS" = "Linux" ]; then
set +e # same reason as the macOS branch below
ldd qemu-system-x86_64 \
| awk '/=> \// {print $3}' \
| grep -vE '/(libc|libm|libdl|librt|libpthread|libresolv|ld-linux[^ ]*)\.so' \
| while read -r so; do cp -Ln "$so" ../out/lib/ 2>/dev/null || true; done
# ⚠️ `patchelf` on the EMULATOR, not on a loader. This project's own
# notes forbid patching a loader's own paths; setting a relative
# rpath on an ordinary executable is the opposite case, and it is
# what makes `out/lib` reachable without an environment variable.
sudo apt-get install -y -qq patchelf
patchelf --set-rpath '$ORIGIN/../lib' ../out/bin/qemu-system-x86_64
for so in ../out/lib/*.so*; do patchelf --set-rpath '$ORIGIN' "$so" 2>/dev/null || true; done
echo "bundled: $(ls -1 ../out/lib 2>/dev/null | wc -l) libraries"
set -e
elif [ "$RUNNER_OS" = "macOS" ]; then
# ⚠️ `set +e` FOR THIS BLOCK, AND THE REASON IS A SILENT KILL.
#
# The first version ran under `set -euo pipefail` and died with exit
# 1 and NO output, right after ninja reported 1564/1564. Every
# command here is in a pipeline ending in `grep`, and a `grep` that
# matches nothing returns 1 — which `pipefail` promotes to the
# pipeline's status and `-e` turns into an exit. A bundling step
# that finds nothing to bundle is not an error; a payload that still
# points outside itself is, and the assertion below is what says so.
set +e
# Mach-O records each dependency by the install name the LIBRARY
# carries, so copying is not enough: every reference has to be
# rewritten, and so does each bundled library's own id.
otool -L qemu-system-x86_64 | tail -n +2 | awk '{print $1}' \
| grep -E '^/(opt|usr/local)/' \
| while read -r dy; do
cp -Ln "$dy" ../out/lib/ 2>/dev/null || true
install_name_tool -change "$dy" "@loader_path/../lib/$(basename "$dy")" \
../out/bin/qemu-system-x86_64 2>/dev/null || true
done
for dy in ../out/lib/*.dylib; do
[ -e "$dy" ] || continue
install_name_tool -id "@loader_path/$(basename "$dy")" "$dy" 2>/dev/null || true
otool -L "$dy" | tail -n +2 | awk '{print $1}' | grep -E '^/(opt|usr/local)/' \
| while read -r dep; do
cp -Ln "$dep" ../out/lib/ 2>/dev/null || true
install_name_tool -change "$dep" "@loader_path/$(basename "$dep")" "$dy" 2>/dev/null || true
done
done
# ⚠️ Rewriting invalidates the ad-hoc signature; re-sign, or the
# binary is killed by the kernel rather than merely warned about.
codesign --force -s - ../out/bin/qemu-system-x86_64 2>/dev/null || true
for dy in ../out/lib/*.dylib; do codesign --force -s - "$dy" 2>/dev/null || true; done
echo "bundled: $(ls -1 ../out/lib 2>/dev/null | wc -l) libraries"
set -e
fi
rmdir ../out/lib 2>/dev/null || true
- name: Configure and build (Windows / MSYS2)
if: runner.os == 'Windows'
shell: msys2 {0}
run: |
set -euo pipefail
cd "$(cygpath "$GITHUB_WORKSPACE")"
mkdir -p build && cd build
"../qemu-${QEMU_VERSION}/configure" \
--target-list=x86_64-softmmu \
--prefix="$PWD/../out" \
--disable-docs --disable-guest-agent --disable-tools \
--disable-vnc --disable-sdl --disable-gtk --disable-curses \
--disable-libssh --disable-spice --disable-smartcard \
--disable-usb-redir --disable-opengl --disable-virglrenderer \
--disable-curl
# ⚠️ THE EMULATOR TARGET, NOT `ninja` — AND ON THIS HOST THAT IS THE
# DIFFERENCE BETWEEN A BUILD AND A FAILURE.
#
# A bare `ninja` builds QEMU's unit tests too, and one of them does not
# link under MinGW:
#
# FAILED: tests/unit/test-vmstate.exe
# test-vmstate.c:48: undefined reference to `qemu_ftruncate64'
#
# 1897 of 2032 targets had already built. That symbol is a QEMU
# portability shim its own test happens to reference and the Windows
# build does not provide; it says nothing about the emulator, which is
# what this workflow is producing.
#
# ⭐ Naming the target rather than patching the test is the smaller
# claim: a package build has no reason to build a test suite, and
# doing so made the whole leg depend on whether upstream's tests
# happen to be portable this release.
# ⚠️ `.exe`, AND OMITTING IT IS NOT A WARNING BUT A HARD STOP:
#
# ninja: error: unknown target 'qemu-system-x86_64'
#
# The Unix legs name the target without a suffix and this one with,
# because meson names the target after the file it produces.
ninja -j"$(nproc)" qemu-system-x86_64.exe
# ⚠️ ASSEMBLED BY HAND RATHER THAN BY `meson install`, AND THE REASON
# GENERALISES BEYOND THIS HOST.
#
# `meson install --no-rebuild` stops at the first GENERATED file the
# install list names but the emulator target did not produce:
#
# ERROR: File 'trace/trace-events-all' could not be found
#
# Naming that file makes it stop at the next one. Dropping
# `--no-rebuild` instead makes meson rebuild everything, which brings
# back the unit test that does not link on MinGW. Neither converges.
#
# ⭐ AND THE INSTALL TARGET IS NOT WHAT A PAYLOAD WANTS ANYWAY.
# Measured on linux-x64: stripping took the emulator from 78.7 MB to
# 27.0 MB while the whole installed prefix went only 392 MB to 343 MB,
# so roughly 316 MB of `meson install`'s output is not the emulator.
# A payload has to SELECT, and doing that here makes the contents
# explicit rather than "whatever the install target happened to copy".
#
# What an x86_64 emulator needs: itself, and the firmware blobs in
# `pc-bios` that its machine types load. Those are prebuilt files in
# the tarball, not build outputs.
mkdir -p ../out/bin ../out/share/qemu
cp qemu-system-x86_64.exe ../out/bin/
cp -r "../qemu-${QEMU_VERSION}/pc-bios/." ../out/share/qemu/
# ⚠️ NOTHING IS DELETED HERE, AND RESISTING THAT IS THE POINT. The
# tempting next step is to prune `keymaps` and the firmware
# descriptors — but which of them a machine type loads at run time is
# a question for the descriptor, and answering it by deleting
# directories until something breaks is how a payload ends up missing
# a blob on somebody else's machine. The whole `pc-bios` tree is
# copied and its size is reported below.
# ⚠️ THE DLL CLOSURE IS THE WINDOWS PACKAGING QUESTION, AND IT IS
# ANSWERED BY MEASUREMENT RATHER THAN BY A LIST. The other four hosts
# resolve their dependencies through an rpath inside the payload; a PE
# has no such thing, so every MinGW runtime library the emulator loads
# has to sit beside it. `ldd` under MSYS2 reports the closure; only
# the ones from the UCRT64 prefix are ours to ship.
ldd qemu-system-x86_64.exe \
| awk '/ucrt64/ {print $3}' \
| while read -r dll; do cp -n "$dll" ../out/bin/ 2>/dev/null || true; done
echo "── the MinGW runtime the emulator needs ──"
ls -1 ../out/bin/*.dll 2>/dev/null | wc -l
# ── The check that the artifact is an emulator, not a file ────────────
#
# ⭐ A BUILT BINARY IS NOT EVIDENCE; A BOOTED IMAGE IS. The index's own
# e2e discipline is "assert the product, not the exit code", and the
# cheapest product here is a multiboot image that prints and powers off.
# It is built by the workflow rather than committed so that nothing in
# this repository is a binary blob whose provenance has to be trusted.
- name: The emulator boots a freestanding image
shell: bash
run: |
set -euo pipefail
QEMU=$(find out -name 'qemu-system-x86_64*' -type f | head -1)
test -n "$QEMU" || { find out -type f | head -40; echo "no emulator was produced"; exit 1; }
"$QEMU" --version | head -1
# A minimal multiboot image. ⚠️ The a.out kludge (flag bit 16) rather
# than plain ELF loading: QEMU's multiboot loader accepts only 32-bit
# ELF images, and this one has to be assembled as 32-bit anyway, so
# the kludge is not strictly needed here — it is used because the
# real consumer (openarch's probe) is a 64-bit image that cannot be
# loaded any other way, and a check that exercised a different path
# would not be checking the same thing.
cat > probe.S <<'ASM'
.set MAGIC, 0x1BADB002
.set FLAGS, 0x00010000
.set CHECKSUM, -(MAGIC + FLAGS)
.section .multiboot,"a"
.align 4
mb: .long MAGIC
.long FLAGS
.long CHECKSUM
.long mb
.long __load_start
.long __load_end
.long __bss_end
.long _start
.code32
.section .text
.globl _start
_start:
cli
movl $msg, %esi
1: lodsb
testb %al, %al
je 2f
movw $0x3F8, %dx
outb %al, %dx
jmp 1b
2: movw $0x604, %dx
movw $0x2000, %ax
outw %ax, %dx
3: hlt
jmp 3b
.section .rodata
msg: .asciz "qemu-x86 probe ok\n"
.section .bss
.space 16
ASM
cat > probe.ld <<'LD'
ENTRY(_start)
SECTIONS {
__load_start = 0x100000;
. = __load_start + SIZEOF_HEADERS;
.multiboot : { KEEP(*(.multiboot)) }
.text : { *(.text*) }
.rodata : { *(.rodata*) }
__load_end = .;
.bss : { *(.bss*) __bss_end = .; }
}
LD
# ⚠️ THE GATE IS "CAN THIS HOST LINK AN x86 ELF", AND MY FIRST VERSION
# ASKED THE WRONG QUESTION.
#
# It checked for a 32-bit COMPILER and let the link run. On Windows
# that passes — MinGW's `cc -m32` happily produces an object — and
# then the link fails, because the linker beside it emits PE and
# nothing else:
#
# ld.exe: unrecognised emulation mode: elf_i386
# Supported emulations: i386pep i386pe
#
# An arm64 or macOS runner fails earlier for a different reason. One
# check covers all of them: ask the LINKER whether it knows the
# emulation, which is the step that actually has to work.
if ! ld -V 2>/dev/null | grep -q "elf_i386"; then
echo "::notice::this host's linker emits no x86 ELF (supported: $(ld -V 2>/dev/null | tr '\n' ' ' | head -c 120)); the emulator's --version is the whole of the check on this leg"
exit 0
fi
cc -m32 -c -o probe.o probe.S -nostdlib 2>/dev/null \
|| { echo "::notice::no 32-bit x86 assembler on this leg; --version is the check"; exit 0; }
ld -m elf_i386 -T probe.ld -o probe.elf probe.o
timeout -k 5 60 "$QEMU" -machine q35 -nographic -no-reboot -kernel probe.elf 2>&1 | tee run.log
grep -q "qemu-x86 probe ok" run.log
# ⚠️ WHAT A PAYLOAD WOULD ACTUALLY WEIGH, MEASURED RATHER THAN ESTIMATED.
# The index's size discipline came from a real incident: a payload was
# 34.81 MB and became 4.62 MB once stripped, and the upload failures that
# had been blamed on the mirror were the size.
- name: What the payload weighs, and what it needs
shell: bash
run: |
set -euo pipefail
QEMU=$(find out -name 'qemu-system-x86_64*' -type f | head -1)
echo "before stripping:"; ls -l "$QEMU"; du -sh out
# ⚠️ STRIPPED, BECAUSE THE UNSTRIPPED FIGURE IS NOT A PAYLOAD SIZE AND
# THE INDEX HAS PAID FOR THAT CONFUSION ONCE ALREADY.
#
# Measured on the first green legs: 392M (linux-x64), 389M
# (linux-arm64), 341M/342M (darwin). Those are debug symbols, not an
# emulator. A previous payload in this ecosystem was 34.81 MB and
# became 4.62 MB once stripped, and the upload failures that had been
# blamed on the mirror were the size.
#
# ⚠️ NOT ON macOS. Stripping a Mach-O invalidates its ad-hoc
# signature, which is the rule this ecosystem's own release pipeline
# follows; the darwin legs report the unstripped figure and say so.
case "$RUNNER_OS" in
Linux)
find out -type f \( -name 'qemu-system-*' -o -name '*.so*' \) -exec strip --strip-unneeded {} + 2>/dev/null || true
echo "after stripping:"; ls -l "$QEMU"; du -sh out
readelf -d "$QEMU" | grep NEEDED || true ;;
macOS)
echo "(not stripped: it would invalidate the ad-hoc signature)"
otool -L "$QEMU" || true ;;
Windows)
find out -type f -name '*.exe' -exec strip --strip-unneeded {} + 2>/dev/null || true
echo "after stripping:"; ls -l "$QEMU"; du -sh out
echo "(the DLL closure is measured in the packaging step, not here)" ;;
esac
# ⭐ WHAT IS ACTUALLY BIG, BECAUSE STRIPPING ANSWERED LESS THAN
# EXPECTED. Measured on linux-x64: the emulator went 78.7 MB → 27.0
# MB, and the whole prefix went 392 MB → 343 MB. So roughly 316 MB is
# not the emulator at all, and a payload that tarred this prefix would
# ship it. The index's asset budget is a per-connection upload rate
# measured at 0.012 MB/s into the CN mirror, with no multipart and no
# resume — so what a payload SELECTS matters more here than what it
# strips.
#
# Listed rather than pruned: which of these an emulator actually needs
# at run time is a question for the descriptor, and answering it by
# deleting directories until something breaks is how a payload ends up
# missing a firmware blob on somebody else's machine.
# ⭐ THE PAYLOAD MUST NOT REACH OUTSIDE ITSELF, ASSERTED RATHER THAN
# ASSUMED. This is the check that would have failed on the first
# build, and it fails loudly instead of at somebody's first run.
echo "── every library the emulator names ──"
case "$RUNNER_OS" in
Linux)
readelf -d out/bin/qemu-system-x86_64 | grep -E 'NEEDED|RUNPATH|RPATH' || true
# Resolve the closure the way the loader will, from the payload.
bad=$(ldd out/bin/qemu-system-x86_64 2>/dev/null \
| awk '/=> \// {print $3}' \
| grep -vE "^$PWD/out/" \
| grep -vE '/(libc|libm|libdl|librt|libpthread|libresolv|ld-linux[^ ]*)\.so' || true)
if [ -n "$bad" ]; then
echo "$bad"
echo "::error::the payload links against libraries it does not carry; installing it would succeed and running it would not"
exit 1
fi
echo "resolved entirely inside the payload, plus core glibc" ;;
macOS)
otool -L out/bin/qemu-system-x86_64 | tail -n +2 | awk '{print $1}'
bad=$(otool -L out/bin/qemu-system-x86_64 | tail -n +2 | awk '{print $1}' \
| grep -E '^/(opt|usr/local)/' || true)
if [ -n "$bad" ]; then
echo "$bad"
echo "::error::the payload names absolute paths into this build machine's package manager; it would fail on any host without the same prefix"
exit 1
fi
echo "no build-machine paths remain" ;;
Windows)
echo "(the DLL closure sits beside the exe; counted above)" ;;
esac
echo "── the ten largest things under the prefix ──"
du -sh out/* 2>/dev/null | sort -rh | head -10
du -sh out/share/* 2>/dev/null | sort -rh | head -10
# ── The publishable archive ────────────────────────────────────────
#
# ⚠️ NAMED FOR THE INDEX, NOT FOR THIS REPOSITORY. `xim`'s descriptor
# builds the URL from `${version}` and a platform/arch pair, so the file
# name has to be derivable from those alone — `qemu-x86-<ver>-<host>.<ext>`
# with the same `linux`/`darwin`/`win32` and `x64`/`arm64` spellings
# `qemu-riscv` uses. A name only this workflow understands would force the
# descriptor to special-case it.
#
# The sidecar carries the same hash the descriptor will quote, computed
# here from the archive that is actually uploaded rather than recomputed
# later from a download.
- name: Pack the archive
shell: bash
run: |
set -euo pipefail
V="${GITHUB_REF_NAME}"
case "$V" in refs/*|main) V="${QEMU_VERSION}" ;; esac
NAME="qemu-x86-${V}-${{ matrix.label }}"
mkdir -p dist
if [ "$RUNNER_OS" = "Windows" ]; then
( cd out && 7z a -tzip "../dist/${NAME}.zip" . > /dev/null )
F="dist/${NAME}.zip"
else
tar czf "dist/${NAME}.tar.gz" -C out .
F="dist/${NAME}.tar.gz"
fi
# ⚠️ `shasum -a 256` rather than `sha256sum`: macOS has the former and
# not the latter, and a fallback chain here is one line against a leg
# that would otherwise fail after a 40-minute build.
if command -v sha256sum > /dev/null; then sha256sum "$F" > "$F.sha256";
else shasum -a 256 "$F" > "$F.sha256"; fi
ls -l dist/
cat "$F.sha256"
- uses: actions/upload-artifact@v4
with:
name: qemu-x86-${{ matrix.label }}
path: dist
retention-days: 14
# ⚠️ ONLY ON A TAG, AND ONLY AFTER THE CHECKS ABOVE. A leg that built but
# did not produce a runnable emulator must not reach a release.
- name: Publish to the release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v2
with:
files: dist/*
fail_on_unmatched_files: true