Skip to content
CI

Measure starting a program that needs an interpreter #99

Sign in to view logs

Measure starting a program that needs an interpreter

Measure starting a program that needs an interpreter #99

Workflow file for this run

.github/workflows/ci.yml at 8e03613
name: CI
# What this workflow asserts.
#
# conformance the suite in the specification package runs against this
# implementation, on both architectures, and every observation
# holds
# surface the exported names are exactly the fifty-one the
# specification lists
# independence the objects reference nothing of a C library but the two names
# no C library defines
#
# The third is the property version 0.3 exists for. A program above openkal may
# supply every name the system's own library supplies; an implementation that
# called one of them would have its calls resolve to the program's, and the
# program's would resolve back here.
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
inputs:
mcpp_ref:
description: "Branch of mcpp-community/mcpp to build and test against (empty = the released pin)"
required: false
default: ""
env:
MCPP_SOURCE_REF: ${{ github.event.inputs.mcpp_ref || vars.MCPP_SOURCE_REF }}
MCPP_VERSION: 2026.8.27.1
XLINGS_VERSION: v2026.8.17.2
XLINGS_NON_INTERACTIVE: '1'
jobs:
conformance:
name: conformance (${{ matrix.os }}, ${{ matrix.toolchain }})
runs-on: ${{ matrix.os }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
# arm64 only, and not because the other architecture is unsupported.
# The build tool has no release for x86_64 on this system --- its
# installer answers "Unsupported release target: macosx-x86_64" --- so
# the suite cannot be built there at all. What can be done there is
# done: the `numbers' workflow measures that kernel's own table on
# both architectures, and the step below compiles every source for the
# other one with the system's compiler.
- { os: macos-14, toolchain: 'llvm@20.1.7' }
steps:
- uses: actions/checkout@v4
# The specification is checked out at the branch under test where it has
# one, so that this run asserts what it is for: that the specification as
# written there and this implementation as written here agree today.
- name: The specification
run: |
git clone --quiet https://github.com/mcpplibs/openkal.git .spec
branch='${{ github.head_ref || github.ref_name }}'
if git -C .spec rev-parse --verify --quiet "origin/$branch" > /dev/null; then
git -C .spec checkout --quiet "origin/$branch"
echo "the specification is at $branch"
else
echo "the specification has no $branch; its default branch is used"
fi
- name: Install xlings
run: |
curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh \
| bash -s "$XLINGS_VERSION"
echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH"
- name: Install mcpp
run: |
xlings update
# ⚠️ THE PIN MAY NAME THE RELEASE THIS RUN IS VALIDATING, which does
# not exist yet — that is the whole point of MCPP_SOURCE_REF. Bootstrap
# from whatever the index has; the step below replaces it with the
# build under review, and the pin is what an ordinary run tests.
if [ -n "${MCPP_SOURCE_REF:-}" ]; then
xlings install mcpp -y -g
else
xlings install "mcpp@$MCPP_VERSION" -y -g
fi
# ⚠️⚠️ TRANSITION: GIVE THE BOOTSTRAP THE glibc ITS BINDING NAMES.
#
# `xim:glibc`'s `latest` moved from `2.44` to `2.44.2`. A payload
# directory is named after the version a request RESOLVED to, while a
# RuntimeBinding carries the version that was DECLARED — and the xlings
# a released mcpp vendors into its own sandbox still declares `2.44`.
# So a clean machine installs `2.44.2`, the toolchain fixup asks for
# `2.44`, and the build stops before anything is compiled:
#
# error: selected RuntimeBinding glibc@2.44 requires payload
# '…/xpkgs/xim-x-glibc/2.44', but it is not installed
#
# ⚠️ On every NEW machine and on none that already existed, which is why
# it is invisible from a developer's own. Measured on `main` as readily
# as on any branch — the index records the same failure verbatim in
# `pkgs/g/glibc.lua` and states the rule it broke: "The index is DATA
# and the client is a PROGRAM: the consumer ships first."
#
# ⭐ REMOVE THIS once a released mcpp resolves it. `mcpp 2026.8.27.1`
# accepts an installed payload whose version REFINES the requested one
# (`payload_dir_for_version`), so a bootstrap from it needs nothing
# here. Until then the missing payload is simply installed.
if [ -x "$HOME/.mcpp/registry/bin/xlings" ]; then
XLINGS_HOME="$HOME/.mcpp/registry" XLINGS_NON_INTERACTIVE=1 \
"$HOME/.mcpp/registry/bin/xlings" install glibc@2.44 -y -g \
>/dev/null 2>&1 || true
echo "glibc payloads present: $(ls "$HOME/.mcpp/registry/data/xpkgs/xim-x-glibc" 2>/dev/null | tr '\n' ' ')"
fi
mcpp --version
mcpp self config --mirror GLOBAL
# ⭐⭐ CROSS-VALIDATION: BUILD THE mcpp UNDER REVIEW AND USE THAT ONE.
#
# Empty in the ordinary run, so this job keeps testing the RELEASED
# mcpp the pin above names. Set it — `workflow_dispatch` input, or the
# repository variable — and the same job runs against that source.
#
# ⚠️ THIS EXISTS BECAUSE THE ORDER USED TO BE WRONG. Several mcpp
# releases went out green and only then turned this ecosystem red: the
# engine's own CI cannot see a defect that appears only in a real
# dependency graph, and this repository could not see the engine until
# it had been published. Validating before the release closes that gap.
#
# The released mcpp installed just above is the bootstrap that compiles
# it; mcpp builds itself and there is no other compiler for it here.
if [ -n "${MCPP_SOURCE_REF:-}" ]; then
src="$RUNNER_TEMP/mcpp-src"
[ -d "$src" ] || git clone --quiet --depth 1 \
--branch "$MCPP_SOURCE_REF" \
https://github.com/mcpp-community/mcpp.git "$src"
# ⚠️ THE CLONE'S OWN WORKSPACE PIN MUST NOT DECIDE WHICH mcpp
# BUILDS IT. `.xlings.json` at mcpp's root pins the mcpp that
# compiles mcpp, and that pin does not move when mcpp is released —
# so a build inside the checkout obeys it and tries to install a
# version the index may no longer carry:
#
# [error] xlings: version '2026.8.17.1' not found for 'mcpp'
# available: 2026.8.25.1
#
# What is wanted here is the source compiled by the mcpp installed
# above, which is exactly what removing the file leaves.
rm -f "$src/.xlings.json"
( cd "$src" && mcpp build --release )
# ⚠️ BOTH SPELLINGS, AND NO `-perm`. The matrix reaches Windows and
# macOS runners too: on Windows the artefact is `mcpp.exe`, and
# `-perm -u+x` is not a question that filesystem answers the way this
# expects. Measured: `Finished release [optimized] in 173.44s`
# followed by "mcpp did not build" — the build had succeeded and the
# search was looking for the wrong name.
#
# `$src` is a FRESH clone each run, so `target/` holds exactly what
# this step just built; `-printf` would be the safer form on a cached
# tree and is a GNU extension this must not use.
built=$(find "$src/target" -type f \
\( -name mcpp -o -name mcpp.exe \) | head -1)
[ -n "$built" ] || { echo "::error::mcpp did not build from $MCPP_SOURCE_REF"; exit 1; }
echo "$(cd "$(dirname "$built")" && pwd)" >> "$GITHUB_PATH"
# ⚠️ Reported, because a PATH entry that does not win looks exactly
# like one that does until something built with the wrong engine.
echo "under review: $("$built" --version) (from $MCPP_SOURCE_REF)"
fi
# The compiler family and version for this row. mcpp keeps its toolchains
# in a sandbox of its own, so this selects rather than installs into the
# system, and `mcpp test' and `mcpp run' have no flag for it --- which is
# why it is set once here rather than passed to each command.
- name: Select the toolchain
run: |
spec='${{ matrix.toolchain }}'
case "$spec" in
msvc*) mcpp toolchain default msvc ;;
*) mcpp toolchain install "${spec%@*}" "${spec#*@}"
mcpp toolchain default "$spec" ;;
esac
mcpp toolchain list
- name: Every interface, every kind of examination
run: |
bash .spec/tools/run-conformance.sh openkal-macos . full,optional
# ⚠️⚠️ CLONING THE SPECIFICATION IS NOT THE SAME AS BUILDING AGAINST IT.
#
# `.spec` is cloned at the top of this job and consumed by the script
# above, which substitutes this manifest itself and RESTORES IT ON EXIT
# --- correctly, since a script that rewrote a checked-in file and walked
# away would leave the tree holding a path. But every step BELOW calls
# `mcpp build` directly, and by then the manifest names `openkal` by
# version again, so they resolved the PUBLISHED specification:
#
# E_NOT_FOUND: package 'compat.openkal@0.9.0' not found in the synced
# index ... the index is current, so this name is either wrong or not
# published yet
#
# ⭐⭐ THE UNIT IS THE STEP, NOT THE JOB, AND NOT THE REPOSITORY. Measured
# 2026-08-28 across the eight repositories of this ecosystem: eight jobs
# in four of them had this shape. An audit that asked "does this job
# substitute?" passed this one, because it does --- and then gives it
# back. These steps are green on `main` and can only be green there,
# because there the published version IS the one under test.
- name: Point at the specification's working tree
run: |
set -euo pipefail
# ⚠️ NOT `sed -i'. This step runs on macOS and on Windows too, and
# BSD sed requires an argument to -i that GNU sed refuses. A temporary
# file is the spelling that holds on all three.
subst() { # subst <file> <relative-path-to-the-specification>
sed "s|^openkal = .*$|openkal = { path = \"$2\" }|" "$1" > "$1.next"
mv "$1.next" "$1"
grep -q "path = \"$2\"" "$1" \
|| { echo "::error::$1 was not substituted"; exit 1; }
}
subst mcpp.toml .spec
# THIS PACKAGE'S OWN TESTS HAD NEVER RUN.
#
# tests/ has held five suites since this implementation was written and no
# workflow invoked them. What ran was the specification's suite, which is a
# different instrument: it examines what every implementation must do, and
# these examine what this one does with the system beneath it. A defect
# visible only to the second kind was therefore invisible here, which is
# what happened to openkal.timeout -- there was no observation of it in
# either place that a wait upon the wrong descriptor did not satisfy.
#
# The step is openkal-linux's, unchanged, including the assertion that
# every suite ran: a suite that discovered nothing reports success.
- name: This package's own tests
run: |
set -euo pipefail
mcpp test 2>&1 | tee tests.log
# The list is derived from the files present rather than written out
# here: a hand-written list names the suites that existed when it was
# written, and one added afterwards escapes the assertion silently.
missing=0
for f in tests/*.cpp; do
name="$(basename "$f" .cpp)"
grep -q "^$name \.\.\. ok" tests.log || { echo "did not run or did not pass: $name" >&2; missing=1; }
done
test "$missing" -eq 0
# The other architecture, as far as this system allows it to be reached.
#
# The system-call numbers agree between the two --- measured, in the
# `numbers' workflow --- and three things do not: the calling convention,
# the register the current context is reached through, and one field of a
# signal context. All three are in src/sys.h and all three are compiled
# here, with the system's own compiler rather than the build tool's,
# because the build tool has no release for that architecture.
- name: Every source compiles for the other architecture
run: |
for f in src/*.cpp; do
clang++ -std=c++2b -c -o /dev/null --target=x86_64-apple-macos13 \
-fno-exceptions -fno-rtti -DOKM_STANDALONE -I .spec/include "$f"
done
echo "every source compiles for x86_64"
- name: The exported surface is complete and contains nothing else
run: |
# Clause 9.3. The list and the checker come from the specification
# rather than from a copy kept here, so that the comparison has one
# source. --complete because this implementation claims every
# interface: a name it fails to export is a failure, not an interface
# it declines to provide.
rm -rf target && mcpp build
# This package's own objects, and not the ones the build tool emits
# beside them: one of those exists to initialise a standard library
# this package does not use, and it is not this package's to answer
# for. They are told apart by name, because the tool places both in
# one directory.
objs="$(find target -path '*/obj/*' -name '*.o' ! -name '*.m.o' ! -name 'mcpp_*' | tr '\n' ' ')"
test -n "$objs" || { echo "no objects were found" >&2; exit 1; }
bash .spec/tools/check-surface.sh --complete .spec/SURFACE.txt $objs
# The property version 0.3 exists for. The assertion is made against the
# objects rather than against the source, because a source can reach a C
# library through a macro.
- name: The objects reference nothing of a C library but the two named
run: |
rm -rf target && mcpp build --features standalone
objs="$(find target -path '*/obj/*' -name '*.o' ! -name '*.m.o' ! -name 'mcpp_*')"
test -n "$objs" || { echo "no objects were found; the check would pass vacuously" >&2; exit 1; }
# The permitted set, and why each entry is in it.
#
# memcpy, memmove, memset, memcmp a compiler emits calls to these
# from ordinary loops. They compute rather than call, so none of
# them can re-enter this implementation.
# clock_gettime_nsec_np,
# pthread_create_from_mach_thread the two names no C library
# defines, which is why they are reachable from here at all.
# __libc_start_main, main, _main the hand-over, undefined here by
# construction.
# kal_* the interface itself.
# __stack_chk_* emitted by the toolchain around
# a frame it protects; supplied by the program, not called by this
# implementation.
# section$start$… / section$end$… ⭐ THE LINKER'S, NOT A LIBRARY'S.
# `src/unwind.cpp` answers `_dyld_find_unwind_sections` out of the
# boundaries of its own image's `__TEXT,__eh_frame` and
# `__TEXT,__unwind_info`, and this object format states those as
# symbols the LINKER synthesises. Nothing defines them in any
# library, so no library can be reached through them — which is
# the property this check exists to establish. Same kind of entry
# as `dyld_stub_binder` in `port/libSystem.tbd`, and the note
# there says why: a requirement of the format rather than of this
# package.
# ___dso_handle likewise the format's: emitted
# beside the weak references above, defined by the image's own
# start files.
permitted='^_?(memcpy|memmove|memset|memcmp|bzero|clock_gettime_nsec_np|pthread_create_from_mach_thread|pthread_create|pthread_join|__libc_start_main|main|kal_[a-z_]+|__stack_chk_guard|__stack_chk_fail|GCC_except_table.*|_ZN3okm.*|__Unwind_Resume|__dso_handle|section\$(start|end)\$__TEXT\$__(eh_frame|unwind_info))$'
# Reported with the object that references it. A symbol without the
# object it came from names a fault and not a place, and the first
# time this check fired the answer was in the object rather than in
# the symbol.
bad=0
for o in $objs; do
for s in $(nm -u "$o" | sed 's/^ *//' | grep -v ':$' | sort -u); do
[ -n "$s" ] || continue
printf '%s\n' "$s" | grep -qE "$permitted" || {
echo "$o references a symbol it must not: $s" >&2
bad=1
}
done
done
test "$bad" -eq 0
echo "the implementation reaches nothing of a C library but the two names that are named"
# A checker is only useful if it fails when it should.
- name: The independence check detects a dependence
run: |
printf 'extern "C" int puts(const char*);\nextern "C" void okm_probe(void) { puts("x"); }\n' > src/probe.cpp
rm -rf target && mcpp build --features standalone
objs="$(find target -path '*/obj/*' -name '*.o' ! -name '*.m.o' ! -name 'mcpp_*')"
if ! nm -u $objs | sed 's/^ *//' | grep -qx '_puts'; then
echo "the probe did not produce the reference it was written to produce" >&2
rm -f src/probe.cpp; exit 1
fi
rm -f src/probe.cpp
echo "a dependence upon a C library is visible to the check"