0.13.1 — the probe now starts a program that needs an interpreter (#30)

And the README paragraph that blamed the emulator is corrected, because the way
it was wrong matters more than the fact.

`examples/subprocess` starts a `#!` script and asserts it runs. ⚠️ The script is
made executable BY THE SHELL, not by this program: `chmod` is refused here, so a
script this port wrote would be refused for its MODE and the observation would
have held for the wrong reason — which is exactly what the first attempt at this
check did, answering EACCES and looking like the defect being hunted.

openkal-linux 0.12.0 carries the fix: `execveat` with a directory descriptor
hands the interpreter `/dev/fd/<dirfd>/<name>`, which a close-on-exec descriptor
does not survive.

Author: Sunrisepeak

README.md

@@ -348,29 +348,49 @@ was measuring in the first place, and the growth does not change its direction.
 
 ## Verification
 
-### ⚠️ Measuring a foreign architecture through emulation, and the one thing it cannot show
+### ⚠️⚠️ A program that needs an INTERPRETER could not be started, and it was first blamed on the emulator
 
-An `aarch64` build of this library runs on an `x86_64` machine through
-`qemu-user` and `binfmt_misc`, and every observation in this repository's probes
-holds there **except the ones that start a program built for the same foreign
-architecture**. That is a property of the emulator and not of this library, and
-it is stated here because the failure looks exactly like a defect:
+**Corrected in openkal-linux 0.12.0.** An earlier version of this section said
+that an `aarch64` build measured through `qemu-user` could not start a program of
+its own architecture, and that this was a property of the emulator. **That was
+wrong**, and the way it was wrong is worth more than the fact:
 
-| what a program starts | under emulation |
+- the symptom appeared only on `aarch64`, where *every* foreign binary needs the
+  `binfmt_misc` interpreter, so *every* start failed at once;
+- the release before it failed identically, which was read as "pre-existing,
+  therefore not ours";
+- a consumer's 108 tests passed on both architectures, which was read as "not
+  reaching users".
+
+⇒ Each of those was true. The conclusion drawn from them was not.
+
+**What it actually was.** `execveat` with a directory descriptor and a relative
+name gives the kernel the program's name as `/dev/fd/<dirfd>/<name>`. For an
+ordinary executable that spelling never surfaces — the kernel holds the file open
+already. For a program that needs an **interpreter** it does: the kernel starts
+the interpreter and hands it that name *to open*, after the replacement, by which
+time a close-on-exec descriptor is gone. The interpreter is told the file does
+not exist.
+
+⭐ Isolated in twenty lines of ordinary C, with nothing of openkal in it:
+
+```
+dirfd WITH O_CLOEXEC       execveat -> ENOENT
+dirfd WITHOUT O_CLOEXEC    STARTED ok
+```
+
+⚠️ **It was never about architecture.** Two kinds of program need an interpreter,
+and both were refused on every system:
+
+| program | needs an interpreter |
 | --- | --- |
-| a native binary of the host — `sh`, `git`, anything on `PATH` | works |
-| **another copy of the foreign-architecture program itself** | the start fails |
-
-⭐ The distinction is the second exec. The kernel runs a native binary directly,
-and the emulated process simply stops being emulated; a foreign one has to be
-re-entered through `binfmt_misc` from inside an already-emulated process, which
-`qemu-user` does not do. Measured: `execveat` returns `ENOENT` for a path that
-exists, while the same call in the same run starts `/bin/sh` correctly, and the
-same failure appears on the release before this one.
-
-⇒ **A consumer measuring itself this way is not affected** unless its tests start
-copies of themselves. A suite that runs helper programs sees nothing wrong: one
-consumer's 108 tests pass identically on both architectures.
+| a `#!` script | yes — on **every** architecture, including a plain x86_64 machine |
+| a binary of another architecture, through `binfmt_misc` | yes — which is why `aarch64` showed it first |
+| an ordinary native executable | no — which is why nothing else failed |
+
+⇒ The emulator only made it *visible*. A consumer running shell scripts on real
+`aarch64` hardware — or on x86_64 — met the same refusal. `examples/subprocess`
+now starts a script the shell made executable, and asserts it runs.
 
 `examples/wordcount` is an ordinary POSIX program whose source mentions nothing
 of any of this. Its three counts are compared against the system's own `wc`,

examples/subprocess/src/main.c

@@ -798,6 +798,64 @@ int main(int argc, char** argv)
 		}
 	}
 
+	/* --- a program that needs an INTERPRETER ---------------------------------
+	 *
+	 * ⚠️⚠️ A WHOLE CLASS OF PROGRAMS COULD NOT BE STARTED, AND NOTHING HERE
+	 * LOOKED. `execveat' with a directory descriptor and a relative name gives
+	 * the kernel the program's name as `/dev/fd/<dirfd>/<name>'. For an ordinary
+	 * executable that spelling never surfaces --- the kernel holds the file open
+	 * already. For a program that needs an INTERPRETER it does: the kernel starts
+	 * the interpreter and hands it that name TO OPEN, after the replacement, by
+	 * which time a close-on-exec descriptor is gone. The interpreter is told the
+	 * script does not exist.
+	 *
+	 * ⭐ Two kinds of program need one, and they are the same defect:
+	 *     a `#!' script                     --- on every architecture
+	 *     a binary of another architecture  --- through `binfmt_misc'
+	 *
+	 * ⚠️ IT WAS FOUND ON aarch64 AND FIRST BLAMED ON THE EMULATOR, because there
+	 * every foreign binary needs the binfmt interpreter and so every start
+	 * failed at once. It reproduces natively with a script, which is what a
+	 * consumer meets on an ordinary machine.
+	 *
+	 * ⚠️ The script is made executable by the SHELL and not by this program:
+	 * `chmod' is refused here, so a script this program wrote would be refused
+	 * for its mode and the observation would hold for the wrong reason. */
+	if (expect_shell) {
+		char here[512];
+		if (!getcwd(here, sizeof here)) here[0] = 0;
+		char script[640], mk[900];
+		snprintf(script, sizeof script, "%s/interp-probe.sh", here);
+		snprintf(mk, sizeof mk,
+		         "printf '#!/bin/sh\\nexit 0\\n' > %s && chmod 755 %s", script, script);
+
+		pid_t m = -1;
+		char* mav[] = { (char*)"sh", (char*)"-c", mk, NULL };
+		int ms = 0;
+		const int me = posix_spawnp(&m, "sh", NULL, NULL, mav, environ);
+		if (me == 0) waitpid(m, &ms, 0);
+		/* The control: unless the shell really made an executable script, the
+		 * observation below would fail for a reason that is not the one sought. */
+		check(me == 0 && WIFEXITED(ms) && WEXITSTATUS(ms) == 0 &&
+		      access(script, X_OK) == 0,
+		      "a shell makes an executable script for this program to start");
+
+		pid_t s = -1;
+		char* sav[] = { script, NULL };
+		int ss = 0;
+		errno = 0;
+		const int se = posix_spawn(&s, script, NULL, NULL, sav, environ);
+		if (se != 0) {
+			printf("note: starting the script reported errno %d\n", se);
+			check(0, "a program that needs an interpreter starts");
+		} else {
+			waitpid(s, &ss, 0);
+			check(WIFEXITED(ss) && WEXITSTATUS(ss) == 0,
+			      "a program that needs an interpreter starts, and runs");
+		}
+		unlink(script);
+	}
+
 	/* --- units, which 0.12.0 added and no probe here ever looked at ------------
 	 *
 	 * ⚠️⚠️ THE WHOLE OF `setpgid'/`kill(-n)' SHIPPED WITH ITS ONLY WITNESS IN

mcpp.toml

@@ -1,7 +1,7 @@
 [package]
 namespace   = "mcpplibs"
 name        = "openkal-musl"
-version     = "0.13.0"
+version     = "0.13.1"
 description = "musl 1.2.5 redirected onto openkal: one C library, ported once, above every implementation of the specification rather than above one kernel."
 license     = "Apache-2.0"
 
@@ -30,7 +30,7 @@ openkal = "0.12.0"
 #
 # The consequence for a program is that it names this package and nothing else.
 [target.'cfg(os = "linux")'.dependencies]
-openkal-linux = { version = "0.11.0", features = ["standalone"] }
+openkal-linux = { version = "0.12.0", features = ["standalone"] }
 
 [target.'cfg(os = "macos")'.dependencies]
 openkal-macos = { version = "0.9.0", features = ["standalone"] }