0.7.0 — the crash a consumer could not localise, and openkal 0.9

.github/workflows/ci.yml

@@ -615,22 +615,41 @@ jobs:
           cxx_runtime = "host-coupled"
           TOML
           sed -i 's/^          //' "$d/mcpp.toml"
-          # ⭐ `symlinkat` IS THE OPERATION AND THE PROBE PRINTS ITS OWN NUMBER.
+          # ⚠️⚠️ THE OPERATION THIS PROBE NAMES MUST STILL BE ABSENT, AND THE
+          # ONE IT NAMED STOPPED BEING SO.
           #
-          # openkal has no atom that creates a link --- the manifest and README
-          # both record that --- so it reaches the default arm rather than a case
-          # with a decision of its own. The number is printed by the probe rather
-          # than computed here, so the expectation comes from the same headers
-          # the dispatcher was compiled with; a number worked out on the host
-          # would be answering for the host's C library.
+          # It was `symlinkat`, chosen because openkal had no atom that creates a
+          # link. openkal 0.9 added `kal_fs_link_create`, this dispatcher gained
+          # a case for it, and the probe --- unchanged, still green in intent ---
+          # began exercising an operation that now SUCCEEDS. It reported zero
+          # diagnostics for a hundred attempts, which is the same reading a
+          # broken diagnostic channel gives.
+          #
+          # ⭐ SO THE ABSENCE IS NOW ASSERTED RATHER THAN ASSUMED. `mknodat`
+          # creates a device node, which openkal does not express and is not
+          # going to; if a case for it ever appears, the grep below fails and
+          # says to pick another operation --- instead of this step passing
+          # while measuring nothing.
+          if grep -q 'SYS_mknodat' port/src/okm_syscall.c; then
+            echo "::error::the dispatcher now handles mknodat, so this probe no"
+            echo "::error::longer exercises an absent operation. Pick another"
+            echo "::error::operation openkal has no atom for and name it here."
+            exit 1
+          fi
+          # The number is printed by the probe rather than computed here, so the
+          # expectation comes from the same headers the dispatcher was compiled
+          # with; a number worked out on the host would answer for the host's C
+          # library.
           printf '%s\n' \
             '#include <sys/syscall.h>' \
+            '#include <sys/stat.h>' \
             '#include <fcntl.h>' \
             '#include <unistd.h>' \
             '#include <stdio.h>' \
             'int main(void) {' \
-            '    for (int i = 0; i < 100; i++) (void)!symlinkat("a", AT_FDCWD, "b");' \
-            '    printf("%ld\n", (long)SYS_symlinkat);' \
+            '    for (int i = 0; i < 100; i++)' \
+            '        (void)!mknodat(AT_FDCWD, "node", S_IFIFO | 0600, 0);' \
+            '    printf("%ld\n", (long)SYS_mknodat);' \
             '    return 0;' \
             '}' > "$d/src/main.c"
           ( cd "$d" && mcpp build --toolchain '${{ matrix.toolchain }}' )

.gitignore

@@ -30,3 +30,8 @@ crash.log
 # What the object-level checks in continuous integration write beside the
 # sources they examine.
 syms.txt
+
+# A working tree of the specification or of an implementation placed beside the
+# sources. No trailing slash: the pattern must match a symbolic link as well.
+.spec
+.impl

README.md

@@ -8,7 +8,7 @@ the claim can be checked rather than repeated.
 
 ```toml
 [dependencies]
-openkal-musl = "0.6.0"
+openkal-musl = "0.7.0"
 ```
 
 It names no implementation and no platform: a C library is the one consumer that
@@ -108,8 +108,10 @@ answer that leaves a program wrong without telling it.
 | memory protection | `mprotect` reports `ENOSYS` | openkal has no operation upon a mapping's protection. musl asks for a guard page below a thread's stack and proceeds without one when told this, so the honest answer is also the one it is prepared for. |
 | out-of-band data | `MSG_OOB`, `MSG_PEEK`, and `POLLPRI` are never reported and `recv` refuses the flags | openkal's transfer operations move bytes and have no second channel and no non-destructive read. |
 | readiness *sets* | `epoll` is not built at all, so the link names it | a set held by the environment is a facility of one kernel rather than a capability. `poll` and `select` ask each descriptor in turn, which is what an interface without a set permits. |
-| symbolic links | `symlink` reports `ENOSYS`; `readlink` reports `EINVAL` for a name that is not one and `ENOSYS` for one that is | `SURFACE.txt` has no operation that creates or reads a link. It *does* have `kal_node_link` and `KAL_FS_PROP_LINKS`, so an implementation can report a link it encounters and cannot make one; the asymmetry is the specification's and is recorded rather than worked around. |
-| ownership and permission bits | `chmod` and `chown` report `ENOSYS`; `stat` reports a mode assembled from what openkal knows | `kal_node_info` carries `writable` — one boolean, not a mode word — and `kal_fs_open_file` takes flags rather than a mode. Mapping the owner-write bit onto it would make `chmod(0600)` succeed and `stat` report something else, which is the shape this port exists to avoid. |
+| ~~symbolic links~~ | **answered since 0.7.0** — `symlink`, `readlink`, and `stat`/`lstat` telling the two questions apart | openkal 0.9 carries `kal_fs_link_create` and `kal_fs_link_read` as operations of `openkal.fs`, and `kal_fs_props` takes the directory, so this port asks whether the volume has such nodes before it uses them. Where it does not, the refusal is what the enquiry already said. |
+| permission bits | `chmod` reports `ENOSYS`; `stat` reports a mode assembled from what openkal knows | `kal_node_info` carries `writable` — one boolean, not a mode word — and `kal_fs_open` takes flags rather than a mode. Mapping the owner-write bit onto it would make `chmod(0600)` succeed and `stat` report something else, which is the shape this port exists to avoid. |
+| the identity of a node | `st_dev` and `st_ino` are the implementation's answer where it has one, and **zero for both where it has none** | ⚠️ They were the constants 0 and 1, so every file compared equal to every other: `std::filesystem::equivalent` on two separately created files answered `true` **with no error**. openkal 0.9 carries an identity and reports whether it knows one; a caller must not read two zeroes as sameness, which is why nothing is invented for an implementation that cannot distinguish nodes. |
+| ownership | `chown` reports `ENOSYS`; `stat` reports 1000 for both | as the row above: a capability-oriented environment has no principal for an owner to name. |
 | a mode given at creation | `open(…, O_CREAT, 0600)` and `mkdir(path, 0700)` **succeed** and `stat` afterwards reports 0666 and 0777 | the row above, in the one place where it does not read as a refusal. openkal opens a file for a purpose and not for an audience, so the argument has nowhere to go. Refusing every mode but the one `stat` will report would refuse nearly every program; what a caller can rely on instead is stated below. |
 | entropy | `getrandom` reports `ENOSYS` where the backend declines `openkal.random` | openkal has no source of one to require, and this port does not invent one. The allocator's cookie and the stack canary are derived from the clock and from an address; neither is a security property here. |
 | a signal delivered anywhere | `raise` and `kill` perform a signal's **default action** and nothing else: terminating signals end the program, ignored ones succeed, stopping ones report `ENOSYS`, and musl's own three (32, 33, 34) are refused, so `pthread_cancel` reports `ENOSYS` | there is no delivery, so there is no handler to reach; what remains of a signal is what it does when no handler exists. `abort` reaches `kal_abort`, which raises the signal on Linux and ends with a distinguished status elsewhere — a parent can tell an abnormal end from an ordinary one on every system. |
@@ -141,10 +143,30 @@ was given would not be confined by having been given it. `/a/b/../c` is
 therefore reduced to `/a/c` before openkal sees it, which is what the program
 means in every case except one that passes through a symbolic link.
 
+**`O_NOFOLLOW` is answered by an enquiry, not by an opening.** openkal states
+that opening resolves and offers no form that declines to, deliberately: a
+program that opens a link in order to read its bytes is asking what
+`kal_fs_link_read` answers. So this port asks `kal_fs_info` with
+`KAL_FS_NO_RESOLVE` first and reports `ELOOP` when the name is a link, which is
+what POSIX says and what a caller passing the flag is distinguishing. Answering
+`ENOENT` instead — which is what resolving a link to an absent target produces —
+is a different answer to a different question, and libc++'s `remove_all` reads it
+as "the entry has already gone" and leaves the tree standing.
+
 **The tables are bounded.** A program may hold 1024 descriptors and 512 open
 descriptions; beyond that it is told so. Allocating the tables instead would
 place them on the allocator, and the allocator obtains its memory through them.
 
+⚠️ **And a program may have started 256 programs it has not waited for.** An
+entry is taken when a program is started and released when it is waited for,
+which is what a process table is; a program that never waits holds entries for
+ever, and the next start reports `EAGAIN` — which is what POSIX says `fork`
+does when the table is full. This bound is stated here because it was not, and
+a caller that met it saw a failure on an operation with no evident relation to
+the ones that caused it: measured, the sixty-fifth `posix_spawn` of a program
+that waited for none, and of one that polled each once with `WNOHANG` and did
+not come back.
+
 ## Asking which operation was missing
 
 `ENOSYS` says that a facility is not here. It does not say which one, and until

examples/posix/src/main.c

@@ -2,6 +2,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <signal.h>
 #include <unistd.h>
 #include <fcntl.h>
 #include <errno.h>
@@ -164,6 +165,177 @@ int main(int argc, char **argv, char **envp) {
 		failures += 2;
 	}
 
+	/* ⭐⭐ THE DISPOSITION OF A SIGNAL IS TOUCHED, WHICH NOTHING HERE DID.
+	 *
+	 * This file had thirty-six observations and three of them were about
+	 * `abort'. It contained no call to `signal' or `sigaction' anywhere --- so
+	 * it examined whether `abort' ENDS the program and never whether a program
+	 * may ASK what a signal is set to. A defect that killed any program doing
+	 * the second passed every one of the thirty-six.
+	 *
+	 * ⚠️ ALL THREE FORMS, AND SIGABRT AMONG THEM. The C library takes a lock for
+	 * any change to that one disposition and blocks signals to take it, so
+	 * SIGABRT reaches code the others do not --- and the enquiry, which changes
+	 * nothing, reached it too. Two of the three forms below would have passed
+	 * while the third killed the process. */
+	{
+		int survived = 1;
+		for (int sig = 1; sig < 32; sig++) {
+			if (sig == SIGKILL || sig == SIGSTOP) continue;
+			struct sigaction seen;
+			memset(&seen, 0, sizeof seen);
+			/* An enquiry, which changes nothing. */
+			const int q = sigaction(sig, NULL, &seen);
+			if (q != 0 && errno != ENOSYS) survived = 0;
+			/* Ignoring, which every environment can express. */
+			errno = 0;
+			signal(sig, SIG_IGN);
+			if (errno != 0 && errno != ENOSYS) survived = 0;
+			/* A handler, which this environment cannot deliver and refuses. */
+			errno = 0;
+			if (signal(sig, SIG_DFL) == SIG_ERR && errno != ENOSYS) survived = 0;
+		}
+		check(survived, "every signal's disposition may be read and written or refused");
+	}
+
+	/* Nodes whose content is another name, where the volume has them. */
+	{
+		/* The names this block uses are its own: the file the earlier
+		 * observations made has been removed by the time this runs, and a probe
+		 * that depended on another probe's leftovers would report an absence as
+		 * a defect. */
+		{ FILE *t = fopen("okm-link-target.tmp", "w"); if (t) { fputs("0123456789", t); fclose(t); } }
+		unlink("okm-probe-link");
+		const int made = symlink("okm-link-target.tmp", "okm-probe-link");
+		if (made == 0) {
+			char target[64] = { 0 };
+			const ssize_t got = readlink("okm-probe-link", target, sizeof target - 1);
+			check(got == (ssize_t)strlen("okm-link-target.tmp")
+			          && strcmp(target, "okm-link-target.tmp") == 0,
+			      "a node's content reads back as it was written");
+
+			/* ⭐ THE OBSERVATION THE PORT MOST NEEDED. Asking resolves and
+			 * opening resolves, so the two agree; asking with the flag reports
+			 * the node itself. They disagreed, and a C++ library above reported
+			 * a link where a caller would have reached a file. */
+			struct stat followed, itself;
+			check(stat("okm-probe-link", &followed) == 0 && S_ISREG(followed.st_mode),
+			      "stat resolves, and reports what the name finally refers to");
+			check(lstat("okm-probe-link", &itself) == 0 && S_ISLNK(itself.st_mode),
+			      "lstat reports the node itself");
+
+			/* ⭐⭐ AND THE THIRD QUESTION, WHICH IS NEITHER OF THOSE TWO.
+			 *
+			 * O_NOFOLLOW does not ask to open the link and does not ask to
+			 * open its target: it asks `is this name a link?' and expects
+			 * ELOOP when it is. openkal offers no opening that declines to
+			 * resolve --- by design --- so this port resolved, and for a link
+			 * to a name that is absent it answered ENOENT.
+			 *
+			 * ⚠️ THAT IS A DIFFERENT ANSWER TO A DIFFERENT QUESTION, AND
+			 * NOTHING NEARBY LOOKED WRONG. Every operation above still held.
+			 * What failed was three layers up: libc++'s `remove_all' descends
+			 * by opening each entry O_DIRECTORY|O_NOFOLLOW and reads ENOENT as
+			 * `it is already gone', so it unlinked nothing and then reported
+			 * ENOTEMPTY for a directory it had just declined to empty. The
+			 * host toolchain removed the same tree.
+			 *
+			 * The answer comes from the enquiry openkal 0.9 added: ask about
+			 * the name itself. Both cases are checked because they fail
+			 * differently --- a live target resolved to a FILE and returned a
+			 * descriptor, which is not an error at all. */
+			int nf = open("okm-probe-link", O_RDONLY | O_NOFOLLOW);
+			check(nf < 0 && errno == ELOOP,
+			      "opening a link with O_NOFOLLOW reports that it is a link");
+			if (nf >= 0) close(nf);
+
+			unlink("okm-probe-target-gone");
+			unlink("okm-probe-dangling");
+			if (symlink("okm-probe-target-gone", "okm-probe-dangling") == 0) {
+				nf = open("okm-probe-dangling", O_RDONLY | O_NOFOLLOW);
+				check(nf < 0 && errno == ELOOP,
+				      "and does so for a link whose target is absent, rather than ENOENT");
+				if (nf >= 0) close(nf);
+				check(unlink("okm-probe-dangling") == 0,
+				      "a link whose target is absent is still removable");
+			}
+
+			unlink("okm-probe-link");
+		} else if (errno == ENOSYS || errno == EPERM) {
+			printf("ok:   this volume has no nodes that name others, which it reported\n");
+		} else {
+			printf("FAIL: making a node that names another (errno %d)\n", errno);
+			failures++;
+		}
+		unlink("okm-link-target.tmp");
+	}
+
+	/* ⚠️ Two different files are two different files. `st_dev' and `st_ino'
+	 * were constants, so every file compared equal to every other and a C++
+	 * library's `equivalent' answered true with no error.
+	 *
+	 * ⚠️⚠️ WHEN THIS FAILS, THE DEFECT IS USUALLY NOT IN THIS PACKAGE. This
+	 * port copies the identity out of `kal_node_info' and puts zero there when
+	 * the implementation does not report one --- which is permitted, and which
+	 * makes every node compare equal to every other. So a failure here says
+	 * "the openkal implementation beneath this one declined to report an
+	 * identity", and the place to look is its `kal_fs_info'.
+	 *
+	 * Measured: it failed on Windows, and openkal-windows was reading a volume
+	 * serial number the object manager had written and then discarding it,
+	 * because the enquiry reported STATUS_BUFFER_OVERFLOW for a volume label
+	 * that did not fit and the implementation read that as a failure. The
+	 * conformance suite could not have said so: an implementation is allowed
+	 * to decline the field, so the suite reports the observation as one it did
+	 * not make. This is the criterion that notices, and it is two packages
+	 * away from the defect. */
+	{
+		struct stat x, y;
+		FILE *fx = fopen("okm-probe-x.tmp", "w"); if (fx) fclose(fx);
+		FILE *fy = fopen("okm-probe-y.tmp", "w"); if (fy) fclose(fy);
+		check(stat("okm-probe-x.tmp", &x) == 0 && stat("okm-probe-y.tmp", &y) == 0
+		          && !(x.st_dev == y.st_dev && x.st_ino == y.st_ino),
+		      "two different files have different identities");
+		unlink("okm-probe-x.tmp");
+		unlink("okm-probe-y.tmp");
+	}
+
+	/* A value POSIX says cannot fail is not a negated error. */
+	check(getpgrp() > 0, "the process group is a number and not a negated error");
+
+	/* The page is the machine's and not the build's.
+	 *
+	 * ⚠️⚠️ AND "POSITIVE POWER OF TWO" WAS TRUE OF THE VALUE THAT BROKE IT.
+	 * This library took `kal_memory_granularity()' as its page size, and an
+	 * implementation for a machine with no memory management unit answers ONE
+	 * --- correctly, since nothing there needs rounding. One is positive and
+	 * one is a power of two, so this assertion held while the allocator asked
+	 * the environment for one-byte extents and the program stopped inside the
+	 * first allocation that needed a new one.
+	 *
+	 * ⭐ SO THE CRITERION IS WHAT THE ALLOCATOR REQUIRES, NOT WHAT THE NUMBER
+	 * LOOKS LIKE. A page smaller than this library's own quantum is not a page
+	 * this library can use, whatever openkal reports. */
+	{
+		const long page = sysconf(_SC_PAGESIZE);
+		check(page >= 4096 && (page & (page - 1)) == 0,
+		      "the page size is a power of two no smaller than the allocator's quantum");
+
+		/* And the property the number exists to have. Several pages, written
+		 * end to end: the allocation this library rounds to `page' and the
+		 * memory it hands back are the same memory. */
+		const size_t span = (size_t)page * 4 + 17;
+		unsigned char *big = malloc(span);
+		int whole = big != NULL;
+		if (big) {
+			for (size_t i = 0; i < span; i++) big[i] = (unsigned char)(i * 31u);
+			for (size_t i = 0; i < span; i++)
+				if (big[i] != (unsigned char)(i * 31u)) { whole = 0; break; }
+			free(big);
+		}
+		check(whole, "several pages are obtained in one allocation and every byte of it holds");
+	}
+
 	printf("-- failures: %d --\n", failures);
 	return failures ? 1 : 0;
 }