openkal-musl 0.1.0: musl 1.2.5 above openkal, on Linux, macOS and Windows - #1
Merged
Conversation
The package was openkal-libc: 243 lines exporting 14 names, doing the two things the specification places outside itself. It was not a C library, and the claim it was written to test - that porting one C library causes the software above it to run on every implementation - cannot be tested without one. musl 1.2.5 is vendored and 1345 of its sources are compiled unmodified. The redirection is one header: musl reaches its kernel through seven inline functions declared once per architecture, and port/include/syscall_arch.h replaces that architecture's copy. Five of musl's own sources are replaced, each because it reads the shape of one environment rather than asking a kernel for something: the auxiliary vector, the program's own ELF headers, the instruction that installs a thread pointer, the system call that creates a thread, and the duplication of the calling image. Four names have no C to fall back on, and one assembly file that assembles for every object format supplies three of them; the fourth needed assembly only in order to issue a system call from a known instruction, and there is no such instruction here. The port layer is 1995 lines. It contains the two structures clause 7.1 forbids an implementation to have - a table of descriptors and a resolver of names - and their being above the boundary rather than below it is the arrangement rather than a violation of it: there is one of each for every environment instead of one per environment. Facilities openkal does not have are refused rather than quietly accepted. mprotect, sigaction with a handler, fork, pipe, poll, readlink, chmod and getrandom all report ENOSYS, and the README records what each costs, because a call that reports success and does nothing is the one answer that leaves a program wrong without telling it. examples/wordcount is an ordinary POSIX program whose source mentions none of this and whose three counts are compared against the system's own wc. examples/posix makes 32 assertions, each observed by its effect.
…r clang as well as gcc The feature the implementation is asked for is renamed: what it selects is not 'a program with no C library' but 'this implementation is the whole of the program's environment', which is a statement about the program and is expressed without reference to what kind of program it is. The link line moves to a per-platform table and gains two statements it needed. The C++ runtime is not embedded: this package is C, the implementation beneath it is C++ only because the specification's declarations are modules, and a runtime attached by the driver would have its own references to a C library resolved by this one. --no-dynamic-linker removes the interpreter clang records even for a static link, which gcc does not, and which the loader refuses to start.
Three properties of that environment stand between musl and it, and each was measured rather than assumed. A long is narrower than a pointer there. musl states its data model in three lines of one generated header, so a second generated header states the other one; two declarations that carry a machine word through a long are widened to the type musl already provides for the purpose, and two sources that do the same are replaced. musl/PATCHES.md lists all of them. A wide character is sixteen bits there and thirty-two in musl. Three of musl's sources write a wide literal; each writes an array instead. The two are still different, deliberately: a program that writes L"..." is told so by the compiler, where narrowing musl's wchar_t would have converted every code point above U+FFFF to the wrong value silently. A weak symbol is not a definition there --- measured with both toolchains, for aliases and definitions, functions and data. musl gives almost every public name to a definition through a weak alias, and provides 46 names as placeholders for another source to replace. The aliases are made strongly and the placeholders are not made at all, decided by the name of the target, which needs no change to musl. One further finding is not about that environment but about this port. musl's per-context state was reached through a variable declared thread_local, which openkal reports as a property a started context has. On one toolchain such a variable is reached through a helper that allocates, and the allocator is what this library provides: the dependency is circular and appears only as a program that starts and never returns. The state is now kept in a table keyed on the identity openkal already gives every context, so this port depends on nothing beyond openkal and runs above an implementation whose contexts have no thread-local storage at all. The wordcount example above this library on Windows reports the same three counts as that system's own tool, and the thirty-two assertions of the posix example all hold there.
…two openkal additions macOS is the third system this library runs on, and reaching it needed two things and no more. A second name for a definition cannot be made there the way it is made on ELF or on PE: the compiler refuses the construct outright. What that format does have is the assembler's own directive, and the measurement that establishes it is in PATCHES.md. The name it makes is strong, so the placeholder list that already existed for the other format applies unchanged --- which is the return on having decided which names are placeholders by the name of the target rather than by a list of sites. The file that supplies what an object format does not is no longer named for one format. On the third, the loader has already run every initialiser in the image by the time control reaches the entry point, so the loops that drive them are absent there and would otherwise run every constructor twice. Three operations are expressed differently from what their names say, and each is recorded in PATCHES.md rather than left to be discovered. execve is starting a program, waiting for it and ending with its status, because openkal has no operation that replaces a running image and clause 3.1 declines to simulate what cannot be supplied. utimensat asks for write access where the interface it implements asks for ownership. A program named without a suffix is tried with one of the environments' suffix second, which is what every C library for that environment does and is here rather than beneath because openkal is deliberately literal about names. utimensat is now answered at all, through openkal 0.5's kal_fs_set_modified. Before this, `touch' reported that the operation was not implemented. tools/working-trees.sh places the specification and the implementation for the system beside this package and points every manifest at them, which is what the continuous integration needs and what a contributor needs for the same reason.
…h working tree is at
…zero and letting the dereference say it An enquiry that finds no entry answered zero, which the caller reads as a pointer to the per-context state. Before the first context registers that is the ordinary state and zero is right; afterwards it is not, and what a reader is shown is a null dereference several frames from the absence that caused it. The debugger is also asked at the right moment: a batch that lists its commands after `run' gets as far as the stop and quits, which is why the first attempt printed one frame and no backtrace.
…e embedded into it The build tool decides whether to embed a C++ runtime, and on one of the three systems its answer is to embed one and to run that runtime's initialiser before anything else in the image, including the entry point --- because that format has no priority-ordered initialiser section and the runtime's streams must be constructed before any global that touches them. A program above this library carries no other runtime. At that moment the C library has not started: it has no per-context state, no auxiliary vector and no allocator. The embedded initialiser reaches the library anyway, through a guarded static and a mutex, and what a reader is shown is a null dereference in the mutex --- six frames from the ordering that caused it. `cxx_runtime = "host-coupled"' says that the build tool embeds nothing. The system's own C++ runtime is named instead, and its initialisers run inside it, on the facilities it was built against rather than on this library's. It is declared by the program rather than here because the build tool reads it from the program: a dependency that declared it would be declaring it for itself. It is one line, it is recorded in the README beside the dependency, and both examples carry it.
…tions asked of them
musl files <bits/float.h> under arch/<arch>/, and for every system musl was
written for the architecture does decide what a long double is. One of the three
systems this port builds for disagrees: on its aarch64 a long double is a
double, where the architecture's own procedure call standard says quadruple.
target __LDBL_MANT_DIG__ arch/<arch>/bits/float.h
aarch64, Apple 53 113
x86_64, Apple 64 64
x86_64, Linux 64 64
aarch64, Linux 113 113
Every routine that takes a long double then reads sixteen bytes out of eight.
The first one a program reaches is the one that formats a floating-point number:
printf("%.3f", 1.5) stopped in frexpl with the value read as infinity, and what
the report named was the arithmetic rather than the header.
port/include/bits/float.h states the compiler's own answer on that combination,
through the macros the compiler defines, so it cannot drift from what is being
compiled --- which is the property the header it replaces did not have.
port/src/okm_float_assert.c asserts the agreement for every target, so a fourth
combination that disagrees is a failed build naming the field rather than a
program that formats a number wrongly.
musl already supports a 53-bit long double: it is what its own arm and riscv64
configurations use, and every conditional in src/math and src/stdio is written
for it. Nothing beyond the one header is involved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Renames this package from
openkal-libctoopenkal-musl— the name says whichC library it is — and replaces the hand-written subset it contained with musl
1.2.5 itself, redirected onto openkal.
The shape of the port
musl reaches its environment through one seam: seven inline functions in
arch/<arch>/syscall_arch.h, plus__get_tpinpthread_arch.h. Everythingelse in musl is written against those.
port/include/supplies both, so 1682vendored files are compiled unmodified and reach openkal instead of a kernel.
musl/PATCHES.mdlists the whole of what is not unmodified: four patchedlines, all of one kind (a machine word carried through a variable declared
long, which is not a machine word on one of the three targets), and ninereplaced sources, each with the reason it is replaced. Five of the nine read
the shape of one environment directly — the auxiliary vector, the program's own
ELF headers, the instruction that installs a thread pointer, the system call
that creates a thread, and the duplication of the calling image.
Three systems, and what each cost
Two findings from Windows are worth stating because neither is visible in any
source:
thread_localvariable.On one toolchain such a variable is reached through a helper that allocates,
and the allocator is what the C library provides.
port/src/okm_context.cisa table keyed on
kal_task_current().longin the dispatcher is a fault waiting on a targetwhere a
longis narrower than a pointer.Three operations expressed differently
Recorded in
musl/PATCHES.mdrather than left to be discovered:forkis absent and stays absent. Duplicating a running image is notsomething every environment can produce, and clause 3.1 declines to simulate
what cannot be supplied.
posix_spawn, and thereforesystemandpopen,are supplied.
execveis starting a program, waiting for it, and ending with its status.A caller cannot distinguish that through this library; there are two images
where a system with the operation would have one. It is what every environment
without the operation does, and two of the three beneath openkal are such.
second, which is what every C library for that environment does.
What runs
examples/posixexamples/wordcountwcmcpplibs/sbaseContinuous integration runs the first two on Linux (gcc and llvm), macOS (llvm)
and Windows (gcc, PE).
Requires
mcpplibs/openkal#1.