lib: Expand list of supported platforms

[toonn]

Motivation for this change

RFC #112 has raised the question of what the source of truth is about the platforms supported in Nixpkgs. RFC #46 specified the authoritative source would be the Nixpkgs manual but this hasn't happened yet.

Recently, lib/systems/supported.nix was added and this seems like a suitable authoritative source. A Nixpkgs manual section could be auto-generated based on this file. However, this file only lists the platforms that the Hydra CI builds for, so we need to add the other tiers and platforms.

I have added the platforms as they are listed in RFC #46 almost verbatim because they include the compiler and libc and further properties like which linker or whether the platform uses static libraries. Should this rather use system triples as in lib/systems/examples.nix, e.g., aarch64-unknown-linux-android? Or perhaps platforms should be represented as attrsets rather than strings?

The RFC also specifies x86_64-illumos while lib/systems/doubles.nix has x86_64-solaris for illumos. Is either one canonical and are there other platforms with multiple identifiers?

This PR is opened to support NixOS/rfcs#113.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• 21.11 Release Notes (or backporting 21.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[Ericson2314]

doubles.nix should be combined with this. I wish I had noticed the original PR adding this file to say that.

[r-burns]

I'm not sure that lib/systems/supported.nix contains sufficient information to serve as the platform tier list. This file is just the system doubles, and the support tier needs to take into account libc (musl/glibc), static/dynamic linking, gcc/clang, ABI, etc. If anything I think lib/systems/supported.nix would be autogenerated from the platform list, rather than the other way around.

[toonn]

@Ericson2314, is that file more authoritative and are the doubles enough information to disambiguate between the platforms or do we need triples.nix?

@r-burns, this is exactly intended to start this discussion. I did note the relative lack of information currently in supported.nix and in the diff you can see I'm including the compiler, linker, etc. However, this is not in a structured way because I'm not sure whether doubles or triples would be enough or whether we should simply give up strings altogether and make an attrset for each platform.

You say supported.nix should be generated from "the platform list" but that list doesn't exist and since the list is all about which platforms are supported at what tiers in Nixpkgs this file looks like the best candidate so far to be augmented into being the official "platform list."

[Ericson2314]

What is all this , and + anyways? I don't know any such syntax yet. We should only do thing we support in parse.nix, where the system is always two components, and the config might be more, and only the entire record is fully generally and not fully serializable as a --separated string.

So yes I do agree with @r-burns. Either we pare this back to be just doubles, or we make it non-string full records, in which case it should be merged with examples.nix.

The underlying problem, as usual, is that Flakes are ignorant about how we've dealt with platforms in Nixpkgs in recent years.

[toonn]

Yes, agreed. I did state this in the OP:

I have added the platforms as they are listed in RFC #46 almost verbatim because they include the compiler and libc and further properties like which linker or whether the platform uses static libraries. Should this rather use system triples as in lib/systems/examples.nix, e.g., aarch64-unknown-linux-android? Or perhaps platforms should be represented as attrsets rather than strings?

[stale]

I marked this as stale due to inactivity. → More info

[toonn]

I've just pushed an update to this PR that attempts to address concerns raised.

Rather than a list of system doubles, which are inadequate as identifiers for the support tiers, it now uses the system attrsets from lib.systems.examples.

This is much closer to the ideal situation but there are still some problems:

1. Neither compiler nor linker are actually specified in these sets.
2. Some platform sets are probably too specific, raspberryPi is used to represent "armv6*-linux", which was specified as a tier 3 platform in [RFC 0046] Platform Support Tiers rfcs#46.
3. Some tiers include platforms that are and are not built by Hydra. Since it's important not to change that list willy-nilly I opted to explicitly allow only the intended systems. This is very verbose.

Item 2 could be addressed with some refactoring of lib.systems.examples, adding a armv6l-linux and then overriding that for raspberryPi for example.

Item 3 does have the benefit that it's clear where a certain tier isn't completely respected. All tier 3 systems should probably be built by Hydra but that's currently not the case for x86_64-unknown-linux-musl.

Pinging @asymmetric @RaitoBezarius, who prompted me picking this back up and might be interested.

[ghost]

Please don't resurrect supported.nix.

We are trying to move away from having a "blessed platform list" in examples.nix that people can fight over, and towards passing arguments to nixpkgsFun.

[toonn]

Then what's the way forward here? Just have the tiers listed in the manual?

Users do want to know how well their platform of choice is supported. And maintainers need a clear reference of what platforms can hold up channels or warrant significant refactors.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-07-24-documentation-team-meeting-notes-66/30885/1

[RaitoBezarius]

Please don't resurrect supported.nix.

We are trying to move away from having a "blessed platform list" in examples.nix that people can fight over, and towards passing arguments to nixpkgsFun.

This is contradictory though with the RFC regarding supported platform tiers. We at least need a way to organize the blessed platform list that is supported by the official nixpkgs CI that outputs signed NARs for cache.nixos.org.

We can call it nixpkgs-ci-platforms.nix if you prefer it.

[ghost]

We at least need a way to organize the blessed platform list that is supported by the official nixpkgs CI that outputs signed NARs for cache.nixos.org.

This is already in the hydra jobset configuration. How else do you think Hydra knows what outputs to build and sign?

Then what's the way forward here? Just have the tiers listed in the manual?

Sure. The RFC is about documentation. Indeed, its one-sentence description of itself is "Document the way to describe the level of support a platform can expect in Nixpkgs."

The only place those lists need to appear as code is in Hydra's job lists (where they already do) and OfBorg's configuration files (where they already do).

Putting it elsewhere is just gratuitous imposition of policy, not mechanism.

[RaitoBezarius]

Sure. The RFC is about documentation. Indeed, its one-sentence description of itself is "Document the way to describe the level of support a platform can expect in Nixpkgs."

The only place those lists need to appear as code is in Hydra's job lists (where they already do) and OfBorg's configuration files (where they already do).

Putting it elsewhere is just gratuitous imposition of policy, not mechanism.

Do you suggest it is possible to consume the Hydra jobset list to auto-generate documentation for the purpose mentioned in the original post of that PR?

If so, that's fine, yes.

[ghost]

Do you suggest it is possible to consume the Hydra jobset list to auto-generate documentation for the purpose mentioned in the original post of that PR?

Probably, but if not then the PR to amend that should be opened against the Hydra jobsets, not lib. lib is one of the very few interfaces that nixpkgs declares to be public and stable.

[toonn]

The problem with the Hydra jobsets is that they're a flat specification of things Hydra builds. AFAIK Hydra only needs the doubles? Adding tier information to the jobsets is superfluous and the tiers need more than just doubles. The documentation would also be more comfortably written against a stable API.

I'm not sure why lib is such a bad location for this information but I also don't care all that much where it goes. If there's an appropiate alternate location, I'm perfectly willing to refactor.

It seems like you're worried about bikeshedding platform tiers more than anything but that's already a thing. There officially is a tier list (even though almost unwritten at this point) and people already want to change the official tier of certain platforms. We should simply require RFCs or whatever due process is decided on for changes to this file.

[ghost]

AFAIK Hydra only needs the doubles?

You're just starting to scratch the surface of why it's a bad idea to try to put this stuff into code.

It's one thing to write documentation that says in prose what non-developer users should expect. It's quite another to specify that precisely enough that it can be used programmatically; it's a minefield we shouldn't step into.

I'm not sure why lib is such a bad location for this information

Because, like I said,

lib is one of the very few interfaces that nixpkgs declares to be public and stable.

Things that turn out to be a bad idea are are easy to revert if they aren't exposed in lib, because we can see all the users of that code -- it's all in-tree. If we have to revert changes to lib things are much more difficult because we can't see all the code that relies on it.

It seems like you're worried about bikeshedding platform tiers

No.