Objective

Discuss with stakeholders and think about how manifest conditions, on-demand installs, the Applications page, and package based vs. immutable OSes should play together. There is already a document for the immutable OS case. It discusses the "what do we want it to be?" aspect as well.

Scope

We need to consider what we want cockpit to be: Monitoring an existing system workload? Then Machines page would just not be shown if libvirt isn't installed. Or do we want Cockpit to add new workloads? Then Machines/Containers/Storage etc. could install their dependencies on demand. But for example, we would never want this for the Networking page, as installing NetworkManager is too intrusive. Likewise, we don't want c-ostree to install rpm-ostree.

As we already have valid on-demand install cases (realmd, PCP metrics), one of Cockpit's ideals is "Discoverability", and Cockpit is often used for learning/experimentation, whenever there is no strong reason to not offer on-demand installation (like for ostree and NetworkManager above), we should err on the side of flexibility and offer it. Let's have some faith in administrators to not blindly install anything they get a button for, and actually consider whether they want a machine to have a container/VM/stratis/etc. workload.

The above reasoning shows that we need to consider this for each page/project individually, and work through them to add and test this feature one by one. We should make this easy and consistent with a new pkg/lib/ component which offers on-demand installation from a "not supported" empty state pattern. But it shows that answering the "existing workload" vs. "configure new workloads" question doesn't have a clear-cut answer, so we best avoid answering it globally, and make that decision on a per-page basis.

This is also compatible with the immutable OS proposal from the above document. That new page can collect the same information from the manifests, and the shared "on-demand install" UI component can adjust its behavior accordingly, i.e. not offer on-demand install on OSTree. That would even be the case naturally, as there is no PackageKit/dnf on these images.

What packages to install

In all cases, the first version assumes that PackageKit is installed. We can refine this later on for dnfdaemon, aptdaemon, and possibly other APIs which are more likely to be present already.

There are three approaches to which packages to install, which are analyzed below.

[COCKPITPKG] Install cockpit-$page

For example, the "Podman containers" page would install "cockpit-podman". Each page would also install cockpit-system for technical reasons (see "Con" point below).

• Pro: Structurally simple. We control the package name in all distros, so don't need any complicated declaration mechanism (in the manifest) that parallels the distro packaging.
• It retains the "automatic dependencies" relationship that apt/dnf maintain to automatically clean up dependencies when the user uninstalls cockpit-$page (manually or on the Apps page)
• Pro: Allows co-evolution of cockpit-$page with the APIs that it talks to. We know that versions in distros have been thoroughly tested against the APIs in them.
• Con: This works fine for things like c-podman or c-machines. If we would ever install cockpit-system that way, it would downgrade or upgrade the Overview page to the version of the target OS, which looks a little odd. As installing cockpit-page would install cockpit-bridge as a dependency, we almost necessarily have to install cockpit-system as well, so that the bridge continues to see the "basic" pages (Overview, Services, and such).

[DEPSONLY] Install the page's dependencies only

We would put the package's dependencies into the manifest, similar to what we already do with stratis. The on-demand install component could then install these, but the cockpit UI itself would continue to come from cockpit/ws or the flatpak.

• Mixed: Always provides the latest Cockpit UI for all machines if the client is current, i. e. the flatpak or the cockpit/ws container. However, for old clients and new servers (e.g. connecting from RHEL 8 or Debian stable to C9S or Fedora) it is the opposite – it would then run old pages against a newer OS.
• Con: It increases our CI requirements and need to carry around special cases considerably. For example, podman has changed its API countless times, and c-podman needed to follow suit. Similarly, Stratis has revved its API several times, and we follow along. Such revs could easily break non-current RHELs like 8.4, 8.6, and 9.0 (all of these are still supported for many years). We would have to continue testing old versions, and keep compatibility code for the older APIs. Same for machines. cockpit-machines needed a workaround for a major distro regression which is now gone from main, but this kind of adjustments happens all the time for libvirt also. In summary: co-evolution of APIs is and remains a thing.
• Con: For the other compatibility direction (old client to new server) this gets worse. We will not constantly update all old Debian or RHEL 8 to accomodate for API changes in newer OSes, or provide an up to date experience.
• Con: As a corollary from the above: Cockpit's main branch already stopped supporting RHEL 7, and so far we plan to branching off RHEL 8 in about a year (this will allow us to drop a lot of hacks, and probably the entire C bridge). But this prevents us from testing the latest versions against RHEL 7 and 8.
• Con: No guaranteed outcome: The dynamic combination of the remote-sent cockpit page version and the installed versions from the distro have never been tested.
• Con: Cockpit would install random packages as explicit choices, so they cannot be cleaned up any more, as there is no notion of "uninstall cockpit-machines".
• Con: The only way to avoid the potential "UI downgrade" would be to never install any cockpit related package. This is incompatible with the "Apps" page, our install instructions, Ansible role, and the user installing some cockpit-* package after trying it out.

[MERGE] Teach cockpit-ws to merge local and remote pages

A hybrid approach would teach our web server to serve all remote pages, and add the local ones which don't exist on the remote bridge.

• Pro: avoid having to install cockpit-system unconditionally, and thus avoid the potential UI downgrade.
• Con: Unknown amount of effort – is that possible with reasonable effort, or would that be too intrusive for the websocket path handlers in c-ws?
• Con: Could lead to weird UI style mix where e.g. some of the pages are PF5 already, and some PF4.
• Con: Fuzzily defined, as "some" pages may be broken (untested API combinations) while the others work fine.

Opinions

• Pitti: [DEPSONLY] has too many drawbacks, so I veto that one. [MERGE] is not obviously better and involves a potentially large and intrusive piece of development in the webserver. [COCKPITPKG] seems to be a good compromise for me, and is a straightforward starting point, as we will always have to expect already existing cockpit packages on the remote host anyway.
• Marius: I vote for [COCKPITPKG]. It is still nice to bootstrap Cockpit remotely on a machine that was installed without it, but I think that should just consist of installing the necessary bits from the OS repositories of that machine.
• MM: For me ideally we would have [MERGE]. But I am just as fine with [COCKPITPKG] as it covers what we need - kickstart people without needing to install anything, help them discover what else they can use Cockpit for and then if they want more, they can just install it. We might want to try PoC for [MERGE] later on just to see if it is plausible.
• lis: If the remote machine has /usr/bin/cockpit-bridge, we just run that, like we always have. If not, then it's weird to try to take content from this system or start installing packages on it, so for me this is very obviously [DEPSONLY]. Our CI complexity goes up some, but also decreases in some ways: we no longer have to build testing images, for example.

Conclusion

For the time being, we keep a separation between two different worlds:

• The "classic distro package" setup requires installing cockpit- on target machines. It will use on-demand installation with the [COCKPITPKG] strategy. It will *not offer the "beiboot" feature.
• The "portable cockpit" setup with the Client flatpak and the cockpit/ws container does not require installation of any cockpit-* packages. It offers bundled pages and beiboot to connect to any remote machine, but will not offer on-demand installation, to avoid changing the remote system in unexpected ways.

I feel that at some point we will need to build a bridge between these two worlds (pun half-intended). But let's revisit that once we get a few months of field testing and feedback for the new beiboot case.