Custom Linux Distro
I have been exclusively using Linux for well over 15 years now, bouncing from various popular distros like Ubuntu, Arch, and Mint. This experience has come to a critical mass in my understanding of the Linux operating system and its ecosystem. This project is a vehicle to deep dive into the various components, discovering and closing any gaps I may have in my knowledge on the subject. Through the years I have built up a wishlist of features to help simplify managing and maintaining my Linux deployments, and intend to realise this list in a novel approach.
While I am confident in my ability to execute this project, I am still learning some of the aspects of the Linux environment. There are some areas where I don’t know what I don’t know. I plan to stream the development of this distro on Twitch and Youtube so that I can hopefully recruit consultation from the community on things I may have missed and for feedback on the decisions I am making in real time. The stream will also serve as an educational resource where the various subsystems of Linux are explained and demonstrated.
This distro will take a different direction from most existing distros. That said, no single feature will be novel and have existed in some cases for decades. A key objective will be to boot to a GUI login screen as quickly as possible. The intent is that the time it takes the user to enter their credentials will mask the time it takes concurrent processes responsible for initialising ancillary resources and services. Another key objective is to make this one of the easiest distros to install, either from an existing Linux installation or from the Windows environment. The installation must also co-exist with the other OS installations and make for a seamless user experience when switching between them.
As for user experience, much of the effort will be put into features that make it easy for a community to support each other. This means good logging and error reporting, as well as making it easy to communicate the current state of the system.
Security will be a key objective but will be flexible enough to not compromise usability. Boot chain validation and process isolation will be the primary mechanisms focused on.
A goal with the project development is to write little to no custom code if possible. It is intended to make good use of existing solutions, strategically choosing technologies that provide a mature and rich feature set while minimising the tech burden of maintenance.
System Requirements
- UEFI compliant firmware
- ACPI system interface
- GPT formatted disk
- 4GB ESP partition ( or the option to reallocate space for a second 4GB ESP formatted partition)
- 16GB of RAM
- AMD, Intel, and possibly ARM CPU 64bit
- Modern GPU (integrated or discreet) from Intel, AMD, or NVIDIA
Brain Dump
The following will develop and be refined as the project progresses but is currently just a brain dump of ideas and resource links.
Development breakdown / Stream programme
- project goals
- immutable
- easily swappable environments/ separation of concerns
- based on containers
- package manager agnostic
- easiest to install
- windows compatibility
- easy to provide support
- make it as easy as possible for the community to support others
- GUI centric, no need for shell to do everything
- speed to login prompt/usable desktop
- separate releases with major graphics cards drivers
- security without compromising functionality
- auditability
- local build of kernel in arch linux container
- mention bootstrap issue
- qemu kvm auto-run
- kernel build parameters
- busybox build in arch linux container
- musl vs glibc vs ..
- kernel modules
- userspace https://en.wikipedia.org/wiki/Linux#Design
- sysfs, efivarfs, procfs, configfs, binfmt_misc, securityfs
- systemd build in arch linux container
- udev startup & “/dev” devtmpfs
- wayland
- networking
- bluetooth
- uefi boot chain, secure boot, tpm
- qemu uefi config
- build system boiler on GH actions 18.
Prior work
The distro will ship as a Unified Kernel Image. At a distance this may look like a modernised TinyCore Linux or Damn Small Linux but those focus on providing a CLI environment.
https://en.wikipedia.org/wiki/List_of_Linux_distributions_that_run_from_RAM
https://www.linuxboot.org/
Reference
Linux Kernel 2.4 Internals
https://makelinux.github.io/kernel/map/
https://makelinux.github.io/kernel/diagram/
https://pubs.opengroup.org/onlinepubs/9699919799/mindex.html
https://www.kernelconfig.io/index.html
- root is ro initramfs
- /etc is rw but in ram
- https://unix.stackexchange.com/questions/77485/can-initramfs-be-paged-out-to-swap-disk
- systemd
- https://systemd.io/
- https://www.freedesktop.org/software/systemd/man/latest/index.html
- systemd-init
- systemd efi (systemd-boot based on
gnu-efi
) - systemd mount instead of fstab
- udev
- systemd-logind
- systemd-journald
- ALL logs are redirected to journald
- look into sharding journal with separate log rotation
- GUI
- compositor + Window Manager + Desktop Environment
- https://www.reddit.com/r/linux4noobs/comments/ksw60b/could_someone_explain_desktop_environment_versus/
- wayland
- https://www.reddit.com/r/linuxquestions/comments/1089ctd/so_what_exactly_is_wayland/
- https://en.wikipedia.org/wiki/Wayland_(protocol)
- https://wiki.archlinux.org/title/Kernel_mode_setting
- https://wiki.archlinux.org/title/Wayland
- https://wiki.archlinux.org/title/GNOME
- https://en.wikipedia.org/wiki/Cinnamon_(desktop_environment)
- https://projects.linuxmint.com/cinnamon/
- https://trello.com/b/HHs01Pab/cinnamon-wayland
- KDE
- Plasma login
- Plasma login manager *
- targeting new laptops
- mouse support in tty
- separate releases for nvidia and amd
initramfs derived from container layers- system to add signed layers at boot via kernel param
- flatpack? ostree?
- custom kernel build, statically linked modules
- https://github.com/NVIDIA/open-gpu-kernel-modules
- https://wiki.archlinux.org/title/AMDGPU
- static ext4, video, encryption, md raid?
- dynamic network, etc
- https://en.wikipedia.org/wiki/Menuconfig
- https://www.kernel.org/doc/html/v6.8/process/changes.html
- https://hub.docker.com/_/archlinux
- CONFIG_DEVTMPFS=n
- secureboot and tpm boot chain
- https://git.kernel.org/pub/scm/linux/kernel/git/jejb/sbsigntools.git/
- https://en.wikipedia.org/wiki/UEFI#UEFI_booting
- https://man7.org/linux/man-pages/man1/objcopy.1.html
- https://man7.org/linux/man-pages/man1/objdump.1.html
- https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface/Secure_Boot#Using_your_own_keys
- https://x86sec.com/posts/2022/09/26/uefi-oprom-bootkit/
- https://en.m.wikipedia.org/wiki/Option_ROM
- https://uapi-group.org/specifications/specs/linux_tpm_pcr_registry/
- https://archiv.infsec.ethz.ch/education/projects/archive/TPMEmulatorReport.pdf
- https://github.com/PeterHuewe/tpm-emulator
- https://github.com/google/pawn *
- fwupd
- efi boot only
- second partition containing rw mounts
- /usr (might want to exclude given flatpack)
- /home
- /var?
- https://linux.die.net/man/7/hier * tmpfs https://serverfault.com/questions/590124/performance-difference-between-ramfs-and-tmpfs
- https://github.com/torvalds/linux/commit/d29216842a85
- swapfile by default /var/swapfile
- network
- nftables only?
- https://en.wikipedia.org/wiki/Netfilter
- https://www.slideshare.net/slideshow/the-linux-networking-architecture/45348971
- good error reporting
- https://www.kernel.org/doc/Documentation/kdump/kdump.txt
- journald reports
- https://www.kernel.org/doc/Documentation/admin-guide/dynamic-debug-howto.rst
- debug output, shelless tty
- https://wiki.ubuntu.com/FirmwareTestSuite
- https://uefi.org/sites/default/files/resources/fwts_uefi_0920_2013.pdf
- https://systemd.io/CATALOG/
- https://cgit.freedesktop.org/systemd/systemd/tree/src/systemd/sd-journal.h
- https://www.freedesktop.org/software/systemd/man/latest/sd-journal.html
- DRM panic / DRM Boot logger
- automated updates
- nothing but binaries and config in initramfs
- disk encryption by default
- damn good tab completion
- glibc based binaries
- https://dmerej.info/blog/post/symlinks-and-so-files-on-linux/
- http://www.etalabs.net/compare_libcs.html
- https://www.gnu.org/savannah-checkouts/gnu/libc/index.html
- https://www.gnu.org/software/libc/manual/html_mono/libc.html#Installation
- https://sourceware.org/glibc/
- https://sourceware.org/git/?p=glibc.git;a=blob;f=elf/dl-cache.c;hb=96429bcc91a14f71b177ddc5e716de3069060f2c#l395 *
- all binaries compiled from source
- git submodules of each project
- coreutils
- https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git https://www.mankier.com/package/util-linux
- kernel
- gnu awk
- https://git.kernel.org/pub/scm/
- efibootmgr
- pulseaudio
- blueman
- https://github.com/flatpak/flatpak/blob/main/CONTRIBUTING.md
- https://git.kernel.org/pub/scm/network/iproute2/iproute2.git
- ntp
- glibc/musl
- build dependencies are excluded
- https://stackoverflow.com/questions/46646625/in-what-library-on-linux-are-the-system-calls-and-how-is-this-library-linked-to
- kernel provided shared libraries, ex: linux-vdso.so
- https://docs.github.com/en/actions/security-for-github-actions/using-artifact-attestations/using-artifact-attestations-to-establish-provenance-for-builds#verifying-artifact-attestations-with-the-github-cli *
- git submodules of each project
windows installer powershell scriptinstall efi via wsl- https://linuxbsdos.com/2024/09/29/2-simple-ways-to-access-the-efi-system-partition-on-windows-11/
- https://oofhours.com/2022/06/29/geeking-out-with-the-uefi-boot-manager/
- https://www.freedesktop.org/software/systemd/man/latest/sd-boot.html#
- use
Get-WmiObject win32_bios
to query the bios version and look up the uefi boot menu button for it https://www.disk-image.com/faq-bootmenu.htm - https://superuser.com/questions/376533/how-to-access-a-bitlocker-encrypted-drive-in-linux
- https://security.stackexchange.com/questions/181539/how-are-bitlocker-fde-keys-stored-in-the-tpm
- https://learn.microsoft.com/en-us/powershell/module/secureboot/set-securebootuefi?view=windowsserver2022-ps
- https://serverfault.com/questions/11879/gaining-administrator-privileges-in-powershell
- https://stackoverflow.com/a/68530475
- https://stackoverflow.com/questions/44919190/windows-equivalent-to-efibootmgr
- windows WSL image release
- https://learn.microsoft.com/en-us/windows/wsl/build-custom-distro
- https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps
- https://unix.stackexchange.com/a/501526
- check for bitlocker and ensure the recovery key is available
- https://www.microsoft.com/en-us/evalcenter/evaluate-windows-11-enterprise
- dbus
- acpi
- https://www.intel.com/content/www/us/en/developer/topic-technology/open/acpica/overview.html
- https://www.kernel.org/doc/Documentation/acpi/namespace.txt
- https://firmwaresecurity.com/tag/acpidbg/
- https://www.slideshare.net/slideshow/acpi-debugging-from-linux-kernel/179373596
- https://cdrdv2.intel.com/v1/dl/getContent/772726
- https://wiki.osdev.org/ACPI
- https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/07_Power_and_Performance_Mgmt/oem-supplied-system-level-control-methods.html
- https://maplecircuit.dev/std/acpi.html
- fix the damn hot backpack issue
- https://www.spinics.net/lists/linux-usb/msg53661.html
- https://github.com/torvalds/linux/blob/e70140ba0d2b1a30467d4af6bcfe761327b9ec95/drivers/platform/x86/asus-wmi.c#L1433-L1458
- https://github.com/torvalds/linux/blob/e70140ba0d2b1a30467d4af6bcfe761327b9ec95/drivers/acpi/battery.c#L738
- https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/07_Power_and_Performance_Mgmt/oem-supplied-system-level-control-methods.html#sws-system-wake-source
- https://mjmwired.net/kernel/Documentation/acpi/debug.txt#144
- https://unix.stackexchange.com/questions/244767/sysfs-alternative-to-proc-acpi-button-lid-lid-state
- https://wiki.archlinux.org/title/Power_management/Wakeup_triggers
- https://man.archlinux.org/man/systemd-sleep.8 *
- resume from suspend
- dhcp
- first-boot config setup optimisations * lpj https://www.kernel.org/doc/html/v4.14/admin-guide/kernel-parameters.html loops_per_jiffy
- microcode loading
- everything in containers
- flatpack
- runtime
- portals
- linker
- network display / screen casting support
- root certs
- time
- timezones
- printers
- cups
- network print
- audio
- bluetooth
- wifi
- dns
- detect captive portals with DoH/DoT active
- block standard DNS ports to prevent leaks
- mDNS
- dns in a container should route through host dns config? (mdns config)
- https://linux.die.net/man/8/nscd *
- firmware
- compress better than half a gig?
- https://github.com/timotheuslin/EFI-BIOS-Resources *
- wine
- rootless?
- internationalization/accessibility
- brail reader
- tts
- memory config
- hugepages
- battery/power management
- power profiles
- devtmpfs vs udev
- https://docs.oracle.com/en/operating-systems/oracle-linux/8/udev/OL8-UDEV.pdf
- https://linux.die.net/man/1/mknod
- https://linux.die.net/man/8/makedev
- https://www.man7.org/linux/man-pages/man3/makedev.3.html
- https://serverfault.com/questions/892134/why-is-there-both-character-device-and-block-device-for-nvme
- sysfs only?
https://github.com/docker/roadmap/issues/593
https://blog.packagecloud.io/the-definitive-guide-to-linux-system-calls/
https://man7.org/linux/man-pages/man2/syscalls.2.html
https://stackoverflow.com/questions/10321435/is-char-envp-as-a-third-argument-to-main-portable
https://en.wikipedia.org/wiki/Netlink
https://busybox.net/
https://busybox.net/downloads/BusyBox.html
https://github.com/brgl/busybox/blob/master/examples/inittab
https://www.qemu.org/docs/master/
https://www.qemu.org/docs/master/system/invocation.html
https://www.qemu.org/docs/master/system/qemu-manpage.html
https://wiki.gentoo.org/wiki/Integrity_Measurement_Architecture
https://sourceforge.net/p/linux-ima/wiki/Home/
https://obsproject.com/forum/resources/background-removal-virtual-green-screen-low-light-enhance.1260/
https://obsproject.com/forum/resources/multiple-rtmp-outputs-plugin.964/
https://droidcam.app/linux/#av
https://www.sheep.chat/en/features/basic
flatpak install com.obsproject.Studio.Plugin.BackgroundRemoval
flatpak install flathub com.obsproject.Studio
flatpak install flathub com.obsproject.Studio.Plugin.DroidCam
sudo apt install linux-headers-$(uname -r) v4l2loopback-dkms
flatpak override --user --device=all com.obsproject.Studio
https://ffmpeg.org/ffmpeg-protocols.html#rtmp
https://obsproject.com/forum/resources/multiple-rtmp-outputs-plugin.964/
https://github.com/Kostr/UEFI-Lessons?tab=readme-ov-file
https://trustedcomputinggroup.org/resource/tpm-library-specification/
https://www.qemu.org/docs/master/specs/tpm.html#tpm-backend-devices
https://www.qemu.org/docs/master/specs/tpm.html#the-qemu-tpm-emulator-device
https://github.com/stefanberger/swtpm/tree/master
https://github.com/stefanberger/libtpms
https://docs.docker.com/build/building/base-images/#create-a-base-image
while inotifywait -e modify Dockerfile; do ./Dockerfile; done
https://stackoverflow.com/questions/30011603/how-to-enable-rust-ownership-paradigm-in-c