raito/lanzaboote
0
0
Fork 0
Code Issues 45 Pull requests 6 Projects Releases 6 Packages Wiki Activity Actions

feat: add CPIO packing for companion files #168

Closed
RaitoBezarius wants to merge 19 commits from cpio-packing into master
pull from: cpio-packing
merge into: raito:master
Conversation 13 Commits 19 Files changed 33 +2099 -59
RaitoBezarius commented 2023-04-30 14:56:18 +00:00 (Migrated from github.com)
Copy link

Depended on #167 #166.

This adds CPIO packing and open credentials and systemd extensions discovery and load.

With this, we should be able to read from drop-in directories, load CPIO archives and combine them as initrds for sysexts (+ measurements).

~~Depended on #167 #166.~~ This adds CPIO packing and open credentials and systemd extensions discovery and load. With this, we should be able to read from drop-in directories, load CPIO archives and combine them as initrds for sysexts (+ measurements).
nikstur (Migrated from github.com) reviewed 2023-04-30 14:56:18 +00:00
blitz commented 2023-05-04 09:40:09 +00:00 (Migrated from github.com)
Copy link

In what situation do we have to create initrds? You only mentioned appending them and cpios can be appended without repacking them.

In what situation do we have to create initrds? You only mentioned appending them and cpios can be appended without repacking them.
RaitoBezarius commented 2023-05-04 13:18:20 +00:00 (Migrated from github.com)
Copy link

In what situation do we have to create initrds? You only mentioned appending them and cpios can be appended without repacking them.

Companion files are not initrd, they are just random files laying around on the ESP.

Therefore, to make them available in your initrd phase, you need to pack them as CPIOs and merge them with the others initrds, hence the need for a CPIO packing phase.

(They are not CPIO on their own at the start, otherwise, it would be cumbersome to manipulate from userspace.)

> In what situation do we have to create initrds? You only mentioned appending them and cpios can be appended without repacking them. Companion files are not initrd, they are just random files laying around on the ESP. Therefore, to make them available in your initrd phase, you need to pack them as CPIOs and merge them with the others initrds, hence the need for a CPIO packing phase. (They are not **CPIO on their own** at the start, otherwise, it would be cumbersome to manipulate from userspace.)
RaitoBezarius (Migrated from github.com) reviewed 2023-05-18 17:25:57 +00:00
rust/stub/src/cpio.rs
RaitoBezarius (Migrated from github.com) commented 2023-05-18 17:25:57 +00:00
Copy link

This is unfinished

This is unfinished
RaitoBezarius (Migrated from github.com) reviewed 2023-05-18 17:26:11 +00:00
rust/stub/src/cpio.rs
RaitoBezarius (Migrated from github.com) commented 2023-05-18 17:26:10 +00:00
Copy link

This is unfinished

This is unfinished
RaitoBezarius commented 2023-05-18 17:26:38 +00:00 (Migrated from github.com)
Copy link

@nikstur I think this CPIO stuff is the good opportunity to introduce our "library crate" for UEFI bootloaders stuff.

@nikstur I think this CPIO stuff is the good opportunity to introduce our "library crate" for UEFI bootloaders stuff.
👍 1
blitz (Migrated from github.com) reviewed 2023-08-25 16:03:26 +00:00
rust/stub/src/cpio.rs
blitz (Migrated from github.com) commented 2023-05-04 09:35:12 +00:00
Copy link

This would be easier to handle if it returns a zero-length vector when zero padding is required. This makes the calling code easier because it removes case distinctions.

This would be easier to handle if it returns a zero-length vector when zero padding is required. This makes the calling code easier because it removes case distinctions.
blitz (Migrated from github.com) commented 2023-05-04 09:36:33 +00:00
Copy link

From an architectural point, it would be great to separate the CPIO writing from any UEFI stuff. So we could split this file into one module that does only CPIO construction in memory (without using any UEFI types) and another module that adds whatever UEFI magic is required.

From an architectural point, it would be great to separate the CPIO writing from any UEFI stuff. So we could split this file into one module that does only CPIO construction in memory (without using any UEFI types) and another module that adds whatever UEFI magic is required.
RaitoBezarius commented 2023-11-15 03:21:37 +00:00 (Migrated from github.com)
Copy link

I decided to be serious about the problem space, so here's my strategy.

pio library

Instead of forcing a CPIO writer inside the linux-bootloader crate, I decided to create a new library called pio which can be used for writing CPIO in no_std, alloc contexts. (name subject to change but like p for pico I/O :>).

It is relying on embedded-io (I/O in no_std) and snafu (for errors in no_std) and rolls its own very basic cursor type for internal purpose. It could do direct I/O via the existing I/O trait and I could move the cursor thing outside, but baby steps.

pio library is a file format library and as with all file format libraries, let's write tests. Writing UEFI tests is not easy, therefore, I decided to move the testing to userspace via cargo test --target x86_64-unknown-linux-gnu, I don't know if I can automate this well enough without xtask, but that should be the way to go IMHO.

As a development dependency, I bring a… CPIO reader (and writer) which also use the same format (newc) which I will use to perform integration testing in various fashions.

I want simple "unit tests" (alignment, read a simple CPIO, etc.), then I want to torture more and more the library as we go on.

Ideally, we could donate this to rust-osdev or whoever is interested in the maintenance at some point.

linux-bootloader library

This library should be responsible to articulate the UEFI APIs to pack CPIO at the high level if I give you a bunch of files via a UEFI filesystem and optionally measure it to the TPM2.

It should also offer a way to merge initrds dynamically.

lanzastub

lanzastub should be discovering system extensions and credentials via linux-bootloader library, measure them if TPM2 is available, send them to the boot linux entrypoint and call the dynamic merger for initrds.

Making this a very simple feature at lanza-level, modulo for new EFI variables to export.

End to end testing

  • build a way to build systemd-credential easily
  • use stage 1 assertions to assert about credentials or sysexts in the initrd?
  • a systemd credential test, put it there, load it, verify you can see the credential in initrd or userspace at least?

I think, if I do this, this PR would be in a good enough shape w.r.t. to testing scenarios.

I decided to be serious about the problem space, so here's my strategy. ## `pio` library Instead of forcing a CPIO writer inside the linux-bootloader crate, I decided to create a new library called `pio` which can be used for writing CPIO in no_std, alloc contexts. (name subject to change but like p for pico I/O :>). It is relying on embedded-io (I/O in no_std) and snafu (for errors in no_std) and rolls its own very basic cursor type for internal purpose. It could do direct I/O via the existing I/O trait and I could move the cursor thing outside, but baby steps. `pio` library is a file format library and as with all file format libraries, let's write tests. Writing UEFI tests is not easy, therefore, I decided to move the testing to userspace via `cargo test --target x86_64-unknown-linux-gnu`, I don't know if I can automate this well enough without `xtask`, but that should be the way to go IMHO. As a development dependency, I bring a… CPIO reader (and writer) which also use the same format (newc) which I will use to perform integration testing in various fashions. I want simple "unit tests" (alignment, read a simple CPIO, etc.), then I want to torture more and more the library as we go on. Ideally, we could donate this to rust-osdev or whoever is interested in the maintenance at some point. ## `linux-bootloader` library This library should be responsible to articulate the UEFI APIs to pack CPIO at the high level if I give you a bunch of files via a UEFI filesystem and optionally measure it to the TPM2. It should also offer a way to merge initrds dynamically. ## lanzastub lanzastub should be discovering system extensions and credentials via `linux-bootloader` library, measure them if TPM2 is available, send them to the boot linux entrypoint and call the dynamic merger for initrds. Making this a very simple feature at lanza-level, modulo for new EFI variables to export. ## End to end testing - [x] build a way to build systemd-credential easily - [x] use stage 1 assertions to assert about credentials or sysexts in the initrd? - [x] a systemd credential test, put it there, load it, verify you can see the credential in initrd or userspace at least? I think, if I do this, this PR would be in a good enough shape w.r.t. to testing scenarios.
👍 1 ❤️ 1
RaitoBezarius commented 2024-01-02 00:14:53 +00:00 (Migrated from github.com)
Copy link

@blitz @nikstur I would appreciate a review on the credentials part, I'm not sure yet what I want to do about local credentials vs. global credentials. I feel like you can always use global credentials and local credentials are almost always useless with lanzaboote except if you use the fat UKIs model? So I suggest removing the code for them from the tool, as it does not make a lot of sense.

For sysexts, I will add more stuff soonish to test them properly.

@blitz @nikstur I would appreciate a review on the credentials part, I'm not sure yet what I want to do about local credentials vs. global credentials. I feel like you can always use global credentials and local credentials are almost always useless with lanzaboote except if you use the fat UKIs model? So I suggest removing the code for them from the tool, as it does not make a lot of sense. For sysexts, I will add more stuff soonish to test them properly.
RaitoBezarius commented 2024-01-02 00:17:01 +00:00 (Migrated from github.com)
Copy link

Note for reviewers: I will work to remove the fork dependency in this PR.

Note for reviewers: I will work to remove the fork dependency in this PR.
RaitoBezarius commented 2024-01-02 00:55:33 +00:00 (Migrated from github.com)
Copy link

Actually, I'm thinking, but I don't think systemd-sysext makes a lot of sense, just merging in /usr or /opt is almost useless for NixOS, so I would rather skip the test for this stuff and leave it as-is.

Actually, I'm thinking, but I don't think systemd-sysext makes a lot of sense, just merging in `/usr` or `/opt` is almost useless for NixOS, so I would rather skip the test for this stuff and leave it as-is.
blitz commented 2024-02-10 20:02:48 +00:00 (Migrated from github.com)
Copy link

I think the separate CPIO module goes in the right direction, but it would make sense to split it off into a separate PR for reviewing.

I think the separate CPIO module goes in the right direction, but it would make sense to split it off into a separate PR for reviewing.
RaitoBezarius commented 2024-02-11 16:13:22 +00:00 (Migrated from github.com)
Copy link

Also split.

Also split.

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
nikstur
Labels
Clear labels   
bug

Something isn't working

  
dependency

Issues related to our dependencies

  
documentation

Improvements or additions to documentation

  
duplicate

This issue or pull request already exists

  
enhancement

New feature or request

  
good first issue

Good for newcomers

  
help wanted

Extra attention is needed

  
invalid

This doesn't seem right

  
question

Further information is requested

  
review-next

   
security

   
stub

   
tool

   
wontfix

This will not be worked on

No labels
bug
dependency
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
review-next
security
stub
tool
wontfix
Milestone
Clear milestone
No items
No milestone
Release 0.4.0
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: raito/lanzaboote#168
No description provided.
Delete branch "cpio-packing"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?