Week 9 ยท Lecture outline
Putting it together, and recovery
Module 12: A whole system. Lab: Lab 9, project build and recovery. Due: Lab 8 and the project checkpoint.
Learning goals
By the end of the week a student can:
- Make an application start on boot as a systemd service with a restart policy.
- Explain how a read-only root filesystem and careful writes keep boot media from being corrupted by a power cut.
- Say what a watchdog and thermal limits do, and use PWM from Linux.
- Recover a board whose boot is broken, and write the recovery as a runbook.
Lecture 1, block 1: starting on boot
- The application is a systemd service, with a restart policy and logs.
- Built on the service unit students wrote in lab 5.
- The demo test: power on and touch nothing.
Lecture 1, block 2: surviving a power cut
- Pulling the power mid-write can corrupt boot media.
- A read-only root filesystem with a small writable area, careful writes and journaling prevent it.
- The project's requirement: it survives having its power pulled.
Lecture 1, block 3: watchdogs, thermals and PWM
- A watchdog restarts a system that hangs.
- Thermals decide whether the board keeps its clock speed under load.
- PWM: a square wave whose duty cycle sets brightness, speed or position; from Linux, the PWM class in
/sys.
Lecture 2, block 1: security basics
- No default passwords.
- SSH keys instead of passwords.
- A plan for updates.
Lecture 2, block 2: recovery
- The boot ROM can always be reached over USB.
- So a broken bootloader or boot media is fixed by starting the board over USB and re-writing its storage.
- Read the console first: the last line names the stage that stopped.
- In the lab, students break only spare boot media, after the TA has tested the recovery path.
Lecture 2, block 3: a runbook
- Written for another student: what they would see, what it means, and each step that brings the board back.
- Tested by following it, not by reading it.
- It is one of the lab 9 hand-ins.
Board demo
- On spare boot media, break the boot configuration on purpose.
- Boot it with the console open, and name the stage from the last line printed.
- Start the board over USB from the phone or with the bench PC's USB boot tool, restore the media with the app or the desktop imager, and boot to the login prompt.
Check questions
Why can a board with a broken bootloader always be recovered over USB?
The boot ROM is fixed in the chip and always runs first. Held in USB boot mode, it takes a first stage over USB, and from there the board's storage can be re-written.
What does a read-only root filesystem protect against, and what does the system still need?
Corruption of the boot media when the power is pulled mid-write. The system still needs a small writable area for the data it must keep, written carefully.