In the world of commercial software that’s essentially impossible.
Any immutable infrastructure lights up license keys and demo restrictions like a spotlight.
Take, for example, Apple’s desktop OS: You used to be able to drag-drop an application to install it. To uninstall you would delete it. Simple. Easy. Stateless. They talked about it a lot and so did Mac evangelizers. But it also meant you could walk in to an Apple store, connect a USB drive in your sleeve to a demo computer, drag, drop, and walk away with full versions of very expensive software.
So, the OS fell prey to the same stateful pitfalls as Windows: places to hide keys, system hooks, etc, etc, etc.
Your best bet these days is likely to manage all software config through a system management tool, keep your data backed up in Time Machine, and “reprovision” your laptop every X days or months.
You can work around this in the event that your commercial software provides a consistent interface for the license key stuff; e.g. via the CLI, a key file (ideally plain text) on-disk, or some scriptable interface.
Then you can store this info in separate key management software (e.g. Unix pass for your local machine) at which point bootstrapping your system could be done relatively automatically.
EDIT: I should add that this is my view of how things should work in the-world-as-it-ought-to-be, which isn’t necessarily the same as the-world-that-we-live-in.
Given iCloud/Apple logins, though, which authenticates and creates the keys necessary for the Mac to run signed apps, wouldn’t the process of wiping and reinstalling simply include either not deleting preinstalled apps or starting from a state that downloads the apps you need and signs you in/enrols your Mac as necessary? The same would be true for Windows, Active Directory, and any other state required to run the system that has to sync or work with a third-party.
I guess to me that’s what the /persist drive was in the example, the need to customize system data beyond the basic install steps. Think of your Application Support folder, or apps that don’t sync to the cloud, or system kexts you might need to run apps.
The biggest problem isn’t software like Apple’s which use the Internet to authenticate how many systems are in use with generous limits, it’s Software like iLok and such that use various stateful properties of your system in an undocumented way so it’s hard to preserve them across installs. If that software uses timestamps, for example, it might be hard to preserve what it needs. Keychain might be another example, I haven’t fully investigated how that interacts with other chips like the T2 in these scenarios.
The problem is that outside of servers, it can be hard to distinguish between files I care about and files I don’t. What I’m hoping we get in the future is a filesystem and (on Windows) a registry, that automatically containerizes all saved state to the app and user it belongs to, ideally with some kind of historical log metadata. I know it’s asking for too much and would never be perfect, but it’s a nice thought. Right now we rely too much on apps to be well-behaved, but it’s the apps that aren’t well-behaved or that share data between themselves and other apps that are the issue. Apps asking for access is one approach, but I’m less concerned for privacy in this scenario than I am for state cleanup. The trouble isn’t reading anywhere though, it’s when an app shared state by updating data in another “container”, then you’d have to create containers to represent files shared between apps, or the container idea disappears and you’re left with metadata about file usage and global state as an alternative... At that point you’re limited by how much work you want to do to keep the system clean, and it might be easier to identify anomalies periodically than to keep a whitelist updated with files to persist...
I can’t remember the particulars right now, but there’s a small team that remotely manages the macbooks of Google employees. I suspect that their tools would be a great fit here.
Unfortunately that was 2013, and https://github.com/google/macops only has a couple commits, so those devops processes are likely very outdated.
The mailing list is dead, but as of mid-2017 this statement was made:
> The main google/macops project is still alive, but there is more development currently going on in the other projects we link to (like santa).
> We still use puppet, as do our Linux and Windows fleet, but none of us actually use puppet infrastructure. We are all working with a standalone (masterless) model.
santa seems to be the only actively-developed macOS management tool on the google github profile, but other tools listed in the talk are actively developed (puppet of course, munki, etc)
They started using Puppet against Mac dev machines, then switched to internal tools I believe. But it was one of the first times it occurred to me that we should manage and use DevOps and SRE practices against user machines and user workloads where possible. The trouble is the tooling isn’t that mature yet, and we can’t assume real-time data or always connected machines, and we don’t have a herd because users only have one machine with them at any time and remote state deletion only works when you know you’re not deleting critical state, which in turn requires better tooling and a greater understanding of user application state persistence behaviour than most are willing to invest time in doing.
This is what makes Chromebooks so easy to maintain though: web apps and sandboxed Android apps can all easily sync to the cloud and compartmentalize their data.
I would pay real money for a linux distro that works as well as ChromeOS or macOS that doesn’t have all the phone-home endemic to both (yes, Macs phone home like mad even with iCloud and all of the analytics off).
People have been doing this for a long time. My team uses Puppet & and Ansible to manage Linux user workstations for our university, and the Windows team uses SCCM + other Windows tools. A big reason why we've stuck with Puppet is is it's flexible enough to manage internal systems + and user workstations fairly well, with a lot of the code being shared with all systems Linux/Unix systems we manage.
Does Homebrew not do this for you? You can install pretty much any popular app use cask installs. If Homebrew works for you, you can use a Brewfile similar to a Gemfile or package.json to handle adding and removing apps from your system?
> But it also meant you could walk in to an Apple store, connect a USB drive in your sleeve to a demo computer, drag, drop, and walk away with full versions of very expensive software
Security permissions/demo versions are the proper solution to this, aren't they?
Permissions fails because if your user has the ability to read the application’s files, they can copy it. If they cannot, then neither can the application itself (aka it cannot run).
Demo versions (as in compiled to be a limited demo) are viable, but was counter to Apple’s image at that time (it doesn’t really show off the experience if you’re getting restricted or nagged).
Read yes, but not write to removable drives. This setting is used on every machine in a VFX company I worked for. Although, I don't know how supported it is across platforms.
Right you have a possible win there. Totally slipped my mind since it’s not currently available.
Apple stores still need to allow people to connect their iOS devices and maybe USB sticks for photos, but they could have created a “read-only” policy and applied it to store machines.
Of course, these days they just distribute through the app store, and those apps put tentacles all over the place. No USB copies would function.
I'd love to get to make further inroads, but I treat my MacBook as something that's disposable, there's nothing on it that I care about.
- Pretty much all my software comes from brew or brew cask
- All my files are stored in either DropBox or Google Drive (split is for legacy reasons mostly)
- All my work is in a remote Git repository
- My local config is also in Git
Anything else isn't backed up. I treat ~/Documents as scratch.
It's not ideal because it still takes some time to configure and log on to a bunch of cloud services, but maybe it says something about moving into management that I spent 90% of my day inside of Firefox, Mail, and Slack...
Fun trivia: the original implementation[0] was 17 lines of Ruby, which was absolutely terrible because I didn't know what I was doing. I'm happy to see that it's grown so much!
Projects are often like stone soup, it takes somebody to kick the process off, and I thank you for that.
I saw that it now supports Whalebrew[1], which seems interesting. One of the issues that I have with Brew is that often my need is temporary and that I want to be able to clean up afterwards (similar to `git stash`).
In the same vein as TA, nix-darwin[1] can be used to configure macOS applications and services. It's not exactly the same as NixOS because the whole system is not entirely managed by Nix but it gets you closer.
There is also Ansible, Chef, Puppet, ... but in my experience the overhead to write and test those config files is even hired than using Nix.
In practice you can actually get extremely close to having Nix manage your whole macOS system with the exception of actual OS-level updates (and particular tools like Xcode, I suppose).
John Wiegley’s system configuration [0] is an example of this, although it is extremely ambitious and definitely not a good place to start for anyone who has used Nix before.
One of the interesting things that Wiegley does for further reproducibility is to have Nix manage and install DMG-based Applications [1] in addition to CLI apps, services, and other system-level things.
I find iOS devices get surprisingly close to this.
If you have iCloud backups, do a final sync, then restore to the device you just bought. This has been the process for the last several iPhones I've bought for my family. At the Apple Store, they walk you through this process so everything is set up before you leave.
It's hard to tell the difference between your old and new device, in terms of software and data, for the most part.
I'm not as far along this curve with macOS as Graham is with NixOS (and I'm a little jealous), but I'm approaching something like what you ask for. Feel free to ask questions; I'll try to remember to check back.
I have an air from 2013; I moved (regrettably, given the keyboard) to a new air around this time last year and used the process as an excuse to force myself to specify everything well enough that I could recover everything I need from stock macOS from a short bootstrap script.
I use yadm (~git) for my dotfiles, Nix for everything Nix can do readily, brew bundle for a few mac apps around the edges, and 3 small restic backups for some project files and such. My dotfiles contain a longer bootstrap script that takes over the heavy lifting once yadm is installed. This script configures most of the settings I need on the way, and prompts me to do the few bits I haven't found a good way to automate. It also collapses ~/Downloads, ~/Desktop, and ~/Documents into a single directory, so that I don't have out-of-sight/mind places for state to hide.
It has been a lot of work, but it has bought me peace-of-mind that I can replace my system quickly (i.e., be ready to work on a new device in ~2h with maybe 20 minutes of actual babysitting?)
That said, I think the basics here are actually fairly low-hanging fruit. You can also make some really quick progress once you know you've got everything essential safeguarded if you're willing to take a dive like Graham has here and let minor losses roll off your back. Access to a second mac also helps a lot; iterating on problems in the actual bootstrap process is slow, and best done while you have a working system you can stay productive on.
I'm currently most of the way through building out the rough inverse of my bootstrap script--a script that audits as much of my "state" as it can; I hope to have it working for a living as part of my backup runs soon. It cleans up and auto-commits anything it can, tells me what is in a clean state, reassures me not to worry about everything I've worried about during past moves, and gives me an explicit checklist for anything that isn't buttoned up.
I think the main things left are forcing me to deal with downloads/desktop/documents as I go (with measures like Graham's being on the table...), and figuring out if there's a sane way to audit for drift in my macOS system/app settings.
I'll number based on your post's paragraphs with a zero index :)
1. Safari is my daily driver, so I'd have to trawl other people's dotfiles/bootstrap scripts for evidence anyone's handling FF in a smart way. But, this might still help: Apple recently changed Safari extensions to be an adjunct to native apps. Before they made this change, I had a section of my bootstrap that used the `open` command to launch the Safari extension page for each extension I used, which made it pretty trivial to click install on each tab.
Another thing you might find tractable is looking for FF config/state files that you can back up and restore (narrowly). This is easier to play with if you have a 2nd system as I mentioned. I figured out, for example, how to restore my macOS Terminal.app windows from my previous install with all of my scrollback history.
2. These are rough, but you're already on the right track. It sucks, so maybe bitch at Apple about it--more voices might move the needle here. I have 3 basic approaches: find a defaults (and, rarely, another command) that does the same thing; figure out which plist file is implicated and diff it before/after changes until I figure out how to force the right change (broadly, I find it better to try preference edits with all involved apps closed); AppleScript it. I have applescripts in my bootstrap for stuff like setting my Finder preferences and enabling specific Safari extensions (now that they're bundled with .apps). If you haven't already tried it, UIElementInspector makes it much more plausible to actually write an AppleScript that has to do UI manipulation.
3. No direct comment on nvram as I've never fiddled with it, though I noticed it does have an option for outputting the current settings and a flag for setting them from a file. For defaults, I'm not sure there's anything better than just not doing it unless you add it to your script. (This is where I was wondering about whether there's a good way to diff my delta against my setup script...)
4. Too vague for me to give a good answer. If you can't find a way to set these from the CLI or hacking around with plists, I'm not sure anything shy of AppleScript will help.
5. What do you mean by wireless configs? Passwords and network names? I pull mine in through iCloud keychain and don't otherwise sweat these.
I experimented with a version of this: https://github.com/mathiasbynens/dotfiles. It worked ok. Still had a few things I had to tweak, but if you have a few things you want to keep it's easy. I basically only use a brewfile + a shell script to set a few defaults and some configs for iterm + kitty that I keep in my personal repo. Makes the most repetitive bits a lot easier. The MAS homebrew stuff didn't work too well when I migrated machines so I still had to install some stuff by hand.
i was planning to try it out (currently i use homebrew), but with Catalina it seems things are somewhat complicated for nixos+macos.
my understanding is that NixOS wants to live in "/nix", and Catalina does not like that.
there are some solutions to this problem but they seem somewhat incomplete (you have to make an unencrypted partition for Nix etc..), though i haven't tried it so i might be wrong :)
I don't know if it's usable for a MacOS laptop, but I use Ansible for my various Linux dev machines, and if you handed me a fresh install, once I got my Ansible directory onto the laptop I could get it completely up and running with a single command.
All the big chunks of data are already isolated into redundant partitions, but it’s the system config that’s tough.
I have a time machine backup, but that’s still not the same as being able to say “I’m gonna wipe my hard drive and start over today”.
So does anyone have have good suggestions on maintaining a MacOS laptop in an immutable way?