This is a tech phenomenon that I keep getting blindsided by no matter how much I try to anticipate it.
Physical work feels difficult. You can look at someone and realize you don’t have nearly as much stamina, and even if you did, it still feels demanding.
Research feels difficult. You’re tasked with thinking about something no one else has considered yet. That rarely happens even outside of science – try to tell a unique joke.
But non-algorithmic programming? You’re telling a machine that precisely follows instructions what you want it to do. At best, you’re a technical translator. You’re not working towards a PhD degree. You’re just wiring things together without creating anything intrinsically new. It looks simple, and so it feels easy.
lol. lmao, even.
Experience shows that it’s anything but easy, but it’s always been hard for me to pinpoint exactly why that is the case. And I think I’ve finally found a good answer.
LithiumI’ve recently started caring about Lithium again and did some work on rough edges.
At the API level, all Lithium does is provide throw and catch functions to simulate typed exceptions with panics or a more low-level mechanism. It’s not a good high-level construct, but it’s a useful tool nevertheless.
As a prototype, it could be implemented in 50 lines max. Obviously, optimizing for performance increases the LoC count significantly, but it still seems like it ought to be manageable.
But there’s 200 commits in this repo. There’s a ton of breakage and various small issues that crop up and suddenly I can’t just say this project is finished, forget about it, and use it as foundation for the next one.
The issuesLithium relies on some low-level rustc mechanisms and nightly features, so it needs a CI to let me quickly react to things changing. I automatically run a CI job every week, but I’m thinking about increasing the rate because I’m uncomfortable with breakage being found a bit later than I’d like.
You might think that breakage mostly happens due to changes to nightly features, but that isn’t the case. Lithium has quite a bit of platform-specific code, so I run CI on many targets. And oh boy, do they work badly.
*-pc-windows-gnullvm to have unaligned thread locals. There’s an unmerged fix. Also, panic! just hangs on i686-pc-windows-gnu under Wine, but I can’t even tell if it’s a Rust bug or a Wine bug.So ultimately, the main reason Lithium is so unstable is external design deficiencies and bugs. It’s logically simple, but the lack of a reliable foundation forces me to use hacks or abandon otherwise good approaches.
Or I can desperately try to fix upstream. I can patch rustc, I can maybe touch unwinders and Cranelift. LLVM is where I draw the line, and it looks like that’s just not enough.
This fragility shows up anywhere you look. A significant chunk of complexity comes from setting up CI. You’d think people had perfected this by now, but apparently not:
aarch64-pc-windows-gnu target. x86_64-pc-windows-gnu and aarch64-pc-windows-gnullvm are both supported. I don’t know why.gnullvm target.cargo test --target <target> has skipped over doctests without me knowing, and then started running them after an update, and that’s revealed problems like having to add rustc flags to both RUSTFLAGS and RUSTDOCFLAGS.In fact, I think that nightly-only and internal compiler features are the least of my worries. They shouldn’t just work, yet they absolutely do. Yes, there’s some code smell like having to special-case Miri, and I’m not happy about relying on std or rustc internals, but them being less of a problem than anything else is telling.
FoundationsIt’s infuriating that the tools that are supposed to help us lead us to our demise. And I think this is a common trope in software development.
If you need to solve a complex algorithmic problem, or if you need to optimize a program, you can usually do that. It might be tricky, you might need to research something or write ugly unsafe code, but then you implement it and it works. You write a clever pile of code and it’s done, until the requirements change significantly.
But reliable products are more than code snippets. They always make assumptions, like a frontend developer assuming the JavaScript runtime works correctly, a Rust programmer trusting LLVM, or the Docker runtime trusting Linux not to be stupid. And if this trust fails, you start to lose your sanity. Nothing you can do is guaranteed to solve the issue. It’s a vibes thing, it’s a “just ship it and hotfix if something breaks” world, it’s pure madness.
And it’s real. Only for societal reasons, only because someone didn’t consider an edge case somewhere and now fixing that requires more effort than anyone wants to invest, but that doesn’t make it any less real. It sucks, and it shouldn’t have happened, and it wouldn’t if we didn’t subscribe to the bollocks “worse is better” ideology, but now we have to live with the consequences.
Computing is now obscure, unreliable sorcery. To program is to harness this inscrutable magic. And when you reframe it this way, it finally feels difficult.