Strange Loop truly was an amazing conference. All the talks there are worth your time. This one is no different.
He's right by the way, this is so sci-fi. Like this is some seriously good software development that's set to make a significant difference and you're what? Optimizing ad placement? I mean, hopefully not, but maybe evaluate if what you're doing is a net positive on society.
Anyway, thoughts. For a long time I've been frustrated at the frankly awful state of quality assurance in software development. Fuzzing is the only testing I really consider as testing at this point. The simulator here often reminded me of Jepsen, since the sensor simulator here is essentially able to sweep through all sorts of scenarios looking for edge cases and issues. Not just crashes, but all sorts of anomalies that go on to affect the entire system over time. Memory leaks, floating point rounding errors, various forms of resource exhaustion.
Why don't developers just do that then? Well, it takes more time and effort and a desire to do a great job instead of check the box. Table driven unit tests that assert add(2, 2) == 4 using a swath of mocks do occasionally find some issues so you can convince yourself it's "good enough." Doing it robustly of course costs more time and money. That in turn means teams under pressure have to systematically pick between building product or building tests. Product building wins because given the conflict, companies only really care about quality in so far as the market forces them to. It's why most developers I've talked to just let AI write tests for their code now. I haven't seen a study yet on this, but I wouldn't be surprised if the results show teams doing this have higher post release fault rates than the status quo (though I could be wrong).
Simultaneously, the only metric people seem to think about when trying to enforce quality by policy (and not culture or incentives) is code coverage, which has two problems. First, code coverage is great at identifying likely problematic code. Code without tests statistically has a high chance of containing bugs (I remember reading something like 70% from one study I can't seem to find anymore). However, if code does have tests, the paper found it was basically no better than random chance as to whether or not it still had bugs. The specific numbers likely vary a bunch between codebases and teams, but the general problem persists. A policy of increasing code coverage may find some bugs, but is more likely (given the time pressures) to create a bunch of mediocre tests that losses the statistical power code coverage actually provides.
So now we get fuzzing for databases (because Kyle Kingsbury has made a career of taking distributed systems defects personally) and spaceships (because people hold public servants to mountainous standards), but you generally skip it for airplanes, security systems, and accounting software. You could spend a lot to have a high quality product, but it turns out people will buy the thing almost regardless of quality if you get the market capture and hostileinteroperability right.
I somehow find myself using Vitter's Algorithm constantly! That's the one the Wikipedia article refers to as "Algorithm R".
Here's a question, what's the fastest way to pick a random line from a really long file?
for line in file: if rand() < 1/++n: samp = line then print(samp).
Reservoir sampling is the magic of picking some fixed size set of items uniformly randomly from an unknown number of items in a single pass. It essentially scales the swap randomness by how many items you've gone through such that it's perfectly equal to the random frequency they would have had if you'd known the number of items you were sampling from to begin with.
Really insightful piece about switching from trying to update the physics using time steps equal to the frame delay (what I normally do) and instead switch to using fixed deltas to integrate under the variable frame rate. Great insight! I'm sure anyone doing physics over the network already knows this, but it was novel for me.
I'd always run into that problem where minimizing the application can cause minutes of frame delay until it's back on screen. I usually just drop the step if it's too big (i.e. pause). They even do that in their final algorithm. One thing I had to play with a bit was the lerp they did using the sub-delta. I worried they had an off by one error but no, they run the accumulator passed the end of the frame time properly. The only artifact is that the first frame has a non-uniform delta. This probably doesn't matter for just about every case, but is a bit interesting.
A long time ago, I decided not to build the Torment Nexus as best I could. It is still possible. It's not perfect. The pickings are slim, and you'll be surrounded by people who are indifferent to suffering. They'd be more than happy to start working on the Torment Nexus if it meant making a few extra bucks. However, there are some software companies out there that are still trying to build things that help society. I'll promise you this though, they won't be a massive company with high pay. You get those by squeezing society for everything you can.
One piece of knowledge to add is that the local minimum described is not actually a problem in high dimensional spaces (like the set of all possible games that could ever be created). The real problem becomes saddle points because it's usually that a few dimensions enter conflict while others are still free to roam but uninteresting in the current state. While the random jumps do help escape the saddle, one of the keys is focusing on something you're not focusing on. Try to move in a random direction by, say, introducing a new constraint or adding a feature from a completely different genre and seeing what happens. In a sense, add a brand new dimension you weren't paying attention to that will pull you in some direction and then see if you found a new path to a great game even if you get rid of the change that finally figured out what's better overall.
I also really enjoyed the bit about having people who are building different prototypes switch prototypes when each thinks they have something interesting. Get them each invested in the other idea instead of just designing their own. Fantastic idea. Way better than try and compromise or do both.
Overall, the boat metaphor makes a great way to remember all these points. Also, notice that absolutely none of this is limited to games. This is all about designing things with a coat of gamedev paint.
I often bring up this bit of Plato whenever someone shouts about kids on lawns. Great to know about in case someone starts droning on about the world they were born into being the only way things could possibly work.
Great talk showing a bunch of simple techniques you can use to take a video game up a level. But be careful with all these visual effects. It can become overwhelming for some people so it's always great if you can turn off unnecessary motion and flashing in the settings. Otherwise, great piece to quickly run through a number of the best little tricks.
If you fell for or keep falling for ontology projects, semantic web, unified type systems, or other grand global metadata projects, this one's for you. Get ready to understand why it is, without a doubt, doomed to utterly fail from the start. Why there are very real systemic forces that will guarantee any large project on such a journey will end in ruin.
An absolute classic paper. Well worth the read. The essential concept is that a self hosted compiler (one that compiles itself from source code) is a system responsible for its own authenticity. You could add a backdoor to a build of the compiler that injects that backdoor when it compiles the compiler. The source code wouldn't need to contain that functionality going forward and nobody's reverse engineering the compiler to look for it. It could sit undetected. Seem too far fetched? Well, it kind of happened in LLVM.
Economic elites and organized groups representing business interests have substantial independent impacts on U.S. government policy, while average citizens and mass-based interest groups have little or no independent influence.
The Rust is fine, but the framework for thinking about software project values is fantastic. Love applying this every time I work on a system to solidify what matters, in what order, and thus better make trade-offs in the overall design. Also, shout out to OpenBSD!
