The Rust is fine, but the framework for thinking about software project values is fantastic. Love applying this every time I work on a system to solidify what matters, in what order, and thus better make trade-offs in the overall design. Also, shout out to OpenBSD!
https://www.muppetlabs.com/~breadbox/software/tiny/
Really cool work creating the smallest ELF headers for a Linux binary.
There's also a recorded talk by the author explaining it all in a fun and succinct way.
https://eprint.iacr.org/2025/1237.pdf
Oh, wow. Somebody call the coroner. It's not even smoke, it's just hot air.
I still like the idea of adding an extra post-quantum crypto routine into protocols like Signal did for applications where the extra compute is offset by long term privacy concerns of the data. Even still, given the likelihood that it's just more p-hacking, I'm definitely more worried about anyone even thinking of moving off of battle tested cryptography for post-quantum at this point.
Flip side, I definitely like the suggestion of some non-trivial primes for people to factor to better demonstrate the capabilities of these machines. Reminds me of a great paper I read, The quest to replace passwords: a framework for comparative evaluation of Web authentication schemes. Watching later papers start to use that framework for their password alternative research really seemed to slow down how often papers would invent both a solution and evaluation criteria at the same time. Fooling yourself is easy. That's why it helps to have others create the tests you aim to pass.
https://paulkedrosky.com/honey-ai-capex-ate-the-economy/
If you expect a bubble pop, you should position yourself to take advantage of the implosion. I wonder what will be left to salvage. What can you do with hundreds of mostly empty data centers and going out of business priced floating point compute modules to repurpose for small scale operations? There will be a lot of spare power in the short run since many of these projects are bringing online significant energy capacity. Food for thought.
Another great lecture about writing, and in this case, speech writing, by Larry McEnerney.
https://lewiscampbell.tech/blog/250718.html
So much this. I've been banging my head against a wall explaining to developers that adding random things from the internet to production code is not a good long term plan for so long my mind is starting to mush.
I love calling this methodology Skyrim Driven Development after the sprawling modding scene for that game. Hundreds of thousands of mods, mostly compatible but with all sorts of weird edge cases, poor coding practices, and shoddy support. All that in an ecosystem where most people just blindly download and patch them into their game without any thought to the consequences.
Oh, is there an new pre-alpha framework written by an eager high school student in their parent's basement? Better use it to rewrite the mission critical system for the multimillion dollar business that's employing you.
Did installing something based entirely on the advice of Stack Overflow or an LLM cause problems? Just try adding and removing things randomly until it works again. Better chmod 777 * just in case.
Look this one says it improves things you didn't even know needed improving. Into the soup it goes.
Oh great, they changed the deal and now I need to pay for continued support. I mean, good on them for making money for their hard work, but damn the people at my company who signed us up for this.
Granted, your own developers building all sorts of action at a distance, implicit execution magic by themselves into the product is also hideously problematic. But at least all that code's in your control. You don't have to hope someone somewhere will keep working on it for free for you forever. Or constantly spend time fixing things some random person on the internet did to break your build for the third time this month.
Again, not all dependencies are bad. It's just that everything in engineering has trade-offs. There are ups and downs, and you must weigh them properly to make good products.
https://arxiv.org/abs/2207.08815
At one point, "just use XGBoost", became a bit of a meme in the machine learning community. Now it's all LLM or bust, no matter how trivial the problem. I even had someone explain recently how they were using them to do binary classification…
All that waste kind of makes you wish people would just go back to spamming XGBoost. You'd likely get better results to boot.
https://guide.handmadehero.org/code/day523/
This is some great slow burn satire. Well worth the watch.
I know we all just accept it, but he's on point. Everyone just gets used to git through brute force memorization and aliases. The data structure can be learned, but it only somewhat helps. Fundamentally should you switch, or checkout, or branch, or merge, or rebase, or reset, or restore, or revert, or rerere, or like… what are we even doing? Most people toss like ten words into documenting their commits. Most times I ignore the commit messages because they contain almost no useful information and instead just look at the diff anyway. Some people write good stuff, but if it's 99% pointless, you learn to ignore it. So why are we even writing it? Git says you must. So we toss git a couple words and move on.
If you take a few minutes and try and look dispassionately at the situation, there's definitely room for improvement here but it's popular because it's subject to the collective action problem. Everyone has to use git because everyone else has to use it. I'd say it has about the same usability as GPG. Consider that PGP is dead and everyone just uses WhatsApp or Signal. There's always room for improvement.
Maybe GOT's on the right track? Try and reuse the protocols and data formats so power users can keep using git but fundamentally keep the complexity on the inside and just let it all fade into obscure details of an otherwise simple and easy to use workflow. We'll have to see. Until someone builds a path out of this, at least we can sit back and laugh.
https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdf
This caused a huge splash so you may have already read it. I think it's generally full of excellent insights. There's been a lot of these types of papers experimenting on LLMs to really dissect them and try and understand how they work scientifically.
I am by no means going to claim to understand LLMs emergent behaviour. However, one thing I'm starting to consider is that this may just be what a type of lossy text compression looks like. I'll probably end up posting the keynote at some point, but I remember John Carmack once saying something to the effect of, "Procedural generation is just glorified compression." When you look at procedural generation from the perspective of a decompresser, they look very similar.
Think of it this way, a compressor is a program that takes some input and creates a different program which, when run through an interpreter (the decompresser), yields a result with desired properties mimicking the original data. That last bit may sound weird, but only if you focus on lossless compression. Lossy compression is where it really starts getting interesting. Lossless has a fairly high lower bound. Lossy, now that's an area where we get subjective and weird.
For example, keep dialing down the quality on the compression. When you do, you start to get less faithful artifacts, things that are derived from the original, that in some way bear statistical relation to the original, but who's form after decompression starts to take on a character of their own. Now imagine that an artist is working on the decompresser so that these artifacts are actually somewhat pleasing and desirable instead of the more typical ugly compression artifacts.
That in mind, doesn't procedural generation sound a lot like substituting the compressor for a human? They take a bunch of input data to craft a program which, when executed, yields a result with desired properties mimicking the original data. Like a level generator, the human is removing all the redundant level geometries and instead creating a statistical model that encodes all this redundant information. Now instead of decompressing to get all possible geometries at once, you instead sample, and decompress only part of the space.
With that in mind, LLMs start to sound an awful lot like a compression algorithm. You generate a large model who's data consists of all this statistical similarity from the original, and in the output you sample the space starting at a point (the prompt) and expand to generate a position in the space of all compressed inputs.
That would explain why they have problems counting the number of "R"s in strawberry or solving the Tower of Hanoi. Nobody's written the solutions in the mountains of input data. Why can they now count the number of letters? The companies behind these are doing all sorts of synthetic input data generation. Farms of people creating questions for answers Jeopardy style. Writing programs to procedurally generate more training data helps bulk out the relatively limited human generated work and add examples of things humans haven't bothered to publish.
This doesn't fully account for everything these models seem to be able to do, but the research on these models also seems somewhat limited still. Science is a very slow process of trying to constantly falsify information. It's trivial to declare them god or slight of hand, but it takes time to exhaustively search for a way they can't possibly be and then conclude that you've searched long and hard enough that it's unlikely they aren't. But you also keep an eye out for anyone who can figure out a way to show they aren't, just in case.
Regardless of what my understanding seems to suggest, my thanks to the researchers here for spending the time to better understand what's going on and give us some great insights into this emergent system.
More understanding for the seemingly sudden shifts and what they could mean. Mark Blyth has had a few works that seem to be slightly ahead of the mainstream, so I tend to really pay attention to what he's thinking about and why he thinks these things matter. Anyone telling you what's in store is very likely to get specifics wrong, but general trajectories are often easier to get right and prep for even if they turn out to not matter.
Also, this is probably one of the first question and answer periods for a talk that had any value what so ever. Some people in the audience there really trying to better understand these ideas and play with them together through dialogue. Not just grandstanding or asking about irrelevant things the person in the audience knows about.
I love when you couple this with the video friendlyjordies did about how Australia, having reelected their Labor government, is now perfectly set to do what America couldn't. Australia has Future Made in Australia, which has a lot of the same investment public-private partnerships the now abandoned IRA had. With the end of one, billions, if not trillions in private investment are potentially up for grabs. If not Canada, at least our sister state has a shot at the prize.
https://borretti.me/article/you-can-choose-tools-that-make-you-happy
I maintain a website that I've built entirely by hand, being read by basically nobody, for no other reason than the joy it brings me and maybe the vein hope I can inspire someone to see what I see, or at the very least, consider reconsidering.
That said, yeah, get weird. Never fool yourself, you're the easiest to fool, but also never give in and never give up. As Bernard Shaw wrote, The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
Nobody would have created many of the incredibly cool things we have if they'd stuck to what was easy and what already worked. They thought they saw what could work better and went for it. Many failed, some were right, a few even paid a high price for their dream. Most of the things around you are mealy an accident of history and incredibly hard work, not some divine truth. Reinvent things now and again for no other reason than you can. Do unconventional things for unconventional outcomes. To see what's possible, and to see if you really can make things better. Build your dreams, and maybe change the world. Probably not, but the avalanche starts with a single snowflake.
https://en.wikipedia.org/wiki/Norman_Borlaug
Another one of those people I brought up in a conversation that I think more people should know about. There may not be anyone less well known who's had as big an impact as Norman Borlaug. If you're alive today, that's partly the fault of Norman Borlaug. Truly one of the greats.
https://addxorrol.blogspot.com/2025/07/a-non-anthropomorphized-view-of-llms.html
Less a learning and more a plea. It's so cringe listening to people talk about statistics like they're omnipotent, going to wipe out all life on earth, or obsolete the terrain for a map. Conversations become much more productive when sophists stop trying to bear witness to the next coming in the output of gradient descent. It's just a tool. It's not magic. Yes, this has significantly surpassed the state-of-the-art NLP models of just a few years ago. No, you shouldn't be using it in lieu of actually learning how statistics work and then picking the right model for the job.
As someone who's been using computational statistics and machine learning for about a decade, I'd years ago written off large parts of the literature as mostly non-replicable p-hacking by people running the software equivalent of the egg-drop experiment. So many "advances" boil down to either moving the goal post by making up a benchmark you're conveniently the best at, overfitting a model and declaring it state-of-the-art, Frankensteining an ensemble for a one percent improvement at ten times the cost, or one of a few other shenanigans researchers have been routinely pulling since AlexNet really shook things up.
As it turns out, I was right that Attention Is All You Need was, in fact, game changing. The game I wasn't really paying attention to was BERT, a model so bloated and of such limited utility, I thought it was mostly a joke at the time. Turns out, if you just keep going with the joke, well past reason, until your model begins to use so much energy that it competes with the fossil fuel industry to see who can cook the planet faster, there's some interesting and useful properties to the joke.
Just please stop telling me it's thinking or alive or concious, or any other biological adjective. We still have neither a scientific nor philosophical understanding of what thinking even is. It's just statistically likely unstructured data. Is it useful? Yeah, in some applications. Is it reliable? It's reliable enough for some applications. Is it efficient? Mostly no, but some applications don't yet seem to have efficient alternatives. Using it for those we do, seems pretty foolish given the situation.
Just try and stop it with the pareidolia.
https://floooh.github.io/2018/06/17/handles-vs-pointers.html
Great breakdown on how to handle memory without uncontrolled malloc()/new everywhere. Instead, create a manager for a subsystem that has its own memory pool. It can then use array indices (either on the pool or a lookup structure) which it issues and accepts through its API.
I'd heard about the technique before elsewhere but this breaks it down and gives some great examples. You've already been doing this with files, windows, and other resources the operating system provides you, so you may even understand the idea intuitively. Either way, a great piece on the technique.
https://antranigv.am/posts/2021/08/2021-08-13-13-37/
I'm the type of person to go listen to a live symphony orchestra once every few years. I went to a performance of a famous classic symphony that was preceded by the premier of a brand new symphony. A brand new symphony, delightful, and all I could think while listening was, oh, it's just a movie score.
When the only work for composers is scoring movies, new symphonies are going to sound a lot like movie scores.
When the only work for developers is SaaS, software's going to all start to look like web shit.
So now the Windows start menu is written in React, otherwise promising image viewers require setting up a database and docker, and developers expect you to install their applications by intentionally opening yourself to remote code execution.
Unless a market and/or business model is soon found to bring about a renascence of desktop application development, I'm finding I have to agree with Casey Muratori that gaming really will become the Irish Monasteries of software development.
https://ericwbailey.website/published/you-must-listen-to-rfc-2119/
This is pretty funny. I love when someone pays an artist just to bring something fun into the world. I do it now and again but we can always use more of that.
Art and fun aside, if you don't know RFC 2199, now's your chance to learn one of the most influential RFCs ever written.
Already know that one by heart? Do you know about the related RFC 6919?
If you already know both of those, well then perhaps I could interest you in RFC 3339, what people usually mean when they say ISO 8601.
I could go on, but I won't.