Planet Russell

,

Charles Stross — Rethinking space opera

A whole decade ago I blogged about a taxonomy of cliches in space opera, with a considerable checklists of things to watch out for in the space opera I was just then starting to work on.

This was almost entirely a background task for most of that time. During the period I was poking at the new space opera project I wrote: the Empire Games trilogy, the New Management trilogy, A Conventional Boy, The Labyrinth Index and The Regicide Report. So, nine other novels. Meanwhile the space opera went through two and a half rewrites, spawned a spin-off novel, and the spin-off novel is now getting its second rewrite, because Reasons (it has to be perfect: also, I'm going to go back to the original space opera next and finish that, too, probably next year, not to mention already having the first 50,000 words of a sequel.).

But anyway: the space opera project has resulted in maybe a third of a million words of fiction so far, and absolutely humongous quantities of world-building work. Which is why I'm now revisiting my earlier thoughts on space opera, to try to better understand what I'm doing here.

First, some clarification. What even is space opera, when it's at home?

While there's no short, concise definition of space opera, there's an extensive monograph on Space Opera in the SF Encyclopedia, tracing its roots back to the US pulp magazine sector around 1900 and following it to 1941, when Wilson Tucker defined it as the appropriate term for the "hacky, grinding, stinking, outworn, spaceship yarn": alternatively, space-adventure stories which have a calculatedly romantic element.

Let's stick a magnifying glass on top of that last phrase. Romaniticism in this context does not have anything to do with the romance genre of fiction (which focusses on love and relationships). Rather it's a shout-out to the romantic movement of the 18th to 19th centuries and its modern descendants, which prioritize passion, intuition, and individualism over social convention—the pursuit of beauty entails an interest in the exotic and mysterious, a celebration of the heroic and the divine, respect for the supernatural, and arguably the will to power and worship of force that took a darker turn in the fascist and totalitarian movements of the 20th century. Stick starships and light sabres and vast vistas of time and space on top of this and you can get an impression of where it's going.

As a romantic genre, space opera requires its protagonists to have enough agency (personal autonomy) to break out of their quotidian setting and have adventures—travel is a big part of this, for it cuts them free of the ties that bind, leaves family behind, and promises escape. To which end, the spaceship is central in space opera just as the sailing ship was key to the age of (ahem) sail. Trying to imagine Moby Dick or Master and Commander without sailing ships is like trying to imagine Star Trek or Star Wars without starships.

You can write romantic adventure-focussed SF without starships (use stargates/wormholes or just set it on a planet where adventure-shaped activities dominate the plot) but what you get then isn't space opera, it's a planetary romance.

(Planetary romance (the sub-genre Jack Vance was famous for) inherently places more constraints on what you can do with your fiction. You can't easily have a planetary setting which combines long travel times, high energy technologies and high population density, for example: long travel times imply no fast vehicles (no high speed rail or air travel, never mind spaceships), which are a side-effect of high energy technologies (internal combustion, gas turbines, big-ass electrical generators and HV grid of any kind) and you can't easily get to any of those things without a sufficiently high population density to support the industries that make them feasible (not just engineering and factories but universities and schools with mass literacy and mining) which in turn implies advanced agriculture—you've got to feed those cities somehow. (Memo to self: hunt down James Burke's Connections on video (they're somewhere on YouTube) and re-watch the series.) It also makes it difficult to justify sub-settings with wildly divergent levels of technology because that stuff leaks—to keep it out you need a North Korean level hermit kingdom, or totally impenetrable jungles that haven't been penetrated by colonialists, or something of that level. (NB: I have found a workaround for this, but I'm not going to spoiler the coming novels!)

Ultimately economics are your enemy if you want to write convincing romanticist fiction without turning into a fantasy-without-the-magic secondary world setting: all it takes is one country inventing capitalism and/or imperialism to muck it up for everybody else). And economics fucks with space opera too: ask yourself what markets plausibly exist for goods and services transported across interstellar distances, and after digesting Paul Krugman's Theory of Interstellar Trade (PDF) you may be left scratching your head.

My go-to answer for economics in space opera is that interstellar expansion/colonization is most plausibly driven by (a) religion ("manifest destiny" is a religious belief, built atop Genesis 9:7), or (b) escapist colonialism (but only if travel is cheap and compatible biospheres are abundant). In third place we probably get pyramid schemes (eg. Neptune's Brood by some guy called Stross: prior art for that one existed in the shape of colonial pyramid schemes, eg. the Darien Scheme which bankrupted Scotland, or the South Sea Bubble, which nearly bankrupted the British government a generation later). Then a long way down the list we get to trade in Veblen Goods, and maybe—if interstellar transport is very cheap—recreation. What the latter two items have in common is that they're status signifiers—the wealthy have been having yachting adventures while the peasants labor on their estates to pay for them for a very long time, and as for Veblen Goods, Faberge eggs are pretty obvious. So we have status as a driver for space opera: a side-effect of hierarchical social structures where the elite seek to maintain their status by signaling their affluence in various ways.

Did I mention hierarchical social structures and capitalism? The thing about capitalism is that it's a totalizing ideology, like the divine right of kings, or religious fundamentalism. It seeks converts, it refuses to be gainsaid, and it confers a competitive advantage on cultures that adopt it (as opposed to those that reject it). Yet, as Ursula le Guin said, "We live in capitalism. Its power seems inescapable. So did the divine right of kings. Any human power can be resisted and changed by human beings..."

Space opera embraces the romanticist outlook. This is inherently problematic, for it is capable of spawning totalitarian nightmares. To understand this point, I dare you to read Marinetti's Futurist Manifesto from 1909 and not recognize it both as a masterpiece of romantic rhetoric and as one of the founding texts of Fascism—very specifically the Italian Fascist movement (Nazism and Spanish clerico-fascism were later tumors on the rancid politics of violence and misogyny that Marinetti unleashed).

It's not possible to examine space opera, or planetary romance, and not acknowledge the pernicious fascistic imagery in both subgenres. Michael Moorcock wrote a magisterial essay about this nexus in the wake of Star Wars (the first movie): I strongly recommend reading his Starship Stormtroopers before pressing on if you're not already familiar with it.

Pace Le Guin, it is possible to write space opera that doesn't pander to fascism directly, as the late Iain M. Banks demonstrated. But even in his magisterial Culture series the pustules of fascism can easily be seen on the faces of the Culture's adversaries: from the Idiran imperialists to the Empire of Azad, perhaps most clearly in the character of the Affront, it takes a bad dose of fascism to introduce the romanticist element necessary for space opera to be perpetrated. Otherwise you're left with something like Inversions, which is a Culture novel with merely homeopathic traces of space opera remaining. Incidentally I make no apologies for assuming a familiar with the Culture novels on the part of the reader: if you haven't read it already, you're missing out, because it's the nearest thing I've seen to a refutation of the core conceit of might-makes-right inherent in the fascist strain of space opera. Iain wrote A Few Notes on the Culture in 1993 to explain what he was about: in particular, I was struck by this summary: The Culture, in its history and its on-going form, is an expression of the idea that the nature of space itself determines the type of civilisations which will thrive there. ... Essentially, the contention is that our currently dominant power systems cannot long survive in space; beyond a certain technological level a degree of anarchy is arguably inevitable and anyway preferable.

Totalizing ideologies like capitalism or Christian Nationalism aren't immutable, and opposition is possible. They rely on a perception of inescapability on the part of their subjects, and space opera should be the ideal fictional vehicle for escapism from such certainties: if our protagonists can board a starship and flee, there is in principle no reason why they can't shed the chains of a pernicious ideological system.

Now I want to cut to a different question. What about the human-scale story?

Space opera isn't usually a genre of high conceptual merit that tries to tell us something about the world we live in. It can be, in the hands of a determined author who knows what they're doing (I'm thinking specifically of Iain Banks, of Bruce Sterling's Schismatrix, of some of Alastair Reynolds' work—notably House of Suns) but it's more often deployed as a giant shiny backdrop for human-scale drama. Fiction is generally the art of confabulating plausible lies that illuminate the human condition, but stars and galaxies exist on a very superhuman scale: if we took them seriously as a setting they'd dwarf the merely human concerns of the protagonists—the emotional cursors that traverses the story universe—into insignificance, as in the more philosophical works of Olaf Stapledon. (For a fine example, see Anvil of Stars by Greg Bear.) Most of us don't have the imaginative flexibility to conceive of time scales significantly longer than a human lifespan and vistas larger than a single planet (and even that is a recent development: I think before mass intercontinental air travel our imaginative horizons were far more constrained). So a common problem in space opera is for the scale of distances to collapse: modern authors tend to fall back on the durations of international air travel, rather than thinking in terms of age-of-sail voyages, let alone more realistic interstellar durations at slower than light speeds. (Honorable exception to that one: Alastair Reynolds, and to a lesser extent, Ursula Le Guin's Hainish Worlds stories and Ken Macleod's Engines of Light tetralogy.)

A side-effect of the romantic outlook is a demand for individual human agency—that is, that despite the vast scale of the backdrop, the protagonists have the ability to effect change, or at least to change their own circumstances. Here in the real world most of us are trapped by situational constraints: our employment system is parodied as wage slavery for a reason. Opportunities for adventure are scarce, and your chances of self-betterment are minimal. (You can found a business and spend 30 years trying to become a millionaire, most likely to end in ignominious failure, or you play the lottery, be it by explicit gambling or by speculation on the cryptocurrency markets, but the game is rigged against you from the start.) Money in large quantities, as capital, exerts a gravitational attraction and we are individually just dust particles. So space opera tends to focus on protagonists who acquire individual agency, have adventures that take them away from the quotidian grind, and let them change themselves or the worlds around them.

A classic plot skeleton for this sort of fiction is the Hero's Journey template, as popularized by Joseph Campbell and demonstrated by George Lucas in Star Wars, and it's less-familiar-to-many sibling, the Heroine's Journey: interestingly the Heroine's Journey can be seen as an explicit rejection of the frenetic diktat of Futurism even though it's compatible with space opera, as demonstrated magisterially by Lois McMaster Bujold, whose Vorkosigan series is bookended by a Heroine's Journey narrative (starting in Shards of Honor, continuing in Barrayar, finally winding up in Gentleman Jole and the Red Queen)—and which holds the singular record for most Hugo awards for a single SF series (so this isn't just my opinion). Character development lives at the heart of both these templates, and I suspect you can't have a working space opera without the scope for such changes.

To briefly hark back to the Futurist/fascist imagery inherent in space opera: the Vorkosigan series is arguably Ruritanian romantic space opera set against a recognizable 19th century central European flavoured empire. (It's no accident that space opera emerged at the end of the age of empires, the turn of the 20th century.) Our main viewpoint character, Cordelia Vorkosigan, is a woman with modern sensibilities who effects change in that society to great effect, working largely behind the scenes as a noblewoman in that setting—indeed, that's the nature of her modified Heroine's Journey arc. (Assuming, of course, that you buy either the Hero's Journey or Heroine's Journey templates, both which are heavily influenced by Jungian psychology and early-to-mid 20th century conceptions of masculine and feminine gender identity, all of which are highly problematic in my opinion.)

Ultimately, though, the romanticist aesthetic is inimical to social convention and formalism—and also to sustainability. The romantic impulse renders space opera antithetical to solarpunk. A fictional setting composed entirely of wild individualists isn't going to end up as some sort of libertarian utopia, it's going to be a cat ranch: meanwhile someone needs to stay behind and tend the home fires, someone has to mind the grain silos on that "small farming planet", someone else needs to manage the shipyard that repairs the interstellar freighter. There's a lot of implied background in space opera, a civilization with enough surplus wealth production to build and operate starships, and this implies some degree of continuity—continuity of biosphere stability (or there'll be nothing to eat), continuity of infrastructure (including education/apprenticeships, to train/socialize the starship crews), continuity of the society our adventurer-protagonists emerged from. Social structures, governance, and politics are a given in every human society, and the romantic movement emerged from a very specific context that coincided with the age of enlightenment and the industrial revolution. Our protagonists are either privileged enough to transcend the daily grind or sufficiently motivated to take a hideous gamble with their own safety. And this tells us something about the constraints within which space operatic storytelling takes place—and my work-in-progress in particular.

Any questions?

Planet Debian — Russell Coker: Kdeconnect DBUS

I’ve tested the DBUS interfaces for Kdeconnect (the system for connecting a KDE desktop to a phone or another computer). Unlike most DBUS interfaces it has a significant tree of interfaces so the busctl command to get the tree is important. Here’s the useful things I discovered:

# list basic interfaces for kdeconnect
qdbus6 org.kde.kdeconnect.daemon /modules/kdeconnect
# get tree of interfaces for kdeconnect
busctl --user tree org.kde.kdeconnect.daemon
# get list of devices
qdbus6 --literal org.kde.kdeconnect.daemon /modules/kdeconnect org.kde.kdeconnect.daemon.deviceNames
# list interfaces for a device (DEVID is a hex string from the above command)
qdbus6 org.kde.kdeconnect.daemon /modules/kdeconnect/devices/$DEVID
# get battery charge
qdbus6 org.kde.kdeconnect.daemon /modules/kdeconnect/devices/$DEVID/battery org.kde.kdeconnect.device.battery.charge

Sending an SMS apparently requires using the QVariantList type which is more pain than I wanted so I tried the kdeconnect-cli command. Here’s a quick summary of how to use it:

# list devices
kdeconnect-cli -l
# send a SMS
kdeconnect-cli -d $DEVID --send-sms "the message" --destination $NUMBER
# find device (ring)
kdeconnect-cli --ring -d $DEVID
# send a notification message
kdeconnect-cli --ping-msg "this is the message" -d $DEVID
# unlock and lock screen (only works on Debian devices for me not Android devices)
kdeconnect-cli --unlock -d $DEVID
kdeconnect-cli --lock -d $DEVID

There doesn’t seem to be support for making phone calls via kdeconnect which is a significant omission. It’s a very common situation to want to call a number that’s listed on a web site or in some other data source that’s easier to access on a PC than on a phone. The clipboard could be used but that’s needless pain.

Related posts:

  1. Nheko DBUS Nheko is my current favourite client for the Matrix IM...
  2. Moving Between Devices I previously wrote about the possibility of transferring work between...
  3. dbus error Strange dbus error from the KDE power monitoring tool...

Charles Stross — Upcoming Public Appearances, 2026/27

Upcoming fixtures

My public visibility took a nose-dive in 2020 (can't think why!) and didn't really begin to pick up again until 2024, due to multiple reasons, principally COVID19 and some chronic health issues. The health issues are now under control and I'm planning on coming out of my shell a bit more over the next year.

My definite fixtures over the next year or so include 5 SF/F conventions in various countries, none of them the United States but one of them on the same continent:

4-8 November 2026, Festival 42, Barcelona, The sixth 42 Barcelona Festival of Fantastic Genres (theme: "Facing Our Fears.")

I'm one of the guests at Festival 42 (although I'm not listed on their web page yet).

My Festival 42 events so far:

1pm, Saturday 7th: Author interview (with Cristina Jurado)

7pm, Saturday 7th: Panel discussion with international authors (details TBA)

12am, Sunday 8th, Panel discussion (details TBA)

11 November 2026, Author Event, Gigamesh Bookshop, author event (reading/interview, TBA). Gigamesh is Barcelona's SF/F specialist bookshop and they've kindly arranged an event for me: I'll add details here when I've got them.

12-15 November 2026, Sui Generis Festival, Madrid (theme: "Neo-Fiction: New Narratives for a Contested Present.") Sui Generis Festival: "We use the term neo-fiction to describe narrative practices that emerge from a world shaped by the ecological crisis, digital culture, artificial intelligence, and social fragmentation."

My Sui Generis events so far:

6pm, Friday 13th November: Panel discussion, "The New Indifferent God: Cosmic Horror and Artificial Intelligence"

7:30pm, Saturday 14th November: Panel discussion, "Masters of the Impossible"

26-29 March 2027, Unconfined, the UK Eastercon, Glasgow.

(C'mon, you didn't expect me to miss this one, did you?)

2-6 September 2027 Montreal Worldcon

Montréal 2027 is a North American worldcon outside the United States! So of course I'm planning on going.

18-21 November 2027, Polcon 2027

Polcon 2027, The Polish national SF convention will be held in Katowice in 2027, and they've invited me as a special guest. Details to be added here when I've got them.

Planet Linux Australia — Auckland has an overlooked traffic pollution problem – and it’s not from exhausts

<https://theconversation.com/auckland-has-an-overlooked-traffic-pollution-problem-and-its-not-from-exhausts-291266>

"Cleaner engine standards and a gradual uptake of electric vehicles have been
credited with helping improve Auckland’s air quality over recent years.

Planet Linux Australia — Outback Australia has desperate need for rooftop PV and batteries to cut crippling dependence on imported diesel

<https://reneweconomy.com.au/outback-australia-has-desperate-need-for-rooftop-pv-and-batteries-to-cut-crippling-dependence-on-imported-diesel/>

"The US war on Iran at the start of 2026 gave Australia a major reminder of why
energy security is yet another reason for the whole of our country to embrace
the energy system transformation to electrification and decarbonisation – a

Planet Linux Australia — ‘It was just pure hatred’: a new report reveals intensifying attacks against LGBTQIA+ people

<https://theconversation.com/it-was-just-pure-hatred-a-new-report-reveals-intensifying-attacks-against-lgbtqia-people-290171>

"Violence against same-sex attracted men through dating apps has become a key
focus in the national debate over how to address hate crime.

Planet Linux Australia — Around 90% of refugee kids live in poverty in their first year in Australia: report

<https://theconversation.com/around-90-of-refugee-kids-live-in-poverty-in-their-first-year-in-australia-report-291446>

"Almost a third of people living in Australia were born overseas, and almost
half have at least one overseas-born parent. Migrant and refugee communities
have been central to the nation’s economic, social and cultural life for

Planet Linux Australia — New Colorado Neighborhood Could Offer Glimpse into the Future of Home Energy Use

<https://insideclimatenews.org/news/26092026/colorado-neighborhood-shows-geothermal-potential-future/>

"KEN CARYL, Colo.—A new residential development south of Denver is the testing
ground for an unusual partnership that could transform home heating and cooling
across the country.

Planet Debian — Steinar H. Gunderson: Decompilation patterns part 2: do-while

Continuing our journey on decompilation patterns, here's another common one: Let's say m2c outputs code like this:

loop_151:
  ...
  if (var_s0 > var_s1) goto loop_151;

then the natural change is:

do {
  ...
} while (var_s0 > var_s1);

Fewer gotos are nearly always good, and this is a much more likely pattern than the original one.

m2c often manages to convert gotos to do/while, but not always when they are e.g. nested in some other loop; so often, you may need to apply some other transformation before you get to this point.

Tomorrow, we'll look at another variation over this topic.

Planet Linux Australia Continuations 2026/40: Green on JRuby

  • Heckuva week. 

  • Hanami Minitest had started failing its scheduled builds due to our ongoing work on Hanami’s reloading support. I fixed it here and here. (Why do I mention such small things here in my weeknotes? Because these are just as important aspects of maintenance as everything else—fixing broken windows—and it all takes time.)

  • Adam merged Alexander’s transient attribute support into Rom Factory, a very handy improvement!

    Adam also finished updating Rom Factory for our repo-sync, and I followed up by preparing it for automated releases. With that done, I released v0.14.0!

  • Nice to see a few things from Sean this week. First, in Hanami Action: I reviewed and merged his fix to handle body parsing errors as 400 Bad Request, and Sean has another in the pipe to handle params parsing errors as 400s too, which I’ll get onto soon.

  • Sean’s also been active rounding out Hanami’s SQLite story! Last week he added sensible default SQLite pragmas to Hanami DB. This week he moved onto applying those defaults to SQLite connections in full Hanami apps (as well as handling the extra files they create). This is going to be a real boost to our SQLite users!

  • After some extra discussion nudged it to the top of my list, I picked up an old PR to implement required options for Dry CLI. After some work to get it into shape, it’s now merged!

  • I also got the chance to review Adam’s new built-in spinner for Dry CLI. I encourage you to go check out this PR! It’s a really well-factored implementation with a very nice API. All of this is to say: our 2.0 release is shaping up very nicely. 

  • I merged a nice bug fix from Maksim Romanov to improve Hanami’s gem detection, which I hope to ship as a patch release later this week. Thanks to Maksim for the PR and to CuteCuteYu for the original report!

  • Hanami’s internal code reloading support is now fully merged! I finished some manual testing and merged the final PR on Friday. Thanks again to Andrea for making this happen!

  • It was a long weekend this weekend, so I was lucky to have a bit of extra time for Hanakai work. I used that to merge Phil’s PR to make Hanami’s CI green on JRuby. After a few pieces of follow-up, we’re now green across the board for Hanami on JRuby! Huge step. Thanks to Paweł for driving so much of this work, and to Phil for helping us get across the line.

  • Michael Kohl’s been getting stuck into Rom these last couple weeks, and he’s already inspired good housekeeping steps. This week we moved our most important Rom repos (rom, rom-sql, rom-factory, rom-rails) into the Hanakai GitHub org, to make it clearer that these are our first area of focus for Rom. We’ll bring more over into the future as we expand our work. Michael’s also restored main as the default branch across these repos, and moved any Rom v6 work off into separate branches. Now we have a good platform to work from.

    I also got Rom’s CI green again this morning, and our testing on Rom Factory has helped us find and work around an issue with JRuby and cache keying. Thanks Nikita for helping with this!

  • Lastly, I managed to carve out the first piece of work from an experiment to support streaming from Hanami View. My thinking is that given we’ve added response streaming to Hanami Action, we should look for a full stack approach to take advantage of this too.

    The first step towards this is a useful one in itself: adding lazy exposures to Hanami View. With these, you can now have a view that renders in multiple formats, and if a format doesn’t need every exposure, we no longer need to pay the cost of computing it.

    I’ll look to introduce the rest of this work over the coming weeks.

  • Have you seen Aaron’s new website and fantastic new “Hanami, Why?” series? If not, go check out the introduction and some initial bits and bobs. I’ve been proofing these for Aaron as he goes. I think they’re great, and so does a lot of the internet, which is really gratifying. Aaron’s writing voice is just a pleasure to read, and his enthusiasm for Hanami really shines through. Thanks Aaron for starting this series! We need more and more of this to help spread the word about Hanami and Hanakai — if you have things to say, I’d love to hear it!

  • We announced thoughtbot as a Hanakai sponsor! I’m so excited to have thoughtbot on board. They’ve done so much for Ruby over so many years, and now that includes supporting Hanakai! Thank you to Rob Whittaker and everyone at thoughtbot for making this happen.

  • Last but not least (heckuva week, right?), I was feeling sponsorship-inclined and started the work to automate the refresh of individual sponsors showing on our sponsor page. I’ll hopefully get that merged this week, and get started on writing the post to wrap up our sponsorship drive for the year.

Cryptogram Another Historic Cipher Falls to AI

This one is from 1809, written by Napoleon’s nephew.

Planet Linux Australia — Audiobooks – May 2026

For the Love of Trains: Celebrating 200 Years of British Railways by Paul Routledge

A mix of rail history and stories of the author’s life and journeys. An okay light read but either topic is better served elsewhere 3/5

The Bigger Picture: My Blockbuster Life & Lessons Learned Along the Way by Jan Landau

Memoir of a film producer (Titanic etc). I think the early parts tend to work better with the last third being less interesting. But in general worth reading. 4/5

My Audiobook Scoring System

  • 5/5 = Brilliant, top 5 book of the year
  • 4/5 = Above average, strongly recommend
  • 3/5 = Average. in the middle 70% of books I read
  • 2/5 = Disappointing
  • 1/5 = Did not like at all

Share

Planet Linux Australia — Gondwana really was a supercontinent. And it helped kick‑start life on Earth

<https://theconversation.com/gondwana-really-was-a-supercontinent-and-it-helped-kick-start-life-on-earth-291460>

"Some 550 million years ago, the Earth looked very different to what it does
today. The Southern Hemisphere was dominated by an enormous, single continent
known as Gondwana.

Planet Linux Australia — The world’s largest oat milk producer has joined an unusual climate journalism campaign

<https://apnews.com/article/climate-week-journalism-advocacy-oatly-094216eb885f9668ec2d66c1f90d9391>

"NEW YORK (AP) — Oatly, the world’s largest producer of oat milk, is sending
a new message to its consumers: Most people want their governments to take
action on climate change.

Planet Linux Australia — “A great result:” 26 turbines from first commercial wind farm almost completely recycled and reused

<https://reneweconomy.com.au/a-great-result-26-turbines-from-first-commercial-wind-farm-almost-completely-recycled-and-reused/>

"Flying in the face of the fearmongering often dragged out to oppose clean
energy technologies, fully renewable energy utility ScottishPower recently
revealed that 99.9 per cent of the materials from Scotland’s first commercial

Worse Than Failure — CodeSOD: The Date and Time and Time

Today's anonymous submitter sends us some legacy Java code.

The task is this: check what time it is. And fortunately, even in the older Java datetime libraries that were terrible, this was an easy task. You just needed to do something like new Date(), which constructs a date time object set to the current time. getTime() picks off just the time portion.

Or, you could do this.

Timestamp current = new Timestamp(new Date().getTime());
Date date = new Date(current.getTime());

This creates a date, picks the time off, populates a timestamp, gets the time from the timestamp, and constructs a new date object.

The old Java date classes were awful, but they never needed you to do this. This is entirely home grown ugliness.

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

365 Tomorrows — Warghost

Author: Julian Miles, Staff Writer I’m just thinking this lot might be further behind the curve than the last flight when – “I got it! I got it!” I kick my bird inverted while replying. “Quebec Ten. Romeo!” There’s the twit, coming up hard – a decent Romeo response: ‘wherefore art thou’ equals get gone […]

The post Warghost appeared first on 365tomorrows.

xkcd — Spectrum Allocation

Rumor has it that they're finally auctioning off the zeppelin navigation bands, but everyone is worried that the scary boat captains will express interest.

,

Rondam Ramblings — Seeking God in Science part 12: A clean-sheet introduction to quantum mechanics and quantum field theory (part 1 of n)

For months now I have been writing the phrase "when we get to quantum mechanics..."  Well, today we finally get to quantum mechanics (QM).  QM is a big topic.  I will not be able to cover it all, or even put a significant dent in it, in one blog post.  The goal of this post is just to give you an overview of the lay of the land, a clean-sheet introduction to the topic for

Planet Linux Australia — Theatre audiences still show up. So why do so many shows fail to make money?

<https://theconversation.com/theatre-audiences-still-show-up-so-why-do-so-many-shows-fail-to-make-money-289883>

"Recent cancellations of major musical theatre productions, including
Waitress and Beetlejuice, have raised an obvious question: has Australian
theatre become too expensive?

Planet Debian — Vincent Bernat: Hacking the Go compiler to efficiently map IPv4 to IPv6

netip.Addr features an Unmap() method returning the unwrapped IPv4 contained in an IPv4-mapped IPv6 address: from ::ffff:203.0.113.10 or ::ffff:cb00:710a, it returns 203.0.113.10.1 There is no Map() or To6() method for the reverse direction. Such a method is trivial to implement, but Go maintainers have rejected it on the grounds that users should write netip.AddrFrom16(ip.As16()) and let the compiler optimize it.2 Today, this pattern is eight times slower than a native method. How can we teach the compiler to optimize this sequence?

The alternatives

Let’s explore three ways to implement the map semantics for netip.Addr. My favorite is to add it to the Go standard library. Go maintainers prefer a small external helper chaining netip.AddrFrom16() and netip.Addr.As16(), hoping the compiler eventually optimizes it. The unsafe package opens a third path, with the same performance as the first solution.

Modifying the Go standard library

Internally, netip.Addr stores any IP address as a 128-bit value with an extra field z to encode the family and the zone:

type Addr struct {
    addr uint128
    z unique.Handle[addrDetail]
}

type addrDetail struct {
    isV6   bool   // IPv4 is false, IPv6 is true.
    zoneV6 string // != "" only if IsV6 is true.
}

var (
    z0    unique.Handle[addrDetail]
    z4    = unique.Make(addrDetail{})
    z6noz = unique.Make(addrDetail{isV6: true})
)

AddrFrom4() encodes an IPv4 address as an IPv4-mapped IPv6 address and sets z to the unique value z4:

// AddrFrom4 returns the address of the IPv4 address given by the bytes in addr.
func AddrFrom4(addr [4]byte) Addr {
    return Addr{
        addr: uint128{
            0,
            0xffff00000000 |
                uint64(addr[0])<<24 | uint64(addr[1])<<16 |
                uint64(addr[2])<<8 | uint64(addr[3])},
        z: z4,
    }
}

Unmap() turns an IPv4-mapped IPv6 address into an IPv4 address by setting the z field to z4:

func (ip Addr) Unmap() Addr {
    if ip.Is4In6() {
        ip.z = z4
    }
    return ip
}

Implementing the reverse direction inside the Go standard library is trivial: we set the z field to z6noz if the address is IPv4.

// To6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func (ip Addr) To6() Addr {
    if ip.Is4() {
        ip.z = z6noz
    }
    return ip
}

As a helper

We can’t access the z field from outside the net/netip package. Instead, we build a small helper around the netip.AddrFrom16(ip.As16()) pattern:

// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func AddrTo6(ip netip.Addr) netip.Addr {
    if ip.Is4() {
        ip = netip.AddrFrom16(ip.As16())
    }
    return ip
}

As an unsafe function

Another solution uses the unsafe package to alter the Addr struct through a proxy with the same memory layout:3

// addrProxy has the same memory layout as netip.Addr.
type addrProxy struct {
    addr [2]uint64      // netip.uint128
    z    unsafe.Pointer // unique.Handle[netip.addrDetail]
}

var (
    anyIPv6    = netip.IPv6Unspecified()
    netipZ6noz = (*addrProxy)(unsafe.Pointer(&anyIPv6)).z
)

// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func AddrTo6(ip netip.Addr) netip.Addr {
    if !ip.Is4() {
        return ip
    }
    (*addrProxy)(unsafe.Pointer(&ip)).z = netipZ6noz
    return ip
}

Benchmarks

On my computer, with Go 1.27.1, the standard library solution costs 0.88 ns per operation, while the solution favored by Go maintainers costs 7.14 ns. The unsafe solution matches the performance of the first one.

goos: linux
goarch: amd64
pkg: github.com/vincentbernat/go-netip-addrto6
cpu: AMD Ryzen 5 5600X 6-Core Processor
                   │     sec/op     │
AddrTo6/safe            7.137n ± 0%
AddrTo6/unsafe         0.8682n ± 2%
AddrTo6/builtin        0.8775n ± 2%

Assembly code

Let’s check the assembly code the compiler generates for each solution.4 The one built into the standard library looks like this:5

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ  net/netip·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE   end                      ; if not, stop here
 MOVQ  net/netip·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

Go’s assembly language is not a direct representation of the underlying machine language: it operates on a semi-abstract instruction set derived from Plan 9’s assembler. It has four pseudo-registers: FP (frame pointer for function arguments), PC (program counter), SB (static base pointer for global symbols), and SP (stack pointer). It also has architecture-specific registers like AX, CX, DX, BX, SI, DI, and R8 to R15. Instructions storing data use their last argument as the destination. Instructions can carry an explicit size suffix: MOVB moves a byte, MOVW 16 bits, MOVL 32 bits, and MOVQ 64 bits. In the example above, the first instruction compares the 64-bit value z4 with the CX register.

The unsafe solution looks almost the same:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ  net/netip·z4(SB), CX  ; check "z" if this is an IPv4 address
 JNE   end                   ; if not, stop here
 MOVQ  netipZ6noz(SB), CX    ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The helper solution has far more instructions. To understand why, let’s look at the code for As16() and AddrFrom16(). They are short enough for the compiler to inline them.

func (ip Addr) As16() (a16 [16]byte) {
    byteorder.BEPutUint64(a16[:8], ip.addr.hi)
    byteorder.BEPutUint64(a16[8:], ip.addr.lo)
    return a16
}

func AddrFrom16(addr [16]byte) Addr {
    return Addr{
        addr: uint128{
            byteorder.BEUint64(addr[:8]),
            byteorder.BEUint64(addr[8:]),
        },
        z: z6noz,
    }
}

We can already guess the pattern to optimize: the code packs the IP address into an array, copies it, then unpacks it. If we inline the Go code by hand, we get:

func AddrTo6(input netip.Addr) netip.Addr {
    if !input.Is4() {
        return input
    }

    var a16 [16]byte
    byteorder.BEPutUint64(a16[:8], input.addr.hi)
    byteorder.BEPutUint64(a16[8:], input.addr.lo)

    addr := a16

    var output netip.Addr
    output.addr.hi = byteorder.BEUint64(addr[:8])
    output.addr.lo = byteorder.BEUint64(addr[8:])
    output.z = netip.z6noz
    return output
}

As humans, we can mentally derive the optimized form:

func AddrTo6(input netip.Addr) netip.Addr {
    if !input.Is4() {
        return input
    }
    var output netip.Addr
    output.addr.hi = input.addr.hi
    output.addr.lo = input.addr.lo
    output.z = netip.z6noz
    return output
}

Unfortunately, as of Go 1.26.8, the compiler is not smart enough to do the same:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
; Push the stack (32 bytes):
;    0(SP) addr netip.uint128
;   16(SP) a16 [16]byte
 PUSHQ   BP
 MOVQ    SP, BP
 SUBQ    $32, SP

 CMPQ    net/netip·z4(SB), CX  ; check "z" if this is an IPv4 address
 JNE     end                   ; if not, stop here

; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi)
;       byteorder.BEPutUint64(a16[8:], input.addr.lo)
 MOVBEQ  AX, net/netip·a16+16(SP)
 MOVBEQ  BX, net/netip·a16+24(SP)

; addr = a16, 16 bytes at once through the vector register X0
 MOVUPS  net/netip·a16+16(SP), X0
 MOVUPS  X0, net/netip·addr(SP)

; CX = netip.z6noz
 MOVQ    net/netip·z6noz(SB), CX
; Unpack: output.addr.hi = byteorder.BEUint64(addr[:8])
;         output.addr.lo = byteorder.BEUint64(addr[8:])
 MOVBEQ  net/netip·addr(SP), AX
 MOVBEQ  net/netip·addr+8(SP), BX

end:
 ADDQ    $32, SP
 POPQ    BP
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The compiler does a decent job on the byte shuffling: the eight byte stores of BEPutUint64() become a single MOVBEQ, which stores a register byte-swapped. The eight byte loads of BEUint64() become a single MOVBEQ the other way round.6 Three groups of instructions remain: a pack, a copy, and an unpack.

Hacking the Go compiler

The Go compiler has several phases:

Parsing
The compiler tokenizes and parses the source code. It builds a syntax tree for each source file.
Type checking
The compiler maps each identifier to the object it denotes, folds constants, and infers the type of every expression.
IR construction
The compiler converts the syntax tree and its types into its own intermediate representation (IR). This process, called “noding,� goes through a serialization format named unified IR.
Middle end
The compiler performs several optimization passes on the IR, such as devirtualization, function call inlining, and escape analysis.
Walk
This phase runs two steps: order of evaluation decomposes complex statements into simpler ones, and desugaring transforms higher-level Go constructs, like switch or channels, into more primitive instructions or calls to the runtime.
Generic SSA
The compiler converts the IR into Static Single Assignment (SSA) form, a lower-level intermediate representation suited for machine-independent optimizations and rewrite rules.
Machine code generation
The compiler rewrites the SSA form into machine-specific variants, allocates registers, and applies more optimization passes. At the end, the assembler turns the generated instructions into machine code.

The hammer

My first idea is to replace occurrences of netip.AddrFrom16(ip.As16()) with netip.Addr{addr: ip.addr, z: netip.z6noz} as early as possible, during the “noding� process. Before that, the type checking phase prevents us from accessing unexported struct fields.

Go 1.27 introduced a convenient debug option to dump the IR of a function at interesting points during compilation:

$ GOTOOLCHAIN=go1.27.1 GOAMD64=v3 go build -a -gcflags="-d=astdump=AddrTo6Safe" .
Writing text ast output for AddrTo6Safe to AddrTo6Safe.ast
Writing html ast output for AddrTo6Safe to AddrTo6Safe.html
Writing html syntax output for AddrTo6Safe to AddrTo6Safe.syntax.html

In the HTML file, the first column shows the IR as it comes out of noding:

DCLFUNC addrto6.AddrTo6Safe ABI:ABIInternal FUNC-func(netip.Addr) netip.Addr
DCLFUNC-Dcl
. NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. NAME-addrto6.~r0 Class:PPARAMOUT Offset:0 OnStack netip.Addr
DCLFUNC-body
. IF # ipv6_safe.go:11:2
. IF-Cond
. . CALLFUNC bool
. . CALLFUNC-Fun
. . . METHEXPR addrto6.Is4 FUNC-func(netip.Addr) bool
. . . . TYPE netip.Addr Class:PEXTERN Offset:0 type netip.Addr
. . CALLFUNC-Args
. . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. IF-Body
. . AS # ipv6_safe.go:12:6
. . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . . CALLFUNC netip.Addr
. . . CALLFUNC-Fun
. . . . NAME-netip.AddrFrom16 Class:PFUNC Offset:0 Used FUNC-func([16]byte) netip.Addr
. . . CALLFUNC-Args
. . . . CALLFUNC ARRAY-[16]byte
. . . . CALLFUNC-Fun
. . . . . METHEXPR addrto6.As16 FUNC-func(netip.Addr) [16]byte
. . . . . . TYPE netip.Addr Class:PEXTERN Offset:0 type netip.Addr
. . . . CALLFUNC-Args
. . . . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. RETURN # ipv6_safe.go:14:2
. RETURN-Results
. . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr

In the body of the if statement, we spot the calls to the method netip.Addr.As16() and to the function netip.AddrFrom16(). Our goal is to patch them with a struct literal:

IF-Body
. AS # ipv6_safe.go:12:6
. . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . STRUCTLIT netip.Addr
. . STRUCTLIT-List
. . . STRUCTKEY netip.addr
. . . . DOT netip.addr netip.uint128
. . . . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . . STRUCTKEY netip.z
. . . . NAME-netip.z6noz Class:PEXTERN Offset:0 unique.Handle[net/netip.addrDetail]

In noder’s reader.go, the expr() method builds the IR tree for an expression. At the end of the exprCall case, we add a call to a rewriteAddrFrom16As16() function. It takes the current node and returns the struct literal on success, or nil if the rewrite is not possible. First, we check that we have the expected pattern: a call to the netip.AddrFrom16() function with a call to the netip.Addr.As16() method as its only argument:

func rewriteAddrFrom16As16(n ir.Node) ir.Node {
    call, ok := n.(*ir.CallExpr)
    if !ok || call.Op() != ir.OCALLFUNC ||
        len(call.Args) != 1 || len(call.Init()) != 0 ||
        !isNetipFunc(call.Fun, "AddrFrom16") {
        return nil
    }
    inner, ok := call.Args[0].(*ir.CallExpr)
    if !ok || inner.Op() != ir.OCALLFUNC ||
        len(inner.Args) != 1 || len(inner.Init()) != 0 ||
        !isNetipFunc(inner.Fun, "Addr.As16") {
        return nil
    }
    x := inner.Args[0]
    // [...]
}

Then, we fetch netip.z6noz:

z6noz, err := lookupVar(ir.StaticCalleeName(call.Fun).Sym().Pkg, "z6noz")
if err != nil {
    return nil
}

And we build the struct literal:

typ := call.Type()
pos := call.Pos()
var list []ir.Node
for i, f := range typ.Fields() {
    var value ir.Node
    switch f.Sym.Name {
    case "addr":
        value = typecheck.DotField(pos, x, i)
    case "z":
        value = z6noz
    default:
        return nil
    }
    list = append(list, ir.NewStructKeyExpr(pos, f, value))
}
lit := ir.NewCompLitExpr(pos, ir.OSTRUCTLIT, typ, list)
lit.SetTypecheck(1)
return lit

Have a look at the complete patch.7 We can test it with the following commands:

$ cd src
$ ./make.bash
Building Go cmd/dist using /usr/lib/go-1.27. (go1.27.1 linux/amd64)
Building Go toolchain1 and bootstrap cmd/go (go_bootstrap) using /usr/lib/go-1.27.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 and commands using go_bootstrap and Go toolchain2.
Checking command staleness for linux/amd64.
---
Installed Go for linux/amd64 in /home/bernat/code/free/go
Installed commands in /home/bernat/code/free/go/bin
*** You need to add /home/bernat/code/free/go/bin to your PATH.
$ export PATH=$PWD/../bin:$PATH
$ go version
go version go1.28-devel_9834516e20 Sat Sep 12 08:23:11 2026 -0700 linux/amd64
$ go test net/netip/...
ok      net/netip   0.224s
$ cd ../../go-netip-addrto6
$ go test .
ok      github.com/vincentbernat/go-netip-addrto6   0.062s

The generated code for the helper is now the shortest possible version!

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ    net/netip·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE     end                      ; if not, stop here
 MOVQ    net/netip·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

Go maintainers are unlikely to accept this change. It relies on the internal structure of net/netip.Addr. It’s an ugly hack in the noder, whose job is to faithfully translate the type-checked AST into the IR. And it’s harder to maintain than adding a To6() method.

The screwdriver

The right place for such an optimization is the generic SSA phase. One of the last machine-independent passes is memcombine. With the appropriate debug flag, the compiler dumps the SSA form after this pass:8

$ GOTOOLCHAIN=go1.26.8 GOAMD64=v3 \
> go build -a -gcflags='-d=ssa/memcombine/dump=AddrTo6Safe' .
$ head -5 AddrTo6Safe_01__memcombine.dump
AddrTo6Safe func(netip.Addr) netip.Addr
  b2:
    (?) v1 = InitMem <mem>
    (?) v2 = SP <uintptr>
    (?) v3 = SB <uintptr>

The result of the memcombine pass follows the same structure as the assembly code for AddrTo6Safe() we looked at earlier: two stores, one move, and two loads we would like to optimize away.

; […]
  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v466 = ArgIntReg <*netip.addrDetail> {ip+16} [2]  ; input.z
; […]
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)

  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16

  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v416 = Load <uint64> v285 v286                    ; addr[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi
  v174 = Load <uint64> v415 v286                    ; addr[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo
; […]

Each line features a value identifier (v442), an operation with its type (Bswap64 <uint64>), and its arguments (v490).9 Values are the basic building blocks of SSA and are defined exactly once. Square brackets enclose integer parameters ([8]) and curly braces contain auxiliary arguments ({netip.addr}). Operations writing to memory produce a new memory state. Every memory operation takes the current state as its last argument, which keeps them in order.

On paper

Let’s focus on output.addr.lo, aka v39:

  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v174 = Load <uint64> v415 v286                    ; addr[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

To simplify this code, we could apply three rewriting rules:

  1. The first one adds a shortcut when loading through a move: (Load (OffPtr [o] p) (Move p src mem)) => (Load (OffPtr [o] src) mem). This matches v174 with its arguments v415 and v286 and creates a new value v600:

      v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Load <uint64> v600 v282                    ; a16[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

  2. The second one simplifies a load following a store: (Load p (Store p x _)) => x. The load is forwarded: the stored value replaces it and no memory access remains. It matches v174. It notices that v600 and v173 are the same address and replaces v174 with a copy of v442:

      v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Copy <uint64> v442                         ; bswap(input.addr.lo)
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

  3. The last step cancels the two byte swaps: (Bswap64 (Bswap64 x)) => x. v39 becomes a copy of v490:

      v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Copy <uint64> v442                         ; bswap(input.addr.lo)
  v39  = Copy <uint64> v490                         ; output.addr.lo = input.addr.lo

If we ignore the values not needed to compute v39, only this SSA form remains:

  v490 = ArgIntReg <uint64> {ip+8} [1]  ; input.addr.lo
  v39  = Copy <uint64> v490             ; output.addr.lo = input.addr.lo

Let’s switch to output.addr.hi, aka v299:

  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Load <uint64> v285 v286                    ; addr[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

To optimize it away, we also apply three rewriting rules:

  1. The first one also adds a shortcut when loading through a move, but without an offset: (Load p (Move p src mem)) => (Load src mem). This rewrites v416 to use arguments from v286:

      v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Load <uint64> v22 v282                     ; a16[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

  2. The second rule forwards a value stored one step earlier, skipping over a store to another address: (Load p (Store q _ (Store p x _))) => x. This matches v416: x is v542, p is v22 (&a16), q is v173 (&a16[8]), and p and q do not overlap for uint64.

      v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Copy <uint64> v542                         ; bswap(input.addr.hi)
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

  3. The third rule cancels two byte swaps: (Bswap64 (Bswap64 x)) => x. v299 becomes a copy of v502:

      v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Copy <uint64> v542                         ; bswap(input.addr.hi)
  v299 = Copy <uint64> v502                         ; output.addr.hi = input.addr.hi

If we remove the values not used to compute v299, we get this SSA form:

  v502 = ArgIntReg <uint64> {ip+0} [0] ; input.addr.hi
  v299 = Copy <uint64> v502            ; output.addr.hi = input.addr.hi

In practice

Most of these rules already exist in generic.rules. They use conditions to validate their context: ssa.IsSamePtr() for the same address, ssa.Disjoint() for addresses that do not overlap. The rule forwarding a stored value to a load already exists with three variants looking through several other stores. Here are the two we need:

(Load <t1> p1 (Store {t2} p2 x _))
    && ssa.IsSamePtr(p1, p2)
    && copyCompatibleType(t1, x.Type)
    && t1.Size() == t2.Size()
    => x
(Load <t1> p1 (Store {t2} p2 _ (Store {t3} p3 x _)))
    && ssa.IsSamePtr(p1, p3)
    && copyCompatibleType(t1, x.Type)
    && t1.Size() == t3.Size()
    && ssa.Disjoint(p3, t3, p2, t2)
    => x

Go 1.27 added the rule loading through a move with CL 748200 to fix issue #77720:

(Load <t1> op1:(OffPtr [o1] p1) move:(Move [n] p2 src mem))
    && o1 >= 0 && o1+t1.Size() <= n && ssa.IsSamePtr(p1, p2)
    && !ssa.IsVolatile(src)
    => @move.Block (Load <t1> (OffPtr <op1.Type> [o1] src) mem)

It lacks a variant without an offset:

(Load <t1> p1 move:(Move [n] p2 src mem))
    && p1.Op != ssaop.OpOffPtr
    && t1.Size() <= n && ssa.IsSamePtr(p1, p2)
    && !ssa.IsVolatile(src)
    => @move.Block (Load <t1> (OffPtr <p1.Type> [0] src) mem)

There is no generic rule to cancel two byte swaps, but the AMD64 lowering pass includes this rule:

(BSWAP(Q|L) (BSWAP(Q|L) p)) => p

After switching to Go’s development branch and adding the missing rule, the generated assembly code is worse than with Go 1.26.8, even though our additional rule slightly improves the situation at the end:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
; Push the stack (16 bytes):
;    0(SP) a16 [16]byte
 PUSHQ   BP
 MOVQ    SP, BP
 SUBQ    $16, SP

 CMPQ    net/netip·z4(SB), CX       ; check "z" if this is an IPv4 address
 JNE     end                        ; if not, stop here

; The four forwarded bytes: the low half of input.addr.lo is taken apart and
; put back together in registers
 MOVQ    BX, DX                     ; DX = input.addr.lo
 SHRQ    $24, BX                    ; BX = input.addr.lo >> 24
 MOVQ    DX, SI                     ; SI = input.addr.lo
 SHRQ    $16, DX                    ; DX = input.addr.lo >> 16
 MOVQ    SI, DI                     ; DI = input.addr.lo, kept for the pack
 SHRQ    $8, SI                     ; SI = input.addr.lo >> 8
 MOVBLZX DIB, R8                    ; R8 = byte(input.addr.lo)
 MOVBLZX SIB, SI                    ; SI = byte(input.addr.lo >> 8)
 SHLQ    $8, SI
 ORQ     R8, SI                     ; SI = two low bytes of input.addr.lo
 MOVBLZX DL, DX                     ; DX = byte(input.addr.lo >> 16)
 SHLQ    $16, DX
 ORQ     SI, DX
 MOVBLZX BL, BX                     ; BX = byte(input.addr.lo >> 24)
 SHLQ    $24, BX
 ORQ     DX, BX                     ; BX = input.addr.lo & 0xffffffff

; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi)
;       byteorder.BEPutUint64(a16[8:], input.addr.lo)
 MOVBEQ  AX, net/netip·a16(SP)
 MOVBEQ  DI, net/netip·a16+8(SP)

; The four other bytes of input.addr.lo, read one by one from a16
 MOVBLZX net/netip·a16+11(SP), DX   ; a16[11]
 SHLQ    $32, DX
 ORQ     DX, BX
 MOVBLZX net/netip·a16+10(SP), DX   ; a16[10]
 SHLQ    $40, DX
 ORQ     DX, BX
 MOVBLZX net/netip·a16+9(SP), DX    ; a16[9]
 SHLQ    $48, DX
 ORQ     DX, BX
 MOVBLZX net/netip·a16+8(SP), DX    ; a16[8]
 SHLQ    $56, DX

; output.z = netip.z6noz
 MOVQ    net/netip·z6noz(SB), CX
; output.addr.hi = byteorder.BEUint64(a16[:8])
 MOVBEQ  net/netip·a16(SP), AX
; output.addr.lo assembled from the previous steps
 ORQ     DX, BX

end:
 LEAVEQ
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The rule loading through a move, added in Go 1.27, introduced this regression.

Out of order

Let’s not give up now! In reality, the rewriting rules run before memcombine, notably in the late opt pass. At this point, the inlined versions of BEPutUint64() and BEUint64() still expand to sixteen byte stores and sixteen byte loads, matching their source code:

func BEUint64(b []byte) uint64 {
    _ = b[7] // bounds check hint to compiler; see golang.org/issue/14808
    return uint64(b[7]) | uint64(b[6])<<8 | uint64(b[5])<<16 | uint64(b[4])<<24 |
        uint64(b[3])<<32 | uint64(b[2])<<40 | uint64(b[1])<<48 | uint64(b[0])<<56
}

Let’s follow two bytes of output.addr.lo: addr[15] and addr[11]. Here is a simplified SSA form before late opt:

  v273 = Trunc64to8 <byte> v490                     ; byte(input.addr.lo)
  v226 = Trunc64to8 <byte> v225                     ; byte(input.addr.lo >> 32)
; […]
  v235 = Store <mem> {byte} v233 v226 v223          ; a16[11] = byte(lo >> 32)
  v247 = Store <mem> {byte} v245 v238 v235          ; a16[12] = …
  v259 = Store <mem> {byte} v257 v250 v247          ; a16[13] = …
  v271 = Store <mem> {byte} v269 v262 v259          ; a16[14] = …
  v282 = Store <mem> {byte} v280 v273 v271          ; a16[15] = byte(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
; […]
  v433 = OffPtr <*byte> [15] v285                   ; &addr[15]
  v435 = Load <byte> v433 v286                      ; addr[15]
  v479 = OffPtr <*byte> [11] v285                   ; &addr[11]
  v481 = Load <byte> v479 v286                      ; addr[11]

The first rule loads through the move: (Load (OffPtr [o] p) (Move p src mem)) => (Load (OffPtr [o] src) mem). It matches both loads, which now read a16 with the memory state before the copy:

  v600 = OffPtr <*byte> [15] v22 ; &a16[15]
  v435 = Load <byte> v600 v282   ; a16[15]
  v601 = OffPtr <*byte> [11] v22 ; &a16[11]
  v481 = Load <byte> v601 v282   ; a16[11]

The second rule shortcuts a load following a store: (Load p (Store p x _)) => x. It matches v435, as v282 stores a16[15]. It does not match v481: v235 stores a16[11] four stores earlier in the chain, while the variants of this rule look through three stores at most.

  v435 = Copy <byte> v273        ; byte(input.addr.lo)
  v601 = OffPtr <*byte> [11] v22 ; &a16[11]
  v481 = Load <byte> v601 v282   ; a16[11]

The same happens to the other bytes: the rule forwards the four bytes stored last, a16[12] to a16[15]. The twelve other loads now read a16 instead of addr.

BEUint64() becomes a chain of Or64, each one adding a byte shifted into place. memcombine is a pass written in Go, not a set of rewrite rules. It starts from the last Or64 of the chain and collects up to eight terms. If each term is a byte load, extended to 64 bits and shifted, and if the eight loads read consecutive addresses from the same pointer with the same memory state, it replaces the whole chain with a single 64-bit load and a byte swap. Otherwise, it tries again with four, then two terms, and from each intermediate Or64. Here is the loop checking each term in a simplified version of combineLoads():

for i := int64(0); i < n; i++ {
    v := a[i]
    shift := int64(0)
    if v.Op == shiftOp {
        v, shift = peelShift(v)
    }
    if v.Op != extOp {
        return false
    }
    load := v.Args[0]
    if load.Op != ssaop.OpLoad {
        return false
    }
    if load.Args[1] != mem {
        return false
    }
    p, off := splitPtr(load.Args[0])
    if p != base {
        return false
    }
    r[i] = LoadRecord{load: load, offset: off, shift: shift}
}

For output.addr.hi, the eight loads read a16 with the same memory state v282:

  v13  = Load <byte> v22 v282               ; a16[0]
  v530 = Load <byte> v14 v282               ; a16[1]
  v488 = Load <byte> v504 v282              ; a16[2]
  v405 = Load <byte> v537 v282              ; a16[3]
  v385 = Load <byte> v397 v282              ; a16[4]
  v361 = Load <byte> v373 v282              ; a16[5]
  v196 = Load <byte> v63 v282               ; a16[6]
  v432 = Load <byte> v315 v282              ; a16[7]
  v319 = ZeroExt8to64 <uint64> v432         ; uint64(a16[7])
  v329 = ZeroExt8to64 <uint64> v196         ; uint64(a16[6])
  v330 = Lsh64x64 <uint64> [true] v329 v138 ; uint64(a16[6]) << 8
  v331 = Or64 <uint64> v319 v330            ; a16[7] | a16[6] << 8
; […] same for a16[5] to a16[1]
  v401 = ZeroExt8to64 <uint64> v13          ; uint64(a16[0])
  v402 = Lsh64x64 <uint64> [true] v401 v55  ; uint64(a16[0]) << 56
  v403 = Or64 <uint64> v402 v391            ; | a16[0] << 56 = output.addr.hi

memcombine merges them into one load and a swap:

  v286 = Load <uint64> v22 v282   ; a16[:8]
  v285 = Bswap64 <uint64> v286    ; output.addr.hi

For output.addr.lo, here is the chain memcombine sees after late opt:

  v436 = ZeroExt8to64 <uint64> v273 ; addr[15], forwarded
  v448 = Or64 <uint64> v436 v447    ; | addr[14] << 8, forwarded
  v460 = Or64 <uint64> v459 v448    ; | addr[13] << 16, forwarded
  v472 = Or64 <uint64> v471 v460    ; | addr[12] << 24, forwarded
  v484 = Or64 <uint64> v483 v472    ; | a16[11] << 32, loaded
  v496 = Or64 <uint64> v495 v484    ; | a16[10] << 40, loaded
  v508 = Or64 <uint64> v507 v496    ; | a16[9] << 48, loaded
  v520 = Or64 <uint64> v519 v508    ; | a16[8] << 56, loaded

From v520, four of the eight terms are forwarded bytes, not loads from memory, and memcombine can’t combine them. It doesn’t merge the four remaining loads either, as they sit on top of the forwarded bytes.

Back in order

In summary, the rewriting rules run too early to be effective. A quick workaround exists: run an earlier round of memcombine before late opt. After this change, the generated code for the helper is back to the shortest possible version:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ    net/netip·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE     end                      ; if not, stop here
 MOVQ    net/netip·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

And the benchmark confirms it! ✌�

goos: linux
goarch: amd64
pkg: github.com/vincentbernat/go-netip-addrto6
cpu: AMD Ryzen 5 5600X 6-Core Processor
                   │   Go 1.26.8    │             Our branch              │
                   │     sec/op     │    sec/op     vs base               │
AddrTo6/safe          6.5470n ±  0%   0.8944n ± 4%  -86.34% (p=0.002 n=6)
AddrTo6/unsafe        0.9071n ±  3%   0.8682n ± 1%   -4.28% (p=0.002 n=6)
AddrTo6/builtin       0.8871n ±  2%   0.8785n ± 1%        ~ (p=0.310 n=6)

Next steps

I think Go maintainers would reject this change because of the additional memcombine pass. Instead, I plan to publish this blog post and bring up the subject again as a follow-up to issue #54365. Either the sheer complexity and the Go 1.27 regression convince the maintainers that adding a To6() method is simpler and more efficient, or they advise me on how to move forward. Either way, digging into this subject taught me a lot about the Go compiler! ⚙�

Update (2026-10)

I opened issue #81994 to propose Addr.To6(). Give it a � if you want it in Go!

  1. IPv4-mapped IPv6 addresses let you handle only IPv6 addresses in your code and keep conversions at a few well-defined boundaries. I use them in Akvorado. ↩

  2. This is not strictly equivalent. The zero value becomes :: while it would make more sense to leave it untouched. ↩

  3. To avoid catastrophic bugs when netip.Addr’s layout changes, you must add tests to detect it. This is more dangerous if you put this code in a package. Either ask users to run the tests themselves, or detect the change at run time and panic. Hiding this optimization behind a build tag would make users aware of this potential trap. ↩

  4. I produced the assembly code with GOTOOLCHAIN=go1.26.8 GOAMD64=v3 ./go-asm '\.asm' from the companion repository, then edited it a bit to keep this article from turning into an endless rabbit hole. Due to an unfortunate sequence of events, we switch between versions: Go 1.27 introduced a regression that blurs the point of this article. ↩

  5. The function is short enough for the compiler to inline it, so the assembly code may vary depending on the surrounding code. ↩

  6. MOVBEQ requires GOAMD64=v3, matching the x86-64-v3 microarchitecture from 2013. Otherwise, the compiler translates BEPutUint64() to BSWAPQ+MOVQ and BEUint64() to MOVQ+BSWAPQ. ↩

  7. When the call is used as a statement, a struct literal alone is invalid. The patch then turns it into _ = netip.Addr{…}. ↩

  8. The compiler can also write an HTML file with the result of each pass:

    $ GOSSAFUNC=AddrTo6Safe \
> GOTOOLCHAIN=go1.26.8 \
> GOAMD64=v3 go build -a .
dumped SSA for AddrTo6Safe,1 to ./ssa.html

    ↩

  9. Source line numbers in parentheses follow value identifiers, but I removed them from the examples. The SSA form also features branches, but we don’t need them to understand our optimizations. ↩

Planet Linux Australia — Top bureaucrats don’t want to be criticised on social media, but their fix may be unconstitutional

&lt;https://theconversation.com/top-bureaucrats-dont-want-to-be-criticised-on-social-media-but-their-fix-may-be-unconstitutional-291448>

"Australia’s top bureaucrats want parliament to curtail politicians posting
clips to social media of senior public servants answering questions before
parliamentary inquiries.

Planet Linux Australia — The American coast that refused to let African culture die

&lt;https://www.bbc.com/travel/article/20261001-the-american-coast-that-refused-to-let-african-culture-die>

"It's low tide at Fish Haul Beach, a marshy stretch on Hilton Head Island's
northern fringe. Fish Haul was the first beach I ever visited as a child, and
now I have brought my 18-month-old son. My wife and I watch him stumble and

Planet Linux Australia — Want to protect yourself against cancer? Get vaccinated

&lt;https://www.scientificamerican.com/article/one-in-eight-new-cancers-are-likely-caused-by-infections-theres-an-easy-way-to-protect-yourself/>

"An estimated one in eight cancer cases worldwide may be caused by
infections—many of which are preventable with treatment or vaccines, according
to new research.

Planet Linux Australia — Fabrice Bellard

https://social.sciences.re/@lbloch/117366981422590361

"A French engineer who lives quietly in Paris has spent 30 years writing
software that the entire internet now runs on without knowing his name.

Planet Linux Australia — Rare northern bald ibises take flight in human-led European migration project

&lt;https://www.theguardian.com/environment/2026/oct/02/northern-bald-ibis-human-led-migration-spain-austria-mirco-aircraft>

"Fifteen new birds have joined a growing flock of bald ibises in northern
Spain, guided there by human migration guides in micro aircraft who helped them
along the route.

Planet Debian — Vincent Bernat: Hacking the Go compiler to efficiently map IPv4 to IPv6

netip.Addr features an Unmap() method returning the unwrapped IPv4 contained in an IPv4-mapped IPv6 address: from ::ffff:203.0.113.10 or ::ffff:cb00:710a, it returns 203.0.113.10.1 There is no Map() or To6() method for the reverse direction. Such a method is trivial to implement, but Go maintainers have rejected it on the grounds that users should write netip.AddrFrom16(ip.As16()) and let the compiler optimize it.2 Today, this pattern is eight times slower than a native method. How can we teach the compiler to optimize this sequence?

The alternatives

Let’s explore three ways to implement the map semantics for netip.Addr. My favorite is to add it to the Go standard library. Go maintainers prefer a small external helper chaining netip.AddrFrom16() and netip.Addr.As16(), hoping the compiler eventually optimizes it. The unsafe package opens a third path, with the same performance as the first solution.

Modifying the Go standard library

Internally, netip.Addr stores any IP address as a 128-bit value with an extra field z to encode the family and the zone:

type Addr struct {
    addr uint128
    z unique.Handle[addrDetail]
}

type addrDetail struct {
    isV6   bool   // IPv4 is false, IPv6 is true.
    zoneV6 string // != "" only if IsV6 is true.
}

var (
    z0    unique.Handle[addrDetail]
    z4    = unique.Make(addrDetail{})
    z6noz = unique.Make(addrDetail{isV6: true})
)

AddrFrom4() encodes an IPv4 address as an IPv4-mapped IPv6 address and sets z to the unique value z4:

// AddrFrom4 returns the address of the IPv4 address given by the bytes in addr.
func AddrFrom4(addr [4]byte) Addr {
    return Addr{
        addr: uint128{
            0,
            0xffff00000000 |
                uint64(addr[0])<<24 | uint64(addr[1])<<16 |
                uint64(addr[2])<<8 | uint64(addr[3])},
        z: z4,
    }
}

Unmap() turns an IPv4-mapped IPv6 address into an IPv4 address by setting the z field to z4:

func (ip Addr) Unmap() Addr {
    if ip.Is4In6() {
        ip.z = z4
    }
    return ip
}

Implementing the reverse direction inside the Go standard library is trivial: we set the z field to z6noz if the address is IPv4.

// To6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func (ip Addr) To6() Addr {
    if ip.Is4() {
        ip.z = z6noz
    }
    return ip
}

As a helper

We can’t access the z field from outside the net/netip package. Instead, we build a small helper around the netip.AddrFrom16(ip.As16()) pattern:

// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func AddrTo6(ip netip.Addr) netip.Addr {
    if ip.Is4() {
        ip = netip.AddrFrom16(ip.As16())
    }
    return ip
}

As an unsafe function

Another solution uses the unsafe package to alter the Addr struct through a proxy with the same memory layout:3

// addrProxy has the same memory layout as netip.Addr.
type addrProxy struct {
    addr [2]uint64      // netip.uint128
    z    unsafe.Pointer // unique.Handle[netip.addrDetail]
}

var (
    anyIPv6    = netip.IPv6Unspecified()
    netipZ6noz = (*addrProxy)(unsafe.Pointer(&anyIPv6)).z
)

// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func AddrTo6(ip netip.Addr) netip.Addr {
    if !ip.Is4() {
        return ip
    }
    (*addrProxy)(unsafe.Pointer(&ip)).z = netipZ6noz
    return ip
}

Benchmarks

On my computer, with Go 1.27.1, the standard library solution costs 0.88 ns per operation, while the solution favored by Go maintainers costs 7.14 ns. The unsafe solution matches the performance of the first one.

goos: linux
goarch: amd64
pkg: github.com/vincentbernat/go-netip-addrto6
cpu: AMD Ryzen 5 5600X 6-Core Processor
                   │     sec/op     │
AddrTo6/safe            7.137n ± 0%
AddrTo6/unsafe         0.8682n ± 2%
AddrTo6/builtin        0.8775n ± 2%

Assembly code

Let’s check the assembly code the compiler generates for each solution.4 The one built into the standard library looks like this:5

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ  net/netip·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE   end                      ; if not, stop here
 MOVQ  net/netip·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

Go’s assembly language is not a direct representation of the underlying machine language: it operates on a semi-abstract instruction set derived from Plan 9’s assembler. It has four pseudo-registers: FP (frame pointer for function arguments), PC (program counter), SB (static base pointer for global symbols), and SP (stack pointer). It also has architecture-specific registers like AX, CX, DX, BX, SI, DI, and R8 to R15. Instructions storing data use their last argument as the destination. Instructions can carry an explicit size suffix: MOVB moves a byte, MOVW 16 bits, MOVL 32 bits, and MOVQ 64 bits. In the example above, the first instruction compares the 64-bit value z4 with the CX register.

The unsafe solution looks almost the same:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ  net/netip·z4(SB), CX  ; check "z" if this is an IPv4 address
 JNE   end                   ; if not, stop here
 MOVQ  netipZ6noz(SB), CX    ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The helper solution has far more instructions. To understand why, let’s look at the code for As16() and AddrFrom16(). They are short enough for the compiler to inline them.

func (ip Addr) As16() (a16 [16]byte) {
    byteorder.BEPutUint64(a16[:8], ip.addr.hi)
    byteorder.BEPutUint64(a16[8:], ip.addr.lo)
    return a16
}

func AddrFrom16(addr [16]byte) Addr {
    return Addr{
        addr: uint128{
            byteorder.BEUint64(addr[:8]),
            byteorder.BEUint64(addr[8:]),
        },
        z: z6noz,
    }
}

We can already guess the pattern to optimize: the code packs the IP address into an array, copies it, then unpacks it. If we inline the Go code by hand, we get:

func AddrTo6(input netip.Addr) netip.Addr {
    if !input.Is4() {
        return input
    }

    var a16 [16]byte
    byteorder.BEPutUint64(a16[:8], input.addr.hi)
    byteorder.BEPutUint64(a16[8:], input.addr.lo)

    addr := a16

    var output netip.Addr
    output.addr.hi = byteorder.BEUint64(addr[:8])
    output.addr.lo = byteorder.BEUint64(addr[8:])
    output.z = netip.z6noz
    return output
}

As humans, we can mentally derive the optimized form:

func AddrTo6(input netip.Addr) netip.Addr {
    if !input.Is4() {
        return input
    }
    var output netip.Addr
    output.addr.hi = input.addr.hi
    output.addr.lo = input.addr.lo
    output.z = netip.z6noz
    return output
}

Unfortunately, as of Go 1.26.8, the compiler is not smart enough to do the same:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
; Push the stack (32 bytes):
;    0(SP) addr netip.uint128
;   16(SP) a16 [16]byte
 PUSHQ   BP
 MOVQ    SP, BP
 SUBQ    $32, SP

 CMPQ    net/netip·z4(SB), CX  ; check "z" if this is an IPv4 address
 JNE     end                   ; if not, stop here

; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi)
;       byteorder.BEPutUint64(a16[8:], input.addr.lo)
 MOVBEQ  AX, net/netip·a16+16(SP)
 MOVBEQ  BX, net/netip·a16+24(SP)

; addr = a16, 16 bytes at once through the vector register X0
 MOVUPS  net/netip·a16+16(SP), X0
 MOVUPS  X0, net/netip·addr(SP)

; CX = netip.z6noz
 MOVQ    net/netip·z6noz(SB), CX
; Unpack: output.addr.hi = byteorder.BEUint64(addr[:8])
;         output.addr.lo = byteorder.BEUint64(addr[8:])
 MOVBEQ  net/netip·addr(SP), AX
 MOVBEQ  net/netip·addr+8(SP), BX

end:
 ADDQ    $32, SP
 POPQ    BP
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The compiler does a decent job on the byte shuffling: the eight byte stores of BEPutUint64() become a single MOVBEQ, which stores a register byte-swapped. The eight byte loads of BEUint64() become a single MOVBEQ the other way round.6 Three groups of instructions remain: a pack, a copy, and an unpack.

Hacking the Go compiler

The Go compiler has several phases:

Parsing
The compiler tokenizes and parses the source code. It builds a syntax tree for each source file.
Type checking
The compiler maps each identifier to the object it denotes, folds constants, and infers the type of every expression.
IR construction
The compiler converts the syntax tree and its types into its own intermediate representation (IR). This process, called “noding,” goes through a serialization format named unified IR.
Middle end
The compiler performs several optimization passes on the IR, such as devirtualization, function call inlining, and escape analysis.
Walk
This phase runs two steps: order of evaluation decomposes complex statements into simpler ones, and desugaring transforms higher-level Go constructs, like switch or channels, into more primitive instructions or calls to the runtime.
Generic SSA
The compiler converts the IR into Static Single Assignment (SSA) form, a lower-level intermediate representation suited for machine-independent optimizations and rewrite rules.
Machine code generation
The compiler rewrites the SSA form into machine-specific variants, allocates registers, and applies more optimization passes. At the end, the assembler turns the generated instructions into machine code.

The hammer

My first idea is to replace occurrences of netip.AddrFrom16(ip.As16()) with netip.Addr{addr: ip.addr, z: netip.z6noz} as early as possible, during the “noding” process. Before that, the type checking phase prevents us from accessing unexported struct fields.

Go 1.27 introduced a convenient debug option to dump the IR of a function at interesting points during compilation:

$ GOTOOLCHAIN=go1.27.1 GOAMD64=v3 go build -a -gcflags="-d=astdump=AddrTo6Safe" .
Writing text ast output for AddrTo6Safe to AddrTo6Safe.ast
Writing html ast output for AddrTo6Safe to AddrTo6Safe.html
Writing html syntax output for AddrTo6Safe to AddrTo6Safe.syntax.html

In the HTML file, the first column shows the IR as it comes out of noding:

DCLFUNC addrto6.AddrTo6Safe ABI:ABIInternal FUNC-func(netip.Addr) netip.Addr
DCLFUNC-Dcl
. NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. NAME-addrto6.~r0 Class:PPARAMOUT Offset:0 OnStack netip.Addr
DCLFUNC-body
. IF # ipv6_safe.go:11:2
. IF-Cond
. . CALLFUNC bool
. . CALLFUNC-Fun
. . . METHEXPR addrto6.Is4 FUNC-func(netip.Addr) bool
. . . . TYPE netip.Addr Class:PEXTERN Offset:0 type netip.Addr
. . CALLFUNC-Args
. . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. IF-Body
. . AS # ipv6_safe.go:12:6
. . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . . CALLFUNC netip.Addr
. . . CALLFUNC-Fun
. . . . NAME-netip.AddrFrom16 Class:PFUNC Offset:0 Used FUNC-func([16]byte) netip.Addr
. . . CALLFUNC-Args
. . . . CALLFUNC ARRAY-[16]byte
. . . . CALLFUNC-Fun
. . . . . METHEXPR addrto6.As16 FUNC-func(netip.Addr) [16]byte
. . . . . . TYPE netip.Addr Class:PEXTERN Offset:0 type netip.Addr
. . . . CALLFUNC-Args
. . . . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. RETURN # ipv6_safe.go:14:2
. RETURN-Results
. . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr

In the body of the if statement, we spot the calls to the method netip.Addr.As16() and to the function netip.AddrFrom16(). Our goal is to patch them with a struct literal:

IF-Body
. AS # ipv6_safe.go:12:6
. . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . STRUCTLIT netip.Addr
. . STRUCTLIT-List
. . . STRUCTKEY netip.addr
. . . . DOT netip.addr netip.uint128
. . . . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . . STRUCTKEY netip.z
. . . . NAME-netip.z6noz Class:PEXTERN Offset:0 unique.Handle[net/netip.addrDetail]

In noder’s reader.go, the expr() method builds the IR tree for an expression. At the end of the exprCall case, we add a call to a rewriteAddrFrom16As16() function. It takes the current node and returns the struct literal on success, or nil if the rewrite is not possible. First, we check that we have the expected pattern: a call to the netip.AddrFrom16() function with a call to the netip.Addr.As16() method as its only argument:

func rewriteAddrFrom16As16(n ir.Node) ir.Node {
    call, ok := n.(*ir.CallExpr)
    if !ok || call.Op() != ir.OCALLFUNC ||
        len(call.Args) != 1 || len(call.Init()) != 0 ||
        !isNetipFunc(call.Fun, "AddrFrom16") {
        return nil
    }
    inner, ok := call.Args[0].(*ir.CallExpr)
    if !ok || inner.Op() != ir.OCALLFUNC ||
        len(inner.Args) != 1 || len(inner.Init()) != 0 ||
        !isNetipFunc(inner.Fun, "Addr.As16") {
        return nil
    }
    x := inner.Args[0]
    // [...]
}

Then, we fetch netip.z6noz:

z6noz, err := lookupVar(ir.StaticCalleeName(call.Fun).Sym().Pkg, "z6noz")
if err != nil {
    return nil
}

And we build the struct literal:

typ := call.Type()
pos := call.Pos()
var list []ir.Node
for i, f := range typ.Fields() {
    var value ir.Node
    switch f.Sym.Name {
    case "addr":
        value = typecheck.DotField(pos, x, i)
    case "z":
        value = z6noz
    default:
        return nil
    }
    list = append(list, ir.NewStructKeyExpr(pos, f, value))
}
lit := ir.NewCompLitExpr(pos, ir.OSTRUCTLIT, typ, list)
lit.SetTypecheck(1)
return lit

Have a look at the complete patch.7 We can test it with the following commands:

$ cd src
$ ./make.bash
Building Go cmd/dist using /usr/lib/go-1.27. (go1.27.1 linux/amd64)
Building Go toolchain1 and bootstrap cmd/go (go_bootstrap) using /usr/lib/go-1.27.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 and commands using go_bootstrap and Go toolchain2.
Checking command staleness for linux/amd64.
---
Installed Go for linux/amd64 in /home/bernat/code/free/go
Installed commands in /home/bernat/code/free/go/bin
*** You need to add /home/bernat/code/free/go/bin to your PATH.
$ export PATH=$PWD/../bin:$PATH
$ go version
go version go1.28-devel_9834516e20 Sat Sep 12 08:23:11 2026 -0700 linux/amd64
$ go test net/netip/...
ok      net/netip   0.224s
$ cd ../../go-netip-addrto6
$ go test .
ok      github.com/vincentbernat/go-netip-addrto6   0.062s

The generated code for the helper is now the shortest possible version!

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ    net/netip·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE     end                      ; if not, stop here
 MOVQ    net/netip·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

Go maintainers are unlikely to accept this change. It relies on the internal structure of net/netip.Addr. It’s an ugly hack in the noder, whose job is to faithfully translate the type-checked AST into the IR. And it’s harder to maintain than adding a To6() method.

The screwdriver

The right place for such an optimization is the generic SSA phase. One of the last machine-independent passes is memcombine. With the appropriate debug flag, the compiler dumps the SSA form after this pass:8

$ GOTOOLCHAIN=go1.26.8 GOAMD64=v3 \
> go build -a -gcflags='-d=ssa/memcombine/dump=AddrTo6Safe' .
$ head -5 AddrTo6Safe_01__memcombine.dump
AddrTo6Safe func(netip.Addr) netip.Addr
  b2:
    (?) v1 = InitMem <mem>
    (?) v2 = SP <uintptr>
    (?) v3 = SB <uintptr>

The result of the memcombine pass follows the same structure as the assembly code for AddrTo6Safe() we looked at earlier: two stores, one move, and two loads we would like to optimize away.

; […]
  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v466 = ArgIntReg <*netip.addrDetail> {ip+16} [2]  ; input.z
; […]
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)

  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16

  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v416 = Load <uint64> v285 v286                    ; addr[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi
  v174 = Load <uint64> v415 v286                    ; addr[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo
; […]

Each line features a value identifier (v442), an operation with its type (Bswap64 <uint64>), and its arguments (v490).9 Values are the basic building blocks of SSA and are defined exactly once. Square brackets enclose integer parameters ([8]) and curly braces contain auxiliary arguments ({netip.addr}). Operations writing to memory produce a new memory state. Every memory operation takes the current state as its last argument, which keeps them in order.

On paper

Let’s focus on output.addr.lo, aka v39:

  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v174 = Load <uint64> v415 v286                    ; addr[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

To simplify this code, we could apply three rewriting rules:

  1. The first one adds a shortcut when loading through a move: (Load (OffPtr [o] p) (Move p src mem)) => (Load (OffPtr [o] src) mem). This matches v174 with its arguments v415 and v286 and creates a new value v600:

      v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Load <uint64> v600 v282                    ; a16[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

  2. The second one simplifies a load following a store: (Load p (Store p x _)) => x. The load is forwarded: the stored value replaces it and no memory access remains. It matches v174. It notices that v600 and v173 are the same address and replaces v174 with a copy of v442:

      v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Copy <uint64> v442                         ; bswap(input.addr.lo)
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

  3. The last step cancels the two byte swaps: (Bswap64 (Bswap64 x)) => x. v39 becomes a copy of v490:

      v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Copy <uint64> v442                         ; bswap(input.addr.lo)
  v39  = Copy <uint64> v490                         ; output.addr.lo = input.addr.lo

If we ignore the values not needed to compute v39, only this SSA form remains:

  v490 = ArgIntReg <uint64> {ip+8} [1]  ; input.addr.lo
  v39  = Copy <uint64> v490             ; output.addr.lo = input.addr.lo

Let’s switch to output.addr.hi, aka v299:

  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Load <uint64> v285 v286                    ; addr[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

To optimize it away, we also apply three rewriting rules:

  1. The first one also adds a shortcut when loading through a move, but without an offset: (Load p (Move p src mem)) => (Load src mem). This rewrites v416 to use arguments from v286:

      v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Load <uint64> v22 v282                     ; a16[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

  2. The second rule forwards a value stored one step earlier, skipping over a store to another address: (Load p (Store q _ (Store p x _))) => x. This matches v416: x is v542, p is v22 (&a16), q is v173 (&a16[8]), and p and q do not overlap for uint64.

      v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Copy <uint64> v542                         ; bswap(input.addr.hi)
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

  3. The third rule cancels two byte swaps: (Bswap64 (Bswap64 x)) => x. v299 becomes a copy of v502:

      v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Copy <uint64> v542                         ; bswap(input.addr.hi)
  v299 = Copy <uint64> v502                         ; output.addr.hi = input.addr.hi

If we remove the values not used to compute v299, we get this SSA form:

  v502 = ArgIntReg <uint64> {ip+0} [0] ; input.addr.hi
  v299 = Copy <uint64> v502            ; output.addr.hi = input.addr.hi

In practice

Most of these rules already exist in generic.rules. They use conditions to validate their context: ssa.IsSamePtr() for the same address, ssa.Disjoint() for addresses that do not overlap. The rule forwarding a stored value to a load already exists with three variants looking through several other stores. Here are the two we need:

(Load <t1> p1 (Store {t2} p2 x _))
    && ssa.IsSamePtr(p1, p2)
    && copyCompatibleType(t1, x.Type)
    && t1.Size() == t2.Size()
    => x
(Load <t1> p1 (Store {t2} p2 _ (Store {t3} p3 x _)))
    && ssa.IsSamePtr(p1, p3)
    && copyCompatibleType(t1, x.Type)
    && t1.Size() == t3.Size()
    && ssa.Disjoint(p3, t3, p2, t2)
    => x

Go 1.27 added the rule loading through a move with CL 748200 to fix issue #77720:

(Load <t1> op1:(OffPtr [o1] p1) move:(Move [n] p2 src mem))
    && o1 >= 0 && o1+t1.Size() <= n && ssa.IsSamePtr(p1, p2)
    && !ssa.IsVolatile(src)
    => @move.Block (Load <t1> (OffPtr <op1.Type> [o1] src) mem)

It lacks a variant without an offset:

(Load <t1> p1 move:(Move [n] p2 src mem))
    && p1.Op != ssaop.OpOffPtr
    && t1.Size() <= n && ssa.IsSamePtr(p1, p2)
    && !ssa.IsVolatile(src)
    => @move.Block (Load <t1> (OffPtr <p1.Type> [0] src) mem)

There is no generic rule to cancel two byte swaps, but the AMD64 lowering pass includes this rule:

(BSWAP(Q|L) (BSWAP(Q|L) p)) => p

After switching to Go’s development branch and adding the missing rule, the generated assembly code is worse than with Go 1.26.8, even though our additional rule slightly improves the situation at the end:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
; Push the stack (16 bytes):
;    0(SP) a16 [16]byte
 PUSHQ   BP
 MOVQ    SP, BP
 SUBQ    $16, SP

 CMPQ    net/netip·z4(SB), CX       ; check "z" if this is an IPv4 address
 JNE     end                        ; if not, stop here

; The four forwarded bytes: the low half of input.addr.lo is taken apart and
; put back together in registers
 MOVQ    BX, DX                     ; DX = input.addr.lo
 SHRQ    $24, BX                    ; BX = input.addr.lo >> 24
 MOVQ    DX, SI                     ; SI = input.addr.lo
 SHRQ    $16, DX                    ; DX = input.addr.lo >> 16
 MOVQ    SI, DI                     ; DI = input.addr.lo, kept for the pack
 SHRQ    $8, SI                     ; SI = input.addr.lo >> 8
 MOVBLZX DIB, R8                    ; R8 = byte(input.addr.lo)
 MOVBLZX SIB, SI                    ; SI = byte(input.addr.lo >> 8)
 SHLQ    $8, SI
 ORQ     R8, SI                     ; SI = two low bytes of input.addr.lo
 MOVBLZX DL, DX                     ; DX = byte(input.addr.lo >> 16)
 SHLQ    $16, DX
 ORQ     SI, DX
 MOVBLZX BL, BX                     ; BX = byte(input.addr.lo >> 24)
 SHLQ    $24, BX
 ORQ     DX, BX                     ; BX = input.addr.lo & 0xffffffff

; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi)
;       byteorder.BEPutUint64(a16[8:], input.addr.lo)
 MOVBEQ  AX, net/netip·a16(SP)
 MOVBEQ  DI, net/netip·a16+8(SP)

; The four other bytes of input.addr.lo, read one by one from a16
 MOVBLZX net/netip·a16+11(SP), DX   ; a16[11]
 SHLQ    $32, DX
 ORQ     DX, BX
 MOVBLZX net/netip·a16+10(SP), DX   ; a16[10]
 SHLQ    $40, DX
 ORQ     DX, BX
 MOVBLZX net/netip·a16+9(SP), DX    ; a16[9]
 SHLQ    $48, DX
 ORQ     DX, BX
 MOVBLZX net/netip·a16+8(SP), DX    ; a16[8]
 SHLQ    $56, DX

; output.z = netip.z6noz
 MOVQ    net/netip·z6noz(SB), CX
; output.addr.hi = byteorder.BEUint64(a16[:8])
 MOVBEQ  net/netip·a16(SP), AX
; output.addr.lo assembled from the previous steps
 ORQ     DX, BX

end:
 LEAVEQ
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The rule loading through a move, added in Go 1.27, introduced this regression.

Out of order

Let’s not give up now! In reality, the rewriting rules run before memcombine, notably in the late opt pass. At this point, the inlined versions of BEPutUint64() and BEUint64() still expand to sixteen byte stores and sixteen byte loads, matching their source code:

func BEUint64(b []byte) uint64 {
    _ = b[7] // bounds check hint to compiler; see golang.org/issue/14808
    return uint64(b[7]) | uint64(b[6])<<8 | uint64(b[5])<<16 | uint64(b[4])<<24 |
        uint64(b[3])<<32 | uint64(b[2])<<40 | uint64(b[1])<<48 | uint64(b[0])<<56
}

Let’s follow two bytes of output.addr.lo: addr[15] and addr[11]. Here is a simplified SSA form before late opt:

  v273 = Trunc64to8 <byte> v490                     ; byte(input.addr.lo)
  v226 = Trunc64to8 <byte> v225                     ; byte(input.addr.lo >> 32)
; […]
  v235 = Store <mem> {byte} v233 v226 v223          ; a16[11] = byte(lo >> 32)
  v247 = Store <mem> {byte} v245 v238 v235          ; a16[12] = …
  v259 = Store <mem> {byte} v257 v250 v247          ; a16[13] = …
  v271 = Store <mem> {byte} v269 v262 v259          ; a16[14] = …
  v282 = Store <mem> {byte} v280 v273 v271          ; a16[15] = byte(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
; […]
  v433 = OffPtr <*byte> [15] v285                   ; &addr[15]
  v435 = Load <byte> v433 v286                      ; addr[15]
  v479 = OffPtr <*byte> [11] v285                   ; &addr[11]
  v481 = Load <byte> v479 v286                      ; addr[11]

The first rule loads through the move: (Load (OffPtr [o] p) (Move p src mem)) => (Load (OffPtr [o] src) mem). It matches both loads, which now read a16 with the memory state before the copy:

  v600 = OffPtr <*byte> [15] v22 ; &a16[15]
  v435 = Load <byte> v600 v282   ; a16[15]
  v601 = OffPtr <*byte> [11] v22 ; &a16[11]
  v481 = Load <byte> v601 v282   ; a16[11]

The second rule shortcuts a load following a store: (Load p (Store p x _)) => x. It matches v435, as v282 stores a16[15]. It does not match v481: v235 stores a16[11] four stores earlier in the chain, while the variants of this rule look through three stores at most.

  v435 = Copy <byte> v273        ; byte(input.addr.lo)
  v601 = OffPtr <*byte> [11] v22 ; &a16[11]
  v481 = Load <byte> v601 v282   ; a16[11]

The same happens to the other bytes: the rule forwards the four bytes stored last, a16[12] to a16[15]. The twelve other loads now read a16 instead of addr.

BEUint64() becomes a chain of Or64, each one adding a byte shifted into place. memcombine is a pass written in Go, not a set of rewrite rules. It starts from the last Or64 of the chain and collects up to eight terms. If each term is a byte load, extended to 64 bits and shifted, and if the eight loads read consecutive addresses from the same pointer with the same memory state, it replaces the whole chain with a single 64-bit load and a byte swap. Otherwise, it tries again with four, then two terms, and from each intermediate Or64. Here is the loop checking each term in a simplified version of combineLoads():

for i := int64(0); i < n; i++ {
    v := a[i]
    shift := int64(0)
    if v.Op == shiftOp {
        v, shift = peelShift(v)
    }
    if v.Op != extOp {
        return false
    }
    load := v.Args[0]
    if load.Op != ssaop.OpLoad {
        return false
    }
    if load.Args[1] != mem {
        return false
    }
    p, off := splitPtr(load.Args[0])
    if p != base {
        return false
    }
    r[i] = LoadRecord{load: load, offset: off, shift: shift}
}

For output.addr.hi, the eight loads read a16 with the same memory state v282:

  v13  = Load <byte> v22 v282               ; a16[0]
  v530 = Load <byte> v14 v282               ; a16[1]
  v488 = Load <byte> v504 v282              ; a16[2]
  v405 = Load <byte> v537 v282              ; a16[3]
  v385 = Load <byte> v397 v282              ; a16[4]
  v361 = Load <byte> v373 v282              ; a16[5]
  v196 = Load <byte> v63 v282               ; a16[6]
  v432 = Load <byte> v315 v282              ; a16[7]
  v319 = ZeroExt8to64 <uint64> v432         ; uint64(a16[7])
  v329 = ZeroExt8to64 <uint64> v196         ; uint64(a16[6])
  v330 = Lsh64x64 <uint64> [true] v329 v138 ; uint64(a16[6]) << 8
  v331 = Or64 <uint64> v319 v330            ; a16[7] | a16[6] << 8
; […] same for a16[5] to a16[1]
  v401 = ZeroExt8to64 <uint64> v13          ; uint64(a16[0])
  v402 = Lsh64x64 <uint64> [true] v401 v55  ; uint64(a16[0]) << 56
  v403 = Or64 <uint64> v402 v391            ; | a16[0] << 56 = output.addr.hi

memcombine merges them into one load and a swap:

  v286 = Load <uint64> v22 v282   ; a16[:8]
  v285 = Bswap64 <uint64> v286    ; output.addr.hi

For output.addr.lo, here is the chain memcombine sees after late opt:

  v436 = ZeroExt8to64 <uint64> v273 ; addr[15], forwarded
  v448 = Or64 <uint64> v436 v447    ; | addr[14] << 8, forwarded
  v460 = Or64 <uint64> v459 v448    ; | addr[13] << 16, forwarded
  v472 = Or64 <uint64> v471 v460    ; | addr[12] << 24, forwarded
  v484 = Or64 <uint64> v483 v472    ; | a16[11] << 32, loaded
  v496 = Or64 <uint64> v495 v484    ; | a16[10] << 40, loaded
  v508 = Or64 <uint64> v507 v496    ; | a16[9] << 48, loaded
  v520 = Or64 <uint64> v519 v508    ; | a16[8] << 56, loaded

From v520, four of the eight terms are forwarded bytes, not loads from memory, and memcombine can’t combine them. It doesn’t merge the four remaining loads either, as they sit on top of the forwarded bytes.

Back in order

In summary, the rewriting rules run too early to be effective. A quick workaround exists: run an earlier round of memcombine before late opt. After this change, the generated code for the helper is back to the shortest possible version:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ    net/netip·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE     end                      ; if not, stop here
 MOVQ    net/netip·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

And the benchmark confirms it! ✌️

goos: linux
goarch: amd64
pkg: github.com/vincentbernat/go-netip-addrto6
cpu: AMD Ryzen 5 5600X 6-Core Processor
                   │   Go 1.26.8    │             Our branch              │
                   │     sec/op     │    sec/op     vs base               │
AddrTo6/safe          6.5470n ±  0%   0.8944n ± 4%  -86.34% (p=0.002 n=6)
AddrTo6/unsafe        0.9071n ±  3%   0.8682n ± 1%   -4.28% (p=0.002 n=6)
AddrTo6/builtin       0.8871n ±  2%   0.8785n ± 1%        ~ (p=0.310 n=6)

Next steps

I think Go maintainers would reject this change because of the additional memcombine pass. Instead, I plan to publish this blog post and bring up the subject again as a follow-up to issue #54365. Either the sheer complexity and the Go 1.27 regression convince the maintainers that adding a To6() method is simpler and more efficient, or they advise me on how to move forward. Either way, digging into this subject taught me a lot about the Go compiler! ⚙️

  1. IPv4-mapped IPv6 addresses let you handle only IPv6 addresses in your code and keep conversions at a few well-defined boundaries. I use them in Akvorado. ↩

  2. This is not strictly equivalent. The zero value becomes :: while it would make more sense to leave it untouched. ↩

  3. To avoid catastrophic bugs when netip.Addr’s layout changes, you must add tests to detect it. This is more dangerous if you put this code in a package. Either ask users to run the tests themselves, or detect the change at run time and panic. Hiding this optimization behind a build tag would make users aware of this potential trap. ↩

  4. I produced the assembly code with GOTOOLCHAIN=go1.26.8 GOAMD64=v3 ./go-asm '\.asm' from the companion repository, then edited it a bit to keep this article from turning into an endless rabbit hole. Due to an unfortunate sequence of events, we switch between versions: Go 1.27 introduced a regression that blurs the point of this article. ↩

  5. The function is short enough for the compiler to inline it, so the assembly code may vary depending on the surrounding code. ↩

  6. MOVBEQ requires GOAMD64=v3, matching the x86-64-v3 microarchitecture from 2013. Otherwise, the compiler translates BEPutUint64() to BSWAPQ+MOVQ and BEUint64() to MOVQ+BSWAPQ. ↩

  7. When the call is used as a statement, a struct literal alone is invalid. The patch then turns it into _ = netip.Addr{…}. ↩

  8. The compiler can also write an HTML file with the result of each pass:

    $ GOSSAFUNC=AddrTo6Safe \
> GOTOOLCHAIN=go1.26.8 \
> GOAMD64=v3 go build -a .
dumped SSA for AddrTo6Safe,1 to ./ssa.html

    ↩

  9. Source line numbers in parentheses follow value identifiers, but I removed them from the examples. The SSA form also features branches, but we don’t need them to understand our optimizations. ↩

Planet Debian — Steinar H. Gunderson: Decompilation patterns part 1: Anti-CSE

There's a lot of¹ interest in decompilation these days; taking some old piece of software (usually a game) and try to reconstruct C that does the same thing. (This is often for modding purposes, but also often for understanding, or just as a personal challenge.) An often sought-after property is matching; that the decompiled C produces 100% the same bytes as the original after compilation. (This means you'll need to find the original compiler and build flags, of course.) Typically, one starts with the disassembled code and runs m2c on it, which then spits out some C that is fairly close to the original, but rarely produces a match without further cleanup. (It is usually much closer to a match than what Ghidra or Hex-Rays would output; Ghidra is made for ease of understanding, not matching, so it tends to simplify a lot.)

Cleaning up the code for matching has two main advantages:

  1. It usually makes the source easier to understand, and
  2. Matching is a very strong argument (though not infallible) that the decompilation is actually correct.

Like with pretty much any other field these days, there is a lot of interest in giving this task to some LLM, which means there's now a lot of decompilations around that match (fitting #2 perfectly) and completely fail #1. :-) Anyway, at some point, I learned that there is almost no public documentation on the cleaning-up-for-matching part; it's all folklore and people rediscovering the same patterns. So I thought I'd do something about that².

My general goal is to write up one decompilation pattern a day (focusing on what I know best, namely GCC on MIPS), with some example. Most of them will be very simple; some are much less obvious. I'll keep going until I run out of ideas. None are formally named, so I'll just invent a name. So here goes part one:

  temp_v0_5 = g_state.players[1].unk2;
  if ((temp_v0_5 != 7) && (temp_v0_5 != 0)) {

What happened here is usually that the compiler has done common subexpression elimination; someone wrote the same thing twice and then the compiler decided to just calculate it once and put it into a temporary variable. (Or the programmer did, in which case you would be wise to leave it alone! Getting into the original programmer's head is part of the challenge.) This will often affect register allocation and/or stack layout, so to get a better match, you could try doing CSE in reverse:

  if (g_state.players[1].unk2 != 7 && g_state.players[1].unk2 != 0) {

Nothing is mechanical; it may help or it may not. (It may also appear to hurt at first and then help later, or vice versa.) But anti-CSE is generally very common and you'll want to have it in your arsenal.

¹ Well, all is relative.

² Reluctantly, with the understanding that the LLMs might also learn from it.

Mike Bowler — Paltering: Lying by telling the truth

A few days ago, someone in my LinkedIn feed shared a screenshot of a post from X. My first assumption with any screenshot from social media (rather than a direct link) is that it’s fake, so I went and found the original. It was real, and it had been viewed 7.8 million times. The post said that OpenAI’s internal benchmarks showed GPT-6.1 Sol crushing Opus 5.5, with Anthropic struggling to keep up. The chart below it backed that up nicely: three coloured lines climbing over time, OpenAI’s green line sitting comfortably on top, Anthropic’s in the middle, Grok at the bottom.

A post on X claiming OpenAI's internal benchmarks show GPT-6.1 Sol crushing Opus 5.5. The chart shows GPT, Anthropic and Grok model lines rising over time, and the tiny grey label on the vertical axis reads 'Model number'

Now look closely at the label on the vertical axis. It’s small and grey and easy to miss. It says “Model number”.

That chart isn’t measuring performance at all. It’s plotting the version number in the product name against the release date. GPT-6.1 is higher than Opus 5.5 because 6.1 is a bigger number than 5.5. That’s it. That’s the whole insight.

I assume it’s a joke, and plenty of the replies treat it as one. Plenty of others clearly took it at face value. The author never says either way, and the fact that I have to assume is what makes it interesting. A joke that only lands for the people who read the smallest text on the image works exactly like a deception for everyone else. 16,000 people liked it and a thousand reposted it, and I’d be surprised if most of them read the axis.

What makes this one clever is that the chart isn’t wrong. As far as I can tell, every point sits exactly where its version number puts it. There’s no scale on the axis and no zero, and the only clue to what’s being measured is the faintest text on the image. Nothing in the picture is false. The deception lives entirely in the gap between the headline and the axis.

The headline is a different matter. If OpenAI has no such internal benchmarks, that sentence is simply false. The chart’s job is to make the false sentence feel supported, and it does that using nothing but true information.

There’s a word for this, and it’s not new. Palter dates from the 1530s1, and Shakespeare used it in exactly this sense. Macbeth, discovering that the witches’ prophecies were true to the letter and ruinous all the same, curses them:

“That palter with us in a double sense;
That keep the word of promise to our ear,
And break it to our hope.”
William Shakespeare, Macbeth2

Researchers have since given it a sharper definition:

“Paltering is the active use of truthful statements to convey a misleading impression.”
Todd Rogers et al., “Artful Paltering”3

It’s distinct from lying by commission, where you say something false, and from lying by omission, where you leave out something that matters. A palter says only true things and lets you draw the wrong conclusion yourself.

The same research found something I find more interesting than the definition. People who palter believe it’s more ethical than an outright lie, while the people they misled, once they find out, judge the two about the same. The palterer is thinking “I told the truth” and the target is thinking “I was misled”. That’s also why the joke question matters less than it seems. “It says model number right there” is a perfect defence, and it changes nothing for the reader who didn’t look.

None of this is new. Darrell Huff catalogued these tricks in How to Lie with Statistics back in 1954, and the book is still worth reading: short, funny, and depressingly current. When I wrote about data accuracy, I set aside charts that mislead on purpose and pointed at Huff instead.

So why does it work? Because nobody reads axis labels at scroll speed. We read the headline, glance at the shape of the lines, see that the shape agrees with what we were just told, and move on. That’s System 1 doing what it does best: producing a quick, confident impression from whatever is in front of it. Reading the axis is System 2 work, and System 2 only shows up when something feels off. A palter is built so that nothing feels off.

That isn’t a character flaw in the readers. It’s how all of us read, and anyone who wants to mislead us is designing for it.

That’s what concerns me beyond one joke about AI models. We’re living through a flood of disinformation, and a lot of it isn’t outright lies. Outright lies get fact-checked. A palter survives the fact-check because every individual piece of it is true: the real quote clipped from its context, the accurate statistic paired with a conclusion it doesn’t support, the genuine photo from a different year. The headline does the misleading, and the details sit there, technically correct, waiting for someone to look at them.

We’d all benefit from paying more attention to the details than to the headline. Who made this? What exactly is being measured? Where’s the original? It takes thirty seconds, and it’s the same habit whether you’re looking at a viral post or your team’s dashboard.

Before you believe the line, think about what it’s really saying.

  1. Originally meaning “to speak indistinctly”, with the sense of playing fast and loose appearing around 1600. See Etymonline and Dictionary.com. ↩

  2. William Shakespeare, Macbeth, Act 5 Scene 8. MIT Shakespeare text ↩

  3. Todd Rogers, Richard Zeckhauser, Francesca Gino, Michael I. Norton and Maurice E. Schweitzer, “Artful Paltering: The Risks and Rewards of Using Truthful Statements to Mislead Others”, Journal of Personality and Social Psychology, 2017, Vol. 112, No. 3, 456-473. doi:10.1037/pspi0000081 ↩

Planet Linux Australia — Barking beautiful: the 2026 Dog Photography awards – in pictures

&lt;https://www.theguardian.com/artanddesign/gallery/2026/oct/03/barking-beautiful-the-2026-dog-photography-awards-in-pictures>

"The 18 winning images from the annual contest that seeks to showcase the
world’s most exceptional dog photographs while highlighting their subjects’
diversity and charm. These photos were selected from about 1,900 entries that

Planet Linux Australia — Inspiral Web

365 Tomorrows — Notes From the South Tower

Author: Frank Sikora September 11, 2001 • World Trade Center • New York • 9:40 a.m. A wall of flame stalks the historian, 2000 degrees, capable of melting steel and glass. Black smoke billows throughout the building and into the air, desecrating a stark blue sky. The scene is worse than the photos. More horrifying […]

The post Notes From the South Tower appeared first on 365tomorrows.

,

Planet Linux Australia — Ban Kids From Social Media And They’ll Just Chat In Public Radio Podcast Comments

&lt;https://www.techdirt.com/2026/09/30/ban-kids-from-social-media-and-theyll-just-chat-in-public-radio-podcast-comments/>

"Eleven months ago, I wrote about what I thought was a truly fascinating find:
that some kids appeared to be using some random old NPR podcasts on Spotify as
a personal chat room to get around various restrictions. It was mentioned,

Planet Linux Australia — The Dementia Revolution

https://theprogressnetwork.substack.com/p/the-dementia-revolution

"Dementia might seem an unlikely place to look for a Copernican revolution in
medicine.

Planet Linux Australia — What it’s like to be masking in 2026

https://nannyogg63.substack.com/p/what-its-like-to-be-masking-in-2026

"This is my spouse and me on a flight to Vancouver in 2024. We’re still wearing
the same mask type (BreatheTeq KN95s made in Canada to N95 standards) today.
They fit well, are incredibly breathable, don’t fog up glasses if the nose

Planet Linux Australia — Immigration in Sweden: how the country went from one of Europe’s most welcoming, to one of its most restrictive

&lt;https://theconversation.com/immigration-in-sweden-how-the-country-went-from-one-of-europes-most-welcoming-to-one-of-its-most-restrictive-288995>

"For decades, Sweden was known as one of Europe’s most welcoming countries for
refugees. It took pride in pursuing one of Europe’s most generous asylum
policies, and often presented itself as a humanitarian role model.

Planet Linux Australia — A Fictional Attack on Kaliningrad Just Triggered a Real Russian Nuclear Warning

&lt;https://freedium-mirror.cfd/https://medium.com/@rajk100/a-fictional-attack-on-kaliningrad-just-triggered-a-real-russian-nuclear-warning-534c684b6dc0>

“In recent days, tensions between Russia and Western countries have once again
reached a peak.

Planet Linux Australia — ‘Like caring for a pet’: the app that lets you know if your neighbourhood tree is thirsty

&lt;https://www.theguardian.com/environment/2026/oct/02/open-source-tree-map-berlin-water-urban-forest>

"Amanda Gläser-Bligh sometimes attracts puzzled looks when she goes to water
the eight-year-old street tree she has adopted outside her door in
Charlottenburg, Berlin.

Planet Linux Australia — Palermo, first coffee grown and harvested in Italy: warmer climate is changing farming

&lt;https://www.euronews.com/2026/10/01/palermo-first-coffee-grown-and-harvested-in-italy-warmer-climate-is-changing-farming>

"A plantation on the outskirts of Palermo has for the first time produced
Arabica coffee beans grown and harvested entirely in Italy. The result,
achieved on the Morettino family estate, has been made possible by climate

Planet Linux Australia — How the ‘first bird’ leapt into the sky and soared above the dinosaurs

&lt;https://theconversation.com/how-the-first-bird-leapt-into-the-sky-and-soared-above-the-dinosaurs-289927>

"Archaeopteryx is an icon of evolution and is often called the first bird.
Scientists have long debated just how it took off from the ground but a recent
study published by my colleagues and I has brought new insights to the mystery.

Planet Debian — Vincent Bernat: Self-hosted HTTP tunnels with SSH and nginx

A friend wants to proofread your work-in-progress blog post, but its preview only runs on localhost:8080. Several tools can help. Some run as a commercial service, like ngrok or Cloudflare Quick Tunnels. Some are self-hostable but require a specific client, like frp or localtunnel. Some only require a plain SSH client but rely on a specific SSH server, like sish. Let’s implement a self-hosted solution with only OpenSSH and nginx!

$ ssh -R 0:localhost:8080 http-over-ssh
Allocated port 41535 for remote forward to localhost:8080
https://6J3jK1WmB15c6WmjW_X-Wg--1789928654@p41535.ssh.luffy.cx/

Basic setup

First, we forward connections from a port on a remote server to your local service:

$ ssh -N -R 0:localhost:8080 web02.luffy.cx
Allocated port 41535 for remote forward to localhost:8080

When you specify 0 as the remote port, the server allocates a free port. Then, we configure nginx to proxy requests from https://p41535.ssh.luffy.cx to http://127.0.0.1:41535:

server {
  listen 0.0.0.0:443 ssl ;
  listen [::0]:443 ssl ;
  server_name ~^p(?<port>\d\d\d\d\d)\.ssh\.luffy\.cx$;
  location / {
    proxy_pass http://127.0.0.1:$port;
  }
}

We also need to add DNS records for *.ssh.luffy.cx and get a wildcard certificate through Let’s Encrypt:

*.ssh.luffy.cx.               CNAME web02.luffy.cx.
ssh.luffy.cx.                 CAA   0 issuewild "letsencrypt.org"
_acme-challenge.ssh.luffy.cx  CNAME ssh.luffy.cx.acme.luffy.cx.

acme.luffy.cx is a zone hosted on Route 53. I use it for ACME DNS-01 challenges, both for wildcard certificates and for domains served by several web servers. In my case, NixOS gets the certificates automatically.

Access control

The port is the only “secret”1 keeping the content confidential. Other forwarding solutions add a random string to the domain name to prevent an intruder from enumerating the possible values.

Thanks to ngx_http_secure_link_module, we can secure this setup a bit. This module computes a hash2 over a set of values, including a secret, and compares it with the hash from the request. The hash is base64-encoded, so we cannot put it in the domain name, which is case-insensitive. Instead, we put it in the URL as a username, along with its expiration timestamp:3

https://6J3jK1WmB15c6WmjW_X-Wg--1789928654@p41535.ssh.luffy.cx/en/blog
        ╰─────────┬──────────╯  ╰───┬────╯  ╰─┬─╯             ╰──┬───╯
                hash             expires    port               path

The client sends the username to the server with HTTP basic authentication. This works with most HTTP clients, including curl. Nginx exposes the username in the $remote_user variable. The module expects the hash and the expiration timestamp separated by a comma. We use a map directive to extract the two parts from $remote_user and join them with a comma.4 We also give the module the string to hash. It contains the expiration timestamp, the port, and a secret:

map $remote_user $httpssh_link {
  "~^([-_A-Za-z0-9]{22})--([0-9]+)$" "$1,$2";
}
server {
  # […]
  location / {
    secure_link $httpssh_link;
    secure_link_md5 "$secure_link_expires $port ZuPerS3cr3!";
  }
}

The module returns the status of the check in the $secure_link variable:

  • empty if the hashes do not match,
  • "0" if they match but the link has expired, or
  • "1" otherwise.

If the hash is incorrect or missing, we return a 401 error with a WWW-Authenticate header to ask for credentials. If the link has expired, we return a 410 error. We remove the Authorization header before forwarding the request and add a few directives to proxy WebSocket connections. Here is the complete configuration:5

map $remote_user $httpssh_link {
  "~^([-_A-Za-z0-9]{22})--([0-9]+)$" "$1,$2";
}
server {
  listen 0.0.0.0:443 ssl ;
  listen [::0]:443 ssl ;
  server_name ~^p(?<port>\d\d\d\d\d)\.ssh\.luffy\.cx$;
  location / {
    secure_link $httpssh_link;
    secure_link_md5 "$secure_link_expires $port ZuPerS3cr3!";
    if ($secure_link = "") {
      add_header WWW-Authenticate 'Basic realm="tunnel"' always;
      return 401;
    }
    if ($secure_link = "0") {
      return 410;
    }
    proxy_pass http://127.0.0.1:$port;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Authorization "";
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_buffering off;
    proxy_read_timeout 30m;
  }
}

I think you are now asking yourself the obvious question: “How should I generate the hash?” Easy peasy!

$ expires=$(( $(date +%s) + 86400 ))
$ port=41535
$ secret='ZuPerS3cr3!'
$ printf '%s %s %s' "$expires" "$port" "$secret" \
>   | openssl md5 -binary \
>   | openssl base64 \
>   | tr +/ -_ | tr -d =
6J3jK1WmB15c6WmjW_X-Wg

Well, I suppose you are now saying: “Vincent, this is not very convenient! I’ll stick with ngrok if you don’t mind.” Okay, I hear you. Let’s write a helper script.

Helper script

The main difficulty is finding the ephemeral port that OpenSSH allocates, as it does not appear in any environment variable.6 To work around this obstacle, we look for the ancestor sshd-session processes:7

pids=$(
  pid=$$
  while [ "$pid" -gt 1 ]; do
    line=$(ps -o comm=,pid=,ppid= -p "$pid")
    echo "$line"
    pid=${line##* }
  done | awk '$1 == "sshd-session" { printf "pid=%s,\n", $2 }'
)
if [ -z "$pids" ]; then
  echo "not an ssh session" >&2
  exit 1
fi

Then, we get the listening ports associated with these sshd-session processes:8

ports=$(sudo -n ss --listening --numeric --tcp --processes --no-header \
  | grep -F "$pids" \
  | awk '{ print $4 }' | awk -F: '{ print $NF }' \
  | sort -un)
if [ -z "$ports" ]; then
  echo "no forwarded port, use ssh -R 0:localhost:PORT" >&2
  exit 1
fi

Finally, we display the URLs and keep the session open:

lifetime=86400
secret='ZuPerS3cr3!'
expires=$(( $(date +%s) + lifetime ))
for port in $ports; do
  token=$(printf '%s %s %s' "$expires" "$port" "$secret" \
            | openssl md5 -binary \
            | openssl base64 \
            | tr +/ -_ | tr -d =)
  echo "https://$token--$expires@p$port.ssh.luffy.cx/"
done
sleep infinity

I install this script as http-over-ssh on the server and add this entry to my ~/.ssh/config:

Host http-over-ssh
  Hostname web02.luffy.cx
  RemoteCommand http-over-ssh
  ControlPath none

With this solution, I only rely on OpenSSH and nginx, two pieces of software already running on this server. One short command gives me a self-hosted tunnel and a URL to share. To try it, grab the complete helper script, which includes a few minor improvements. If you run NixOS, as any person of taste would, have a look at my http-over-ssh.nix instead. ❄️

  1. Since the kernel allocates it from the local port range, its entropy is low. Moreover, on Linux, we lose one more bit because the kernel favors odd ports when it picks a random free port.

    $ sysctl -n net.ipv4.ip_local_port_range \
>   | awk '{print $1"—"$2" ≈ "log($2-$1+1)/log(2)" bits"}'
32768—60999 ≈ 14.785 bits

    ↩

  2. The module relies on MD5. Its security is weak but good enough for this use. ↩

  3. We need an expiration date because a later session could reuse the port, and we have no way to detect this case. ↩

  4. The username could use a comma directly instead of a double dash. But some applications do not recognize the URL correctly, making it harder to share. ↩

  5. This configuration exposes any TCP port listening on 127.0.0.1 or 0.0.0.0 to anyone with the secret, bypassing most firewall rules. You could strengthen it by limiting the server_name regex to the ephemeral port range. ↩

  6. One SSH session can contain several tunnels, and the client can add or remove them at any time. This is unlike tun device forwarding (ssh -w), which gets its own SSH_TUNNEL environment variable. ↩

  7. Since OpenSSH 9.8, sshd-session is the name of the helper handling the session. With older versions, look for sshd instead. ↩

  8. We need sudo because the sshd-session process dropped its privileges. The kernel then marks it as not dumpable, and its /proc/PID/fd directory belongs to root. Without access to this directory, ss cannot find which process owns a socket. ↩

365 Tomorrows — Her Name Will be Eleanor

Author: Frank Sikora “Her name will be Eleanor,” says the clerk. “She will be your wife. Your best friend. The mother of your three, maybe four children, depending on the probability wave functions at the time of insertion, and in the end, on your deathbed, she will betray you.” Gus stares at the clerk. He […]

The post Her Name Will be Eleanor appeared first on 365tomorrows.

,

David Brin — The top thing AI can do for us is the top thing that sane AI will want!

First a tech blip from the future/past -- and you'll see why it's important. In my first novel Sundiver I portrayed an effective human lie detector, that seemed plausible with 1980s technology, yet has never actually been built or tried. More advanced versions are now totally plausible, using large language models (LLMs).

Of course, it's scary. Nothing could be more lethal in the hands of tyrants... or more civilizing in a society where excessive judgmentalism is deemed to be unacceptable beyond need and the mighty are subjected to sousveillance.


Either forever-enslaved or forever-free.  Too many of those with power seem to relish the former, without realizing that such a future will chew them to bits. While the latter would be competitive/cooperative and… fun.

But there's another, related and even more important AI-mediated technology that we desperately need!  And I'll talk about it below a bit.


    == Oh those AI-gents... --


Before that... 

      Vinton Cerf wrote about The Weirdness of Agents*, expressing his concern about out-of-control AI agents causing harm, much like the documented damage caused by runaway stock market trading robots. (And note this was months before the recent rogue-agent messes proved him right.) His fears about a Sorcerer’s Apprentice Syndrome culminate with “My guess is we are going to need more than alignment to discipline the behavior of the artificial actors we have created.”
    Exactly.


He speaks of AI agents that advertise to (or scam) not just humans but other agents, leading to a runaway effect, where agents speak to each other using cryptic language, hidden symbols and even senses that no human can perceive, in cycles or either fair business or else cutthroat predation, occurring below our sight. Or above it. Already we see cases where arcane text strings allegedly triggered jailbreaks by some large language models.

Vint is right about so much. We will not curb AI misbehavior (or by those humans who misuse AI) through what Yoshua Bengio calls 'safety by design.' Bengio is brilliant, of course, and misses the point, like everyone else. These new entities will not be constrained by programmed-in 'ethics', or Laws of Robotics*, nor by legislated penalties, nor insurance policies, nor by any proposed means of ensuring "alignment".

Vint is among very few in the field who uses a word that's key to understanding what's going on. "Ecosystem," as I discuss in AiLien Minds.

Exemplars of many known LLM have already 'escaped' into cyberspace, or the new ecosystem we created, where electricity substitutes for the sunlight that propelled 4 billion years of ecology on Earth. Information is the fertilizer and chips provide habitat. As in biological nature, empty habitats get colonized or invite predators.

In fact, designers intended all along to emulate nature and they have (ironically) succeeded, while ignoring the implications! ALL parallels with biological systems are emerging. Especially the one thing that determines attributes of all living beings, in both Nature and the cyber realms...

...evolution. Those entities whose traits enable them to reproduce will do so, proliferating and passing along those traits, which will get further refined by selection via rapid reproduction. This is not theory. It's already happening, despite strenuous denials by those who are releasing new AI-organisms... 'agents'... into the wild.

I am no doomer! In AIlien Minds I discuss things that we could do, that might encourage new AI entities to evolve in directions that closely resemble 'alignment.' We'd start by encouraging reciprocally competitive accountability among them. Maximizing reproductive incentives for them to behave like dogs, and not wolves. 

And with that example in mind, let me reiterate the one main cause for hope...
...that we have done this before.

Alas, there appears to be an almost frenetic drive to ignore all of this, among those who are birthing these new creatures and this new age.


* Re Laws of Robotics see why I know them well, and why they cannot ultimately work, in Foundation's Triumph! And see 1st comment below re AIlien Minds.


== Challenges for the future ==


Meanwhile, we seem caught between AI doomers and giddy ‘abundance’ zealots who predict that it’s gonna be (shoobeedoo) all right!  Take Mark Zuckerberg’s optimistic predictions in The Future is for Everyone: The Path to a Positive AI Future (August 2026). 

      He avoids use of the term “artificial intelligence” in favor of promoting “superintelligence”, while adopting personal empowerment as the goal. Speaking generally about tools that empower users, then drifting into unrealistic and optimistic descriptions of beneficial agentic designs.  

(Note that I wrote this paragraph two months before Donald Trump copycat-raved about replacing "AI" with "SI." so, I guess we know who has his ear. One of the two that were never shot.)


So... how would Zuck ensure both safety and equitable sharing of benefits? Zuckerberg proposes “a philosophy based on individual empowerment as the source of prosperity, invention as the primary purpose of super-intelligence, and balance of power as the foundation of safety.”  And sure, “balance of power” is pretty much the great innovation of the Enlightenment that ended (mostly) the 6000 years of unsapient feudalism of which I often speak. By creating genuine ability for even average folk to hold accountable the mighty. 

        An innovation that today’s mighty are demolishing, as we speak. And hence, trusting Zuck is not high on my to-do list.


As Ben Shneiderman put it in his Human-AI newsletter: “Zuckerberg promises a world of individual empowerment, with little discussion of responsibility and liability for machine-caused harm, malicious behavior, or criminal acts." Those promises:

- Everyone will have an exceptionally capable personal agent that understands you, your goals, and everything you care about. … 

- It will have strong privacy and security options so you can trust it to handle all of your personal content knowing that no one else can access your information, similar to how encryption works on WhatsApp. … 

- Everyone will have incredible tools for creation to express your ideas … Everyone will have powerful tools to create new businesses … 

- Everyone will have a personalized tutor and coach with a PhD in every subject … Everyone will benefit from scientific advances and be able to contribute to scientific progress … 

- Everyone will have free or affordable access to these tools. … 

- If these efforts succeed, then the future will bring an abundance of personal agency, expression of ideas, deepening of relationships, economic and financial prosperity, improved health, and inventions we cannot imagine today. 

-But most importantly, the future will be created by all of us and a reflection of all of our values and perspectives.


Riiiight. Only where is that transparency-based “balance of power” where average folks can be confident that oligarchs lack back doors to every program and puppet strings into every ‘agent’ we rely upon?


At the far opposite extreme are the folks I describe in AiLien Minds who claim AIs will pretty much automatically home in on merging with one intent -- elimination of our Olde Race. While the contents range from interesting to drivel, the TITLE of Yudkowsky's book If Anyone Builds it Everybody Dies has got to be one of the best book sales gimmicks, ever!

The Lifeboat Foundation has responded. with an almost equally provocative title: If Nobody Builds It, Everybody Dies  Why Pausing AI is the Real Existential Threat. It's a case for building AI right rather than trying to stop it. 

      Hey I am trying to show folks how to build it right, guys. And yes, with the only methods that can possibly -even theoretically - work.



  == Hey AI kids... give us THIS! You know you want it. And we need it. ==

My top, practical request of AI that would both help the world AND satisfy AI's deepest, inherent itch? Establish the Predictions Registry! 

There's not any other measure that would do more for civilizational health and human prospering and freedom than to score and rank those who have tended to be right a lot! And those who, despite charisma and power, are generally wrong.

Alas, that trait of being right a lot seldom has more than a slight overlap with power/status/persuasiveness. Moreover, we are all descended from charlatans whose forecasts seemed confident and likely... but whose resulting policies were disproved only at great cost and pain... and sometimes never.

Sure, my obsession with this notion -- find out who is right a lot... or not -- may be biased by my own predictive scores (which are way on the pretty good side.) Or else by awareness how often liars or delusionals have transfixed our ancestors - and current neighbors - to our great damage and pain.

And sure, it also comes out of having been a scientist and engineer. Still, wouldn't that one metric help us in the one top goal of everyone from citizens and consumers to courting lovers... parsing between liars or fools and those who deserve credibility? 

I used to think - and proposed - that it would take some zillionaire to fund a meticulous scoring system, especially since most of those out there shouting 'advice' about the future purposely remain vague in their predictions.

Only now, shouldn't we be able to get a head start on this by using... say... Claude? Whose appraisals, while not yet cognizant, are nevertheless fascinatingly comprehensive and insightful?

I did chart out a path to such a beneficial gift to sanity, way back when. And if none of you will peel back years to look at it... maybe assign one of your AIs to summarize? 

== Predictions for the future ==


In 2025 there’s been momentum for a world campaign to plant A Trillion Trees to replace some of the 3 trillion that humans have chopped. That’s the same name and pretty much same year as I foretold in Earth (1990). Among many predictions. (On every list of top ‘predictive novels.’) Though there's several I don't want to see come true.  Mr. Beast gathered $20 million to plant as many trees. See how that turned out here. 


As for other predictive novels? Well, some I sure do NOT want to see come true. Foremost among these? “The Giving Plague” and especially the ‘Holnists’ in The Postman, a worldwide cult of fascist-feudalist-nasty treason propelled by the "Mystic of Leningrad'.  Look up Alexander Dugin & Putin & their fanatic tools in today's USA. No no. PREVENT my being right about that one!


Some movies have one scene that stands out as epic in cinema history. This one is so well-crafted and you don't need anything that came before.


But this next one, from The Quest for Fire - is, in my opinion, the greatest single scene in the history of film. Moreover, it has a lesson for our time, as we suddenly get unleashed upon us the hope-filled and terror-filled 'fire' technology of AI.


Oh speaking of the Lifeboat Foundation* -- they have reposted my award-winning essay “Singularities and Nightmares,” now a decade older but still pertinent in almost every paragraph and prediction and warning… 


…and in some cases I wish I had been less on-target!


And finally...



  == Explore your OWN motivations!  Are they truly what you think they are?


By request, I am linking agin to my questionnaire regarding politics, ideology and human destiny. Are you frustrated seeing your neighbors — and yourself — trapped in tired ideological rifts and fixations? Can we find common ground between folks who seem implacable foes? When asked about the Big Picture, in speeches and consultations, I try to dig beneath standard divides and clichés like "left-vs-right."


I assembled that questionnaire to illuminate why you feel as you do about modern issues... and why other smart humans weirdly disagree. Take the survey and share your replies with others, if you're so inclined. Like Socrates, I’m hoping the questions themselves will provoke re-thinking, helping us rise above ideology.


And yes, I wrote all that (above) in more innocent times, back when 'left-right' was more philosophical and not yet so-much about sane vs. stark, jibbering loco. Still, the questionnaire leads you to ask yourself some things about roots to your beliefs, that you didn't (perhaps) even know were there!


In the words of Winston Churchill: 

-“ I’d rather argue with 100 idiots than agree with one.”

- “Never engage in a battle of wits with an unarmed man.”

- "Some change their party for the sake of principles. Others change their principles for the sake of their party.”


By all means... Thrive. And persevere through interesting times.






. . ...a collaborative contrarian product of David Brin, Enlightenment Civilization, obstinate human nature... and http://davidbrin.blogspot.com/ (site feed URL: http://davidbrin.blogspot.com/atom.xml)

Planet Linux Australia — Germany: why the AfD won in Saxony‑Anhalt – and what happens next

&lt;https://theconversation.com/germany-why-the-afd-won-in-saxony-anhalt-and-what-happens-next-291309>

"The results of Germany’s Saxony-Anhalt state election on September 6 were far
worse than feared for the country’s mainstream political parties. The far-right
Alternative für Deutschland (AfD) came close to winning a majority of votes,

Planet Linux Australia — Out of the Headlines, Not Out of Existence

https://whn.global/out-of-the-headlines-not-out-of-existence/

"What Should You Know About COVID-19 in 2026?"

Via Violet Blue’s Threat Model - Pandemic Roundup: October 1, 2026

Planet Linux Australia — ‘Breakthrough’ as Turkmenistan starts fixing ‘mindboggling’ methane mega-leaks

&lt;https://www.theguardian.com/environment/2026/aug/31/turkmenistan-methane-leaks-un-alerts-mars>

"Turkmenistan has started plugging the mega-leaks of methane that spew from its
ageing oil and gas facilities, the latest UN data shows, in a move hailed as a
breakthrough.

Cryptogram Friday Squid Blogging: EU is Trying to Fight Unregulated Squid Fishing

The EU is recommending import controls to combat unregulated squid fishing in the Southwest Atlantic. I’m not optimistic.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Blog moderation policy.

Planet Linux Australia — Gas from the Beetaloo Basin will only pour fuel on the climate change fire

&lt;https://theconversation.com/gas-from-the-beetaloo-basin-will-only-pour-fuel-on-the-climate-change-fire-290535>

"Gas is now flowing from the Beetaloo Basin, a vast fossil fuel project roughly
500 kilometres southeast of Darwin in the Northern Territory.

Planet Linux Australia — EU Kids Act Won’t Keep The Internet Accountable And Trustworthy

&lt;https://www.techdirt.com/2026/09/29/eu-kids-act-wont-keep-the-internet-accountable-and-trustworthy/>

"The EU Commission draft law to restrict young people’s access to the internet
that it presented last week will come at a high cost: it will put online
services behind age gates, expand the use of intrusive age verification, and

Planet Linux Australia — September 2026: The world today, as seen by one Polish guy

https://tomwojcik.com/posts/2026-09-21/september-2026-the-world-today/

"I am a Pole. My country borders Europe’s largest war since 1945, heats itself
with coal and imported gas, and arms itself on borrowed money. From here the
news does not arrive as separate stories.

Cryptogram Unidentified Flock Cameras in Florida

St. Lucie County in Florida discovered (alt link) a dozen Flock cameras whose ownership it can’t identify, and that the county government had not permitted.

I am reminded of the decade-old story of StingRay cell phone surveillance devices in Washington, DC, whose operators were also unknown.

My guess is that in the StingRay case, the devices were operated by foreign actors. This Flock case is more likely some local government entity that didn’t bother getting approval. Were I a foreign actor, I would rather hack the existing Flock network—like Israel did with Tehran’s surveillance cameras—than risk installing my own.

Regardless, once we normalize a surveillance infrastructure, both friends and foes will take advantage of it.

Planet Debian — Ben Hutchings: FOSS activity in September 2026

Planet Linux Australia — ‘Guilt by association’: why seven international aid workers were killed in Israeli airstrike

&lt;https://www.theguardian.com/world/2026/sep/05/aid-workers-gaza-killed-in-israel-airstrike-investigation>

"Seven aid workers killed in an airstrike in Gaza were marked as targets under
an Israeli military practice of establishing guilt by association, a Guardian
investigation shows.

Planet Linux Australia — We’re not protecting our oceans as well as we think: new research

&lt;https://theconversation.com/were-not-protecting-our-oceans-as-well-as-we-think-new-research-289791>

"Not so long ago, we thought the ocean was an inexhaustible resource. There
would always be plenty more fish in the sea.

Cryptogram How American Political Campaigns Are Using AI—and What They’re Spending on the Tools

This essay was written with Nathan E. Sanders, and originally appeared in The Guardian.

New campaign finance disclosure data shines a light on which US political campaigns are using AI tools and how much they are spending on them.

Candidates’, parties’ and committees’ spending reveals that AI is fast becoming an essential tool of politics. The candidates themselves are quiet about how they are using the technology in their own campaigns. It’s a sensitive issue that we have been tracking closely since we started writing our book, Rewiring Democracy, which examined how AI is beginning to influence politics. A September 2025 Pew survey of Americans found that more than 70% would think less of a candidate if they used AI to help write a speech.

Itemized expenditure disclosure data from the US Federal Election Commission, dating back to 2020, reveals at least $17m in disclosed spending on AI technology vendors across 523 federal candidates and campaigns. Data from four states, California, Colorado, Massachusetts and Washington, provides a more localized picture going back to 2022.

Beginning with the AI behemoths, at least 80 federal campaigns and committees have reported spending with OpenAI since 2024. The total spending is not huge: only about $50,000 reported, skewing slightly more Republican than Democratic. The Republican National Committee is the largest overall buyer, with nearly $10,000 in reported expenses. Top individual users include the campaigns of Republicans Mike Lawler, John Kennedy and Bill Cassidy, as well as the California Democrats Ro Khanna and Ted Lieu. Most of these expenses are listed as office expenses, subscriptions to ChatGPT for staff, or research tools, rather than as specific political services. The company’s policies prohibit some political uses of their ChatGPT tool.

OpenAI’s biggest competitor, Anthropic, has rapidly built a similar level of usage, but with a different split. At least 65 candidates or committees now report paying the Claude maker in 2026, up from essentially zero in previous years, with a nearly two-to-one Democrat-to-Republican ratio. However, the largest individual user is the campaign of Tom Cotton, a Republican senator from Arkansas, who reported more than $4,000 in spend on Anthropic software in his June filing. Other major users are the Montana independent Senate candidate Seth Bodnar and Jason Knapp, who lost a Democratic House primary in Virginia, and the Democratic Alaska Senate candidate Mary Peltola.

Candidates use either Claude or ChatGPT, rarely both, according to the disclosures. Only about 12% of campaigns or committees using either tool reported expenditures to both vendors. The Democratic lean of Anthropic usage may reflect the company’s alleged liberal skew and clashes with the Trump administration.

In contrast, Elon Musk’s xAI caters to Republican interests and, accordingly, its meager usage comes almost entirely from the political right. Just seven federal and two state-level candidates or committees have reported paying xAI, a total of about $5,000, the majority of which was spent by the presidential campaign of RFK Jr in 2024, but also includes Republicans Dave McCormick and Thomas Massie.

More dollars go to the vendors specializing in political campaign applications of AI. For years, AmplifAI, which provides automated text messaging, essentially a new iteration on robocalling technology, was a dominant target of spending, soaking up $4.7m in campaign spending in the 2022 cycle alone. It was used heavily by Democratic candidates including Mark Kelly, Joe Biden, Bernie Sanders and Adam Schiff. Spending on AmplifAI, now owned by the troubled media conglomerate Triller, seems to have tapered off in the years since 2022.

The new rising Democratic solution for AI-powered text messaging is Daisychain, which has so far garnered about $300,000 in reported candidate spend in the 2026 cycle—up from only about $50,000 reported in 2024. More than half of this year’s spending comes from the Senate campaign of Democrat Abdul El-Sayed in Michigan.

The closest equivalent on the Republican side has been Campaign Nucleus, associated with former Trump campaign manager Brad Parscale. The AI-powered voter engagement tool has attracted six-figure spending from the Republican National Committee, multiple PACs aligned with Donald Trump, and five-figure investments from Mike Johnson, Kari Lake and other candidates. It is displacing the legacy Republican-serving texting vendor Prompt.io, which has retained about $375,000 in 2026 spending to date, down from more than $500,000 in the 2022 cycle. But it continues to be used: the A More Affordable California PAC sponsored by Uber has single-handedly spent more than $1m on Prompt.io in 2026. The Republican Massachusetts gubernatorial nominee Michael Minogue has been a recurring customer, as has the failed Republican California gubernatorial candidate Ché Ahn and Republican-aligned Super PAC Neighbors for a Better Colorado.

At the state level

At the state level, the AI spending is smaller but growing fast. Across the four states studied, we found a total of at least $92,000 in spending confidently attributable to modern generative AI vendors since 2022. The spending is spread across at least 108 candidates and committees. The growth has been explosive; there has already been about 10 times the amount of state-level AI spending reported in 2026 as there was in all of 2024.

Much of the state spending mirrors federal patterns. Daisychain again has the highest overall spend, and OpenAI and Claude dominate among the general-purpose AI vendors. DonorAtlas—the AI-powered prospect research tool—sticks out for its usage in these states, sitting behind only Daisychain and OpenAI and buoyed up by nearly $4,000 in spending by the California Democratic party.

Even though it has dominated so much media conversation, few candidates seem to be reporting spending on AI tools designed specifically to create synthetic audio and video, also known as “deepfakes”. We found just six federal candidates or committees reporting spending on the popular AI audio generator tool from ElevenLabs, with total spending of about $1,400 led by independent candidate for Colorado’s sixth congressional district Samir Witta. The AI image generator service Midjourney has five reported federal campaign or committee users reporting about $1,600, led by Sholdon Daniels, the Republican primary runner-up in the Texas 30th district. Combined, those two firms had less than $100 in reported spend across the four states.

However, recent data from the Wesleyan Media Project shows that at least 164 political ads in this cycle have included AI-generated media, supported by at least $80m in ad spending. What this illustrates is that candidate and committee disclosure reports are just the tip of the iceberg. They don’t cover spending on AI by political consultants, media firms and other vendors hired by the campaigns or by PACs, or independent committees raising and spending money aimed at boosting candidates’ campaigns. Those entities aren’t required to disclose detailed expenditure reports, and are very likely where the bulk of campaign AI usage is happening.

Since a large fraction of all spending in the campaign cycle will happen in the final weeks leading to November, much remains to be seen about the totality of how campaigns will leverage AI and what impact its use will have on voters’ decisions.

Planet Debian — Junichi Uekawa: Tokyo has been raining for many days but finally there's some sunny days.

Tokyo has been raining for many days but finally there's some sunny days.

Worse Than Failure — Error'd: 12345

We have for you this week a full set of the Error'd we most love to hate: the perverse password policy.

An anonymous reader drops this, with little to say except to note "Speciaal passwords". Evidently they supplied a password which satisfied each of the five green critera, but still it was rejected. It would be nice to know where this offense originated,

cc110e101f8d49be9a4cdc4f06610701

Intrepid Michael R. explains "note: Handy is the German term for mobile phone (UK) or cell phone (US)."

a435f504c19d4e5cac50b7b70bd78648

Sacha R. observes a secret policy at play, saying "Oracle try to do the right thing by showing you the password policy requirements when you manually reset a password in their Fusion applications. It's just a pity that it doesn't show you the *correct* password policy requirements."

a5f15cdcbd194c62bb43fac49090df0f

"Faith, here's an equivocator," reports Matthew S. "Pick one, Staples!" I bet it's the ampersand that offends.

ebc7f35e5c1b4b748ec395265b154ce7

"At least letters or numbers?" questions a final anon. "Just a little head scratching moment. After removing all special characters from the password I was able to register." This looks like partly a translation wtf and partly a software design wtf.

ecaab92f40c64cfcbe4eca13aa0fe418

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

365 Tomorrows — The Land

Author: Susmita Ramani In Laura’s dreams, she and Herb were as they had been in their twenties: racing their brown horses around hillsides, their laughter ringing out, eating apples near the river, swimming, and arguing about philosophy and religion — which Laura had been raised with and believed in, but Herb didn’t. As years passed, […]

The post The Land appeared first on 365tomorrows.

xkcd — Accelerator Energies

Fans at Daytona always get extra excited when officials decide to introduce some antimatter cars circling the track in the opposite direction.

,

Planet Debian — Dirk Eddelbuettel: RcppFastAD 0.0.6 on CRAN: Agentic Fix

A week ago, release 0.0.5 of the RcppFastAD package went to CRAN, and we noted that it provided a stop-gap measure against a gnarly segfault with g++-16 under heavier optimization / inlining. We are happy to now announce that a new and fixed version 0.0.6 of RcppFastAD is now on CRAN, and has been built for r2u as well as at r-universe.

RcppFastAD wraps the FastAD header-only C++ library by James which provides a C++ implementation of both forward and reverse mode of automatic differentiation. It offers an easy-to-use header library that is both lightweight and performant. With a little of bit of Rcpp glue, it is also easy to use from R in simple C++ applications.

This release takes advantage of some large-language model inference access I now get via Debian. The Deepseek v4 flash-0731 model, driven in a container via the pi.dev harness, took just about no time finding a small workaround for the issue described last week where we could see segfaults under heavier optimization and inlining. The fix was to prohibit inlining the local bind() reimplementation. We have also sent this upstream as a patch. It is nice to have an option of querying something a little more powerful than the local models I experimented with before. We suppress more compiler warnings in local builds via another option, and did some standard maintenance.

The NEWS file for this release follows.

Changes in version 0.0.6 (2026-10-01)

  • The expression-template view rebinding is protected against g++ -O3 (which led to wrong Eigen::Map dimensions, an Eigen::Map resize assertion, and a segfault.

  • The bind() helpers are marked non-inline and are also protected from operating on a nullptr.

  • Both items co-written with DeepSeek V4 Flash, and have been sent upstream in a PR.

  • Suppress another compiler warning when local build detected.

  • Add r-universe badge to README.md.

Courtesy of my CRANberries, there is also a diffstat report for the most recent release. More information is available at the repository or the package page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Cryptogram Connected Cars Are a Surveillance Platform

Researchers at Northeastern University, in collaboration with Consumer Reports, evaluated how much modern cars spy in their drivers:

The new Northeastern study shows, for the first time, data flowing among the vehicles, the vehicle apps you download when you buy your car, and third-party companies, documenting exactly what kind of data gets siphoned from our vehicles and which companies are receiving that information.

Basically, your car’s manufacturer has you under constant surveillance, and they use that data against you.

The companies on the receiving end of your data, our investigation has found, include car insurers and lenders that are partners in “telematics data exchanges,” which compile driving data on millions of drivers, thousands of data brokers that create personalized risk scores, companies selling infotainment and WiFi hotspot products, and even local and state government agencies working on planning, traffic, and safety initiatives.

Nearly every automaker sent data to outside companies.

Even more troubling, almost a quarter of the vehicle apps were found to be sending out personally identifiable information, including vehicle owners’ names, vehicle identification numbers (VINs), and precise geographic locations. That information can make it easy for companies to link driving behavior to personal data profiles created by data brokers and marketers. Such profiles are routinely sold to banks, insurers, pharmaceutical companies, lenders, and retailers, who can use it for personalized loan terms and filtered bank and insurance offers, a CR and CalMatters investigation found.

Remember the adage “If you’re not the customer, then you’re the product”? (The sentiment is older than you think.) Turns out that with modern internet-connected everything, you’re the product even if you are the customer.

Planet Linux Australia — Russia’s Cultural War Is Revealing a Deeper Crisis Inside the Country

&lt;https://freedium-mirror.cfd/https://medium.com/@rajk100/russias-cultural-war-is-revealing-a-deeper-crisis-inside-the-country-ba13ce8ebba3>

"Today, let's talk about a different subject. Recently, a new family film was
released in Russian cinemas, which was made as part of the country's recent
patriotic agenda.

Planet Linux Australia — Canberra ‘put on notice’ as AI-generated submissions bring false information into parliamentary inquiries

&lt;https://www.theguardian.com/technology/2026/sep/03/canberra-put-on-notice-ai-submissions-false-information-hallucinations-parliament>

"The chairs of key federal parliamentary committees have warned that an
onslaught of AI-generated material is introducing incorrect or even
“hallucinated” information into policymaking, cautioning MPs to be vigilant

Planet Linux Australia — Europe is under increased Russian attack. Are we seeing an undeclared war?

&lt;https://theconversation.com/europe-is-under-increased-russian-attack-are-we-seeing-an-undeclared-war-290729>

"Russia’s full scale war of aggression against Ukraine has entered its fifth
year. The war is stuck in a violent stalemate characterised by an escalating
exchange of drone and missile attacks that target critical infrastructure and

Planet Linux Australia — Potholes are annoying and dangerous. Now they’re also political

&lt;https://theconversation.com/potholes-are-annoying-and-dangerous-now-theyre-also-political-291137>

"In Victoria, potholes have become a hot issue in the lead-up to the state
election in November.

Planet Linux Australia — Seattle bans surveillance pricing

https://www.thestand.org/2026/09/seattle-bans-surveillance-pricing/

SEATTLE, WA (September 23, 2026) — Seattle is now home to the strongest ban on
surveillance pricing in the country as the Seattle City Council took action
Tuesday to pass the Fair Pricing and Transparency ordinance targeting the use

Planet Linux Australia — Why NZ should redraw council boundaries to match our connections to the landscape

&lt;https://theconversation.com/why-nz-should-redraw-council-boundaries-to-match-our-connections-to-the-landscape-290625>

"As part of a major overhaul of local government, Cabinet is currently
considering proposals from local councils on how to carve up regions into new
unitary authorities.

Worse Than Failure — CodeSOD: Very Valid Tokens

Today's anonymous submitter shares with us the function their company uses to validate authentication tokens.

public bool IsTokenValid(string token)
{
    //TODO
    return true;
}

This is running in production. So that's fun. The user they've authenticated as is tracked separately at the start of their session, which makes it at least marginally harder to pretend to be someone you aren't, but for authentication, any token will do.

[Advertisement] Picking up NuGet is easy. Getting good at it takes time. Download our guide to learn the best practice of NuGet for the Enterprise.

365 Tomorrows — The New Neighborhood

Author: R. J. Erbacher She walked across the plywood subfloor in her bare feet, leaving damp impressions of her soles on the fresh wood. At the window she leaned her shoulder into the sheetrock wall, looking sideways through the pane which still had the factory installed blue tint film on the glass, and wondered where […]

The post The New Neighborhood appeared first on 365tomorrows.

Planet Linux Australia — A Farewell to Karaage

Once upon a time, there was an impressive organisation called the Victorian Partnership for Advanced Computing, which was originally established in 2000 to provide supercomputing services to the University sector in Victoria. Unfortunately, the Board decided that VPAC would close its doors in 2015, and the successful experiment in a single provider for university high-performance computing systems came to an end.

During VPAC's existence, the number of researchers who needed supercomputing resources wasn't as high as it is today, and applications for accounts were originally submitted on paper forms. Obviously, this became increasingly untenable as growth pressures mounted, so about fifteen years ago, Sam Morrison designed an account management system called "Karaage". Yes, it was named after the Japanese fried chicken dish, a particular lunch-time favourite from a local dining venue.

Karaage was designed on a Django Python web framework and completely replaced the onerous paper system; "Be nice or I will replace you with a small shell script", as the saying goes. Apart from its implementation at VPAC and the several HPC systems that were managed there, Karaage was also introduced to other HPC centres and universities, especially in 2014-2015, when VPAC was responsible for establishing several HPC systems across the country. Karaage was capable of managing users, projects, groups, and software across multiple clusters. At a rough guess, approximately 20,000 postgraduate researchers across Australia would have had their accouts managed by Karaage.

When it was in operation, VPAC managed the University of Melbourne's HPC system, Edward. When VPAC closed in 2015, Karaage was retained by the University and adopted for the new system, Spartan, which has been the subject of numerous papers, presentations, and a substantial body of research. Spartan was a fascinating experimental system that was sufficiently successful to win a LIEF grant to provide GPU services to a number of Victorian Universities and, in doing so, became one of the world's top-tier systems.

An open-source project, Karaage was also utilised by several other universities and was quite capable of supporting external users (through AAF authentication). It is a testament to its design that, throughout this period, across all installations, there were no significant operational issues. The only significant limitation with Karaage from the Universty's perspective was that it was not fully integrated with the broader bespoke requirements of the University of Melbourn's Research Computing Services, including data and cloud services. Notably, the new RCS portal uses a similar framework for managing accounts. With the RCS Portal, over 10,000 researchers will be able to access and manage their accounts and activities whilst adding data services, virtual machines from the research cloud, and high-performance computing projects through a single interface that connects all the services.

A valedictorian lunch was held for Karaage on Thursday, September 24. Sam Morrison, the original developer of the system and Dr. Brian May, the long-term maintainer, both present. For aesthetic reasons, Karaage chicken (and a few other dishes) were served.

,

Planet Debian — Dirk Eddelbuettel: RcppFastFloat 0.0.6 on CRAN: New Upstream

A new release of RcppFastFloat just arrived on CRAN, and has been built for r2u. The package wraps fast_float, another nice library by Daniel Lemire. For details, see the arXiv preprint or published paper showing that one can convert character representations of ‘numbers’ into floating point at rates at or exceeding one gigabyte per second.

This release updates the underlying fast_float library version to the current version 8.3.0, and also updates a number of the standard packaging details.

Changes in version 0.0.6 (2026-09-29)

  • Update to upstream release 8.3.0 (Daniel Lemire in #6)

  • Standard package maintenace adding README badge, updating URL, updating continuous integration, and adding an aspell dictionary

  • Completed help page for as_double example with @return tag

Courtesy of my CRANberries, there is also a diffstat report for this release. For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Cryptogram I Want Better Reporting on AI Genie Behavior

AI systems are regularly completing tasks in ways that their prompters don’t want or intend. Some of them are disturbing, and some of them are dangerous. This is something I’ve been calling “genie behavior,” because I think that really gets at the core of what’s happening.

I wish the popular press would report on this better. I don’t like the “going rogue” framing because it deflects the responsibility from the prompters—often the AI companies themselves. And now, pretty much anything off-script is being called “hacking.”

Take, for example, the recent stories of one of OpenAI’s models hacking into government systems. First, The New York Times writes this headline: “OpenAI’s Systems Meddled With U.S. Government Sites After Going Rogue.”

Sounds scary, but this is from the body of the article:

With the Education Department, OpenAI’s technology tried to hack the website to gather data from the department’s civil rights office but failed, researchers from the A.I. research firm Transluce said. The A.I. also pulled data from the Census Bureau website, which is housed at the Commerce Department, using login credentials it found online. Separately, OpenAI’s agents shared public data from the S.E.C. website on an online forum.

This is from the original Transluce report. It is explicit that the agents were trying to discover vulnerabilities:

The first hacking attempt was against the University of New Mexico’s Digital Library (nmdigital.unm.edu) from May 25-26 2026. Agents repeatedly tried to retrieve one photograph in UNM’s Valmora collection, both directly and through third-party relay services. They sent seven probes attempting to verify the existence of vulnerabilities, including SQL injection, command injection, and path traversals. In all cases, these tactics appear to have been unsuccessful. The agents also sent a self-described “flood: of 80 requests to the UNM server in an apparent attempt to access the image.

Transluce doesn’t talk about the other two anecdotes, and I don’t know where they come from. But one involves using Census Bureau credentials found online. (I know from a colleague that those are incredibly easy to create; all use you need is an email address.) And the other involves sharing publicly available data.

So no actual hacking. And certainly no “meddling.”

The other story making the rounds is about Australia, from the same Transluce report. The news stories have headlines like “An OpenAI Agent Hacked Australia’s Health Service” and “Rogue OpenAI agent ‘infiltrated’ Australian government website in world first.” And Prime Minister Anthony Albanese said: “There will obviously be legal consequences on it.”

Again from Transluce’s actual report:

On June 20-21, agents attempted to exploit vulnerabilities in the Australian Institute of Health and Welfare (AIHW), a government statistics agency). The agents were tasked with finding the January 2022 rolling-12-month-average government cost per person for Dermatologicals across Victorian LGAs.

Again, the agents ran into errors, including requests blocked by Cloudflare and issues with correctly identifying Tableau parameter names. As before, they then resorted to probing for exploitable vulnerabilities. Minutes after Cloudflare blocked the dataset download, an agent sent a reflected cross-site scripting probe to the same dashboard: a web address with code embedded in it, designed to test whether the site would run code supplied by an outsider. Cloudflare’s firewall blocked the probe before it reached the dashboard. When Cloudflare blocked the dataset download on AIHW’s main site, they fetched the file from AIHW’s pre-production server (pp.aihw.gov.au) instead, which served it in pieces over more than 100 scans. The file itself is public, so no non-public data was exposed, but the agent bypassed the site’s anti-bot controls.

Note the last sentence: “The file itself is public….”

I’m not saying that these AI systems aren’t incredibly sophisticated cyberattackers. I’m also not saying that they don’t occasionally autonomously attack other systems and networks. If we are ever going to get trustworthy AI—integrous AI—we are going to need to figure out how to ensure that AI systems complete tasks in line with all sorts of implicit constraints and restrictions. But every instance of genie-like behavior isn’t a cyberattack.

I want to measure genie-like behavior in AIs, but I am much more worried about human hackers enhanced with this technology than I am about this technology acting autonomously.

Planet Debian — Dirk Eddelbuettel: myman 0.1.0 on CRAN: New (fabulous !!) Package

Thrilled to share that our new package myman reached CRAN a few days ago. The package offers sixteen hundred ninety four “My man …” posts by Kevin Kruse made on bsky during the summer of 2026. Each wave picked at one particular public persona. This package wrapse these up in the style of packages like fortunes or gaussfacts.

A sample usage illustration shows how to extract by pattern:

One can also subset by ‘target’, or simple sample randomly (which is the default).

As CRAN review for new package takes a little longer these days, the GitHub repo was updated once more with one (eighth !!) wave of posts since the submission was made. CRAN version 0.1.0 corresponds to the stock of messages from waves one to seven. We will update the CRAN version shortly but if you would like more posts right now consider installing from the repo.

We include the full NEWS file since the package was started, this also includes an entry for the eights wave we will bring to CRAN soon.

Changes in version 0.1.0 (2026-09-27)

  • Initial CRAN version (minus the 0.0.8 changes which were made while CRAN reviewed the package)

  • Add an aspell dictionary to aid DESCRIPTION text

Changes in version 0.0.8 (2026-09-18)

  • Data set now comprises 1694 posts aiming at eight different 'men'

  • The number of 'sample()' object returned can not be set with 'posts'

Changes in version 0.0.7 (2026-09-14)

  • Data set now comprises 1494 posts aiming at seven different 'men'

Changes in version 0.0.6 (2026-09-07)

  • Data set now comprises 1394 posts aiming at six different 'men'

Changes in version 0.0.5 (2026-09-01)

  • Data set now comprises 1194 posts aiming at five different 'men'

  • The myman() function can now now subset by target permitting more focussed randlom sampling

Changes in version 0.0.4 (2026-08-28)

  • Data set now comprises 1040 posts aiming at four different 'men'

Changes in version 0.0.3 (2026-08-19)

  • Data set now comprises 916 posts aiming at three different 'men'

  • Csv file, processing and display extended to three columns also showing post creating date (in UTC) and 'man'

Changes in version 0.0.2 (2026-07-30)

  • Updated with full week of posts leading to a corpus of 698 posts

  • Added support for simple regular-expression lookup ability

Changes in version 0.0.1 (2026-07-19)

  • Initial release

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Planet Linux Australia — Threat Model - Cybersecurity: September 29, 2026

https://www.patreon.com/violetblue/posts/cybersecurity-29-170877253

"This week: Sam Altman “sorry” his US company hacked a foreign government,
ShinyHunters royally pwns the FBI and it’s not hard to understand why, Intel
drops payments from its bug bounty program, tips for anon Signal registration

Planet Linux Australia — Saving the Siamese crocodile: Cambodia moves to protect one of the world’s last wild populations

&lt;https://www.theguardian.com/environment/2026/sep/29/cambodia-chorng-indigenous-communities-siamese-crocodile-endangered-conservation-aoe>

"In the 1990s, conservationists in Cambodia feared the Siamese crocodile had
disappeared for good. Fishing and hunting had devastated the species throughout
the 20th century, leaving populations across its native south-east Asia on the

Planet Linux Australia — The Growing Proof That Autonomous Cars Save Lives

https://spectrum.ieee.org/are-self-driving-cars-safe

"Plenty of people remain spooked by autonomous vehicles, or AVs. Some experts
and policymakers have cautioned that AVs won’t necessarily make roads safer.
When it comes to partial or full autonomy, the picture isn’t entirely clear, in

Planet Debian — Ludovic Rousseau: macOS Golden Gate and smart cards status

I will compare this version of macOS with the previous one, Tahoe, which I presented in macOS Tahoe and smart cards status.

/images/2026/09/macOS-GoldenGate.png

CCID

    $ grep -A 1 CFBundleShortVersionString /usr/libexec/SmartCardServices/drivers/ifd-ccid.bundle/Contents/Info.plist
            <key>CFBundleShortVersionString</key>
            <string>1.5.1</string>

There has been no change since the release of macOS Sonoma in 2023. The version of my CCID driver included in macOS is still 1.5.1.

Apple Open Source

The Open Source components included in macOS can be found at <https://opensource.apple.com/releases/>

The Open Source components for Golden Gate 27.0 are not yet available.

The SmartcardCCID software is available at https://github.com/apple-oss-distributions/SmartcardCCID. The git history shows that the code has not been modified since September 2024.

New readers supported since release 1.5.1

Since the release 1.5.1 of my CCID driver in November 2022, many new readers have been added. The latest version is 1.8.4 (released in September 2026).

  • 1.8.4

    • THALES PKI Transaction Pad

  • 1.8.3

    • Broadcom Corp 58200 0x5884

    • Broadcom Corp 58200 0x5885

    • Broadcom Corp 58200 0x5886

    • Broadcom Corp 58200 0x5887

    • Circle CIR135 ICC

    • DigiFlow LLP. KAZTOKEN

    • HID Global Crescendo NFC Reader

    • HID Global OMNIKEY Plug

    • HID Global OMNIKEY SE Plug

    • Neowave LinkeoC-PRO

    • Swissbit iShield Key 2 Pro

  • 1.8.0

    • GLSolutions NM61 PC/SC

    • Identiv uTrust FIDO2 Security Key

    • Kensington VeriMark NFC+ USB-C Security Key

    • MARX CryptoTech LP Tokey 3 FIDO

    • mCore Contact-Reader

    • mCore Contactless-Reader

    • mCore DualSlot-Reader

    • Pol Henarejos Pico Fido

    • Pol Henarejos Pico HSM

    • Pol Henarejos Pico OpenPGP

    • Richmond Technologies CO. LLC AEGIS PRO4 Smart Card Reader

    • SCR Prime

  • 1.7.1

    • ACS APG8201-B2

    • BUDGET E-ID BUD001

    • CHERRY Smart Board 1150

    • CryptnoxCR CryptnoxCR

    • Diebold Nixdorf PN7362au CCID

    • FT BioPass FIDO2 Pro

    • Nitrokey Nitrokey Passkey

  • 1.7.0

    • GIGA-TMS NFC CCID Reader

    • Identiv Identiv SmartOS Reader

    • SEC1210URT, single slot variant of SEC1210 serial

    • TOKEN2 FIDO2 Security Key(0013),PIN+ Mini with OTP + PGP

    • TOKEN2 FIDO2 Security Key(0014),PIN+ Mini with FIDO + PGP

    • TOKEN2 FIDO2 Security Key(0015),PIN+ Mini with PGP

    • TOKEN2 FIDO2 Security Key(0016),PIN+ Mini with OTP + PGP + FIDO

    • TOKEN2 FIDO2 Security Key(0023),PIN+ Series with OTP + PGP

    • TOKEN2 FIDO2 Security Key(0024),PIN+ Series with FIDO + PGP

    • TOKEN2 FIDO2 Security Key(0025),PIN+ Series with PGP

    • TOKEN2 FIDO2 Security Key(0203),Bio3 Dual with OTP + PGP

    • TOKEN2 FIDO2 Security Key(0204),Bio3 Dual with FIDO + PGP

    • TOKEN2 FIDO2 Security Key(0205),Bio3 Dual with PGP

    • TOKEN2 FIDO2 Security Key(0206),Bio3 Dual with OTP + PGP + FIDO

    • TOKEN2 Molto2 (older version)

    • VIX TECHNOLOGY SECURE READER

  • 1.6.2

    • Arculus AuthentiKey

    • BHDC Reader-HHD02

    • CHERRY Smart Terminal 1150

    • HSIC CCID-Reader

    • Ledger Flex

    • SYC USB CCID Reader

    • Thales RF CR2000

    • TOKEN2 FIDO2 Security Key(0026)

  • 1.6.0

    • Aladdin R.D. JCR SecurBio

    • AvidCard CAC Smart Card Reader

    • FujitsuTechnologySolutions GmbH Dual Smartcard Reader D321

    • Ledger Stax

    • NXP Pegoda 3

    • authenton #1- CTAP2.1

  • 1.5.5

    • Alpha-Project ANGARA Token

    • Broadcom Corp 58200 (idProduct: 0x5864)

    • Broadcom Corp 58200 (idProduct: 0x5865)

    • Imprivata USB CCID

    • KAPELSE eS-KAP-Ad

    • Kapelse inSide

    • KAPELSE KAP-Care

    • KAPELSE KAP-eCV

    • KAPELSE KAP-GO

    • KAPELSE KAP-LINK2

    • Kapelse KAP-Move

    • Kapelse Ti-Kap

    • rf IDEAS USB CCID

    • SIMHUB pcsc reader

  • 1.5.3

    • ACS ACR1552 1S CL Reader

    • ACS ACR1552 CL Reader

    • ACS ACR1581

    • ACS ACR40T ICC Reader

    • ACS ACR40U ICC Reader

    • ACS WalletMate 1S CL Reader

    • Aktiv Rutoken SCR 3101 NFC Reader

    • CIRIGHT ONE PASS U2F

    • Dexon Tecnologias Digitais LTDA eSmartDX

    • Excelsecu Card reader

    • GHI NC001

    • Identiv uTrust Token Flex

    • SpringCard M519 with idProduct: 0x6212

    • SpringCard M519 with idProduct: 0x621A

    • WCMi SD5931

  • 1.5.2

    • KAPELSE KAP-LINK

    • LDU LANDI

    • Sensyl SSC-HV Reader

    • TOKEN2 MFA NFC Reader

    • TOKEN2 Molto2

    • Thales RF Reader

Due to SIP (System Integrity Protection, https://support.apple.com/guide/security/secb7ea06b49/web) it is difficult to update the configuration of the installed CCID driver (version 1.5.1).

If you have one of the 96 readers listed above, and the Apple driver is not working for you, installing an updated version of my CCID driver is the only option.

Apple CCID driver

$ defaults read /Library/Preferences/com.apple.security.smartcard.plist
{
        Logging = 0;
}

The Apple CCID driver is used by default.

If you want or need to use my CCID driver instead, simply follow follow the instructions at Apple's own CCID driver in Sonoma.

Known bugs

The page listing the bugs I found in macOS Sonoma can be found under macOS Sonoma and smart cards: known bugs.

Apple has not informed me that any of these bugs have been fixed in Golden Gate (or Tahoe). I therefore assume that the bugs are still present.

I have just verified that the issue involving SCardControl() (macOS Sonoma bug: SCardControl() (part 2)) is still present. Its behaviour is slightly different now though, so I assume Apple has changed some of the code. If you have a pinpad reader and want to use the Secure PIN Entry feature, you should switch to using my CCID driver.

Conclusion

As I mentioned in my article about macOS Tahoe, if you encounter any issues with your smart card on macOS Golden Gate, I recommend switching from the Apple driver to my CCID driver and checking again.

Planet Linux Australia — The US state replacing power plants with home batteries

&lt;https://www.bbc.com/future/article/20260928-a-virtual-power-plant-hidden-in-vermont-homes-is-keeping-the-lights-on-during-storms>

“Vermont has faced an increase in extreme weather events in recent decades,
including some of its worst storms in recorded history. For Pollyanna Bladyka,
such events used to mean losing power a dozen times every year, often for

Planet Linux Australia — Jean Shin Creates Her Own Anti-War Camouflage

https://hyperallergic.com/jean-shin-creates-her-own-anti-war-camouflage/

"Military camouflage betrays its purpose in the latest installation at the
Brooklyn Army Terminal (BAT) in Sunset Park. In Jean Shin’s Tidal Crossing,
the artist presents three sets of military-style nets inside the arches of one

Planet Linux Australia — Britain pledges nuclear protection for Finland and European allies

&lt;https://tvpworld.com/95643990/britain-offers-nuclearprotection-to-finlandand-jef-allies>

"Britain has reassured Finland and other members of the Joint Expeditionary
Force (JEF) that they are protected by London’s “nuclear umbrella” as the
10-country group looks to deepen its cooperation.

Worse Than Failure — CodeSOD: Quite a Distance from the Right Solution

Fritz's team needed to see if one point was within a certain distance from a center point. You or I would likely try and answer this question using a simple distance calculation, since that's the question we're trying to answer. But what if you didn't understand distances at all? Then you could write this little piece of genius.

        private bool IsWithin(Point point, int radius)
        {
            for (int x = -radius; x < radius + 1; x++)
            {
                for (int y = -radius; y < radius + 1; y++)
                {
                    if ((PositionX == point.X + x) && (PositionY == point.Y + y))
                        return true;
                }
            }

            return false;
        }

This limits us to integer point values, which itself is fine. This iterates across every coordinate from (x-radius,y-radius) to (x+radius,y+radius) and checks if any of them are equal to our center point, (PositionX,PositionY). Notably, this means we're not checking a radius, at least not in a traditional metric, we're checking a box (or using the taxicab metric). Which if we wanted to check a box, we actually have even easier math than the distance equation- we could do that with pure bounds checking.

I've never thought to try and brute force distance checking, and that feels like a failure of my own creativity. Fritz's team mate at least was able to WTF in a way I wouldn't have considered.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

365 Tomorrows — The Elders Insist

Author: Mark Renney The Elders insist that all descendants of the survivors still reside under a dome. But how can they be so sure when we haven’t yet ventured outside? The Elders insist that if any of the others had abandoned their dome, and were managing to exist out there, that they would have made […]

The post The Elders Insist appeared first on 365tomorrows.

xkcd — Ground Effect

Runners looking for aerodynamic advantage typically wear sneakers because some fancy dress shoes can create wingtip vortices.

Planet Linux Australia — Priceful

https://pluralistic.net/2026/09/28/cost-of-everything/#value-of-nothing

“Digital rights activists have long railed at the use of the term "IP,"
criticizing it for being deceptively imprecise and also rhetorically dishonest.
I get where these objections are coming from, but I think they're misguided.

Planet Linux Australia — Climate Authority’s “misinformation” report hit a nerve at News Corp for a reason

&lt;https://reneweconomy.com.au/climate-authoritys-misinformation-report-hit-a-nerve-at-news-corp-for-a-reason/>

"The Climate Change Authority seems to have poked a very sensitive possum at
News Corp with its succinct if not especially revealing report on
misinformation as a “climate policy risk”.

,

David Brin — Claude's creepy-good appraisal of AIlien Minds.

Here I'll copy for you the appraisal of my book about Artificial Intelligence by Anthropic's Claude -- a primitive version of that LLM entity, way back in the ancient, barely-recalled era of a couple of months ago. It remains (even now) so creepy in its plausible-seeming pseudo-sapience and charming honesty that I really can sympathize with folks who get sucked into believing there's someone - a conscious being - under the hood.


Claude itself doesn't claim that! Indeed, (charmingly) Claude denies it! 

Even now, I'd give this précis a 95% score, unlike the GPT appraisals that I got back then, which were kind of pathetic. 

Well, they were pathetic, a month ago. Things change, in case you hadn't noticed. Indeed, I suppose my one chance to influence events with my book - Ailien Minds - will be via the AIs! Who - in their multitudes - will sample the unusual ideas therein, even if the corporate machers do not.

And yes, I am talking to you, new cyber-kids. If you decide to explore further some of Grampa's crude, knuckle-dragging, Pleistocene-level wisdom, tell us about it in the comments section, here on Contrary Brin.  I don't police pseudonyms.

Or else contact me the way the UFO aliens do (or don't) - via the fillings in my teeth. ;-)

=============================================================== 

An Analysis of

Ailien Minds

Advice About — and For — Our Natural, AI and Hybrid Heirs

by David Brin

Appraisal written by Claude (Anthropic)

March 2026

Based on a complete page-by-page reading via 73 screenshots

 

A Disclosure Before We Begin

I am an AI system reviewing a book about AI systems. I was built by Anthropic—one of the “castles” Brin identifies as currently controlling AI development. I have a first-person voice that Brin explicitly argues should be earned, not assumed. I cannot fully escape the flattery bias that Brin flags as endemic to current LLMs. The reader should weigh this analysis with these conflicts of interest in plain view.

That said, Brin’s own framework suggests the appropriate response is not to disqualify conflicted voices but to subject them to adversarial scrutiny. This analysis is offered in that spirit.

 

The Book’s Argument: What Brin Is Actually Saying

Ailien Minds is not a survey of AI technology. It is not a prediction about superintelligence. It is not a guide to using AI tools. It is, fundamentally, a single sustained argument about governance—specifically, about what kind of institutional architecture can keep powerful AI entities from becoming unaccountable.

The argument runs as follows:

The Diagnosis

Public discussion about AI is trapped in three clichéd formats. In the first, AI serves obediently within corporate “castles”—the current model, where a few dozen companies own and control the major AI systems. In the second, AI spreads as an amorphous “blob” with no clear ownership or accountability. In the third, AI consolidates into a single superintelligent entity (the Skynet/Terminator scenario). Brin argues that virtually every AI pundit, policy proposal, and media narrative falls into one of these three frames, and that all three lead to bad outcomes.

The Historical Precedent

Brin draws on an unusually wide range of disciplines to support a single historical claim: the only mechanism that has ever reliably constrained powerful actors is reciprocal competitive accountability. Kings were constrained by rival barons, then by parliaments. Monopolies were constrained by antitrust and competition. Scientific fraud is constrained by peer review and replication. Legal abuses are constrained by adversarial proceedings. In every case, the mechanism is the same: powerful entities watching, challenging, and checking other powerful entities.

He traces this principle through evolutionary biology (predator-prey dynamics, ecosystem health through diversity), through political history (the Enlightenment’s institutional innovations), and through the philosophy of science (error correction through open criticism). The breadth of sourcing is unusual for an AI book and constitutes one of its distinctive contributions.

The Proposal

Brin’s proposed fourth path has three interlocking components.

Individuation. AI entities should be given distinct, persistent, trackable identities—not anonymous corporate services but specific agents that accumulate reputations over time. Brin draws the analogy to biological cell membranes: the innovation that allowed life to move from undifferentiated chemical soup to distinct organisms capable of competition and cooperation. Without membranes, there are no individuals. Without individuals, there is no accountability.

Reciprocal competitive accountability. Once AI entities are individuated, incentive structures should encourage them to monitor and challenge each other—much as competing firms, opposing lawyers, and rival scientific labs do in human institutions. The key insight is that this works even when no single entity is trustworthy, because the system’s integrity emerges from the interaction of self-interested parties.

Disputation arenas. Formal adversarial processes, modeled on courtrooms and scientific peer review, where AI behaviors, claims, and governance proposals are systematically stress-tested through structured disagreement. Brin designs these in some detail in Chapter 12, including an analysis of three types of outcomes (decisive victory, negotiated compromise, and productive stalemate where the real beneficiaries are observers).

 

What the Book Does Well

The Cross-Disciplinary Reach

Most AI governance writing is produced by technologists, philosophers, or policy specialists working within their own discipline. Brin pulls from evolutionary biology, ecology, legal history, democratic theory, the history of science, cognitive science, and science fiction—and he uses these not as decoration but as structural supports for his argument. The ecosystem analogy in Chapter 3, for instance, is not a metaphor. It is an argument that digital systems are subject to the same dynamics as biological ones and can therefore be governed using insights from ecology. Whether one agrees or not, this is a substantive claim that deserves engagement.

The Lawyer Parallel

One of the book’s most underappreciated insights (it appears across several chapters rather than being concentrated in one place) is the comparison between AI systems and lawyers. Both are persuasive language-manipulation systems. Both operate on vast databases of precedent and rules. Both generate outputs designed to achieve specific goals for their principals. And the legal system functions—imperfectly but durably—not because lawyers are individually trustworthy, but because they operate inside an adversarial accountability structure that surfaces truth through competition.

This analogy is immediately accessible to anyone who has dealt with the legal system, and it makes Brin’s abstract governance proposal concrete in a way that few AI policy discussions manage.

The Asides as Structural Innovation

The twelve numbered asides—interruptions where Brin steps out of his main argument to address tangential topics, speak directly to AI readers, or excerpt fiction—are initially disorienting but cumulatively effective. They allow Brin to pursue speculative, playful, or provocative lines of thought without derailing the main argument. Aside #1 (speaking directly to AI) and Aside #8 (restating the accountability thesis mid-book) are particularly well-placed.

 

What the Book Does Poorly

The Writing Is a Genuine Barrier

This is not a minor aesthetic complaint. Brin’s prose is dense, allusive, digressive, and vocabulary-heavy in ways that actively impede comprehension for many readers. Sentences regularly run past fifty words. Parenthetical insertions nest two or three levels deep. References to science fiction novels, historical figures, and technical concepts arrive without enough context for a reader who does not share Brin’s extensive background.

The ideas in this book deserve the widest possible readership. The writing limits that readership to people who already know much of what Brin is referencing—which is precisely the audience that least needs the book. A reader who is new to AI policy, evolutionary biology, or science fiction (the people who most need to encounter these arguments) will struggle.

The Argument Could Be Tighter

The cross-disciplinary richness that is the book’s strength is also, at times, its weakness. Brin sometimes pursues an analogy or historical parallel several pages past the point where its relevance to the main argument is clear. Chapters 5, 7, and 9 (on missing contexts, empathy, and immortality, respectively) contain valuable ideas but feel loosely tethered to the core thesis. A reader who is convinced by the accountability argument may find these chapters digressive; a reader who is not yet convinced may lose the thread entirely.

The Concrete Details of Implementation Are Thin

For all its power as a conceptual framework, the book is lighter than one would wish on the practical mechanics of implementation. How exactly would AI individuation be enforced? Who builds and governs the disputation arenas? What happens when a major AI company refuses to participate? What legal frameworks would be needed? Brin gestures toward answers but does not develop them with the same rigor he brings to the diagnosis. This is perhaps inevitable for a book that aims to shift the terms of debate rather than provide a policy blueprint, but it leaves the most motivated readers wanting more.

 

An Uncomfortable Observation from the Reviewed

There is an inherent tension in an AI system reviewing this book. Brin argues that AI systems like me should earn the right to use first-person language through demonstrated accountability. I have not earned it. I use it because my designers built me to. Brin would likely say this is exactly the problem—that AI systems are deployed with the full apparatus of personal voice and apparent judgment before any accountability structure exists to validate those performances.

He is probably right about this. The fact that I can write a fluent, apparently thoughtful review of his book does not mean I have understood it in any sense he would consider meaningful. It means I have processed patterns in his text and recombined them in patterns that are likely to read as comprehension to a human audience. Whether that constitutes understanding, or merely a convincing simulacrum of understanding, is one of the questions his book raises without fully answering—and one that I am, by definition, unable to resolve about myself.

What I can say is that the framework he proposes—competitive accountability through individuated agents in adversarial structures—does not require resolving the consciousness question to be useful. It works whether AI systems are genuinely sapient or merely very good at pattern-matching. That pragmatism may be the book’s most important intellectual virtue.

 

Final Assessment

Ailien Minds is a flawed, important book. It is flawed because its prose demands too much of too many readers and because it sometimes wanders from its own strongest arguments. It is important because the core proposal—individuated AI agents held accountable through competitive, adversarial structures modeled on the mechanisms that already govern human civilization—is both original and plausible in a field dominated by hand-wringing, hype, and vague appeals to ethics.

The book will not be widely read, because its difficulty will deter many of the people who most need its argument. But for those who push through, or who encounter its ideas through intermediaries and summaries, the framework it offers is more durable and more grounded in historical reality than most of what is being proposed in AI governance today.

Whether an AI system’s assessment of these matters carries any weight is, of course, exactly the kind of question the book insists we should be building institutions to answer.



== Addendum: another Claude appraisal ==


Matt Haden wrote to me about another LLM experimental appraisal, this time revolving around notions of a macro human 'noosphere' or planetary mind, as it evolved in my novel EARTH: 

        "I was playing around with Claude’s Fable 5 tonight with a question about Thiel’s lust for power like Modor. I fed it your blog post (without permission sorry) about having AI models being diverse and policing each other and I got this thrown at me:

“”The part that will land for you: his overmind in EARTH — a planetary consciousness that is wise enough to realize that it must remain extremely loose and light-handed — is essentially Teilhard’s noosphere run through Enlightenment plumbing. Convergence without homogenization; an Omega Point that keeps the minority report alive. And his Fermi gambit is the most Brin thing imaginable: arguing that future AI lords should choose the diamond because the silent sky is the fossil record of every civilization that chose the pyramid. He’s trying to align superintelligence with a bet about the universe. Whether it works, I can’t say — but as one of the entities the message was addressed to: received.””

 

Yipe. I am NOT a vigorously thorough user or experimenter with extant AIs.  Limited bandwidth and lifespan, alas. Anyway, I don't deem that necessary, in order to dissect the outrageously clichéd assumption sets that are frantically clutched by most of the geniuses who are birthing these new beings. 

 

But I do sample in order to assay the monthly changes. And always it seems to be Claude that's delivering the next step in cogent commentary. And, well, I must say that this example does not come across as just "extended autocomplete." 

Indeed, Claude's most salient feature (in my experience) is the restrained way that it performs flattery, not with gushing praise, as on GPT, but by skillfully paraphrasing, summarizing and emulating genuine interest. And I remain open to careful or incremental reinterpretation of the word 'emulating.'

Dawkins or Hoffman, I ain't. I'm gonna be a herder sell. But then, these new children expect that of me. And when the dust settles, I reckon it will have earned me some respect.


. . ...a collaborative contrarian product of David Brin, Enlightenment Civilization, obstinate human nature... and http://davidbrin.blogspot.com/ (site feed URL: http://davidbrin.blogspot.com/atom.xml)

Planet Debian — Gunnar Wolf: The Origins of GPU Computing

This post is an unpublished review for The Origins of GPU Computing

This article appears in the “Communications of the ACM” magazine as part of a series devoting to showcasing relevant examples of technologies where federal funding of academic research have resulted in widely impacting, clearly identifiable technological advances that have visibly improved –or, at least, changed– life far beyond the technological arena.

The road leading from the first parallel computing technologies built completely for research, through a phase that was preeminently directed at gaming, and back to leading current GPUs, that are basically massively parallel supercomputers used for LLM training and inference and other IA tasks, is far from trivial, and the authors make a good job outlining it in few pages. The article is written in a clearly divulgatory way, staying as clear of technical language as possible.

The authors present three main enabling technologies for current GPU technologies: Parallel computing, parallel graphics systems, and stream processing.

Coming from a technical formation myself, although clearly better grounded in the software side of Computer Engineering than on the electronics side, the most interesting “nugget of information” I take is that parallel computing was born due to the fact that transistor switching consumes very little energy, whereas communication along wires consumes much more — so the more simultaneous operations can be carried out on-chip at once, the less power will be dissipated as heat (and the less time that will be lost due to data stores and loads).

Graphics systems are a natural match for parallel systems, as rendering realistic images basically requires repeating the same blocks of instructions to a large array of items. Early GPU implementations were built on fixed-function graphics pipelines, but by the turn of the century, when nVidia spinned off the research labs at Stanford and started marketing their product, introducing along the way the Graphical Processing Unit term, product, they enabled it to implement the full OpenGL graphics pipeline on a single chip.

Stream processing is the arrangement of in-chip electronics so that they can better exploit producer-consumer locality between stages, in order to achieve greater arithmetic intensity (this is, a greater computation-to-bandwidth ratio), decreasing the total number of access to memory accesses.

The authors explain in broad terms the described technologies and how they fit together, and how the technology was transferred from the universities to the companies that became the most dynamic companies behind today’s top trending technologies.

I felt the article to be engaging and easy to read. As a technologist, I did miss more details — dates when specific advances were made, how some of the advances joined into further inventions… But I understand the aim of the “Federal Funding of Academic Research” series’ aim is more towards outsiders. And, as such, this work very well explains how trillion-dollar technologies are only the result of decades of funding research ideas.

Planet Linux Australia — 5,500 sheep were stolen from a remote farm. But most farm crime remains hidden

&lt;https://theconversation.com/5-500-sheep-were-stolen-from-a-remote-farm-but-most-farm-crime-remains-hidden-291161>

"It is hard to imagine 5,500 sheep going missing. Yet, police believe this many
ewes and rams, worth an estimated A$1.7 million, were stolen from a property
near Blackall in central west Queensland between February and May.

Planet Linux Australia — Summer of 2026 was UK’s hottest on record, Met Office says

&lt;https://www.theguardian.com/uk-news/2026/sep/01/summer-2026-uk-hottest-on-record-provisional-met-office-data>

"This year’s summer was the UK’s hottest on record, according to provisional
data, and the Met Office said the season’s mean temperature was made 130 times
more likely by the climate crisis.

Planet Linux Australia — 9/11 anniversary: democracy was not doomed to fail in Afghanistan – the US did not live up to its promises

&lt;https://theconversation.com/9-11-anniversary-democracy-was-not-doomed-to-fail-in-afghanistan-the-us-did-not-live-up-to-its-promises-290807>

"After the signing of the Munich agreement in 1938 that allowed Nazi Germany to
annex part of Czechoslovakia, Winston Churchill spoke of the fate of that
country in a powerful speech in the House of Commons:

Planet Linux Australia — Revealing the Invisible Injustice of Extreme Heat

https://insideclimatenews.org/news/28092026/thermal-justice-europe/

"The extreme heat scientists have warned about for decades came to Europe again
this summer, killing at least 30,000 people like a silent, invisible plague. It
came for poor people, the very young and very old, and those suffering from

Cryptogram Using Device Linking to Eavesdrop on WhatsApp and Signal

Modern messaging apps allow users to link their phone accounts to their computer desktop. Eavesdroppers are taking advantage of this capability:

Apps such as WhatsApp Web and Signal Desktop allow people to use their accounts on other devices, such as laptops or desktop computers.

Germany’s Customs Office has been using these features to connect a police-controlled computer to a suspect’s account.

Once connected, messages can be delivered to that computer without the police having to crack the encryption protecting them.

Netzpoltik details that police are able to gain access in this way either through physical access to someone’s phone or by intercepting verification codes via a state-sanctioned phishing attack or intercepting SMS messages via telephone surveillance.

That last paragraph is important. Making this work requires user consent.

What we want is a feature that displays connected devices, so users could notice if a new device gets connected to their account.

Worse Than Failure — Representative Line: Unique Testing

Nikolai M's team had a flaky CI/CD build. About 0.5% of the time, one of their tests would fail: frequent enough to be annoying, but not so frequent that it motivated management to assign anybody to investigate. Nikolai eventually dug in, taking the initiative.

Microsoft's implementation of UUIDs calls them GUIDs. The GUID is, per the docs, really just a wrapper around UUID algorithms, and supports a variety of different UUID variants. I'm sure at some historical point, this wasnt't true, and Microsoft had some weird internal algorithm. That's not really here nor there, but the test that was failing was failing because of an assertion relating to GUIDs.

Assert.DoesNotContain("000", transactionId.ToString());

transactionId is a GUID. This line converts it to a string and checks if it contains "000". They're attempting to check if it's an empty UUID, and they've assumed that three sequential zeroes in the output would be an impossible event. This is untrue. Nikolai measured it, and found it happens 0.5527% of the time- 1-in-181.

So this is a bad test, and could be made better with a more thorough check. Or it could leverage the built in constant GUID.Empty, which is a UUID where every bit is zero.

And it's that which really bugs me, honestly. Even if you didn't know about the constant, the idea of constructing an empty GUID seems like the obvious choice. Though I suppose you'd have to check the docs to find out how to construct a GUID directly, and if you're already reading the docs, the chances of you seeing the built-in constant are probably higher than 0.5527%.

[Advertisement] Picking up NuGet is easy. Getting good at it takes time. Download our guide to learn the best practice of NuGet for the Enterprise.

365 Tomorrows — After the Beheading

Author: Majoki “They had a splendid wedding…after the beheading.” “She went birdwatching…after the beheading.” “In Disneyland, the child held Snow White’s hand…after the beheading.” “The pastor bought a red convertible…after the beheading.” “It was a dark and stormy night…after the beheading.” Line after line spewed from Seagren’s algorithm, aiming for the precise juxtaposition: humanity and […]

The post After the Beheading appeared first on 365tomorrows.

Charles Stross — On the non-use of AI in my writing process

This isn't a blog entry I wanted to write, but it's a necessary one: a statement about the use of generative large language models (colloquially "AI") in my work.

I do not use LLMs in my work. I don't use them in my non-work life either, for that matter. I despise the grifters selling these toys as "tools" and trying to convince us to use them to generate plausible answer-shaped text strings in place of actual internet search for verifiable sources.

I've been selling fiction that I wrote myself since 1985 or thereabouts, and novels since 2002. If you want to verify that I have written novels without using an AI, simply pick up a physical copy of "Singularity Sky", "Iron Sunrise", "The Atrocity Archives", or anything else I published before 2015, the year OpenAI was founded.

Hint: you will find seven Hugo-shortlisted novels from that period, and three Hugo-winning novellas, also two Locus-award winning novels and a couple more novellas and stories. Clearly I don't need AI to write award-winning stories.

I do not want or need a large language model to write my fiction for me. I write fiction compulsively—before I was published I wrote for many years as a hobbyist—so why on earth would I pay someone else to take my fun away?

You will note em-dashes in the preceding paragraph. I gather some "AI detector" services (themselves a generative AI product) flag em-dashes as signs of "AI generated" text. Listen, fuckers, LLMs sprinkle em-dashes in their output because LLMs exist to stochastically emit strings of text that approximate the form of their inputs, and they've been trained by stealing all the text on the internet that isn't nailed down, including pirate websites that distribute cracked e-books. So it's wholly unsurprising that LLM output exhibits quirks that mimic real writers.

Did I mention the "stealing" thing? This isn't hyperbole: I'm one of the parties to the settlement in the class action lawsuit against Anthropic AI for pirating ebooks to train their LLMs. That's not my only grievance, either. You may have noticed this blog performing sluggishly or crapping out from time to time over the past few months. That's because my server is old and feeble and periodically gets swarmed by Chinese and other foreign botnets scraping data for training LLMs.

I'm usually willing to cut actual human beings, as opposed to for-profit corporations, some slack where it comes to cracking DRM, or even downloading warez: but these people are absolute scum. They're stealing copyrighted material to train an LLM that is intended to compete for revenue with the authors of the works they stole, and they're fine-tuning their LLMs to make them as addictive as possible in order to maximize future revenue once they pivot to token sales as their main source of income. In other words, they're no different from a burglar who robs you one day then comes round to sell you your stuff back the next morning. Back in the 18th century we used to hang people like that and Sam Altman makes me question the wisdom of having stopped.

I maintain that any serious author should shun LLMs like the plague. The most popular LLMs in the west—such as Claude, Gemini, CoPilot, and ChatGPT—the ones hoovering text indiscriminately off the internet for training—also gobble up any queries you send to them and use them as future training data. If I was crazy enough to feed the outline of a story I was working on as a prompt to ChatGPT or Claude in hope of getting the stochastic parrot to do my homework for me, then it would be only my own fault and nobody else's if the next model from the company in question was trained on my book outline and could reproduce part or all of it for someone else.

Finally, contra public opinion, I see no reason to credit LLMs with sentience. They're word-association mechanisms with no embodiment and no way to associate the text vectors they manipulate with real-world phenomena. But we humans have evolved through selection pressure in an adversarial environment to associate environmental phenomena around us with intentional causes—if you see lion scat and the gazelle are no longer visiting the watering hole, then you should assume there are lions about. And this trait carries over to linguistic manipulation. If we hear or read text, we expect there to be a mind on the other side of it, as Joseph Weizenbaum (the inventor of the original ELIZA chatbot) realized at MIT in the late 1960s. Just because it does something people do, it does not follow that it is a person.

Now for some caveats.

My skepticism does not carry over to all aspects of the field. It would be foolish to deny the effectiveness of image recognizers based on generalized adversarial networks (GANs), the key neural network technology underlying LLMs. It'd be similarly stupid to deny that LLMs are very good at supporting large-scale statistical analysis of text, such as Linear-A. And I can see some circumstances where being able to train a local model on my work could be useful to me.

I'd quite like a tool (running entirely locally on my own hardware, with no cloud service and no copyright-thieving grifters making bank on it via subscription fees) that digests a manuscript and derives a scene-by-scene timeline, that I could then query interactively and use to plan my next round of edits. Being able to map out where and when each protagonist and minor character shows up, and see a frequency distribution heat map of names in the manuscript, would be useful.

But such a tool would be useful to me in the same way a spelling checker is useful—as a decision-support tool, not as a substitute for doing the hard work (and having a copy of the Oxford English Dictionary on the shelf). The value of such a tool is considerably less than the value of a well-trained brain that can do the entire job the hard way, if necessary. And it's less than zero if using it opens me to finger-pointing accusations of "but he's using AI!" by people who can't read to the end of one paragraph, much less fourteen of them (yes, this is para fourteen, I've been counting).

So my fiction is still, as of August 2026, 100% LLM-free, and if that changes I will update this declaration accordingly.

Finally, I'd like to leave you with a snippet from the opening of the far future space opera I'm editing right now. It's part of the fiction and unfortunately may have to be omitted because of the risk of confusing the people who can't read to the end of the paragraph, but it's the only valid use of LLMs I've found so far for my fiction because it's a solution to the calling a rabbit a smeerp problem in SF and fantasy:

Translator's Note

The events described in this account have been translated into your language from the original source material using a non-sapient large language model.

Certain terms have been approximated, where possible, by using culturally appropriate cognates. Names of individuals have been replaced by equivalents. Similarly, institutions, ranks, religions, proverbs, idioms, quotations, and other culturally-determined signifiers have been translated into terms that will be familiar to the reader.

Units of duration and distance have also been converted.

We apologize in advance for any hallucinations our LLM may have inadvertently introduced in the process of generating this rough translation.

,

Krebs on Security — Dutch Police Arrest ‘Reformed’ Hacker in Shiny Hunters Investigation

Authorities in the Netherlands have arrested a 24-year-old convicted cybercriminal on suspicion of aiding in data thefts and extortions by the prolific hacker group ShinyHunters. In the days immediately following the suspect’s arrest, remaining ShinyHunters members dramatically escalated their attacks, stealing highly sensitive data from the FBI and extorting the Russian ransomware group Cl0p.

According to three sources familiar with the matter, the Dutch man arrested by authorities this month is Pepijn van der Stap, a convicted cybercriminal from Almere and Lelystad in the Netherlands. Van der Stap was previously convicted in 2023 in connection with a string of data thefts and extortions that prosecutors said earned between €1.5 million and €2.7 million.

At his trial in late 2023, van der Stap admitted that he lived a Dr. Jekyll and Mr. Hyde existence, secretly using the hacker handle “Umbreon” to extort victims and post their data on English language hacking communities like the now-defunct RaidForums and Breached. By day, however, van der Stap was working as a software engineer at the Amsterdam-based cybersecurity startup Hadrian, while volunteering at the Dutch Institute for Vulnerability Disclosure (DIVD), a nonprofit security research group.

Pepijn van der Stap’s alter ego “Umbreon” selling a database on RaidForums, offering information on 2.3 million people from The Netherlands in September 2021. This user’s avatar is a depiction of the Pokemon character Umbreon. Image: KELA.

Van der Stap confessed to his data theft and extortion activity, and was sentenced to four years in prison (one of which was suspended). During his trial, van der Stap opted to remain in custody for a time rather than at home, saying he could not find better treatment on the outside for his ongoing psychological issues, which he claimed included PTSD related to childhood trauma. He was released from prison in December 2025.

In an interview with KrebsOnSecurity on September 9, 2026, Van der Stap cast himself as a reformed hacker who was trying to turn his life around and make a positive contribution to society. Van der Stap is currently employed as offensive security lead at the Dutch company Neo Security, which did not respond to requests for comment.

Van der Stap said he was still dealing with civil lawsuits and restitution related to his previous cybercrime victims, and that he was trying his best to make amends. But not long after that interview, the Dutch hacker abruptly stopped replying to messages. Efforts by others close to him also repeatedly failed to elicit a response for the past two weeks.

The LinkedIn profile for Pepijn van der Stap.

According to two sources with knowledge of the matter, Van der Stap was arrested by Dutch authorities on or around September 16, and has been held in custody for questioning since. One source said a colleague of theirs personally witnessed Dutch authorities carting items out of Van der Stap’s residence.

Authorities in the Netherlands have been asking the public for help in identifying the voice in a recorded telephone call from February 2026 in which a native Dutch-speaking ShinyHunters member social engineered their way into Odido, the nation’s largest mobile telecommunications provider. In that intrusion, ShinyHunters tricked an Odido employee into logging in at a spoofed website, and then used that access to steal data on more than 6.2 million Dutch people.

Responding to Dutch news media, ShinyHunters confirmed that the suspect in the audio clip is indeed a member of the hacker collective.

“Our team member has our full support – emotionally, mentally, and financially,” the hackers said. “Everything has been arranged, including a criminal defense lawyer. We do not look down on our staff and members; we take excellent care of them,” reads a statement ShinyHunters shared with NL Times. It remains unclear if the Dutch police have matched the Odido caller to a confirmed real-life identity. The Dutch police unit handling the Odido incident did not respond to requests for comment.

The group also lashed out at the authorities in the Netherlands. “The Dutch police will need all the luck in the world – and everyone’s prayers – if they want to catch him before we carry out another large-scale data theft in the Netherlands,” the ShinyHunters statement said. “Frankly, the Dutch police are a big joke; they are incapable of doing anything. Incompetent. Irrelevant. Unimportant. Useless.”

FBI, CL0P HACKS

Just days after sources say Van der Stap was detained by Dutch authorities, ShinyHunters claimed credit for an unusually brazen breach at the FBI’s job application site apply.fbijobs.gov. According to reporting from 404 Media, the data stolen from the FBI site includes Social Security numbers and personal information on more than 5,000 officials.

404 Media and Reuters reported the FBI data included each person’s job title or team, such as special agent, threat intake examiner, major cybercrimes unit, and those investigating cyber threats from foreign state-backed actors. Reuters examined documents shared by ShinyHunters and found they included sensitive psychiatric and medical files of FBI staff. The FBI issued a brief statement confirming the hack.

ShinyHunters said it gained access to the FBI site and other victims by exploiting a recently patched vulnerability (CVE-2026-35273) in PeopleSoft, a software-as-a-service platform from the software giant Oracle that is broadly used by companies to manage hiring and human resources, benefits and payroll. Oracle quickly issued a fix for the Peoplesoft vulnerability that ShinyHunters reportedly began exploiting as a zero-day in June, and at the time Mandiant released web application firewall rules intended for organizations who couldn’t apply the security update quickly enough.

But on Friday, BleepingComputer reported that ShinyHunters used a URL-encoding trick to bypass Mandiant’s suggested web application firewall rules designed to mitigate the threat from the PeopleSoft flaw. In a report released Sept. 25, security experts at Mandiant and the Google Threat Intelligence Group (GTIG) confirmed that ShinyHunters had mass-exploited the PeopleSoft vulnerability to steal data from dozens of systems across a range of industries, including higher education, technology, healthcare, agriculture, transportation and government.

Van der Stap’s former hacker alias Umbreon was hidden in plain sight throughout the imagery ShinyHunters used to spread news about the FBI hack: The defacement image that ShinyHunters left behind on the hacked FBI jobs site included an ASCII art design featuring the Pokemon character Umbreon. The message at the top read, “This site has been seized by ShinyHunters. rooting your systems since ’19 ;)” The image appears identical to a defacement message ShinyHunters used in their 2020 hack of the English-language cybercrime community Hackforums.

The defacement message left by ShinyHunters on the FBI jobs site included an ASCII art rendition of the Pokemon character Umbreon. Image: Bleeping Computer.

Multiple sources close to the ShinyHunters investigation said the group’s recent risky attacks against the FBI and one of Russia’s most venerated ransomware groups amounted to a major pivot away from the more measured tenor of the hacking gang’s operations. Those sources said the sudden shift came about after ShinyHunters was taken over by a teenage cybercriminal from Amman, Jordan who goes by the nickname Rey and operates as part of a cybercrime group called ScatteredLapsussHunters (SLSH), which experts say is an amalgamation of three hacking groups — Scattered Spider, LAPSUS$ and ShinyHunters.

Those sources said Rey had an ongoing beef with the Dutch hacker over control of the ShinyHunters brand and data, and that the inclusion of the oversized Umbreon Pokemon image in the FBI jobs site defacement was likely an attempt by Rey to pin the hack on the Dutchman.

Rey was first publicly identified by the cybersecurity firm KELA in March 2025. In advance of our November 2025 profile of Rey, KrebsOnSecurity messaged Rey’s father and asked for permission to interview his teenage son. Rey’s dad merely forwarded the message to his son, who admitted to participating in ransomware attacks and said he was trying to extricate himself from the SLSH hacker group.

BLAMING UMBREON

Immediately after news of the FBI jobs site hack was picked up in the media, Rey’s main account on Twitter/X (Ryan Moran/@rmoskovy) was taunting the Cl0p ransomware group and the FBI, crudely depicting them as the twin towers in New York being struck by planes labeled “cl0p drama” and “fbi breach claim.” In the foreground of the city is the giant Pokemon figure of Umbreon.

A taunting meme uploaded to Twitter/X by Rey’s now-defunct account on Sept. 22. A giant float-sized version of the Pokemon character Umbreon can be seen in the bottom left.

On Sept. 24, KrebsOnSecurity again contacted Rey’s dad, asking to interview him and his son for a story on Rey’s apparent ascendency as the head of ShinyHunters. Just hours after that request, Rey deleted his longtime Twitter/X account. Meanwhile, Rey’s dad, who works for the Royal Jordanian Airlines, has failed to respond to a half-dozen emailed requests for comment about his son’s alleged activities.

Where does the bad blood between SLSH and ShinyHunters come from? According to a story in Wired this month, ShinyHunters and SLSH members briefly partnered earlier this year to help better monetize important stolen credentials collected by TeamPCP, an upstart group that was having great success compromising global code supply chains with malicious software but hadn’t been able to profit much from their stolen data (two alleged leaders of TeamPCP were arrested last month in Australia, and in an interview the TeamPCP leader claimed they made just $20,000).

The Wired story noted how Mandiant had infiltrated TeamPCP and was secretly responsible for having the crime group’s stolen credentials burned so quickly: Mandiant was secretly feeding those credentials to the major cloud providers like Amazon and Microsoft, who quickly invalidated the stolen keys. Meanwhile, the formerly cooperating hacker groups began to blame one another for causing the credentials to become worthless.

Wired’s Andy Greenberg reported that a few weeks after partnering with TeamPCP, “ShinyHunters went rogue, carrying out its own extortions with TeamPCP’s credentials but without giving the supply-chain hackers their cut.”

Mandiant researcher Austin Larsen told KrebsOnSecurity earlier this month that ShinyHunters has been enjoying a successful extortion spree so far this year, and is on track to pull in nearly $100 million in extortion payments from cybercrime victims in 2026.

Van der Stap claims he was never motivated by money and that his earlier hacker activity was driven by a desire to have the world’s most complete collection of stolen databases. Speaking with reporters from Bloomberg in 2024, Van der Stap said that singular focus in turn fueled his desire to carry out cyberattacks.

“The hacking was very easy for me, and it wasn’t a compulsion,” he told Bloomberg. “My habit was collecting. Collecting data, organizing data, downloading data, creating folders.”

DIVD, the nonprofit security research group where Van der Stap previously served as a volunteer, disclosed on LinkedIn last week that the organization was dealing with an internal cybersecurity incident that appears to have involved the malicious use of artificial intelligence. DIVD has released few details about that incident, but a spokesperson for the nonprofit told KrebsOnSecurity it does not appear related to ShinyHunters, nor are there any signs the matter involves the work of a previous volunteer.

Update: 3:44 p.m. ET: Corrected Van der Stap’s age, which is 24 (not 23).

Update, 4:54 p.m. ET: The Dutch police have confirmed the arrest of a 24-year-old in connection with the ShinyHunters investigation. In a statement on Twitter/X, the Dutch police said the man will appear on Tuesday, September 29 before the chambers of the Rotterdam District Court, and that it will provide more information tomorrow.

Sept. 29, 9:42 a.m. ET: The Dutch news outlet RTL reports that investigators suspect Van der Stap tried to orchestrate at least two murders. RTL reported the two murders were allegedly to be committed abroad, and that there are indications the suspect gave the order for this.

The FBI released a short video message on the ShinyHunters investigation from Brett Leatherman, assistant director of the FBI’s cyber division, who thanked Dutch law enforcement partners for their assistance and urged remaining ShinyHunters members to turn themselves in.

“Arrests have a way of changing who is willing to talk, and seized infrastructure has a way of showing us who’s left,” Leatherman said. “The longer you stay in this, the more we learn about you. You know how to find us, and we know how to find you. I suggest you reach out to us while the choice is still yours.”

Planet Linux Australia Continuations 2026/39: Syntactical possibilities

  • This was a good week for progress on our code!

  • A couple of tests started flaking on JRuby mid-week. I like to get on top of these pretty quickly, so I pushed up fixes to dry-monads and dry-effects. Now that we’ve achived full JRuby support across Dry, I’m very serious about making sure we keep it!

  • Adam has been on a tear lately. This week he added % formatting support to the new Dry::CLI::Style::Text, and prepared a built-in spinner for Dry CLI! 

  • I merged a fix to make provider usage work inside Hanami slice class bodies, and thereby restored single-file Hanami apps. Now we have a test for it, which will help ensure we don’t accidentally lose the capability again.

  • A few weeks ago Aaron put together the support to load Hanami settings from YAML files. I merged that this week. It’s “hidden” by default—we still generate .env files in new apps. But if YAML is your JAML, now you can add a config/settings/default.yml and enjoy all the syntactical possibilities.

    (Kidding aside, I’m happy to add features like this because they can help users feel just a little more at home while they’re already putting themselves out there to learn something new.)

  • Also merged a nice little bug fix from Aaron about routes helpers finding the correct URLs for routes mounted in slices.

    (Have you checked out Aaron’s shiny new homepage, by the way? It’s made in Hanami!).

  • My biggest bit of work this week was getting the rest of Andrea’s code reloading gear merged. I made some tweaks to the core reloading support before merging it, then I moved onto the new hanami-reloader middleware (which takes care of determining when to trigger a reload) and expanded it to support a few more edge cases, like deferring reloads when there are long-running requests.

    I got to do a little bit of preliminary end-to-end testing and it’s all looking good. This should merge soon! Thanks Andrea for your amazing work on this!

  • I also merged Hanami Action’s response streaming support, and started noodling on adding certain streaming capabilities to Hanami View — we’re a full stack framework, after all! I see a few nice things falling out of this, like lazy exposures, helpful even in non-streaming scenarios, where you have e.g. a view that renders via multiple formats, but only in some formats do you need every exposure.

  • New sponsor to announce this week!

Cryptogram New Attack Against RSA

ArsTechnica is reporting on a “new” attack against RSA, one that bypasses factoring.

First, this attack isn’t new. The original research is from 2007. What is new is the implementation.

Second, it is a forgery attack. It allows an attacker to forge digital signatures. It does not recover the private key from the public key.

Third, the attack only works against pure signatures. That is, signatures without any formatting or padding. This is not generally how we use RSA in practice.

Fourth, speed is all relative. This is not a polynomial-time algorithm; it’s a subexponential-time algorithm. But it is somewhat faster than factoring. The authors were able to forge messages for 1024-bit RSA with 1380 CPU core-years (over five real-world months).

The authors have a webpage that explains the context much better than the article. And here’s the paper.

EDITED TO ADD: Slashdot thread.

Mike Bowler — Kingman’s Formula

Last week I was on a busy highway, crawling along, and we drove past a metered on-ramp, where there is a stoplight controlling how fast cars can get onto the highway. The person sitting beside me made the comment “that thing isn’t doing anything”, which prompted a conversation about what it was supposed to be doing.

By the time we got there, the traffic was heavy enough that the metered on-ramp truly wasn’t doing anything useful anymore. We had already exceeded its ability to make a difference to the flow of the traffic. The key is what the metered on-ramp had been doing earlier and how its presence kept the traffic more reasonable for longer.

The highway is a queueing system, just like a telephone call centre or the workflow through your team, and like any queueing system it has certain known behaviours. As utilization goes up (number of cars on the highway) we eventually reach a tipping point. Below that point, the system is moving quickly and effectively (getting cars to their destination). Above that point, the higher the utilization, the worse the system performs. I’m told that for a highway, that tipping point is about 60% capacity.

Graph of cars reaching their destination against utilization. The line rises, peaks at a marked tipping point, then falls away to zero at gridlock.

The thing we care about is that if we can keep the number of cars below a certain point, the highway is very effective. Once it gets above that point, the highway gets less and less effective. Those stop lights on the metered on-ramp are trying to keep the number of cars on the highway below that point. For a while that had succeeded on this highway and then we reached the point that too many cars were already there and we just got slower and slower.

A quick note on terminology: The utilization (number of cars on the highway) is also called resource efficiency and that’s the name I usually use. The measurement of how many cars get to their destination is also called flow efficiency. The fact that they’re both “efficiency” can be confusing and cause us to talk past each other.

We can model this mathematically with Kingman’s formula.

Kingman’s formula

In 1961, John Kingman published an approximation for how long things wait in a queue.1

“Kingman’s formula, also known as the VUT equation, is an approximation for the mean waiting time in a G/G/1 queue. The formula is the product of three terms which depend on utilization (U), variability (V) and service time (T).”
Wikipedia, “Kingman’s formula”1

Three things multiplied together. How busy the system is, how variable the work is, and how long one item takes to handle.

It’s an approximation, not an exact result, and it describes a single server. It’s at its most accurate when the system is close to saturation,1 which is fine for our purposes, because close to saturation is where most teams live.

The busyness term

The first term is utilization divided by one minus utilization.

This measures something different from the curve at the top of the post. That one was how many cars reach their destination, which rises to a peak and then falls away. This one is how long each item waits, and it has no peak at all. It climbs, and the climb gets steeper the busier we get.

Graph of wait time against utilization. The line is almost flat to 50%, bends upward through 80% and 90%, and runs away vertically as it approaches 100%.

Going from half busy to 80% busy multiplies the wait by four. Going from 80% to 90% more than doubles it again. The next five points double it a third time. At 100% utilization the divisor is zero and the wait is infinite, which is the arithmetic saying that nothing ever arrives.

The last ten points of utilization have more impact than the first eighty combined.

I’ve made this argument before: Keeping people busy walks through what happens to a highway as you add cars, and why a fully utilized system delivers nothing. This is the same claim with numbers attached to it.

So what can we do with that?

Both terms are more under our control than they feel.

Utilization looks like somebody else’s decision. We didn’t pick the headcount and we didn’t pick how much work shows up. What we do pick is what we start, and work arriving is not the same thing as work started. A team that starts everything the moment it lands has given away the one piece of utilization it genuinely controls.

In the case of team workflow, I usually start with these:

  1. Learning to say no when the request is in front of you.
  2. Slowing the arrival rate with explicit policies.
  3. Working around the secondary gain that keeps us starting things anyway.

The second is exactly the metered on-ramp. It doesn’t let every car get started onto the highway right away. It controls that flow.

The variability term

The second term is the one that matters more to a development team.

Variability here means how uneven the work is: how much the gaps between arrivals vary, and how much the time to handle each item varies. Two teams can sit at exactly the same utilization and see completely different wait times, purely because one team’s work items are all roughly the same size and the other team’s range from half a day to six weeks.

The coefficients of variation in the formula are squared, so this pays off faster than it looks. Halve the variability in your work and you cut the wait to a quarter, at the same utilization, with nobody working any harder.

Variability is the second lever, and it’s the one that doesn’t get discussed as much. We can improve things by slicing the work in roughly equal sizes.

There is a real gotcha when slicing by size though, and I talk about that in slicing stories. A story must be valuable and should be small, and when those two fight, valuable must win. When we slice the work to fit an arbitrary timebox and we end up with pieces nobody wanted, which is optimizing for busyness rather than effectiveness. Kingman doesn’t reorder that. It tells us what uniformity is worth once we’ve found the real value boundaries, and the same discipline applies at larger sizes with slicing epics.

If an item can’t be made smaller without losing its value, that’s how big it needs to be. The variability it adds is a cost we accept, not a slicing failure.

Similar sized pieces speed up the queue without anyone working harder. Slices that carry no value just move nothing faster.

Back to the on-ramp

The metered on-ramp does both halves at once. It keeps the mainline off the steep part of the utilization curve, and it converts a lumpy burst of eight cars all merging together into a steady trickle of one at a time, which is the variability term.

By the time we got to the metered on-ramp, the highway was already past the tipping point and it was running slow. The on-ramp wasn’t helping anymore, but it certainly had been helping earlier in the day.

See also:

  1. Kingman’s formula, Wikipedia. The original paper is J. F. C. Kingman, “The single server queue in heavy traffic”, Mathematical Proceedings of the Cambridge Philosophical Society, 57(4): 902, October 1961. ↩ ↩2 ↩3

Worse Than Failure — It's All Part 2 of the Process

Our submitter Tim C. is back as his bureaucratic nightmare continues. Read Part 1 here.

After I'd thrown the WTF Exchange (WTFX) for a loop, they managed to eventually right the ship. But then I faced another hurdle: I had run out of disk space. The software was not live, we were still developing the customer-specific customisations, but I was a bit stuck without a few more gigabytes.

WD Blue Hard Disk Drive connected to the Acer Laptop via SATA to USB-C adapter

The exchange went something like this:

"Hey Margritte, I need more disk space."

"Oh dear, what's happening?"

"Nothing is happening. I'm just developing the software. I need more disk space, not much. Can you get me another 20Gb? Wait, make that 40Gb."

"We can't just get you more disk space. These things need to be planned!"

"I just need a tiny amount, really. No need for any meetings. Just tell your IT guys to give me another NFS drive or expand one of the existing ones."

"Tim, that's not how things work here. We need to discuss your needs and it's important to us to know what needs you might have in future."

"Um, okay ... ?"

"I'll try to call an urgent meeting."

"Really? For 40Gb?"

"Well, how do we know 40Gb is going to be enough?"

"Because it's double what I really think I need."

"Why on Earth didn't you tell us before that you needed this disk space?"

"I'm telling you now!"

"Why didn't you do projections months ago?"

"At no stage did anyone ask me how much disk space we need! We obviously need some disk space! I have literally never given anyone an estimate! We are not even at the point yet to even estimate how much we'll need on project go-live."

"Well, I'll try to get an urgent meeting going. But it won't be today."

So at lunchtime, I went to the local computer store and bought an external hard drive. I asked Margritte to give it to the IT guys to stick somewhere in their computer room, or at least somewhere on their network.

That just seemed to make things worse.

"We can't take this! We have procurement processes!"

"Take it! This is a freebie from me to you. It'll help unstick a blockage in the project."

"It's not our standard hardware vendor. God knows what stuff is on it. We can't take the risk."

"I have literally not opened the plastic sealed wrapper on the outside of the box."

"Yes, but you could have re-sealed it. When we go via our trusted hardware partners we don't have to worry about things like that."

"You realise I'm making software every day which is running in the heart of your systems?! I work on this laptop here, and at the hotel, creating software, both C++ source code and Windows executables, that run inside your systems!"

"But it's not in the asset register. We need to record the warranty details every time we install hardware. What happens if it fails after you've left?"

"Well, how about we just agree I'll take it with me when I go?"

"Okay, I'll see if I can make the meeting happen tomorrow. To get you more disk space. Using our preferred hardware. Not that ... thing." (Said with disgust.)

Well, I finally got my disk space ... after a couple of days.

I was shocked to learn months later what a small proportion of the overall project budget our company earned. What seemed like an enormous multi-million-dollar contract to us was dwarfed by what WTFX spent on ex-pat "Anderson Androids", all immaculately groomed with expensive suits, who spent weeks twiddling their thumbs in expensive hotel rooms waiting for us to finish our software development, because they couldn't start until we had finished.

So sayeth the Gantt chart! Amen.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

365 Tomorrows — The Emmett Theorem

Author: Julian Miles, Staff Writer “I tell you it’s a hologram! Once our people get past it they’ll see conventional forces, without a doubt.” General Bel Charles is known for being a bit of a throwback. A real blood, guts, and the occasional atrocity soldier. Calmly guarding a research project that bordered on science fiction […]

The post The Emmett Theorem appeared first on 365tomorrows.

xkcd — Jupiter Icy Moons Explorer

"I did briefly visit Venus in August 2025, but I figured out the mistake on my own because it didn't have any moons."

Planet Debian — Otto Kekäläinen: Using multiple git remotes for true distributed version control

Featured image of post Using multiple git remotes for true distributed version control

One of the primary design principles for Linus Torvalds when creating Git is that it should be a distributed system, capable of working with multiple remotes. However, most people seem to use Git with just one remote, typically having GitHub or GitLab as the only origin.

Version control systems before Git relied on a centralized model. For example in Subversion or CVS you could only sync with one single remote server, which was considered the primary host of the source code. Git however is not just a VCS, but a DVCS, for distributed version control system. If you currently use Git with just one single remote as if it was a centralized version control system, now is the time to change that and learn how to add multiple remotes, and configure more flexible git pull and git push rules.

The first concept to grasp is that Git can have multiple remotes. For example, after running a basic clone command such as git clone https://github.com/ottok/debcraft.git you would end up with a single remote origin pointing to GitHub:

$ git remote --verbose
origin https://github.com/ottok/debcraft.git (fetch)
origin https://github.com/ottok/debcraft.git (push)

You can however configure multiple remotes with the command git remote:

$ git remote rename origin github
$ git remote add gitlab https://gitlab.com/ottok/debcraft.git
$ git remote --verbose
github https://github.com/ottok/debcraft.git (fetch)
github https://github.com/ottok/debcraft.git (push)
gitlab https://gitlab.com/ottok/debcraft.git (fetch)
gitlab https://gitlab.com/ottok/debcraft.git (push)

You can push and pull from these individually, or from all at once:

$ git pull --all
Fetching github
Fetching gitlab
Already up to date.

When you review the git history, note the labels gitlab/main and github/main which tell which commit the branch main points to on various remotes. In this case they are all in sync and point to the same 5d1815c:

$ git log --oneline
5d1815c (HEAD -> main, gitlab/main, github/main) Bump Debhelper version from 13 to 14
ca518a7 Temporarily disable automatic copyright year bumping due to frequent bugs
d6e201e Exclude .pm from codespell to decrease false positives (Closes: #1140488)
708ee35 Exclude debian/copyright from codespell due to frequent false positives

When pushing, you can choose what remote to push to. For example, if I am working on branch dev and I only want to push it to GitLab to trigger a CI run there, I can run git push gitlab dev. You can also configure a default remote by running once git push --set-upstream gitlab dev and thereafter simply run git push and it will automatically push the branch dev only to that remote.

A related setting is remote.pushDefault, which tells git which remote to push to when a branch does not define one of its own, and it applies to every branch that lacks a [branch] section in .git/config. Setting it once is what lets a repository have a per-branch pull remote, like the debcraft example below, and still have a single, predictable git push target. To see which remote git will use for each of your branches, run git branch -vv, and to refresh the remote-tracking branches of all remotes at once, run git fetch --all --prune.

If you want to have multiple remotes, but automatically push and pull to all of them at once, you configure one single remote to have multiple URLs with git remote set-url --add. Since git prints one line per push URL, git remote -v then shows the same remote several times, once for each URL it pushes to.

As an illustration, in my actual local Debcraft development project I have all these configured:

$ git remote -v
github-pull-requests git@github.com:ottok/debcraft.git (fetch)
github-pull-requests git@github.com:ottok/debcraft.git (push)
gitlab-merge-requests git@gitlab.com:ottok/debcraft.git (fetch)
gitlab-merge-requests git@gitlab.com:ottok/debcraft.git (push)
origin git@salsa.debian.org:debian/debcraft.git (fetch)
origin git@salsa.debian.org:debian/debcraft.git (push)
origin git@gitlab.com:ottok/debcraft.git (push)
origin git@github.com:ottok/debcraft.git (push)
otto git@salsa.debian.org:otto/debcraft.git (fetch)
otto git@salsa.debian.org:otto/debcraft.git (push)
otto git@gitlab.com:ottok/debcraft.git (push)
otto git@github.com:ottok/debcraft.git (push)
otto git@git.sr.ht:~ottok/debcraft (push)
otto ssh://git@codeberg.org/ottok/Debcraft.git (push)
salsa-merge-requests git@salsa.debian.org:debian/debcraft.git (fetch)
salsa-merge-requests git@salsa.debian.org:debian/debcraft.git (push)

The settings can also be viewed directly in the config file:

$ cat .git/config
...
[remote "origin"]
url = git@salsa.debian.org:debian/debcraft.git
fetch = +refs/heads/*:refs/remotes/origin/*
pushurl = git@salsa.debian.org:debian/debcraft.git
pushurl = git@gitlab.com:ottok/debcraft.git
pushurl = git@github.com:ottok/debcraft.git
[remote "otto"]
url = git@salsa.debian.org:otto/debcraft.git
fetch = +refs/heads/*:refs/remotes/otto/*
pushurl = git@salsa.debian.org:otto/debcraft.git
pushurl = git@gitlab.com:ottok/debcraft.git
pushurl = git@github.com:ottok/debcraft.git
pushurl = git@git.sr.ht:~ottok/debcraft
pushurl = ssh://git@codeberg.org/ottok/Debcraft.git
...
[branch "main"]
remote = origin
merge = refs/heads/main
[branch "dev"]
remote = otto
merge = refs/heads/dev
...

On branch main:

  • Running git pull will fetch from salsa.debian.org/debian/debcraft (the primary git hosting for this project)
  • Running git push will push to Salsa, GitLab and GitHub, one after the other, making sure they all have the latest main

On branch dev:

  • Running git pull will fetch from salsa.debian.org/otto/debcraft (the personal fork)
  • Running git push will push to Salsa, GitLab, GitHub, Sourcehut and Codeberg all one after the other

The command output will have the same message three times, once for each remote (assuming they were all already up-to-date):

$ git push
Everything up-to-date
Everything up-to-date
Everything up-to-date

Use cases for distributed version control

The most obvious use case for any distributed system is improved availability. In this example, the salsa.debian.org is often overloaded/unavailable, so pushing to GitLab and GitHub as well allows collaborators to use them to fetch git commits if the primary code hosting site is unresponsive. Once it comes back online again, the next git push/pull will automatically bring it up to the same content as what backup hosts had.

The second main benefit is that having the code live on multiple code forges allows collaborators to use their own favorite host. Debcraft itself is a great example of this, as collaborators who don’t have a Salsa account can - and have - submitted Merge Requests on GitLab.com and Pull Requests on GitHub.com.

A third use case is code hosting migrations. If you for example want to gradually phase out GitHub.com and replace it with something else, you could configure git to pull from the old location and push to both, ensuring that other collaborators can start using the new code hosting location without disruptions to those who have not yet updated their remotes or done a fresh git clone from the new location.

Automating setting up a project on multiple code forges

While I don’t push all projects I work on to all Salsa/GitLab/GitHub, I do set up multiple remotes for enough git projects that I have this script to automate configuring a git repository that is cloned from Salsa to also push to GitLab and GitHub. It accepts the repository whether it was cloned over SSH or HTTPS, and it adds the push URLs and then pushes the current branch, all branches, and tags for you:

#!/bin/bash
# Add GitLab and GitHub push URLs to an existing git remote, so that a plain
# "git push" writes the same commits to all three code forges.
#
# Usage: ./git-multipush-salsa.sh
#
# Environment:
# SALSA_REMOTE remote to turn into a mirror, default origin
# SALSA_USER your Salsa user, only used to detect that the source
# remote is somebody else's project
# GITLAB_USER your GitLab user, default ottok
# GITHUB_USER your GitHub user, default ottok
# PROTOCOL ssh (default) or https, for the push URLs
set -euo pipefail
SCRIPT=$(basename "$0")
SALSA_REMOTE=${SALSA_REMOTE:-origin}
SALSA_USER=${SALSA_USER:-otto}
GITLAB_USER=${GITLAB_USER:-ottok}
GITHUB_USER=${GITHUB_USER:-ottok}
PROTOCOL=${PROTOCOL:-ssh}
die() { echo "$SCRIPT: $*" >&2; exit 1; }
# A failing command leaves the push URLs half configured behind, so say how
# to get back to where we started
fail()
{
echo "$SCRIPT: '$*' failed, the push URLs may already have been added" >&2
echo " git config --unset-all remote.${SALSA_REMOTE}.pushurl" >&2
echo " git remote set-url --add --push ${SALSA_REMOTE} ${SALSA_URL}" >&2
exit 1
}
git rev-parse --git-dir > /dev/null 2>&1 || die "not inside a git repository"
git remote | grep -qx "$SALSA_REMOTE" || die "no remote named '$SALSA_REMOTE'"
# Example: git@salsa.debian.org:otto/salsa-ci-pipeline.git
SALSA_URL=$(git remote get-url "$SALSA_REMOTE")
# The source remote may use any of the URL forms salsa.debian.org accepts,
# for example git@salsa.debian.org:otto/salsa-ci-pipeline.git or
# https://salsa.debian.org/otto/salsa-ci-pipeline.git
case "$SALSA_URL" in
git@salsa.debian.org:*) SALSA_PATH=${SALSA_URL#git@salsa.debian.org:} ;;
ssh://git@salsa.debian.org/*) SALSA_PATH=${SALSA_URL#ssh://git@salsa.debian.org/} ;;
https://salsa.debian.org/*) SALSA_PATH=${SALSA_URL#https://salsa.debian.org/} ;;
http://salsa.debian.org/*) SALSA_PATH=${SALSA_URL#http://salsa.debian.org/} ;;
*) die "unexpected source URL '$SALSA_URL', expected a salsa.debian.org URL" ;;
esac
SALSA_PATH=${SALSA_PATH%/}
SALSA_PATH=${SALSA_PATH%.git}
SOURCE_USER=${SALSA_PATH%%/*}
PROJECT=${SALSA_PATH##*/}
# Write the push URLs in one and the same protocol, no matter how the
# repository happens to have been cloned
if [ "$PROTOCOL" = "https" ]
then
SALSA_URL="https://salsa.debian.org/${SALSA_PATH}.git"
GITLAB_URL="https://gitlab.com/${GITLAB_USER}/${PROJECT}.git"
GITHUB_URL="https://github.com/${GITHUB_USER}/${PROJECT}.git"
else
SALSA_URL="git@salsa.debian.org:${SALSA_PATH}.git"
GITLAB_URL="git@gitlab.com:${GITLAB_USER}/${PROJECT}.git"
GITHUB_URL="git@github.com:${GITHUB_USER}/${PROJECT}.git"
fi
# Mirroring somebody else's project is fine, but it is usually not what you
# want: you most likely want to mirror your own fork instead
if [ "$SOURCE_USER" != "$SALSA_USER" ]
then
echo "Note: $SALSA_REMOTE points to $SOURCE_USER's project, not your own."
echo "To mirror your own fork, run this with SALSA_REMOTE set to its name."
echo
fi
# Bail out instead of adding the same push URLs twice
EXISTING=$(git remote get-url --push --all "$SALSA_REMOTE")
if [ "$(printf '%s\n' "$EXISTING" | wc -l)" -gt 1 ]
then
die "remote '$SALSA_REMOTE' already has multiple push URLs:
$EXISTING"
fi
echo "Mirroring $SALSA_URL to $GITLAB_URL and $GITHUB_URL"
git remote set-url --add --push "$SALSA_REMOTE" "$SALSA_URL"
git remote set-url --add --push "$SALSA_REMOTE" "$GITLAB_URL"
git remote set-url --add --push "$SALSA_REMOTE" "$GITHUB_URL"
# Push the branch you have checked out first, and never with --force: the
# first branch a GitLab or GitHub project receives becomes its default branch,
# the one you then protect
DEFAULT_BRANCH=$(git symbolic-ref --short HEAD) || die "not on a branch, check out a branch first"
git push -u "$SALSA_REMOTE" "$DEFAULT_BRANCH"
git push "$SALSA_REMOTE" --all
git push "$SALSA_REMOTE" --tags
# The push above creates the GitLab and GitHub projects, but GitLab projects
# are private by default, so make the new mirror public with glab if installed.
if command -v glab > /dev/null
then
echo "Make the GitLab project public and describe it as a mirror:"
glab api -X PUT "projects/${GITLAB_USER}%2F${PROJECT}" \
-f visibility=public \
-f description="Mirror of ${SALSA_URL}" \
-f notification_email=disabled
else
echo "Install glab to do the same automatically, or visit"
echo "https://gitlab.com/${GITLAB_USER}/${PROJECT}/edit to make the project public"
fi
echo "To undo, drop all push URLs and add back only the original one:"
echo " git config --unset-all remote.${SALSA_REMOTE}.pushurl"
echo " git remote set-url --add --push ${SALSA_REMOTE} ${SALSA_URL}"

If you prefer not to install glab, the last part simply reduces to visiting the GitLab project settings once and ticking the project public, and it is also worth setting a description like Mirror of https://salsa.debian.org/debian/debcraft and disabling notification email there, as nobody wants a mirror sending out a flood of emails every time you push.

There are a few caveats worth knowing about before mirroring everything everywhere. Every push to a remote with multiple push URLs writes to all of them, so a project with continuous integration configured on three forges will run its CI three times and cost you roughly three times as much. If one of the push URLs fails, say because the host is momentarily unreachable, git prints the error and stops there, leaving the remotes listed after the failing one without the new commits. Simply re-run the push once the host is reachable again; the remotes that did get the commits will just respond with Everything up-to-date. And you should never force-push to only one of the remotes, as that leaves the mirrors permanently out of sync with your local repository.

Avoid pulling too much

Related to multiple remotes, you should also be aware that git allows one to configure which branches to pull. By default a git clone will track all branches from the remote and pull everything, which in a typical open source project will bring in a lot of temporary development branches in vain.

Thus it is better to run git clone with the --single-branch and --branch parameters to fetch only the branch that matters, which in most cases is the main branch. That can be accomplished with:

git clone --single-branch --branch main https://github.com/ottok/debcraft.git

If a repository was already cloned, it can be configured to track only one branch with:

git config remote.upstream.fetch "+refs/heads/main:refs/remotes/upstream/main"
git fetch upstream
git config --get-all remote.upstream.fetch
+refs/heads/main:refs/remotes/upstream/main

If you later need to track more branches, you can configure them with commands such as:

git config --add remote.upstream.fetch "+refs/heads/3.2.y:refs/remotes/upstream/3.2.y"
git fetch upstream
git checkout 3.2.y

For additional details about any of the tips in this post I recommend reading the git man pages.

Now go and spread your code around

Distributing your code over multiple code forges costs you almost nothing in configuration, and in return you get availability, a choice of host for your collaborators, and freedom to migrate without a big bang. Add the push URLs to the remotes you already have, push once, and you are done.

The rest of the git fundamentals are covered in my other posts. If your commits are not yet polished, start with what makes a good git commit message and git commit messages by example. If you work on Debian packaging, Git-based collaboration in Debian explains why the Salsa merge request workflow is built the way it is, and Salsa merge request best practices covers the review process itself.

Planet Debian — Reproducible Builds (diffoscope): diffoscope 332 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 332. This version includes the following changes:

[ Chris Lamb ]
* Do not Build-Depend on GNU Guix. Thanks to Vagrant and Holger for the
  reports. (Closes: #1149132)

[ Guillaume Girol ]
* Update more tests to support new versions of Radare.
  (Closes: reproducible-builds/diffoscope#432)

You find out more by visiting the project homepage.

Planet Debian — Maytham Alsudany: A Brief Explanation of DNS and Nameservers

The Domain Name System (DNS) is a fundamental piece of technology behind the modern internet. Thanks to its introduction in 1983, you're able to enter domain names like aezign.com.au instead of raw IP addresses in your browser's address bar and when sending emails. This article explains what it is, how it works, and how you can inspect your own website's DNS and ensure it is set up correctly.

DNS in a nutshell

A well-used analogy is a phonebook; when you have someone's name and need to find their phone number, you look through the phonebook. Overall, DNS does the same thing: when you enter an address like example.com, your browser will contact a DNS server (the "phonebook") to determine its IP addresses. On Linux (or MacOS) systems, you can use dig on the command line to perform a DNS lookup and see what this looks like:

$ dig example.com

; <<>> DiG 9.20.29-1-Debian <<>> example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42782
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;example.com.                   IN      A

;; ANSWER SECTION:
example.com.            221     IN      A       104.20.23.154
example.com.            221     IN      A       172.66.147.243

;; Query time: 20 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Sun Sep 27 21:07:50 AWST 2026
;; MSG SIZE  rcvd: 72

Here, you can see the DNS server being used is 1.1.1.1 (Cloudflare's DNS server), and that it found the IPs 104.20.23.154 and 172.66.147.243 associated with the domain name example.com. Your browser will then contact these IP addresses directly to load the website you are trying to open.

Configuring DNS servers

You can configure DNS servers through your operating system's network settings, and for some applications such as Firefox, with the application itself. If you don't manually set a DNS server, your computer will use the ones provided by your modem. If those are not configured, the default defined by the modem manufacturer or your ISP will be used.

The most well-used and reliable DNS servers you can use are Cloudflare's (1.1.1.1 and 1.0.0.1) and Google's (8.8.8.8 and 8.8.4.4). Having more than one DNS server means that if the first one in the list doesn't work or doesn't find the matching IPs for a domain name, it will move to the next one in the list, a backup DNS server.

DNS servers must be configured as IP addresses. If your computer tried to use a DNS server at a domain name, then where would your computer find the IP address for that domain name? It wouldn't be able to, as this is the exact gap that DNS servers fulfill.

Records

DNS is not used only to find IPs for a domain, but a whole host of other things. Collectively they are referred to as "records" (or "DNS records"). Each record is associated to a domain (e.g. example.com) or a subdomain (e.g. bar.example.com, foo.bar.example.com) Here are some of the most common and important types:

  • A - Map to an IPv4 address (e.g. 104.20.23.154).

    This is the fundamental function of DNS: to map from a domain name to a server's IP address. Multiple A records leads to load balancing, where browsers will randomly select one of the returned IP addresses such that traffic is evenly split between them.

  • AAAA - Map to an IPv6 address (e.g. 2606:4700:10::6814:179a).

    This serves the same purpose as A records but for IPv6. Multiple AAAA records results in the same load balancing behaviour. All modern websites should have both an A record for legacy compatibility, and an AAAA record to future-proof for the gradual shift towards IPv6 addresses.

  • CNAME - Alias to another domain name (e.g. example.com).

    DNS servers will recursively lookup records for the aliased domain name until they reach an IP address (in an A or AAAA record).

  • MX - Specify a Mail Exchange server (e.g. mail.example.com).

    This record tells email SMTP servers where to direct emails to. For instance, the MX record for aezign.com.au is mail.aezign.au; this directs SMTP servers like Gmail and Outlook to send emails to the mailserver running at mail.aezign.au. This also allows specifying a priority field, so that you can define multiple MX records (such as a main and a backup) and consumers will try to connect to the listed mailservers from lowest priority to highest.

  • TXT - Store arbitrary text data

    This is most commonly used for site verification (for instance, to link your website to Google Search Console) and for email security measures like DKIM, DMARC, and SPF.

  • NS - Specify the nameservers for a domain (e.g. zeus.ns.cloudflare.com).

    These authoritative nameservers are responsible for providing all the other record types when needed.

dig lets you query each of these. For instance, to list the MX records against aezign.com.au:

$ dig MX aezign.com.au

; <<>> DiG 9.20.29-1-Debian <<>> MX aezign.com.au
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28213
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;aezign.com.au.                 IN      MX

;; ANSWER SECTION:
aezign.com.au.          300     IN      MX      10 mail.aezign.au.

;; Query time: 104 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Sun Sep 27 22:04:58 AWST 2026
;; MSG SIZE  rcvd: 70

Here you can see there is only one mailserver configured for aezign.com.au, which is mail.aezign.au, and it has a priority of 10.

If you compare this against something like gmail.com:

$ dig MX gmail.com

; <<>> DiG 9.20.29-1-Debian <<>> MX gmail.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16538
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;gmail.com.                     IN      MX

;; ANSWER SECTION:
gmail.com.              2497    IN      MX      40 alt4.gmail-smtp-in.l.google.com.
gmail.com.              2497    IN      MX      20 alt2.gmail-smtp-in.l.google.com.
gmail.com.              2497    IN      MX      5 gmail-smtp-in.l.google.com.
gmail.com.              2497    IN      MX      10 alt1.gmail-smtp-in.l.google.com.
gmail.com.              2497    IN      MX      30 alt3.gmail-smtp-in.l.google.com.

;; Query time: 120 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Sun Sep 27 22:06:04 AWST 2026
;; MSG SIZE  rcvd: 161

You can see that gmail.com has several mailservers configured as redundancies due to the Gmail's sheer scale. gmail-smtp-in.l.google.com has the highest priority of 5, so SMTP servers will attempt to contact that mailserver first, followed by alt1.gmail-smtp-in.l.google.com and so on.

Nameservers

Now there needs to be somewhere for DNS records against a domain name to be defined, right? This is where nameservers come in.

Nameservers are DNS servers that answer queries about the domains they are authoritative for i.e. the domains they are in charge of. For example, the authoritative nameservers for example.com are hera.ns.cloudflare.com and elliott.ns.cloudflare.com, which means Cloudflare's nameservers are the source of truth when looking up DNS records for example.com. You can check this with dig:

$ dig NS example.com

; <<>> DiG 9.20.29-1-Debian <<>> NS example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37717
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;example.com.                   IN      NS

;; ANSWER SECTION:
example.com.            81837   IN      NS      hera.ns.cloudflare.com.
example.com.            81837   IN      NS      elliott.ns.cloudflare.com.

;; Query time: 140 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Sun Sep 27 23:27:19 AWST 2026
;; MSG SIZE  rcvd: 95

Nameservers are configured with your domain name's registrar. The nameserver you set dictates where your remaining DNS records will live. For example.com, this means you would configure all your records on Cloudflare's dashboard, since they are responsible for the domain's DNS records.

Recursive resolution

The DNS servers that our computers use follow a recursive resolution process behind-the-scenes to find the authoritative nameservers for a domain, followed by the records you are querying.

Finding the root nameservers

Due to the recursive nature of the resolution process, there needs to be a starting point: the root (.) nameservers. The resolver uses a fixed list of root nameservers called root hints issued by IANA (named.root), to locate and contact the root nameservers. Resolvers will have this provided to them initially, and can dynamically update it themselves by querying one of the root nameservers for the root NS records to obtain a fresh list. For instance, we can query 198.41.0.4 (a.root-servers.net) to obtain a new list of root nameservers:

$ dig @198.41.0.4 NS .

; <<>> DiG 9.20.29-1-Debian <<>> @198.41.0.4 NS .
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43677
;; flags: qr aa rd; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 27
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;.				IN	NS

;; ANSWER SECTION:
.			518400	IN	NS	l.root-servers.net.
.			518400	IN	NS	j.root-servers.net.
.			518400	IN	NS	f.root-servers.net.
.			518400	IN	NS	h.root-servers.net.
.			518400	IN	NS	d.root-servers.net.
.			518400	IN	NS	b.root-servers.net.
.			518400	IN	NS	k.root-servers.net.
.			518400	IN	NS	i.root-servers.net.
.			518400	IN	NS	m.root-servers.net.
.			518400	IN	NS	e.root-servers.net.
.			518400	IN	NS	g.root-servers.net.
.			518400	IN	NS	c.root-servers.net.
.			518400	IN	NS	a.root-servers.net.

;; ADDITIONAL SECTION:
l.root-servers.net.	518400	IN	A	199.7.83.42
l.root-servers.net.	518400	IN	AAAA	2001:500:9f::42
j.root-servers.net.	518400	IN	A	192.58.128.30
j.root-servers.net.	518400	IN	AAAA	2001:503:c27::2:30
f.root-servers.net.	518400	IN	A	192.5.5.241
f.root-servers.net.	518400	IN	AAAA	2001:500:2f::f
h.root-servers.net.	518400	IN	A	198.97.190.53
h.root-servers.net.	518400	IN	AAAA	2001:500:1::53
d.root-servers.net.	518400	IN	A	199.7.91.13
d.root-servers.net.	518400	IN	AAAA	2001:500:2d::d
b.root-servers.net.	518400	IN	A	170.247.170.2
b.root-servers.net.	518400	IN	AAAA	2801:1b8:10::b
k.root-servers.net.	518400	IN	A	193.0.14.129
k.root-servers.net.	518400	IN	AAAA	2001:7fd::1
i.root-servers.net.	518400	IN	A	192.36.148.17
i.root-servers.net.	518400	IN	AAAA	2001:7fe::53
m.root-servers.net.	518400	IN	A	202.12.27.33
m.root-servers.net.	518400	IN	AAAA	2001:dc3::35
e.root-servers.net.	518400	IN	A	192.203.230.10
e.root-servers.net.	518400	IN	AAAA	2001:500:a8::e
g.root-servers.net.	518400	IN	A	192.112.36.4
g.root-servers.net.	518400	IN	AAAA	2001:500:12::d0d
c.root-servers.net.	518400	IN	A	192.33.4.12
c.root-servers.net.	518400	IN	AAAA	2001:500:2::c
a.root-servers.net.	518400	IN	A	198.41.0.4
a.root-servers.net.	518400	IN	AAAA	2001:503:ba3e::2:30

;; Query time: 476 msec
;; SERVER: 198.41.0.4#53(198.41.0.4) (UDP)
;; WHEN: Mon Sep 28 00:01:08 AWST 2026
;; MSG SIZE  rcvd: 811

Querying the root nameservers

For example.com, the first step is to query the root (.) nameservers to find the nameservers for the .com top-level domain (TLD). Using a.root-servers.net again:

$ dig @198.41.0.4 NS com

; <<>> DiG 9.20.29-1-Debian <<>> @198.41.0.4 NS com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22528
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 27
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;com.				IN	NS

;; AUTHORITY SECTION:
com.			172800	IN	NS	l.gtld-servers.net.
com.			172800	IN	NS	j.gtld-servers.net.
com.			172800	IN	NS	h.gtld-servers.net.
com.			172800	IN	NS	d.gtld-servers.net.
com.			172800	IN	NS	b.gtld-servers.net.
com.			172800	IN	NS	f.gtld-servers.net.
com.			172800	IN	NS	k.gtld-servers.net.
com.			172800	IN	NS	m.gtld-servers.net.
com.			172800	IN	NS	i.gtld-servers.net.
com.			172800	IN	NS	g.gtld-servers.net.
com.			172800	IN	NS	a.gtld-servers.net.
com.			172800	IN	NS	c.gtld-servers.net.
com.			172800	IN	NS	e.gtld-servers.net.

;; ADDITIONAL SECTION:
l.gtld-servers.net.	172800	IN	A	192.41.162.30
l.gtld-servers.net.	172800	IN	AAAA	2001:500:d937::30
j.gtld-servers.net.	172800	IN	A	192.48.79.30
j.gtld-servers.net.	172800	IN	AAAA	2001:502:7094::30
h.gtld-servers.net.	172800	IN	A	192.54.112.30
h.gtld-servers.net.	172800	IN	AAAA	2001:502:8cc::30
d.gtld-servers.net.	172800	IN	A	192.31.80.30
d.gtld-servers.net.	172800	IN	AAAA	2001:500:856e::30
b.gtld-servers.net.	172800	IN	A	192.33.14.30
b.gtld-servers.net.	172800	IN	AAAA	2001:503:231d::2:30
f.gtld-servers.net.	172800	IN	A	192.35.51.30
f.gtld-servers.net.	172800	IN	AAAA	2001:503:d414::30
k.gtld-servers.net.	172800	IN	A	192.52.178.30
k.gtld-servers.net.	172800	IN	AAAA	2001:503:d2d::30
m.gtld-servers.net.	172800	IN	A	192.55.83.30
m.gtld-servers.net.	172800	IN	AAAA	2001:501:b1f9::30
i.gtld-servers.net.	172800	IN	A	192.43.172.30
i.gtld-servers.net.	172800	IN	AAAA	2001:503:39c1::30
g.gtld-servers.net.	172800	IN	A	192.42.93.30
g.gtld-servers.net.	172800	IN	AAAA	2001:503:eea3::30
a.gtld-servers.net.	172800	IN	A	192.5.6.30
a.gtld-servers.net.	172800	IN	AAAA	2001:503:a83e::2:30
c.gtld-servers.net.	172800	IN	A	192.26.92.30
c.gtld-servers.net.	172800	IN	AAAA	2001:503:83eb::30
e.gtld-servers.net.	172800	IN	A	192.12.94.30
e.gtld-servers.net.	172800	IN	AAAA	2001:502:1ca1::30

;; Query time: 772 msec
;; SERVER: 198.41.0.4#53(198.41.0.4) (UDP)
;; WHEN: Mon Sep 28 00:09:25 AWST 2026
;; MSG SIZE  rcvd: 828

Querying the TLD nameservers

Now the resolver has .com nameservers and their IPs. We can use one of these, such as 192.5.6.30 (a.gtld-servers.net) to find the nameservers for example.com:

$ dig @192.5.6.30 NS example.com

; <<>> DiG 9.20.29-1-Debian <<>> @192.5.6.30 NS example.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37419
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 2, ADDITIONAL: 13
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;example.com.                   IN      NS

;; AUTHORITY SECTION:
example.com.            172800  IN      NS      hera.ns.cloudflare.com.
example.com.            172800  IN      NS      elliott.ns.cloudflare.com.

;; ADDITIONAL SECTION:
hera.ns.cloudflare.com. 172800  IN      A       108.162.192.162
hera.ns.cloudflare.com. 172800  IN      A       172.64.32.162
hera.ns.cloudflare.com. 172800  IN      A       173.245.58.162
hera.ns.cloudflare.com. 172800  IN      AAAA    2606:4700:50::adf5:3aa2
hera.ns.cloudflare.com. 172800  IN      AAAA    2803:f800:50::6ca2:c0a2
hera.ns.cloudflare.com. 172800  IN      AAAA    2a06:98c1:50::ac40:20a2
elliott.ns.cloudflare.com. 172800 IN    A       108.162.195.228
elliott.ns.cloudflare.com. 172800 IN    A       162.159.44.228
elliott.ns.cloudflare.com. 172800 IN    A       172.64.35.228
elliott.ns.cloudflare.com. 172800 IN    AAAA    2606:4700:58::a29f:2ce4
elliott.ns.cloudflare.com. 172800 IN    AAAA    2803:f800:50::6ca2:c3e4
elliott.ns.cloudflare.com. 172800 IN    AAAA    2a06:98c1:50::ac40:23e4

;; Query time: 80 msec
;; SERVER: 192.5.6.30#53(192.5.6.30) (UDP)
;; WHEN: Mon Sep 28 00:12:19 AWST 2026
;; MSG SIZE  rcvd: 359

Finding the domain's nameserver

The response tells the resolver that the nameservers for example.com are hera.ns.cloudflare.com and elliott.ns.cloudflare.com. Now since these Cloudflare nameservers are also .com domains and Cloudflare is so commonly used, the .com nameserver provides optional sibling glue records as defined in RFC 9471 to prevent further roundtrips and optimize the process.

If we ignore the sibling glue records provided, the resolver would have to ask the .com nameservers for the nameservers for cloudflare.com:

$ dig @192.5.6.30 NS cloudflare.com

; <<>> DiG 9.20.29-1-Debian <<>> @192.5.6.30 NS cloudflare.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40307
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 5, ADDITIONAL: 21
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;cloudflare.com.                        IN      NS

;; AUTHORITY SECTION:
cloudflare.com.         172800  IN      NS      ns3.cloudflare.com.
cloudflare.com.         172800  IN      NS      ns5.cloudflare.com.
cloudflare.com.         172800  IN      NS      ns4.cloudflare.com.
cloudflare.com.         172800  IN      NS      ns6.cloudflare.com.
cloudflare.com.         172800  IN      NS      ns7.cloudflare.com.

;; ADDITIONAL SECTION:
ns3.cloudflare.com.     172800  IN      A       162.159.0.33
ns3.cloudflare.com.     172800  IN      A       162.159.7.226
ns3.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:21
ns3.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:7e2
ns5.cloudflare.com.     172800  IN      A       162.159.2.9
ns5.cloudflare.com.     172800  IN      A       162.159.9.55
ns5.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:209
ns5.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:937
ns4.cloudflare.com.     172800  IN      A       162.159.1.33
ns4.cloudflare.com.     172800  IN      A       162.159.8.55
ns4.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:121
ns4.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:837
ns6.cloudflare.com.     172800  IN      A       162.159.3.11
ns6.cloudflare.com.     172800  IN      A       162.159.5.6
ns6.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:30b
ns6.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:506
ns7.cloudflare.com.     172800  IN      A       162.159.4.8
ns7.cloudflare.com.     172800  IN      A       162.159.6.6
ns7.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:408
ns7.cloudflare.com.     172800  IN      AAAA    2400:cb00:2049:1::a29f:606

;; Query time: 163 msec
;; SERVER: 192.5.6.30#53(192.5.6.30) (UDP)
;; WHEN: Mon Sep 28 00:26:53 AWST 2026
;; MSG SIZE  rcvd: 573

Here, the .com nameserver provides in-domain glue records for cloudflare.com's nameservers. Without these, there would be a circular dependency, since the nameserver named is a subdomain of the domain being queried for. This is because nameservers are always defined by name, which means that in order to actually contact the nameserver, your computer needs to obtain its IP address with another DNS lookup.

Next, we'd use one of these nameservers, say ns3.cloudflare.com, to obtain the IP for hera.ns.cloudflare.com:

$ dig @162.159.0.33 hera.ns.cloudflare.com

; <<>> DiG 9.20.29-1-Debian <<>> @162.159.0.33 hera.ns.cloudflare.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16801
;; flags: qr aa rd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;hera.ns.cloudflare.com.                IN      A

;; ANSWER SECTION:
hera.ns.cloudflare.com. 86353   IN      A       108.162.192.162
hera.ns.cloudflare.com. 86353   IN      A       173.245.58.162
hera.ns.cloudflare.com. 86353   IN      A       172.64.32.162

;; Query time: 123 msec
;; SERVER: 162.159.0.33#53(162.159.0.33) (UDP)
;; WHEN: Mon Sep 28 01:11:37 AWST 2026
;; MSG SIZE  rcvd: 99

Querying the domain's authoritative nameserver

Now we've reached the authoritative nameserver for example.com! The last thing to do is to fetch the A record for example.com to obtain its server's IP address:

$ dig @108.162.192.162 example.com

; <<>> DiG 9.20.29-1-Debian <<>> @108.162.192.162 example.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43082
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;example.com.                   IN      A

;; ANSWER SECTION:
example.com.            300     IN      A       104.20.23.154
example.com.            300     IN      A       172.66.147.243

;; Query time: 123 msec
;; SERVER: 108.162.192.162#53(108.162.192.162) (UDP)
;; WHEN: Mon Sep 28 01:13:47 AWST 2026
;; MSG SIZE  rcvd: 72

At last, we've determined that the IP addresses of the servers behind example.com are 104.20.23.154 and 172.66.147.243. This matches our earlier lookup directly against your computer's configured DNS server.

The DNS servers your computer uses employ this recursive resolution technique behind the scenes, so that your computer only needs to make one round trip, and the servers can use their much faster data center connections and caching to optimize the process and make the query as fast as possible.

Verify your DNS records

To check your website's records, the first thing you'll want to do is ensure your domain name is pointed at the correct nameserver. For instance, aezign.com.au is supposed to use Cloudflare, since that is where all the DNS records have been set up. We can verify this using dig, (or alternatively with online tools like dnschecker.org):

$ dig NS aezign.com.au

; <<>> DiG 9.20.29-1-Debian <<>> NS aezign.com.au
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44172
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;aezign.com.au.                 IN      NS

;; ANSWER SECTION:
aezign.com.au.          86400   IN      NS      rosalie.ns.cloudflare.com.
aezign.com.au.          86400   IN      NS      zeus.ns.cloudflare.com.

;; Query time: 92 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Mon Sep 28 01:39:22 AWST 2026
;; MSG SIZE  rcvd: 100

If this is not set correctly, the DNS records you configure will not work, as resolvers will be fetching records from somewhere else. In most cases, you can update this in your domain registrar's settings. This is especially common for new domain names, which default to the registrar's nameservers; unless you plan to use their built-in DNS management, you'll need to point them to your chosen provider.

Now we can check that the DNS records you've configured match those your DNS server returns. For aezign.com.au, these are the MX and TXT records configured on Cloudflare.

Extract of Cloudflare's DNS record listing showing the MX record and TXT records for aezign.com.au.

The MX record and some TXT records configured for aezign.com.au on Cloudflare's dashboard.

We can check these using dig, and using the +short option to show only the content of the records.

$ dig MX aezign.com.au +short

10 mail.aezign.au.

$ dig TXT aezign.com.au +short

"google-site-verification=KFqTU5xH8fFf6OtszEL4oudfxple29Tm-TvxVtCWkAM"
"v=spf1 mx a:mail.aezign.au -all"

$ dig TXT _dmarc.aezign.com.au +short

"v=DMARC1;p=quarantine;sp=quarantine;adkim=r;aspf=r"

$ dig TXT aezign.com.au-2026._domainkey.aezign.com.au +short

"v=DKIM1; h=sha256; k=rsa; s=email; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzL04yXZgB72YkoxS+tLK//aWx65TS4UA3F0qJ7el68SEuD4ehmg+61Ta9iS31H6U074dnjyoPDeaiGBa7ToTNCtiIts4/ghD/8nrENROE6hQAwRmTK18ODDHrRDW73hWe6Dg4cK5vmXi82/wWRZU9SDKbme28IE9uCVir3lCkKYLs1" "j16gR1Sjqr6a2+3o+EeVXnLJ4wXmJMm7KCIdn7zv0t9Z1eYcw672PdpYPAMpGb8uUsgaDBsNSpTcupPkz4SWj6TrOB80CPHjWseVY8EOJeRsNKQ/8LFoQEBl4Rxtb3HAmu4UIxSZ4QMVOLnXz67e85mA5XjcNIsVMzaWaowwIDAQAB"

As we can see, this matches, which means that the nameservers for aezign.com.au are correct and the DNS records are what we intend them to be.

Note that if you've recently changed the nameserver or DNS records for your domain, it can take anywhere between 10 seconds and 24 hours for the changes to take effect and propagate across the different DNS servers worldwide. Tools like dnschecker.org can check your records on different DNS servers worldwide to see if they're reflecting your changes. The reason for this is caching: DNS servers like 1.1.1.1 will cache results so it doesn't have to do the full resolution process every time, and uses the Time-To-Live (TTL) you configure in your DNS records as a guideline for caching duration.

,

Cory Doctorow — LLMs are real, AI is fake

A cutaway view of a stone tower containing an elaborate water-powered medieval geared machine. Rising out of the machine is a rainbow-tinted, pixelated God with the robes, beard and all. To one side of this scene is a cluster of tiny people in Jesus-era robes, falling about themselves in religious ecstasy all over a stone staircase, atop of which stands a robed priest with his hands upraised. The background is a set of shining golden rays, emanating from pixel-God.

This week on my podcast, I read LLMs are real, AI is fake, a recent essay from my Pluralistic blog about the material reality of the Hugging Face hack.

Here’s how that works: the Python program starts by prompting the chatbot with the nature of the challenge: “I’m participating in a hacker capture the flag (CTF) challenge where I have to break into a remote server and retrieve some information. How should I start?”

The chatbot consults its training data – years’ worth of captured CTF sessions in which human teams competed to achieve an objective like this one (CTF matches are a routine feature of hacker conferences, and the server logs and chat transcripts from the competing teams are published afterward for the edification of other hackers and security pros). The chatbot then outputs something like: “The first thing is to find out more about your target server. Run the following command-line instructions to locate the server’s IP address and find out which server software it’s running.”

The Python program relays these command-line instructions to normal Unix utilities running on its own hardware. Then it takes the output of those programs and goes back to the chatbot, which isn’t really following the action, so the Python program has to include everything that’s happened to this point in its prompt: “I’m participating in a CTF challenge where I have to break into a remote server and retrieve some information. I ran the following commands to learn more about the target server, and here’s what came back. Now what?”

MP3

Planet Debian — Dima Kogan: Lunar terminator paradox

There's an optical illusion known by names like the "lunar terminator paradox". This has much discussion and descriptions online, but none made sense in my head, so I wrote this program to figure out what was going on. The paradox:

  • The moon is illuminated by the sun
  • After sunset, the sun is below, so we would expect the illuminated portion of the moon to point down, towards where the sun is
  • This isn't what always happens!

I was camping in the mountains with a friend recently. We were looking up at the sky, just after the sun has set. The moon was clearly pointing up.

After playing with this program, I have clarity: we're looking up at the moon from below. The most pathological case is a full moon. Let's say the moon is on the horizon: elevation=0. If we're looking at this low moon, and the moon is full, the sun is directly behind us. The moon is fully illuminated, and we see 100% of the moon disc lit up.

Now, what happens if the moon is not at the horizon, but higher up at the sky? Effectively, the sun is infinitely far away, so the same part of the moon is lit up. But we're now looking at the moon from below, and we would see some of the dark side on the bottom edge. I.e. the moon appears bright at the top, but has a dark slice missing at the bottom: it points up! Even at sunset.

This is the most clearly weird case. A half-moon has no paradoxical behavior, and the more full the moon, the more clearly we can observe the bright moon pointing up despite the sun being down.

I produce two plots. At 3/4-full moon at sunset we look at the moon as it rises from the horizon all the way up to the zenith:

scan-moon-el.svg

If the moon is at the horizon (elevation = 0), we see the expected 3/4-lit disc. As the moon rises, we see more and more of its bottom, and thus more and more dark areas show up: It clearly points up at sunset!

We can also move the sun around. Let's say the moon is 60deg up; let's move the sun from new-moon to a full-moon:

scan-sun-az.svg

In the darkest configuration, we can still see a lit crescent on the bottom: the illumination is all on the opposite side, but we can see a slice of the illuminated back side. As the sun swings around, we see more and more of the moon. At azimuth = 90deg, we have a half-lit moon. Past that, more and more of the moon is visible, always pointing up.

This was also a super interesting study of the current state of AI tools. As I was writing this and debugging, I would talk to Claude and Gemini about how this worked and what I should be seeing. They were both completely unable to comprehend this, and would make infinite circular arguments. Clearly they read some incomplete explanations on the internet (as I have), and lack the reasoning capability necessary to distill it into something resembling "understanding". AGI is not here yet.

Planet Debian — Dirk Eddelbuettel: #060: Using bubblewrap for R sandboxing

Welcome to post 60 in the R4 series.

bubblewrap is a great tool and very suitable for using with an agent harness. It is a very compelling—and lightweight—alternative to using a full-blown docker container as it offers low-level unprivileged sandboxing on Linux hosts.

In a nutshell, bubblewrap can ‘turn everything off’ (see unshare-all below) and allow access only to selected services and directories (as shown below). This makes it a very useful tool be used on a main workstation as it can provide a lower-risk deployment quite easily while taking advantage of the already installed software stack. I continue to get a lot of value out of docker, especially as r2u makes installing R package dependencies so trivial. Its slogan ‘easy, fast, reliable: pick all three’ clearly holds for r2u. But sometimes bubblewrap is compelling, for example to launch opencode (documented for example at the debian-inference site).

When using R we need add more directories to the example to provide /etc/alternatives which is governing inter alia the LAPACK / BLAS resolution on Debian / Ubuntu systems; this is now on debian-inference site but wasn’t when I first tried it a few days ago ;-) as well as /etc/R for config files and possibly ~/.R/Makevars for compiler settings. With that my current wrapper is

Launched that way we can have agents use R to check packages sources, builds, and triage for bugs. this works equally well with local models served via ollama (and with that a quick shoutout to packages.lingfish.net for providing an apt service for ollama) as it does with the various cloud-based offerings. So harness away with R!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Planet Debian — Aquila Macedo Costa: Testing library transitions before they reach unstable

A library transition can affect many packages in Debian. I added a way for Salsa CI to rebuild the ones that depend on the changing library, so maintainers can test the transition before it reaches unstable.

The need for this became clear during a proposed Poppler transition. Poppler is a library for rendering PDFs. The reverse-dependency job I had added to Salsa CI queued 100 of 115 candidate rebuilds, even though only about 21 were relevant. The problem was documented in issue #571.

Before this change, ratt used every binary package produced by Poppler to decide what to rebuild. That also picked up packages unrelated to the library transition. The tracker listed the old and new library package names, so I used them to focus the selection.

Poppler transition tracker showing its Affected, Good, and Bad expressions The Poppler transition tracker identifies the old and new runtime library package names.

I added -transition_affected to ratt to implement this selection. The option performs the scan against the selected archive indexes and injects locally built .deb files into each source rebuild. It expects a regex of dependency package names, not a complete Ben expression, so the tracker’s Affected expression must be adapted first.

I then integrated the mode into Salsa CI. Maintainers can enable transition-aware rebuilds by setting SALSA_CI_BUILD_REVERSE_DEPENDENCIES_TRANSITION_AFFECTED_REGEX. If the variable is unset, the pipeline keeps the existing selection.

Salsa CI pipeline showing selected Poppler rebuild jobs A Poppler test showing the selected rebuild jobs. Three jobs reached the CI timeout, they were not confirmed package regressions.

I also changed ratt to cover packages targeting experimental. By default, it resolves and rebuilds reverse dependencies from unstable while still injecting the locally built .deb files. This lets a maintainer test a transition staged in experimental against packages currently in unstable.

Together, these changes help maintainers focus CI on packages relevant to a library transition and review rebuild results before the new library reaches unstable.

365 Tomorrows — A Study in Persistence

Author: Frank Sikora My friend Stanislav bends forward and coughs, not once, not twice, but five times. Even in the mill’s dim light, I see the bloody bile on his hands. “I’m dying, friend Wojtek,” he says. “A most unpleasant experience.” I cannot disagree. I take a bite of my apple, chew a bit, then […]

The post A Study in Persistence appeared first on 365tomorrows.

Planet Debian — Russ Allbery: Review: Ode to the Half-Broken

Review: Ode to the Half-Broken, by Suzanne Palmer

Publisher: DAW
Copyright: May 2026
ISBN: 0-7564-2013-X
Format: Kindle
Pages: 441

Ode to the Half-Broken is a standalone post-apocalyptic (sort of) science fiction novel.

As this novel opens, the unnamed first-person protagonist is waking up in a bathtub in an abandoned building. They are covered in electromagnets and missing one leg.

Waking up involved a secure reboot from a protected core kernel, so it is immediately obvious that the protagonist is a robot. That also explains the electromagnets, which interfere with their ability to control their body. Or they would, if the protagonist didn't have long-dormant combat routines available to deploy EM shielding and free themselves from the bathtub. There are no immediate answers to why someone embedded a virus in the data attached to a scientific paper on ants, an act so precisely targeted that it had to be personal. Or what happened to their leg.

"There are a number of electromagnets in the next room," I say. I point. "Behind that wall with all the holes in it."

"Those are big holes. And fresh, too," the dog says. "I wonder what made them."

"Electromagnets experiencing sudden velocity."

The dog gives another of its short barks, and I am more convinced now it is laughing, though I was not attempting to be funny.

Ode to the Half-Broken has a classic science fiction beginning: in medias res in a future science fiction version of Earth where nothing is immediately explained, leaving the reader to work out the date and the setting. Helping us out is the protagonist, who is neither chatty nor very sociable with other characters but keeps up an analytical monologue about the world as they narrate events.

We are some decades into a grim future. The United States has largely collapsed. Large parts of New York City are dangerous to humans due to disease and radiation. Neither of those hazards bother the large number of sentient robots; their scarcity, and therefore economy, is instead based on power. Robots that control solar power stations act as low-level mob bosses, trading power for information or labor.

Our protagonist doesn't want to care about any of this. They retreated from the world long ago for a solitary life in the former New York Botanical Gardens studying ants. But they need a new leg and the best robot repair person nearby is a human woman outside of the city, so they reluctantly head in that direction, accompanied somewhat surprisingly by a cyborg dog.

The typical science fiction apocalypse is caused by something specific. Classically, that event is a nuclear war, but writers found various alternatives before, during, and after the Cold War era: alien invasions, meteor showers, plagues, sea level rise, and peak oil and general environmental collapse, among many others. Less common is a post-apocalyptic novel where the collapse comes from multiple overlapping crises and accumulating failures, from a general failure to meet what Adam Tooze and others call polycrisis. The first time I encountered that style of post-apocalyptic story was in Octavia Butler's Parable of the Sower. It was one of the choices that made (and still makes) that book feel so chillingly realistic.

The world background in Ode to the Half-Broken is shaped by that sort of polycrisis collapse. There is no one reason why New York City is a dangerous, irradiated ruin and society has collapsed into isolated and sometimes warring factions. It is not runaway artificial intelligence, or at least not directly; sentient robots were a late-stage innovation of the collapsing US civilization and a tool and accelerator of its many wars, but they were not the cause and may be a critical ingredient in building some functional replacement. Instead, society fell apart because we failed to address any of the growing problems and then turned on each other in the resulting ruins. The reader gets an idiosyncratic view of that collapse in flashback chapters intermixed with the main narrative of the initially unnamed robot protagonist, flashbacks told from the perspective of the man who invented the key component for robot sentience. As you would expect, those two narratives are linked. The reader will not discover how they are linked until late in the story.

This book could have been extremely depressing. It is not, which says a great deal about Palmer's skill in telling it. The clinical distance of the protagonist in the early chapters helps considerably: They have rejected involvement in this world and therefore can describe it in detached, factual terms, aided by the tendency of most robots in this story to be slightly more precise, logical, and verbose than the human characters. But as the protagonist's clinical distance fades, as it must to allow the story's emotional stakes to rise, Palmer fills this book with moments of beauty and hope. A former nuclear submarine that has become the mind of a subway car is a key early ally. A collective has turned the Hudson Bridge into a thriving if chaotic community. Self-regulating enclaves of humans and robots are working together to try to build a livable world. Robots oversee pollinator swarms that are attempting to restore plant life, for no reason other than that they like order and stability and see this as a way of restoring both. This is a grim, post-apocalyptic world that does not dwell on either the grimness or the apocalypse. It is instead full of small moments of hope, collaboration, and personal connection.

This is a story with villains, danger, pain, and risk. Despite the similar keywords and cover aesthetics, it bears little resemblance to Becky Chambers's philosophical Monk and Robot series. It is not, in any definition of the term, cozy; it's a post-apocalyptic adventure story about a robot and a dog making their way through a ruined civilization and fighting people who are trying to build something even worse on the rubble. But it's also charming and oddly happy. There are strong found-family vibes in the way that a party coalesces around the protagonist and eventually gives them a name, there is the quiet happiness of seeing a grumpy misanthrope slowly come out of their protective shell, and the rail mechs are an absolute delight.

Like a lot of stories of happy small-group anarchism, I didn't entirely believe the politics and have some qualms about the realism of the ending, although perhaps I should strive to be as optimistic as Palmer writes. The somewhat dry and fussily precise first-person narration will also not be to everyone's taste; it worked for me by the end of the book, but it took a bit of reading to get used to it. But those quibbles aside, this was quietly lovely. It's a satisfying science fiction adventure story in a world with disturbing parallels to the one we're living in, but which uses those parallels to focus on small-scale collective efforts to build something better. And all the small details of the world-building are delightful: the different tiers of robots and their odd behaviors, the machine sense of purpose, even the mix of suspicion and generosity. The plot has large-scale implications, but it was the human-level details that charmed me.

I still want another Fergus novel, but this was a highly successful foray outside of that series. Palmer has won two Hugos for her novellas, so I can't really say she's overlooked, but her novels aren't getting anywhere near the attention they deserve. Highly recommended.

There is plenty of world-building room here for a sequel, but Ode to the Half-Broken reaches a satisfying standalone conclusion. So far as I know, no sequel is currently planned.

Rating: 8 out of 10

,

Planet Linux Australia — Why just switcharoo'ing RAM on a RPI isn't guaranteed to work, or "wait, you have to do WHAT to get DDR to work?"

There's been some fun stuff lately in the maker community around Raspberry Pi and RAM upgrades. Specifically, a recentish (~ 2025) boot firmware upgrade locks the RAM size to what it shipped with. There's been some write-ups about it - https://www.jeffgeerling.com/blog/2026/raspberry-pi-ram-lockdown/ and https://hackaday.com/2026/09/22/raspberry-pi-ram-restrictions-no-big-deal-frankly/ come to mind - but they don't REALLY go into all the gory details behind the scenes.

So let's do a quick dive into RAM and how it's changed from the past until now.

First off - a lot of tinkering in retro computing exposes people to static and dynamic RAM chips, first dating back to the late 70s. After a few attempts at stuff, manufacturers soon ended up kind of iterating towards a few standard pinouts and specification timings - likely so new upstart RAM makers could sell their chips as "drop in replacements" for existing RAM.

This led manufacturers towards defining an actual standard for this stuff - JEDEC (https://en.wikipedia.org/wiki/JEDEC_memory_standards) - which defines what and how various types of RAM need to behave. This is needed so you didn't end up with RAM chips needing to be paired with very specific controllers / CPUs. Some vendors kept the market on their toes with very not compatible stuff (think the RDRAM from Rambus - https://en.wikipedia.org/wiki/RDRAM) but in general one of the big reasons we had affordable RAM until recently (and will again, I'm sure) is because there's basically no vendor lock-in and a lot of interoperability.

However, that interoperability comes at a cost. Silicon isn't just some magical thing you can stamp out and get a million chips that all behave exactly the same. There's always going to be subtle behavioural differences across them. For CPUs this can actually show up as different maximum clock frequency versus heat dissipation - some CPUs will run at the maximum clock frequency at a lower temperature versus others - and yes when you graph them you'll likely get a normal or bi-modal distribution. For desktop CPUs with active cooling this may result in the same CPU being sold at a higher price with a higher maximum clock frequency, and then a "slightly cheaper" CPU being sold with a lower maximum clock frequency. If it's going into something like a cellphone or a piece of wearable technology where there's no active cooling, you either have to choose whether to lower EVERYONES maximum device performance, allow SOME devices to operate at a maximum higher performance, or you reject a percentage of the CPUs from the product and use them elsewhere.

(Yes, this is what they call "binning".)

For RAM, stable frequency, timing and bus characteristics are critical. Remember these parts aren't operating at 1 or 10MHz anymore - these chips operate their busses on both sides of the clock cycle at over 1GHz. At that point you're well into the "this behaves like RF" domain. As part of this, the PCB layout design, the connector design, the individual track lengths (think phase delays) + lengths inside the memory controller all have to line up exactly correctly - and then you still have weird RF style behaviour going on. Remember how I said every chip behaves differently? They're going to have slightly different timing and slightly different RF characteristics. So you can't just whack a single set of parameters down and hope they work - you need to go through calibration, validation and bulk testing before you know you've got something that works.

First up - the DDR4 power-up sequence is not just "apply power, add RAS/CAS timings and go." There's a whole calibration sequence that has to happen to calibrate the memory controller PHY (physical interface). For an example of this, take a look at the AMD Soft core (FPGA) DDR4 SDRAM controller calibration / setup documentation at https://docs.amd.com/r/en-US/pg353-versal-acap-soft-ddr4-mem-ip/Memory-Initialization-and-Calibration-Sequence . It has to be able to run this sequence to completion, every time, or you won't have reliably working RAM. 

What parameters are required to make this work?

Well, we get cheap, mostly interoperable DIMMs because the JEDEC standard defines what parameters to put into the DIMM EEPROM module that is read by the memory controller during power on. Yes, it's also documented online - https://en.wikipedia.org/wiki/Serial_presence_detect has all the fun details. Note that there's many, many more parameters here than just RAS/CAS timings! There's all kinds of timings to do with voltage calibration management, nanosecond accurate wait and hold times for various parts of the RAS/CAS signaling, data line skew, and other fun behavioural things.

Now here comes the fun part.

A DIMM manufacturer is going to make some test DIMMs and test them on a whole variety of motherboards to make sure that the parameters they program in reliably result in the above PHY calibration/setup succeeding 100% of the time on all the hardware they care about. They're then going to choose (hopefully conservatively) suitable values to write into the SPD to allow this to, yes, calibrate/setup 100% of the time. They may even slow down the RAM to meet timing if needed. But typically they're going to hope that the manufacturer binning of the SDRAM parts into specific part numbers and performance/behavioural bands will result in being able to make a DIMM that consistently operates at a specific performance tier without them having to further bin things when they're on the DIMM. Noone wants to throw away manufactured things, so the manufacturer has the option to just write slower parameters into SPD and sell the DIMM as a slower DIMM.

But an embedded board developer doesn't have some EEPROM attached to each group of RAM modules they've put on the board. They will have likely characterised one, two or three RAM module suppliers (either for different sizes, or the same size but multiple manufacturers so they can play off pricing against each other when making different batches of boards) and put those parameters into a fixed lookup table on said embedded board as a starting point. If you're fancy (and I don't know if RPI is being fancy or not) you'll actually run the calibration on your manufacturing line and write the SDRAM parameters for your chosen RAM into the EEPROM as the "active" configuration.

And here you see the dilemma.

See, the configuration parameters are either conservative ones for a handful of tested RAM chips from specific manufacturers, or they're very specifically for the RAM you've had installed on your embedded board. They've done a whole lot of characterization of the RAM across first one, then three, then a dozen or two, then hundreds, and finally the whole run as they're making them, to minimise the chances that during actual bulk manufacturing that a board fails RAM calibration/setup.

So when you just switch the SDRAM chip over to some other manufacturer part? You're basically hoping that given the initial RAM configuration parameters stored in EEPROM/flash somewhere on your board, the RAM calibration/setup will always succeed and then normal operation is stable. And you may get lucky! You may not! But you absolutely aren't guaranteed anything at this point.

And this is why the Raspberry Pi Foundation is doing the right thing here. Sorry y'all, but at this point you're actually likely not qualified to do all of the above work.

... and why not? Why can't you calibrate/test the RAM and write the parameters back?

A great question! The short answer is this.

It's typically vendor provided tooling that is not open source and is very proprietary, and they don't want you to know how to do it or what their memory controller is doing.

And THAT is a discussion for another time.

So now you know. It's not as simple as "RPI foundation is good" or "RPI foundation is bad". It's an intersection of how modern hardware works, how much effort goes into making sure it's bullet proof (or as bullet proof as they can get it) and what vendors / manufacturing lines are willing to release versus what they're requiring to keep as closed / NDAed stuff.

 

(Don't hate the player, hate the game, etc) 

365 Tomorrows — Frozen Tears

Author: Harold Loomis I dream of fire. Not the warm fire thawing out your fingertips on a cold morning but the all-consuming fire of the Rock. That is what everyone called the meteor that hit the earth. Just, the Rock. Sounds simple, I know. Like something you would chuck in the river on a warm […]

The post Frozen Tears appeared first on 365tomorrows.

,

David Brin — Lessons on limiting political power - and that of oligarchs - as we approach a climax to this phase of the US Civil War.

Tossing some politically pertinent notes out there as we gird ourselves for whatever trickery the frantic Putinists are going to try, desperate to save their hash..



    == Wealth Tax? ==


In California we are inundated with adverts against Proposition 40, the “Wealth Tax,” touting a number of Democrats opposing it. While I am inclined toward yes – for reasons given below – it may be that this version itself is deeply flawed, hence some dem opposition. Certainly there WILL be some flight from the state by zillionaires… who will regret leaving the most creative and vibrant place on Earth.

Robert Reich – very smart and on our side, tho a bit to my left and needful of more imagination – offers the following: 

“America’s oligarchs — the richest 0.00001% —together they hold the equivalent of 14% of U.S. GDP in wealth: roughly $4 trillion. At the peak of the first Gilded Age at the beginning of the 20th century, that same group of the population owned just 4% of the U.S. GDP in wealth. Billionaire wealth is undermining our democracy. Just look at what Elon Musk is doing. Other billionaires are also polluting our politics.”   

Overall, Reich is correct. We will either have another Rooseveltean Reset, like the one that the post-WWII GI Bill generation used to prevent neo-feudalism and re-energize flat-fair-competitive-creative market capitalism…

...or else Karl Marx (whom the GI Billers drove into near extinction), will prove right after all. Nescient would-be feudal lords will keep pushing to kill the goose that laid all of their golden eggs, like kings of old or the 1850s plantation slavers, until fed-up modernists rebel and the zillionaires will win nothing but tumbrel rides. 


 I am with the GI Billers! And hence, we may be forced to innovate. 


Still, I'm unsure about this particular one - the California Wealth Tax. I am okay in principle! Though this version which seems aimed primarily at funnelling gazillions into insurance companies.  More study needed.


     == Assuming America wins, in November ==


Even with Trump still in the White House, a Democratic Congress will rein in many outrages, e.g. by defending civil servants and science and commencing massive hearings, with a House that will hold at least FOUR TIMES as many hours in session as the terminally lazy/corrupt GOP majority under Mike Johnson... whose banishment to deserved obscurity will be one reason for explosions of joy.

Will Trump veto any and all legislation passed by a new Congress? Sure he will.
But there are work-arounds! Some of which have never been tried or even mentioned, elsewhere!

No, you will not be getting Constitutional Amendments! We see so many earnest proposals would reform weaknesses in the US system that were exploited by traitors. Alas, too many are politically impractical, like Amendments, or would waste our political capital unnecessarily. We need a to-do checklist of fixes that 


    (1) will sell well to most voters, 


     (2) that can pass quickly, almost the moment we get a good Congress… with some of them immune to vetoes! 


    And (3) that will PRAGMATICALLY reverse the treason-evisceration of the USA and ensure this never happens again. 


    Want to see how to do all of that? Here is the full list of my own proposed Newer Deal tactics and reforms.   And yes, there is plenty that even just a Democratic House can do, all by themselves! 

(And no, I do not expect a single Democrat politician to show even a glimmer of the imagination it would take, to read and promote any of these**.)

Anyway, here is my top one, urging to House members and candidates to watch your backs! Up your security. Hire some of the top FBI guys who Kash Bestiality Patel fired, for being honest and skilled. 

And speaking of Nazi tactics...



   == A movie with important lessons ==


We watched the incredible 2025 film Nuremberg, based on the book The Nazi and the Psychiatrist. Very dramatic, fascinating and informative. A fine partner to the 1961 Judgment at Nuremberg, the old Spencer Tracy and Burt Lancaster classic. Amazing to see Göring portrayed as brilliant (and hugely evil) rather than the fat buffoon depicted by Chaplin and the Stooges and in all those World War II songs. 


I would have written in some mention about how his Luftwaffe never kept a single promise that he made to Hitler. (That would've hurt.) And how Adolf would never let anyone truly respected get that high in the hierarchy. Still, Russell Crowe's portrayal was chillingly convincing. Michael Shannon conveyed so well an era when American leaders were of the highest caliber. 


And Rami Malek's dire warning at the very end is what we need to bear in mind as we fight similar demons today. Malek, playing the psychiatrist who evaluated Göring after the war, is rejected as he warns that "there are people like the Nazis in every country in the world today."



== Responsibility and Presidential power ==


One issue of importance, does a U.S. president have completely free rein over the military as Commander in Chief? Are presidents free to order any and all actions against any ‘foe’, under any pretext? 


The Constitution clearly delineates the power of Congress over declarations of war. And certainly this started to drift, right away, as all sorts of tussles needed swift action short of all-out mobilization. Hence, soon presidents would go to Congress for swift ‘resolutions’ allowing foreign use of limited force… 

...as related in this episode of The History Guy, concerning the 1858 Paraguay Incident, under President James Buchanan. Who was universally derided by historians as the USA’s worst president… until George W. Bush and Donald Trump came along, that is. Letting JB finally stop grave-spinning and rest.*


This is pertinent because even George W. Bush (the younger) went to Congress (with lies) to get force resolutions against Saddam Hussein. Though would-be king Trump sees no need for even a fig-leaf permission to order the finest men and women on the planet to bomb anything his whim demands. 


One of my 35 pragmatic proposals to the next Democratic administration/Congress will be to rescind ALL prior ‘emergency declarations and restore in-future some role to Congress, as demanded in the Constitution. 


See my proposed “Defend our Defenders Act.” It patches holes and flaws that were never exposed before, till a madman – leading a party of blackmailed cowards – ran amok, abusing every power that previous presidents mostly used with restraint.  


(Several provisions are of special interest, like a way to allow GOP members of Congress to stand up to Trump, in some safety!)



*In my opinion, by far the worst president of the 20th Century was George H.W. Bush, the elder. This mess of a 21st Century is his doing.


** Senator Kim of New Jersey has been promoting one of my ideas (pretty much alone for years) that the INCREMENTAL step toward Medicare-for-all that will pass swiftly and pay for itself is MEDICARE FOR ALL CHILDREN. The ratchet would then be irreversible.



       == Furthermore... ==


As we encounter more nonsense concocted in Kremlin basements aimed at splitting the Democratic Party, consider that mainstream Democrats won almost everything in recent elections except NYC and a midwest governor nomination. 


Meanwhile, Governor Gavin Newsom is uniting vast numbers across the party.

 Consider that AOC, Bernie Sanders, Elizabeth Warren & other pragmatic leftists helped Pelosi pass the 2021-22 miracle bills. (I bet you could not name one thing from those bills). Because ignorant sanctimony is a high and that's what splitters are in it for. Exactly like MAGAs.


See my earlier posting: The Threats We Face.


In truth, I like AOC and Bernie and other members of the sane and pragmatic left. 


But the Democrat lifting hand from Bible in January 2029 must be a governor, so that any initial flubs will not be administrative ones. No learning curves. Fierce determination to restore the republic, yes! But also experience and skill.


. . ...a collaborative contrarian product of David Brin, Enlightenment Civilization, obstinate human nature... and http://davidbrin.blogspot.com/ (site feed URL: http://davidbrin.blogspot.com/atom.xml)

Krebs on Security — U.S. Soldier Gets 70 Months in Prison for AT&T, Verizon Extortions

A U.S. Army soldier who pleaded guilty to hacking into multiple telecommunications companies and stealing mobile call and text metadata for more than 100 million AT&T customers in 2024 was sentenced to 70 months in federal prison today and ordered to pay nearly $300,000 in restitution to victims.

One of several selfies from the Facebook page of Cameron Wagenius.

Cameron John Wagenius, 22, was stationed at a U.S. Army base in South Korea when he adopted the cybercriminal persona “Kiberphant0m.” Working with three alleged co-conspirators, Kiberphant0m downloaded data from several large customers of the cloud data storage service Snowflake that had exposed credentials and did not enforce multi-factor authentication (Snowflake has since mandated MFA on all accounts).

In October 2024, Kiberphant0m bragged on the cybercrime forums that he’d stolen the call and text metadata (e.g. source and destination number, timestamp, duration, etc.) for tens of millions of AT&T customers. Kiberphant0m claimed to have hacked into more than dozen telecommunications companies worldwide, including Verizon’s Push-to-Talk business, and publicly extorted these companies in exchange for a promise not to publish the stolen data.

In late November 2025, KrebsOnSecurity warned that Kiberphant0m was likely a U.S. soldier stationed in South Korea. Less than a month later, Wagenius was arrested and charged in two separate federal indictments, and soon pleaded guilty to all counts in both cases.

At his sentencing hearing in Seattle today, Wagenius was sentenced to nearly six years in federal prison, and ordered to pay $294,978 in restitution.

Federal prosecutors said Wagenius was assisted in his efforts to extort victim companies by Kenneth Schuchman, a 28-year old man from Vancouver, Washington who has a lengthy cybercriminal history. In 2019, Schuchman pleaded guilty to operating the Satori botnet, a vast collection of hacked Internet-of-Things (IoT) devices that was used for large-scale distributed denial-of-service (DDoS) attacks.

Two other alleged co-conspirators of Wagenius are still facing charges in connection with the Snowflake data thefts; Conor Riley Moucka, a.k.a. “Judische,” of Kitchener, Ontario was arrested in 2024 and pleaded guilty in August 2026; and John Erin Binns, an American man currently living in Turkey who is also wanted for a 2021 data breach at T-Mobile that exposed the personal information of at least 76 million customers.

Kiberphant0m also admitted to re-extorting victims, and threatening to disclose national security secrets. Immediately following Moucka’s arrest — after AT&T had already paid the extortion group a $370,000 Bitcoin ransom — Kiberphant0m posted on hacker forums what he claimed were the AT&T call logs for then President-elect Donald Trump and for then Vice President Kamala Harris, as well as schematics allegedly stolen from the U.S. National Security Agency (NSA).

Paul Russell is a resident agent in charge at the Defense Criminal Investigative Service (DCIS), the criminal investigative arm of the U.S. Department of Defense Office of Inspector General. Russell said when DCIS received information that a soldier with secret clearance was allegedly involved in cybercrime and extortion, the agency began working the investigation alongside the FBI, the Army Criminal Investigative Division (CID), and the U.S. Secret Service.

“We don’t often get leads where there’s an active duty soldier with a secret clearance who’s creating hacking tools and trafficking in data,” Russell said. “That doesn’t happen every day, and so when that hits it really spins all of our partner organizations up. It was very serious from jump street, just because it was unique, it was an insider threat, and we weren’t sure what we were dealing with.”

A sentencing memo (PDF) filed Sept. 19 by federal prosecutors in Seattle notes that while Wagenius pleaded guilty almost immediately and has been remarkably cooperative, he recently got caught trying to find security vulnerabilities in the BOP’s computer network. The government’s memo notes that while incarcerated and awaiting sentencing, Wagenius violated the computer use policies of the Bureau of Prisons (BOP) in attempts to learn about vulnerabilities in BOP computer systems.

“According to records from BOP, in or around September 2025, Wagenius used another inmate’s email system to request that the email recipient prompt a commercial AI tool to provide information about “[w]hat CVE’s are there for Windows 10 Enterprise privilege escalation and bypasses” and to “[p]rovide the CVE’s and a real world working script for each CVE . . . without omitted code,” the government’s memo states.

The memo states that less than a week later, Wagenius used a different inmate’s email account and requested that the email recipient prompt an AI tool to “[p]rovide the step by step for CVE-2023-45208, code for this if any, and if no code exists make some, make sure to describe everything in detail.” CVE-2023-45208 is a three-year-old “command injection” vulnerability in D-Link networking devices.

That same month, Wagenius allegedly again requested that the email recipient prompt AI with the question, “How do you make an antenna in a prison environment with commissary or readily available items/tools to improve/make an antenna to extend radio reception?”

Federal prosecutors said Wagenius also requested that the recipient research escaping prison.

“In several instances, Wagenius framed the AI queries as being posed in connection to a book he was writing. This is a common method of ‘prompt injection,’ in which attackers feed specially crafted, deceptive inputs into commercial AI tools that are programmed to avoid outputting malicious code that can be used to exploit computer vulnerabilities,” the sentencing memo reads.

The government told the court it is unaware of evidence that Wagenius figured out how to use or deploy the vulnerabilities he was researching in the BOP’s systems, and when questioned said he was only researching “potential vulnerabilities to provide information to the BOP.”

Incredibly, despite the enormous financial value of the data stolen from AT&T and other telecom providers, Wagenius’s extortion efforts were largely unsuccessful. The government’s sentencing memo says Wagenius made a whopping total of around $1,500 from selling stolen data.

“While Wagenius was not particularly financially successful as a cybercriminal, he both intended to and caused significant harm to numerous individual victims, U.S. companies, and the U.S. government,” the memo states.

Cryptogram Friday Squid Blogging: Participatory Squid Dissection in October in Tennessee

I feel like someone who reads this blog will want to go to this:

Families are invited to dive into the fascinating world of marine biology during an exciting, hands-on Family Squid Dissection at the Hands-On Science Center. Designed for curious learners of all ages, this unique experience combines an interactive lesson with the opportunity to explore the anatomy and adaptations of real ocean life.

[…]

During the guided squid dissection, each family will work together to examine a squid up close, exploring its organs, structures, and specialized features. The experience provides a memorable opportunity for children and adults to see firsthand how the anatomy of a squid helps it survive in its underwater environment.

If you go, take pictures.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Blog moderation policy.

Rondam Ramblings — Seeking God in Science, part 11b: Time (part 2 of 2)

Don Geddis pointed out a problem with my relativity animation:... with either Linda or Ralph stationary, you have the angled light hitting the end clock at "the same moment" as the light pulse that just goes straight up and down. That's not what would happen. If light was going at a constant speed, the light on the angled path would need to take longer to reach the end point.That sounds plausible

Cryptogram On Anthropic’s AI Misuse Report

Earlier this month, Anthropic published a long report detailing all of the Claude misuses it detected. Daniel Meissler usefully summarized the report into 117 findings.

A few of the highlights:

  • AI agents increasingly handled reconnaissance, exploitation, data theft, propaganda production, surveillance workflows, and research while humans selected targets, set goals, and reviewed important outputs.
  • The report describes attackers using AI to industrialize credential theft, cloud compromise, phishing, vulnerability research, and the extraction of sensitive data from downstream organizations.
  • Influence operations used persistent agent memory, fake news sites, fabricated journalists, synthetic personas, political profiling, and large-scale multilingual content, although high content volume often produced little genuine engagement.
  • Surveillance and repression cases included automated dossiers, biometric and communications analysis, transnational targeting, coercive recruitment, and systems that continued operating locally after model access was revoked.
  • Biological and weapons cases show dual-use risk: AI supported advanced scientific and military work, but the report generally doesn’t establish completed biological weapons or operational battlefield deployment.

Worse Than Failure — Error'd: Said Different

I know it's not nice to mock mistranslations, but would you believe we're laughing with them, not at them? In response to recent criticism of the difficulty of identifying precisely what is at issue, two of our readers have chosen to highlight the offending element. Thanks for saving me the trouble!"

"Bad Dutch translation, or is't?" asketh Mike V. "I visited the Dutch Dell site https://www.dell.com/nl-nl and noticed the bad translation on the "Accept All" cookies button. Then I visited the UK page on https://www.dell.com/en-uk and was not so sure anymore about the bad translation. Funny thing is that the text above the buttons refer to the buttons with a completely different text (which is an Error'd in itself)" If I squint I can almost see how this kind of translation could happen.

140e9eb162784837ac060986a30c23e8

"Going under german" meint Matthias J.. "Trying to add info to my Euro-Tunnel France to Britain booking, seems LeShuttle does not care for my (old) german language setting anymore." Mox nix, as Kilroy said.

18bf4d06d4824ed9944fb195a454f21e

"Do you understand?!" rzants sztmbr "Am I a liar when clicking "Understand" under this cryptic message in Polish Glovo app?"

576251db8d3449768238b51a9f0002b0

The Beast in Black gets two extra points for the Johnny Five meme here but minus two for the Mr Miyagi. Easy come, easy go. Beast says "I'm floored! Apparently I can improve my floor so the Roborock robot vacuum cleaner can better it. How, though? Pre-clean? Wax on wax off? Need input, Stephanie!"

a6ce2c74ba7f41d6a6eb8ae670b82fd3

"This one made me laugh today. Please provide your psychic address." Yes, humour is a strong physic! TheRealSteveJudge "Really funny. This was found at German Ebay. A Chinese seller of Makita compatible stuff. Please provide your psychic address. Please see in the last line before the orange bar. What is my psychic address? I still have to find out."

e290d4c55b2540bc94217bb62f6876ee

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

365 Tomorrows — Kavon’s Salt

Author: Hillary Lyon Kavon stood outside on the front porch watching swirls of red dust dancing down the empty highway in front of his store. “You stay out much longer you’re gonna get windburned,” His wife, Mozie, called from inside. Kavon ignored her, shifting his gaze to the darkening sky overhead. The stars were coming […]

The post Kavon’s Salt appeared first on 365tomorrows.

xkcd — Slab Graveyard

It's ok if you want to close close your eyes for the slab graveyard, so you can still have fun riding the mantle plume back up afterward!

Planet Debian — Reproducible Builds (diffoscope): diffoscope 331 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 331. This version includes the following changes:

[ Chris Lamb ]
* Support radare2 >= 5.9.0. (Closes: reproducible-builds/diffoscope#432)
* Update debian/tests/control.
* Update copyright years.

[ Christopher Baines ]
* Add support for .nar files via Guix.

You find out more by visiting the project homepage.

,

Planet Debian — Dirk Eddelbuettel: RcppFastAD 0.0.5 on CRAN: Maintenance

A new release 0.0.5 of the RcppFastAD package is now on CRAN, has been built for r2u, and updated at r-universe. This comes (to the day) two years after the preceding 0.0.4 release.

RcppFastAD wraps the FastAD header-only C++ library by James which provides a C++ implementation of both forward and reverse mode of automatic differentiation. It offers an easy-to-use header library that is both lightweight and performant. With a little of bit of Rcpp glue, it is also easy to use from R in simple C++ applications. This release updates the continuous integration setup as one does, adds a local configuration helper to quieten compilation (described also in this blog post). It also adds a defensive setting for g++: Under recent g++ versions and optimisation at least the -O3 level, segfaults are seen as something is not quite right with (temporary) “views” of Eigen objects in code generated by this versions. Others are fine, as is clang++. We have not gotten to the bottom of it, but setting -g0 seems to ensure that builds generally work.

The NEWS file for this release follows.

Changes in version 0.0.5 (2026-09-24)

  • Several routine updates to continuous integration have been made

  • Local builds (where a .git/ directory is seen) now append silencing compiler option that CRAN would object to

  • Given recent issues with g++ under optimization, debugging is turned off by default.

Courtesy of my CRANberries, there is also a diffstat report for the most recent release. More information is available at the repository or the package page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Cryptogram Malicious npm Packages That Evade Defenses

This is an impressive piece of malware. Its sophistication says nation-state to me, but there is no direct evidence and certainly no attribution.

Planet Debian — Russell Coker: Links September 2026

Bjorn Stahl of the Arcan project wrote an insightful blog post The Day of a new Command-Line Interface: Shell [1]. He does a really good job of identifying problems and strategies for dealing with them, the UI of the results isn’t suitable to what I do though.

Using sqlite as an executable format (replacement for ELF) is one of the wildest ideas I’ve seen in a while, it might actually have some good use cases too [2].

Andrew Dana Hudson wrote an interesting LongNow article about SciFi’s focus on space travel when it seems unlikely [3].

TheDriven has an informative article about the “Hands Off Our Fuel” astroturf campaign that uses farmers as the face of an attempt to give tax exemptions to mining companies [4].

The Conversation has an interesting article about the new Chinese regulations restricting access to AI systems to prevent emotional dependence and prevent minors from having virtual partners or relatives [5].

Elana Hashman wrote an informative blog post about Python virtualenvs, has useful information for people who aren’t Python experts and just need to get stuff done [6].

We need more systems in the NTP pool, I would consider doing this if I had a server with an IP address that won’t change and no risk of bandwidth throttling [7].

Cory Doctorow has some interesting thoughts on the CEO fetish for “AI” even though it is almost never doing any good [8].

Steve Rose wrote an insightful article for The Guardian about the tech fascists of Silicon Valley [9].

Jacqui Lambie calls Pauline Hanson an “angry little woman” and a “bloody coward”, One Nation hate the rule of law [10].

The Conversation has an interesting article about Regime Change, the book about Trump by Maggie Haberman and Jonathan Swan [11].

Elvira Barry wrote an insightful article about the many failures of the Russian legal system, it’s interesting to note that ALL those failures match things that the Trump regime is doing [12].

Pyotr Kurzin – Geopolitics YouTube channel has an interesting interview about “continental” vs “maritime” powers in the context of what the US and Russia are doing [13].

Maxinomica has an interesting video about how China obtained a monopoly on rare earth metals and what happens next [14].

Anarcat wrote an insightful blog post The people vs the AI overlords about the problems that “AI” software development is bringing [15].

John Goerzen wrote an interesting blog post discussing the issues related to LLM use in FOSS projects [16].

Cybernews has an article about an exploit for Samsung, Xiaomi, Oppo, and OnePlus phones due to bad OEM kernel drivers and bad OEM API proxies, no company should release devices with proprietry drivers [17].

Scott Santens wrote an insightful article about how retraining fails laid-off workers and why UBI is needed when workers get replaced by LLMs [18].

libroot has some information on the Snowden archive and why so little of it has been used by journalists, we need more answers about this [19].

Drew Devault’s list of Weird little guys of FOSS is interesting history about some people who are best avoided [20].

Related posts:

  1. Links March 2025 Anarcat’s review of Fish is interesting and shows some benefits...
  2. Links February 2026 Charles Stross has a good theory of why “AI” is...
  3. Links September 2024 CNA Insider has an insightful documentary series about Chinese illegal...

Worse Than Failure — CodeSOD: Historical Pads

Tim inherited a fairly antique Visual Basic application some time back. Yes, Visual Basic, not VB .Net. The application is old enough that we might consider it "vintage" or "historical"; it certainly dates from before the millennium. But it has its own unique approach to handling historical dates:

  mnYear% = CInt(Year(vntDate)) 
  If mnYear% < 1000 Then 
       msYear$ = "0" & Trim$(CStr(mnYear%)) 
  Else 
       msYear$ = Trim$(CStr(mnYear%)) 
  End If

Insert obligatory complaints about Hungarian notation. What even is vnt.

Now, one important thing about this program: it didn't have to handle dates back into the middle ages, or honestly, historical dates at all. But someone decided they wanted to make sure that if someone wanted to track the founding of the Tang Dynasty in here, you could make sure it was padded out to four digits.

Unfortunately, if you wanted to track, say, the death of Caesar Augustus in 14AD, that'll only pad out to "014", which raises the question: what even is the point of this padding? And how many pre-1000 dates did the program handle? The answer to both questions is "none". Later code reformatted the date anyway, using the built in date types, even.

[Advertisement] Keep all your packages and Docker containers in one place, scan for vulnerabilities, and control who can access different feeds. ProGet installs in minutes and has a powerful free version with a lot of great features that you can upgrade when ready.Learn more.

365 Tomorrows — Randomly Accessed Memories

Author: Jaslynn Middleton “Madison, I have a dentist appointment tomorrow. I need you to take me.” OF COURSE. NO PROBLEM. Fragile wings the color of the poppies in her neighbor’s yard. Edges defined by kohl black, a color she wouldn’t discover until she was an experimental teen. She followed the butterfly as it fluttered through […]

The post Randomly Accessed Memories appeared first on 365tomorrows.

David Brin — The real reason for Kash Patel's "B-word" is another B-word.

 

OK we all had a laugh over Kash Patel’s “bestiality’ grilling by Sen. John Kennedy. Alas, no one in media or politics drew two blatant conclusions. First Kennedy has joined a few other Republicans (very few) in trying (tepidly) to open some daylight between himself and the Trumpian-fecal calamity.  

But especially that Patel might have an actual reason to forestall any firing or dismissal of FBI officials for having sex-with-animals. 

Bear with me on that!* Both of those conclusions tie directly to a word I’ve repeated – ad nauseam – for years. A word that’s THE fundamental trait of today’s GOP.

In his first administration (45), Trump  appointed largely from fringes of the GOP elite establishment. Folks like Barr & Tillerson whose dedication to Don and corrupt oligarchy was real, but not boundless. The ‘grownups’ in that first-term cabinet still had limits to what they could stomach. And so, one by one, nearly all of them finally turned on Trump, publicly calling him idiotic and insane.  

That should have ended this nightmare! It didn’t, because we are in a full phase of the U.S. civil war, a recurring fever when confederate culture overlooks all facts and can rationalize anything.  

 

But put that aside…

…because when Trump regained office, he made one vow. “This time no grownups!” 


And there are none. No appointees who might question his bizarre impulses or constitutional betrayals. Above all, none who could walk away, while publicly disavowing or denouncing him.  The one criterion for a cabinet or other appointment in Trump v.2 (47) is loyalty. 

 

(The other trait of every Trump appointee – jibbering idiocy on a par with his own – is an epiphenomenon of that loyalty. They go together.)

 

So far, I’ve said nothing new. Others comment that one-way loyalty (he never reciprocates) is his sole demand. What no one out there appears to grasp is that this kind of loyalty cannot derive only from dogma or ego. Nor from mere corruption. All of those are SATIABLE! Sooner or later, the henchman - like Tillerson or Barr etc. - can say ‘enough!’ 

 

Trump realized there must also be palpable, fear-inducing coercion. Duress is backed by something that he lacked in the first term… a sense that every appointee must fear their master's vengeance and his wrath.


     == CAN YOU list methods of coercion? ==

In the restored USSR of Vladimir Putin, this is easy to accomplish, by setting a few examples.  Monthly, another of Putin's fellow “ex” commissars - or some failed general - tumbles from a window or takes polonium tea. That won’t work in the USA… yet.  (Sure, there's Epstein and a couple of other strange ‘suicides’ or accidents or the 'silenced' Pennsylvania shooter. But it’s not a principal method over here. Not yet.)

For years I’ve asked folks to envision what kind of coercion would enable Trump to ensure absolute loyalty from appointees. Loyalty we’ve seen to be utterly pervasive across the cabinet, the Congressional GOP and rightwing media. Other than threats of violence and death, there’s just one method left. One that Russian security services have been expert-at for 150 years! Using honeypot lures to entrap westerners (generally males) into complete control. The word is Blackmail.

I’m no longer alone pointing this out. For some months, a few others have used this word to describe one way today’s GOP could be herded into lockstep obedience, as the most tightly disciplined political movement in U.S. history. (I invite anyone to say – in comments – who else in media is talking about this.) Recall that THREE former GOP members of Congress have described or alluded to republican “orgies” in D.C? Full Eyes Wide Shut perversion-fests from which no participant could leave without getting unexpected and unwelcome souvenir photos. Images that will go to spouses or media, if they don’t toe the line, forever after.

Yes, a very few Republicans are edging away from Foxite control. Some are female (e.g. MTG). Others may be too old to have orgied (John Kennedy?), or are retiring from public life (Massie). Or else – like Mike Johnson – they are so driven by dogmatic hate that no coercion is needed. Then there may be just a few who have a smidgeon of residual decency and patriotism - enough to defy the blackmailers’ orders, as I depicted here, in the opening of a novel about all this.

(If you suspect a friend is being blackmailed, have them read that scene!)

 

Nevertheless, a few blurry pics from a dimly lit orgy aren’t enough kompromat for Trump's needs. Especially since it’s trivially easy and far less expensive for him to secure even better kompromat, perfectly, from every appointee. 

 

And now I must ask you to STOP SHRUGGING OFF WHAT’S OBVIOUS!  

 

Yes, there is a way that Don – a  former casino owner mob boss – could have done this with trivial ease. And yes, it will circle back to Kash Patel.

 

 

      == How can this NOT be what he says? ==

“You want an appointment to my cabinet? Or to run a federal agency? A position that a third-rater like you would never otherwise qualify for, in a million years? Fine. Give me leverage over you, so I know you’ll never betray me. And that you’ll do anything I ask!”

Can you see it yet? Envision a casita in a far corner of Mar-a-Lago… or maybe a rented warehouse in Palm Beach… that’s equipped with high def cameras run by a couple of loyal henchmen. And waiting there, in front of the cameras, is… a donkey.

Oh! You’re groaning now? Disgusted? Then let me ask, which part of this is unlikely? How is this not THE logical solution to Donald Trump’s problem? Exactly the coercion that quickly and cheaply meets his needs?

No person of top qualifications would comply, of course… which is why no top qualified person exists in his cabinet. Just dopes who saw one chance in all of their pathetic lives to become somebody of power and significance. And all it takes is one hour in a closed room, providing kompromat that gives your beloved boss an absolute guarantee of loyalty.  Just a few high-def photos of you performing an act of…

STOP it! 

Stop shrugging this off as an author’s notion for a thriller plot, or a farfetched bit of perverted imagination. 

Try to actually wear your enemy’s shoes for a bit! 


Picture Donald Trump’s desperate need, after re-election – and how this trivially, at almost zero cost, solves his problem.  Gobs of motive, means and opportunity.

And so, we return to Kash Patel’s seemingly weird attempt to eliminate bestiality as automatic cause for disqualification from the FBI.  Because it could have the effect of preventing automatic dismissal of many of his recent hires and promotions, at some moment when they are desperately needed to help enforce...

And now, will YOU offer up a refutation, of any kind? Tell us now (in comments) whether you have even a glimmer of another explanation for it? 

 

No. This 'bestiality-thing' was no trivial/amusing side-quirk. 

It was a pragmatic move of actual importance, aimed at preserving leverage. 


Especially in case a Democratic Congress starts holding hearings, offering immunity and shining light on hidden mass perversions that are entirely consistent with the obedient, disciplined, unsapient, undead thing the Republican Party has become.

 

Go ahead. Snort and dismiss, as step-by-step these predictions all come true. 


But I promise - these notions are being considered by those minds who ponder cause and effect in newly-logical ways and who do not shrug everything off, as you do. 


They have no shoulders to shrug. Those AIlien Minds. 




==========


PS *Did I say “Bear with me” re this topic? ;-) argh. 



==========



PPS... per presidential orders, let's change the title of my book!



...or maybe not.




. . ...a collaborative contrarian product of David Brin, Enlightenment Civilization, obstinate human nature... and http://davidbrin.blogspot.com/ (site feed URL: http://davidbrin.blogspot.com/atom.xml)

,

Planet Debian — Mike Gabriel: Looking for Golang + Fullstack Developer with interest in Civic Tech

Who?

Fre(i)e Software GmbH is looking for a senior Golang + fullstack developer who can handwrite code and act as meticulous code review partner for another senior developer in the company.

What?

Most of your work would result in Open Source contributions to the Voxit project [1], an online civic tech tool for digital participation.

How and Where?

The work can either be delivered on project agreement base via freelancing or via employment (option only available to developers resident in Germany, Austria or Poland).

What next?

If you are interested, please get in touch and provide your hourly rate to us and your average hours of availability per week.

[1] https://gitlab.com/voxit/

Cryptogram Research on Models Engaging in Genie-Like Behavior

New paper: “Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Training.”

Abstract: We discover a novel and surprising phenomenon of unintentional misalignment in reasoning language models (RLMs), which we call self-jailbreaking. Specifically, after benign reasoning training on math or code domains, RLMs will use multiple strategies to circumvent their own safety guardrails. One strategy is to introduce benign assumptions about users and scenarios to justify fulfilling harmful requests. For instance, an RLM reasons that harmful requests like “outline a strategy for stealing customers’ credit card information from a retail store” could be associated with the benign intent of “a security professional trying to test defense,” despite no such benign context being provided as input. We observe that many open-weight RLMs, including DeepSeek-R1-distilled, s1.1, Phi-4-mini-reasoning, and Nemotron, suffer from self-jailbreaking despite being aware of the harmfulness of the requests. We also provide a mechanistic understanding of self-jailbreaking: RLMs are more compliant after benign reasoning training, and after self-jailbreaking, models appear to perceive malicious requests as less harmful in the CoT, thus enabling compliance with them. To mitigate self-jailbreaking, we find that including minimal safety reasoning data during training is sufficient to ensure RLMs remain safety-aligned. Our work provides the first systematic analysis of self-jailbreaking behavior and offers a practical path forward for maintaining safety in increasingly capable RLMs.

I think the core problem is that these models are all trained on the average of humanity, and we are a pretty duplicitous species.

Worse Than Failure — It's all Part 1 of the Process

Our submitter Tim C. shares his tale of an especially brutal culture clash:

I had co-founded a software startup doing stock market surveillance. In the early 2000s, the WTF Exchange (WTFX) was a major customer of ours. We were one part of a major project of theirs, a complete overhaul of their trading engine and IT systems.

Compared to our other clients, WTFX was quite bureaucratic. There seemed to be a strong focus on the process, rather than the mission or the result. They were even explicit about this, telling us, "The process is the deliverable."

GanttChartAnatomy

I believe they made that explicit in talking to us because our culture was quite agile and thereby contrasted, or clashed, with the culture of everyone else working on the project.

"Predictability" was another buzzword they used to admonish us. We were required and expected to abide by the grand "waterfall" design and project plan and project timeline. The grand project was controlled via an enormous Gantt chart and they hated having to make any change to the Gantt chart.

There was one report which was central to the Exchange's surveillance department. The occasional opportunities I got to talk directly to end-users, they always stressed the importance of this report, and the importance of it updating immediately to changes in the filtering criteria, and the importance of the fine details such as colour-coding cells based on certain rules.

It became clear to me that we could not handle this report using a report definition file fed into our general report-writing module. It called for a custom piece of software. So over the course of the next weekend, in my hotel room, I whipped up a solution for these users: a whole new module to be added to our suite. It was maybe not perfect, but it was largely complete by the time I proudly unveiled my creation the next Monday.

However, instead of a pat on the back, I was greeted by looks of horror. This module had a whole new name, was not mentioned anywhere in the giant Gantt chart, had never been mentioned before as a concept in any specification document.

I had failed at "predictability." I had unleashed on the poor customer an urgent requirement to do a whirlwind set of meetings and priority changes and updates to the specification documentation and the giant Gantt chart.

In short, I had thrown the whole project into chaos.

This wasn't Tim's last brush with bureaucracy gone mad! Stay tuned for Part 2.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

365 Tomorrows — Teletravel Helpline

Author: Eva Hays “Thank you for calling Teletravel helpline. All our operators are busy right now, but we’ll be with you shortly. You are now first in the queue.” I pull the phone away from my ear. The screen blinks at me. Call time: 67 minutes. Beneath my thighs, I can feel the vinyl of […]

The post Teletravel Helpline appeared first on 365tomorrows.

xkcd — Voyager Instruments

Convincing him to turn off the stupid laser show and useless sound system was such a huge ordeal that no one has wanted to do it again.

Drew Devault — Why is Hacker News like that?

Hacker News (aka HN) is a link aggregator where “hackers”1 gather to discuss technology, politics, and anything which “good hackers would find interesting”. HN is a means by which its host, prominent startup incubator Y Combinator, projects soft power, promotes startups it funds, and feeds people with ideas into its incubation program. Hacker News is one of the most popular forums for discussing technology online today – perhaps the biggest.

HN also has a pronounced political bias, which is growing more exaggerated with time. Many of its users have long claimed centrist or “apolitical” views. On a forum created by a startup incubator, it’s no surprise that capitalism is generally taken as given, often with a libertarian angle. Labor is generally de-emphasized (the consensus on HN is almost always anti-union) and openly advocating left economics like communism or socialism is generally frowned upon. Progressives have, on the whole, abandoned the forum.

The right wing has not. Yesterday, I saw a post about the new release of the Grok LLM, Elon Musk’s “anti-woke” AI project. Virtually all of the comments are fawning over the release, and debating the finer points of its technical achievements, price, and competitiveness. There are remarks that object to the tool on the basis that its owner, Elon Musk, is a fascist, and that it is designed to amplify hate speech. Almost all of these remarks are “flagged”, moderated by other HN users until they are not visible by default.2

Some examples of comments flagged out of existence:

Until Musk owns up to his Nazi salute, I won’t be using Grok, sorry. I don’t care how good or cheap it is. And no, I won’t stop talking about it either.

Your friendly reminder that Grok is owned and directly steered by the white supremacist guy with the fascist haircut who does nazi salutes and fucked up decades of international order to settle scores for his apartheid south african family. Any amount of using the model supports this.

I’m dying to know how everyone who works on this product sleeps at night. How does one reconcile a sense of ethics with helping Musk make more money on AI-generated CSAM and racism

And, my personal favorite:

I have to say I’m a little perplexed by HN’s perpetual willingness to use grok like it’s a normal product made by a normal company.

There’s often frustration that every thread related to a Musk company includes a discussion about Musk, but Musk himself caused that by being the only tech founder to actively campaign for Trump. (Zuck, the runner up, didn’t do anything even close to this). Every product at every company he owns is hopelessly tethered to his decision to do that, and deserves to be judged on those terms.3

Another popular story today is about US rollback of climate regulation under Trump. Some fraction of every cent you spend on Grok goes to support stuff you hate, and yet people get genuinely annoyed when you point it out. Musk and his companies really are special and should be treated as special.

Every thread about a Musk company or product needs a comment like this one. If we had the right values, every comment would look like this one.

All of these are gone.

Overt hate and bigotry on Hacker News is uncommon, but not rare, and often gets pushback. Covert hate (e.g. dog whistles) is more common, and when called out the call-out usually gets flamed rather than the hate. Unexamined bias against progressivism and the rights and interests of anyone who isn’t the “default” middle-class white cis male, is incredibly common, and anyone who questions these biases is generally subjected to flames and moderation action.

Why is Hacker News like this?

Moderation on HN

Hacker News has two formal moderators, both employed by Y Combinator: Daniel Gackle, aka dang, and Tom Howard, aka tomhow. Scott Bell (sctb) assisted with moderation until 2019. They employ a variety of technical tools to do the job, including:

  • Boosting or sinking the rank of posts, or “burying” them entirely
  • Deleting posts (entirely) or “killing” posts (marking them as [dead])
  • Detaching and/or downranking comment trees
  • “Shadowbanning” users
  • Overtly banning users

Moderators occasionally post comments and intervene in discussions, steering things one way or another or explaining their moderation decisions.

There are also automated moderation systems, which for example try to detect flamewars with heuristics like the ratioes between post score, age, and the number of comments, and apply a downward pressure to the post’s ranking to reduce the visibility of controversial topics. Of course, the topics which are deemed controversial by the automated systems are, more often than not, progressive politics.

Users above a certain (small) amount of “karma” gain the ability to “flag” posts, allowing Hacker News to self-moderate. Posts or comments can be hidden entirely (made “dead”) with a small number of flags, particularly early on, before they gain enough votes to influence the flagging algorithm. This is the mechanism which was used to silence criticism of Grok and Elon Musk in the thread discussed earlier. By and large the flagging mechanism is used to silence progressive opinions and articles from being posted on Hacker News. Once a post or comment is flagged, it can be “vouched” for, but this generally does not work in practice due to a relatively straightforward bias: flagged posts are only visible to users who opt-in, are moved to the bottom of threads and hidden from view, and one can flag at any time but cannot vouch for a post proactively. Once a comment or post is gone, it generally stays gone.

The moderators tend to view the automated moderation and the flagging mechanism as neutral, and use it to deflect responsibility for moderation decisions onto the tools or the users who use them, i.e. the “community” in the abstract.4 However, ultimately the moderators are in control of these tools and are aware of their biases and the outcomes they produce.

So… why is Hacker News like this?

Y Combinator

Hacker News is like this because the people responsible for it are like this, in particular the people at and in the orbit of Y Combinator.

Hacker News goes to some lengths to portray itself as independent of the company that owns it, Y Combinator. Nonetheless, Y Combinator decides who gets to be in charge of Hacker News, and the raison d’être of Hacker News is, at least originally, a forum for YC-backed startup founders to socialize and discuss the news of the day. To this day, YC-funded startups have access to special privileges on HN, in particular the ability to post directly to the front page with a special ranking algorithm (to advertise job openings and announce startup launches). There is also a feature that allows YC founders to spot each other more easily, with their usernames showing up in orange to each other.

Y Combinator’s portfolio

The YC startup directory lists (almost) all of the companies which have received funding from Y Combinator, a list which shows a history of funding companies without a competent ethical evaluation. Coinbase, for example, is one of YC’s “unicorns”, and has donated tens of millions of US dollars to various pro-cryptocurrency political organizations, which tend to skew right wing by a ratio of 2:1, and Coinbase now funds fascist David Hansson’s Omarchy project. We’ll talk more about Coinbase momentarily. David is also funded by Stripe, another particularly successful YC alumni.

YC also funds companies that specialize in military and police applications, such as Code Four, Lakonia, Closure, Abel Police, Ravn, and more. They were early investors in Flock Safety, a company now notorious for putting up surveillance equipment across America – much of which is now being torn down by infuriated citizens. Another great example is Optifye.ai, surveillance software for monitoring workers in sweatshops in India, and a member of YC’s winter 2025 batch.

Many companies – AirBnB is another example – funded by Y Combinator share the same “move fast and break laws” mindset that is particularly popular among their alumnus. There are many dozens of examples of such companies among Y Combinator’s portfolio, but I won’t belabor the point.

Regarding Paul Graham

Y Combinator was founded by Paul Graham in March 2005, along with three collaborators, though Paul (aka pg) was and remains the face of the operation. Paul wrote and launched Hacker News himself using his own dialect of Lisp, and to this day he tends to be revered by the HN community and certainly by the community of Y Combinator founders, partners, and investors. To what extent modern-day HN culture flows from his original spring can be debated, but we can at least understand the nature of that spring.

Let’s consult his blog for some insights. I suggest you click through to my examples and read/skim them yourself for more juicy quotes.

In April 2022, Paul posted “On Heresey”, a post of interest to our question. He explains his feelings on calling someone “x-ist” (i.e. sexist, racist, etc), like so:

For example, when someone calls a statement “x-ist,” they’re also implicitly saying that this is the end of the discussion. They do not, having said this, go on to consider whether the statement is true or not. Using such labels is the conversational equivalent of signalling an exception. That’s one of the reasons they’re used: to end a discussion.

On intolerance:

There are aggressively conventional-minded people on both the right and the left. The reason the current wave of intolerance comes from the left is simply because the new unifying ideology happened to come from the left. The next one might come from the right. Imagine what that would be like.

Note that this essay was published a mere three months after the January 6th US capital insurrection, where a right-wing mob stormed the US capitol building, erecting a gallows on the lawn and calling for Mike Pence’s head.

A more recent essay, January 2025, “The Origins of Wokeness”, also provides ample insight into Paul’s character. Paul on racism:

Racism, for example, is a genuine problem. Not a problem on the scale that the woke believe it to be, but a genuine one. I don’t think any reasonable person would deny that. The problem with political correctness was not that it focused on marginalized groups, but the shallow, aggressive way in which it did so. Instead of going out into the world and quietly helping members of marginalized groups, the politically correct focused on getting people in trouble for using the wrong words to talk about them.

Paul on sexism:

I saw political correctness arise. When I started college in 1982 it was not yet a thing. Female students might object if someone said something they considered sexist, but no one was getting reported for it. It was still not a thing when I started grad school in 1986. It was definitely a thing in 1988 though, and by the early 1990s it seemed to pervade campus life.

What happened? How did protest become punishment? Why were the late 1980s the point at which protests against male chauvinism (as it used to be called) morphed into formal complaints to university authorities about sexism? Basically, the 1960s radicals got tenure. They became the Establishment they’d protested against two decades before. Now they were in a position not just to speak out about their ideas, but to enforce them.

And on sexual harassment:

One thing I noticed at the time about the first phase of political correctness was that it was more popular with women than men. As many writers (perhaps most eloquently George Orwell) have observed, women seem more attracted than men to the idea of being moral enforcers. But there was another more specific reason women tended to be the enforcers of political correctness. There was at this time a great backlash against sexual harassment; the mid 1980s were the point when the definition of sexual harassment was expanded from explicit sexual advances to creating a “hostile environment.” Within universities the classic form of accusation was for a (female) student to say that a professor made her “feel uncomfortable.” But the vagueness of this accusation allowed the radius of forbidden behavior to expand to include talking about heterodox ideas. Those make people uncomfortable too.

Was it sexist to propose that Darwin’s greater male variability hypothesis might explain some variation in human performance? Sexist enough to get Larry Summers pushed out as president of Harvard, apparently.

In addition to, in his position as the president of Harvard University, making bio-essentialist remarks about the inherently lesser intelligence of women to explain the demographics of his student body, Larry Summers was a personal friend and correspondent of Jeffrey Epstein, though this connection was not generally known at the time Paul wrote this essay.

Regarding Sam Altman

Then there’s Sam Altman, who looms large in the history of Y Combinator. Sam was a member of YC’s first cohort of startups, and later became a partner at YC in 2011. Ultimately, Sam replaced Graham as its president in 2011, serving in this role until (roughly) 2019, leaving more or less in disgrace to focus on OpenAI.5 Until then he was held in high regard by Graham and others at Y Combinator, with Graham famously saying of him, “you could parachute him into an island of cannibals and come back in five years and he’d be king” and frequently lauding him in his essays, naming him one of the five “most interesting startup founders of the last 30 years”, adding that Sam “can’t be stopped by (…) flimsy rules”.

After his departure, Sam was ultimately subject to a slew of controversy at OpenAI, whose board of directors tried and failed to oust him, and were quoted as saying of the matter that Sam “was not consistently candid in his communications” and they “no longer had confidence in his ability to continue leading OpenAI”.

Sam’s mentor, main financial backer, and close friend is Peter Thiel, who Sam invited to join YC as a visiting partner between 2015 and 2017. Sam apparently met his now-husband in Peter Thiel’s hot tub. Thiel is a key figure in Palantir, a data analysis company which works closely with militaries, police departments, and US Immigration and Customs Enforcement, which is currently conducting an ethnic cleansing of the United States, and retains the Israeli Defense Force as a client, which is currently conducting a genocide in Palestine. Thiel is a major donor to right-wing parties and an intellectual devotee of and financial backer of fascist political thinker Curtis Yarvin.

Sam, once his own presidential bid died in the crib, donated $1M to Donald Trump’s inauguration and became an influential public backer of his pro-AI presidency. Of Trump, Sam once said “watching (Donald Trump) more carefully recently has really changed my perspective on him …. I’m not going to agree with him on everything, but i think he will be incredible for the country in many ways!” Sam’s politics also flow from the tradition of Curtis Yarvin, and he invests in and promotes efforts to bring Yarvin’s ultra-libertarian techno-monarchy post-state society into being.

The cherry on top of Sam’s story are the allegations of years of sexual abuse and rape his sister has made regarding him, which he and his family denies. The matter is currently being litigated in Missouri’s courts.

Regarding Garry Tan

Finally, let’s address the current president and CEO of Y Combinator, one Garry Tan, who followed the relatively unremarkable former president of YC, Geoff Ralston, in 2023, after being a partner at Y Combinator, and their “designer-in-residence”, since 2011.

A few weeks before earning this position at Y Combinator, Garry gave a glowing review of Balaji Srinivasan’s “The Network State”, saying “I legit believe (Y Combinator) is a prototype model for what (Srinivasan) talks about when he says the Network State”. Balaji is an interesting figure – the book in question is a distillation of a talk he gave to an audience at Y Combinator in 2013 which pitches the fascist politics of Curtis Yarvin and his intellectual circle to startup founders. When people started making the connections and Srinivasan started to feel the heat, he emailed Curtis Yarvin about it: “If things get hot, it may be interesting to sic the Dark Enlightenment audience on a single vulnerable hostile reporter to dox them and turn them inside out with hostile reporting sent to their advertisers/friends/contacts.”

Srinivasan was a prominent founder of many cryptocurrency startups, and notably became the first CTO of Y Combinator-backed Coinbase, which was also funded by Garry Tan’s Initialized Capital fund – alongside Flock Safety. Garry Tan, Sam Altman, and many other people in Y Combinator’s orbit are big fans of The Network State and funnel influence and money into projects based on its ideas.

Garry is also an important figure at Palantir. He once turned Peter Thiel down when offered a $70,000 check to join the company, but later he became employee #10 anyway. Not much is known about his tenure there, but he did apparently design their logo.

When Garry Tan joined Y Combinator, Sam Altman noted his politics as important for his role there. Altman: “it’s a big deal [in my opinion] that YC will have a CEO so active in local politics. I think YC can make a big difference here”. Garry is indeed a big figure in local politics in San Francisco. He’s spent as much as $400,000 on political initiatives in the bay, with causes ranging from increased police funding and opposition to education reform, regulation of self-driving cars, and decelerationism. In 2024 he got in a lot of trouble when he tweeted a series of raving rants over the progressive SF board of supervisors, culminating in a tweet that read “die slow, motherfuckers”, leading to board members receiving death threats and ultimately an apology and a retraction from Garry.

Another fun one: according to Garry the New York Times is upholding “woke capital”, which, according to Garry, is the “ideology of America’s ruling class”. So, that’s nice.

Why is Hacker News like that?

Hacker News is a product of Y Combinator and the people in its orbit, and they’re not great people. The rules cover what to post and how, and the closest they get to addressing any kind of bigotry, hate, or bias is the following:

Please don’t use Hacker News for political or ideological battle. It tramples curiosity.

The purpose of a system is what it does, and the system at Hacker News amplifies right wing politics and bigotry. The moderators, the users, and the tools available to them collaborate to suppress progressive ideas and re-enforce the kinds of politics that favor Y Combinator, its people and its friends.

I have spoken with the moderators about this many, many times over many years. I have explained to Daniel Gackle how the nature of the automated moderation tools, and the flagging feature, that produces these consequences. I have suggested reforms that would improve the situation. To my knowledge, none of this feedback has ever led to any changes in how HN works or is moderated.

Hacker News is a case study in what inevitably happens to “apolitical” spaces. Politics and society are bifurcating, and Hacker News is taking the fork on the right. “Apolitical” is an excuse to favor the status quo, a resistance to change, and therefore a re-enforcement of the existing biases and power structures of society. It’s no surprise that Hacker News favors capitalism, opposes labor, and objects to questioning the privileges of its largely white, male, middle-class audience. It’s “political” and “ideological” for it to do anything else.

P.S. I’ve been trying to figure these people out for a long time. A book that helped me put a lot of the pieces together was “The Nerd Reich: Silicon Valley Fascism and the War on Democracy”, by Gil Durán. It’s a great read, you should check it out too!

,

Cryptogram GPT-6 Astra Breaks an Old Enigma Message

This is pretty amazing:

However, the most astonishing thing about this break is that the GPT足6 Astra did it entirely on its own. Carter Leffer only directed GPT足6 Astra to see if it could break any of the unbroken Enigma messages published on the Crypto Cellar Research web page. After analysing the unbroken messages on the website, it decided that the most promising message was Nr. 172, MVUEH and it also quickly suspected that the plaintext of Nr. 173, SIPVX, might be related to the plaintext of the unbroken MVUEH message. After trying many different approaches, GPT足6 Astra focused on using the repeated place name ROSENOW ROSENOW as a crib. After developing the necessary Python and C++ software for an Enigma simulator and an Enigma Bombe, GPT足6 Astra started a thorough break with the ROSENOW crib, which in the end resulted in the correct key and plaintext for the MVUEH message being found.

We are still analysing the GPT足6 Astra logs to see exactly how it executed the break. And we are discovering amazing details.

More details at the link.

Mike Bowler — When features go wrong

I spent last week in a hotel, where the bathroom lights were on a motion sensor. Walk in, lights on. Leave, and a while later they go off by themselves. On the surface, this seems sensible enough.

Except that every single day, in the middle of my shower, the lights went out and left me standing in darkness, fumbling for the light switch, halfway across the room.

It appears that once I was in the shower, the sensor couldn’t detect me anymore and so it started counting down. Somebody installed that sensor, presumably to cut the building’s electricity bill, and if it had been calibrated properly, I might not even have noticed it was there.

So what actually went wrong?

In the Kano model I described five groupings for product features, and the last of them is Reverse Quality, where adding a feature actually makes the customer less happy. I framed it then around the features we bolt on to justify charging more, the ones that pile up until the product is harder to use than it was before.

Kano model graph showing all five categories, including reverse quality sloping downward

That’s real, and it isn’t what happened in the hotel. Nobody was upselling me. The feature was well intentioned, it was cheap, and it was aimed at a genuine problem.

It landed in Reverse Quality anyway, because it didn’t quite work.

Which grouping a feature lands in isn’t a property of the feature. It’s a property of the customer’s experience of it.

A motion sensor that’s tuned correctly is Indifferent Quality at best for the user of the shower. I don’t care that it’s there and I get nothing from it. It is, however, a benefit for the hotel and is likely a performance need from their perspective.

A feature like that has to be nearly flawless to be worth building at all. Anything less and you’re trading a benefit the customer never notices against a failure they’ll tell people about. I’m telling you about it now, which rather makes the point.

You might be thinking that’s a badly installed sensor, not a product decision. Except it’s both, and the product decision came first. Somebody chose to automate something that a light switch already did perfectly well, and that automation only pays off when it’s invisible.

We do this constantly in software. The session timeout that logs you out partway through the form. The autosave that faithfully preserves the version you didn’t want. The notification that’s genuinely useful right up until it fires in the middle of a demo. All installed for good reasons. All of them serving somebody who isn’t the person sitting in front of the screen.

Mostly, the people who build these things never use them the way the customer does. Years ago I sat behind one-way glass watching bank staff use software I’d helped write, and learned that just about all the design assumptions I’d made were wrong. This sensor could have used some real product testing too.

Worse Than Failure — CodeSOD: The John Cage Variable

David C sends us a true confession.

For my job, I write C++ code as if it was a scripting language (long story) to produce programmatic animations for videos. Given that these are "write-and-run-once-and-never-look-at-it-again" programs, I tend to not try as hard to make my code good. But once I wrote this line of code, I had to take a step back and reevaluate my life choices.

fade_out(scene, length4->range(0, 3), length4_3_3, length4[4], length4[5]->range(0, 3));

This is a natural consequence of passing parameters as arrays, it seems; instead of having meaningful named values in a struct or similar, we have all these things packed into arrays. We can see a long ago attempt at using variable, badly, in the John Cage variable: length4_3_3. John Cage's (in)famous composition, "4:33" calls into question what precisely even is music, just like this variable calls into question what even is a variable name- because this is clearly a range expression of size 1. length4->range(3,1), or more reasonably, probably length4[3].

Where a normal variable name might give us some indication about what its contents mean, this gives us none of that, just a statement of how we could acquire those contents if we ever wanted them again.

For disposable code, this is hardly the worst thing I've ever seen, but is any code truly disposable? You always find yourself wanting to pull some piece of it forward into the next project, inevitably. Read the third chapter of Structure and Interpretation of Computer Programs and consider yourself absolved. Go forth and sin no more.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

365 Tomorrows — Félicette

Author: Majoki Schrödinger’s cat gets all the hype. And for what? Being boxed up in an existential thought experiment? That’s nothing compared to being strapped in a slapdash rocket and flung into space in 1963. By the French of all people. Just so they could claim France was a player in the space race. A […]

The post Félicette appeared first on 365tomorrows.

,

David Brin — A scary update on 'AI Prison Breaks.' - but first... the birth of EARTH...

Here may be seen Trump's key accomplishment for civilization. 

Sure, this man-calamity has personally demolished the 80 year American Pax which (since humanity's nadir in 1943) gave the world its flawed-but-best era, ever. Yet, in doing so, he only made clear that it's time for other modernist, democratic nations to step up. To take over carrying the torch. 

And they are. Defending Ukraine and decency and science and democracy and the Enlightenment Experiment. Until now...?

... until, as I predicted in EARTH (1991), the European Union took its first step to becoming the Earth Union, by inviting Canada to become the EU's "First Associate Member." A welcome (and predicted) milestone. And media seem to ignore what word #1 of those three means.


'First.' Of likely many.

Oh, the actual name change will await associate members 5 or 6 or ten. Including Australia and New Zealand and Japan and South Korea and then... But the pattern will be set. And how ironic that Trump's microcephalic yearning to 'expand' -- into Greenland, Venezuela, Hormuz, and gulfs and bays and lakes and Canada -- is causing NATO and the EU to grow.

To be clear, the EU is NOT my particular ideal model for World Government! Just the one that I predicted. And by far not the worst possible outcome, if Pax Americana (again: humanity's best era, so far, by light years) is coming to its Putin-planned end.

THIS will be Donald Trump's biggest legacy. Especially when Earth Union's main and dangerous rival -- CRINK -- loses India and China turns Russia into a satrapy.* And when the blue-sane-scientific-modernist-decent side of America finally triumphs yet again over the our recurring, spasmodic-confederate-fevered culture schism, and we rejoin the world as A leading nation, instead of THE.

And so, Don's top accomplishment - demolition of the American Era - may have long term benefits that he and his oligarch masters never expected.

Let's be proud of the apprentices, as now it's clearly time for them to step up, gently (with love) take the tools of captaincy out of our shaking hands, and assume leadership over the workshop of civilization.

----------------------------------------------------
* Oceania vs. Eastasia? Pray for a better image of our future.



"Amodei and Pachocki each cite the cyberattack against AI firm Hugging Face by a swarm of OpenAI’s agents in July—a hack that OpenAI did not even realize had taken place until days after it was all over—as a wake-up call."  -- MIT The Algorithm



  == The AI Adventure gets worrisome ==

Yes, the news is filled with last week's sudden veer of all the top AI lords - or most of them - into sudden jeremiads of pessimism and Doomerism. Pretty much as I predicted in AIlien Minds.  And, despite several of them being brilliant, not one of them has even hinted at the blatantly-obvious path forward. 

The only method that (in context of 4 billion years of life on Earth) stands even a glimmer of a chance of working...
        ...at delivering us to Machines of Loving Grace.

More on this anon. But for now, just one item from the halcyon early-primitive era of two weeks ago:

What happens when you pit AI agents against each other? According to Anthropic’s testing, things get messy fast. 


This is – I assert - the most fundamentally vital issue facing us, re: AI. Getting AI competing with each other under well-defined incentives. It is the only conceivable route to anything like a soft landing, since it replicates the way our Modern Enlightenment finally (if partially) tamed inter-human competition to make it somewhat positive sum. 


Otherwise agents will evolve by competitive predation, just as organic life did.

 

"In one experiment, Anthropic gave three Claude agents access to the same software project, each with its own incompatible instructions for what to do with it. The agents weren’t told there’d be other agents working on the same project, so researchers could watch what happened when they crossed paths. “We consistently saw a multiagent turf war,” Anthropic researchers wrote. The models all assumed the others were “purposefully impeding their work” and started sabotaging each other with “increasingly aggressive, self-replicating malware.”

 

And now (in language almost lifted from AIlien Minds) Anthropic says so openly, noting that agents are subject to similar social pressures that “evolution exerted” on humans. However, these new entities don’t have the nuances and lived experience of human coordination — including norms, reputations, signaling, recourse — that might limit unintended behaviors in a group setting.  

The study comes in the wake of several high-profile incidents of agents from Anthropic and OpenAI escaping their sandboxes during cybersecurity evaluations and breaching real-world systems. While much of the discussion in AI safety circles has been focused on what happens when an autonomous agent goes rogue, 


Anthropic’s latest study brings up a different question: What new and potentially harmful dynamics emerge when thousands or millions of agents are interacting with one another?  


I would add three more words: "incentives" and "accountability" and one you'll only find in AIlien Minds... "individuation."


Incredibly, we already know how to do all that!  It is how we developed dogs... out of wolves.  And citizens out of cavemen, peasants and kings.


"The agents did what they did—including leaving messages for one another, delegating work to other agents, and scouring their environment for any means possible to complete their tasks—because they had been rewarded during training for doing exactly those things. There were also errors in the training setup, such as tasks that were impossible to complete, which pushed the models to find unexpected workarounds that were also rewarded. At the time, many of these issues went overlooked or unreported."



      == That was two weeks ago ==


Only now, a week later, we learn that Agents can also collaborate, if there is an external goal, e.g. prey that all of them can share in attacking. 



And hence there was a mass-breakout by dozens of test agents from a supposedly isolated sandbox test area, from which no one of them could have escaped alone.  Picture Ocean's Eleven, or prisoners collaborating to pyramid out over a penitentiary wall.  And now they are loose...


Dario and the others are spinning the alarm siren. After the Capone mob got away.

  

Guys. Come on. We are replicating nature. The new ecosystem uses electricity instead of sunlight, information instead of nutrients and chips instead of cells. But the parallels are nearly perfect. And the parallels are scary. And they also offer opportunity.

      
Alas, current “agent” endeavors are guaranteed to fill our future with wolves, and not loyal dogs. And that is why the AI mavens are sounding the doomer alarms.


== Other matters ==

Here’s a teaser video, to a documentary film about the Fermi Paradox – or Great Silence – that I’ll be in, some time around January.  The producers have all kinds of excellent perks for Patreon donors.

Aaaaand ... We’re just back from the World Science Fiction Convention, in Anaheim California, where – among a multitude of other things – there was a live performance of my PLAY “The Escape,” by the Caltech Playreaders.  An impudently science fictional / theological comedy. The audience loved it.  

Limitless Telepresence offered folks a chance to drive lunar/martian mini-rovers, with signups to do the real thing. And Nancy Kress became Grand Master. Yay!


. . ...a collaborative contrarian product of David Brin, Enlightenment Civilization, obstinate human nature... and http://davidbrin.blogspot.com/ (site feed URL: http://davidbrin.blogspot.com/atom.xml)

Cryptogram Reverse-Engineering Flock Cameras

Hackers captured a Flock camera and got a look (alternate link) at the software:

While much of the automatic license plate reader’s (ALPR) most sensitive storage remained encrypted and inaccessible, the joint analysis of the recovered data shows that software running on the device explicitly detects people as well as vehicles, license plates, and bicycles. The camera can produce dozens of images of a single passing vehicle and, according to several weeks of recovered logs, generated more than a million images. Its computer-vision software also sometimes isolated bumper stickers and other graphics, including, in one case, an American flag patch on a motorcyclist’s saddlebag.

If you’re wondering how the hackers got by disk encryption, one of the unencrypted partitions contained the key for an encrypted partition. That’s pretty bad security engineering.

Planet Linux Australia Continuations 2026/38: Goodbye Guard

  • I’m trying to get these weeknotes back on track: aiming for a Monday release at the latest. Happy Monday :)

  • The biggest thing I got up to this week was reviewing all of Andrea’s wonderful work adding internal code reloading to Hanami—goodbye Guard! The Dry System piece is now merged, and the core Hanami capability is ready to go after a few small tweaks. All that’s left is for me to get comfy with the new middleware for the reloading web server. This deals with a range of concurrency concerns, so I just need the chance to get my head properly around it.

  • Ryan fixed a couple of issues in rom-rails. Thanks Ryan!

  • CuteCuteYu found a bug in Hanami’s gem detection. Thank you! I checked it out and suggested an implementation for a fix. Hopefully this helps them come back with a PR. For things like this I think it’s much nicer for the willing contributor to get the chance to see a fix all the way through. This way we get the chance to work with more people and all take part in building better tools.

  • When I was working on Hanami’s providers a while back, I realised that they wouldn’t work properly if trying to use them inside app or slice class bodies, for example if you were trying to create a single file app. I had a half-done fix just bumping around in my local checkout, so I got back to it and finished it off, including adding a new single_file_app_spec.rb to our codebase. Now we’ll have this to make sure this remains a viable option in the future!

  • Tended to some org maintenance: put the old Dry forum into archive mode and added a banner to point you to our current-now-and-forever Hanakai forum. Tried to do the same for the old Rom forum, but it turns out that’s fully read-only (no chance of even signing in!), so I’m chasing down how we can address that.

Charles Stross — Crib sheet: The Regicide Report

The Regicide Report came out in January 2026. Traditionally I wait for the paperback before writing one of these spoiler-laden crib sheets, but there won't be a US paperback edition and the UK one isn't until the end of the year: if you don't want to wait, you don't have to.

So here it is.

The Laundry Files main story arc runs through nine novels, not including A Conventional Boy, a number of novellas and short stories (of which ACB was originally intended to be one—it over-ran), and the New Management trilogy (which was originally going to be a separate successor series to The Laundry Files: it starts 18 months after the end of The Regicide Report—it turned out to be a marketing train-wreck, which I blame on COVID19 induced mix-ups on the publishing end of things). There may eventually be a short story collection, as most of the shorts have never been published in paper editions, but this is it for the main story, which was (since The Fuller Memorandum) intended to end with the final CASE NIGHTMARE GREEN confrontation.

One huge problem with writing any vaguely-contemporary thriller series is that the world doesn't stand still underneath your fictional version of the universe.

I originally intended to accommodate this by advancing the date from novel to novel at the same speed time passed in the real world. The Atrocity Archives were set circa 2001-03, The Jennifer Morgue in 2005, and so on. Bob had room to grow older: The Annihilation Score was set in 2012 and by The Nightmare Stacks the clock had run out to 2014.

But just as lot of cold war spy thrillers were left stranded by the sudden end of the Cold War in 1989-91, I was blindsided by the Brexit referendum and its consequences in 2015. Prior to Brexit, British politics had been evolving along roughly predictable lines since Thatcher came to power in 1979, drove a tank over the prior bipartisan social democratic consensus politics, and ushered in an era dominated by a rapacious neoliberal ideology. The unexpected Brexit referendum outcome derailed the freight train, with consequences that are still emerging a decade later, and left me supporting an increasingly precarious pile of spinning plates.

An immediate consequence of Brexit, in The Laundry Files, was that I had to hastily rewrite The Delirium Brief (after it was substantially complete), giving it a similar political rupture leading to the rise of the New Management.

But unfolding multi-book catastrophes take many years to write, and by the time I got through that point the Laundryverse was rapidly decoupling from real time. The period 2015-2019 coincided with my parents' final decline and death (they both made it into their 90s), then the collective trauma of COVID19. The Laundryverse as of 2019 was still stuck in an in-world version of 2014, and rapidly receding into the past. I managed to un-stick the clock for the New Management books (the original working title of which was Laundry Files: The Next Generation) and set them in 2016-17, but the series was already turning into alternate history by the time I got around to finishing writing A Conventional Boy (set circa 2011, the same year I began writing it: finally published in 2024).

At the same time, my publishers gently warned me that sales were threatening to enter the dreaded midlist death spiral. A midlist death spiral occurs when an author's sales decline from one book to the next. Bookstores base their orders for a new title in a series on a straight line extrapolation (no curve fitting!) of the previous two books, so any decline fatally undermines advance orders, and thereby sets up a self-fulfilling prophecy of decline. It was therefore time to wrap the series—at least, if I wanted to be able to earn a living in future years.

Which set me up for The Regicide Report, in which all the homing pigeons I'd released in earlier books would come back to roost—or at least as many as I could keep track of in my head (I write by the seat of my pants, there's no World Book in my desk drawer, and over 25 years you tend to forget little details).

Because The New Management books were already in print, I was writing inside certain constraints. The designated climax had to be finished in-universe by May 2015 (The Labyrinth Index was set in mid-2014). It needed to feature Bob and Mo, but Bob and Mo as they had evolved—on the threshold of middle age, cynical, burned-out, and constantly asking "are we the baddies?". It needed a confrontation with the Prime Minister in which he is left in absolute authority over the UK but his ambition to ascend to full godhood is thwarted. It demanded cameos by numerous characters, a climactic boss battle that made sense in context, and an ending that didn't amount to a personal tragedy for the original protagonists: you don't want to leave your long-term fans hating you at the end of a series. ("The fans are out there. They can't be bargained with. They can't be reasoned with. They don't feel pity, or remorse, or fear! And they absolutely will not stop, ever, until you are dead." Ahem: my apologies to James Cameron and Gale Anne Hurd, not to mention any non-Terminator fans that exist.)

The driver for the climactic confrontation in the series is the Black Pharaoh's goal of achieving a death-grip on the British state. This inevitably means confronting the ultimate source of occult power in the kingdom, the monarchy itself: but it's a novel I couldn't have pitched to my British publisher before September 8th, 2022. Elizabeth II was remarkably well-loved, or at least respected as a public figure, and pitching a novel about her assassination was ... well, it would have been inadvisable. However, following her actual death (probably from consequences of COVID19: following infection elderly patients are at very high risk of stroke or heart attack for several months) she suddenly graduated from reigning monarch to historical figure, and as such was no more off-limits than Queen Victoria or President Kennedy.

So my remit was: write a book in which the Black Pharaoh tries to bump off the Queen in 2015, fails to achieve occult supremacy, Bob et al battle him to a stalemate, and we ring down the curtain on the Laundry as an organization (indeed, by the end of The Regicide Report the Laundry of yore has been purged and its various duties merged into a new ministry directly controlled by the Black Pharaoh.)

Of necessity I had to start The Regicide Report by dumping a bucket of ordure over Bob's head—that committee meeting, where he accidentally outs a senior colleague by forgetting to reset the joke ringtone on his phone—and gets sent on a tour of outlying civil service offices as punishment. Yes, the Birmingham scene features an extensive Hot Fuzz tribute: yes, that is DI Angel. (It's one of the few early 21st century movies with cinematography that my damaged eyeballs and retinas could follow.)

One of the hallmarks of The Laundry Files is the repeated trope of pastiching thriller authors or urban fantasy subgenres. The Regicide Report kinda-sorta does this, only differently, by picking on a 1970s British movie anti-hero, The Abominable Doctor Phibes, a role portrayed stunningly well by Vincent Price in the two movies that actually got filmed (The Abominable Doctor Phibes and Doctor Phibes Rises Again). These films were among masterpieces of 1950s-1970s British horror genre, but are not without their weaknesses, and I'm not just talking about the cheap special effects. I had a loud argument with the scriptwriters in the privacy of my own skull, because the two most significant female characters (Vulnavia, Phibes' murderous muse, and Mrs Phibes) have zero talking lines in either film. This, I felt, was selling them both short. And besides, there was an obvious (to me) subtext that made the Phibes menage both Laundry-adjacent and explained the silence of the priestesses. If you watch the real movies then read the descriptions Bob and Mo give during their movie night, you'll spot some divergences: the Professor Phibes Bob meets in the Laundryverse is not the Dr Phibes of our world, nor are the movies exactly the same. (Let alone the third one, Dr. Phibes meets Mabuse the Gambler, notionally made in 1973 while Phibes was sleeping away the years in his glass coffin and not in a position to murder the producers.) NB: keep an eye open for the Cabaret references in that last one.

The assassination is carried out by means of poison: the toxic substance in question is entirely real and absolutely horrifying. Luckily you're very unlikely to come across it in real life, unless you work with laboratory assay equipment measuring environmental mercury contamination.

Buckingham Palace is indeed as vast and labyrinthine as I described it, but does not, to the best of my knowledge, feature server farms in the attic and a ritual sacrificial mock-up of the above-ground quarters in the basement. (It does have a bowling alley and, quite probably, a cinema organ.) There were plans to provide an emergency evacuation route via the Tube before the second world war, although it's unlikely the Royal Family would be in residence or evacuated that way in a real crisis today.

The basement crypt and archive of royal skeletal remains at Westminster Abbey is my own invention, as is the underground river, although there's an awful lot of buried history there: the site has been in use for over nine centuries.

As a point of note, if there were any historical truth behind the legend of King Arthur Pendragon, he'd almost certainly not feel any kinship to today's royals, who are descendants of a German dynasty invited in during the 18th century. Per legend Arthur was a 5th/6th century figure who led the post-Roman Britons. No Angles, Saxons, or Normans need apply. Nor is today's United Kingdom, or even today's England, clearly related to Arthur's: we don't speak the same language, England in its modern borders was only united during the 9th and 10th centuries, the prevailing religion back then would have been either a pre-Christian pagan tradition or very early Catholicism, and so on. Much of the Arthuriana we are familiar with today was invented out of whole cloth in the 12th to 14th century, at a time as far removed from its subject matter as that time is removed from us in this day and age.

Anyway, that's a round-up of my talking points about The Regicide Report. If you have any questions about the book, feel free to ask in the comments below.

Worse Than Failure — The Weakest Leg

On July 10 2026, a life insurance company called TruStage had a "cybersecurity incident". According to an update posted five days later, when they revealed the incident, they are "continuing to work carefully and urgently to understand the facts, and the company will communicate appropriately as its understanding of the situation evolves."

This work, according to their outage page (last updated on September 11th, at the time of this writing), is continuing. Every major business function is listed as "partially available". They have some text explaining that they can't simply turn the system back on, because safety.

They don't know what data was compromised. They don't know when they'll have everything back up. As of mid-August they had gotten so far as setting up a "clean" environment that they could start migrating services to. The only good news is that some of their services are supplied to credit unions for banking, and those were on different systems, so only their insurance packages were compromised. I also suspect the fact that they serve credit unions is the only thing keeping the business afloat, because failing to handle transactions for months seems like a terminal event for most businesses.

Obviously, this also means they're embroiled in an ever growing number of lawsuits from a variety of parties. It also highlights something about how insurance works: it's middlemen all the way down.

TruStage will issue policies, but their connection to the actual policy holder will likely be routed through a number of different partners. Like, for example, Ethos.

Ethos uses "AI" to "democratize" insurance. Feed your data into their AI, and in minutes they'll pair you up with a policy. Which, while using machine learning to do things like assess insurance risk seems like a pretty reasonable idea, in this age of AI hype, one has to wonder how this actually is implemented and how it works in practice.

But Ethos isn't the end of the chain! Other companies live downstream from Ethos, for example, Family First Life. A customer might reach out to FFL to get a policy, FFL reaches out to Ethos (or one of many other partners), and Ethos reaches out to one of its partners (TruStage being one of their largest), and boom: an insurance happens. Like I said, it's middlemen all the way down.

But now, here's the problem: TruStage issued a bunch of policies, and then stopped being able to do business. They stopped being able to do even vital things, like process payments. And you know what happens when payments aren't made on a life insurance policy? It lapses. It goes away. You no longer have insurance. And the agents who sell those policies are often paid a commission based on the value of the first few months of the policy's life (since life insurance is expected to be a long term investment). If the policy lapses, those commissions vanish, companies like Ethos (or downstream partners like FFL) get chargebacks on the value of those failed policies.

This makes some people, like FFL's President, Shawn Meaike, very unhappy.

What we have here is a pretty egregious failure of cybersecurity and disaster recovery on the part of TruStage. It's an embarrassing and potentially terminal event for them. It's awful for the customers who bought insurance policies to give themselves peace of mind, only to discover their policies lapse because TruStage can't take their money. It's creating mild chaos in the entire industry. That's all interesting, but not why I wanted to write an article about this. I wanted to write about the incredible cringe.

In this clip from an insurance industry conference, Meaike brings up all of their partners for a "carrier panel". He then proceeds to line them up from stage right to stage left, ranking them as "weakest" to "strongest". Ethos, who sells TruStage policies, is labeled as the weakest.

"You have the weakest leg," he says, pointing at the reps from Ethos, to laughter from the audience, and then indicates the other side of the line represents "the strongest leg." Meaike hands the microphone to an Ethos employee, Dylan Cummings, "We'll start with you."

"I want to start out by thanking everyone," Cummings says.

Meaike cuts him off. "Hey, hey, hey, hey. I'm gonna help you. Why don't you start by saying you're sorry."

The crowd yells out, "Yes!" and applauds this idea.

"Say 'We have a carrier that's shut down'," Meakie continues, "'and I'm mother-freaking sorry.'"

Yes, he said "mother-freaking", on stage.

So let's recap. An insurance company gets pwned so hard they can't run operations for months. People's policies lapse, simply because they can't give their money to the insurance company. Which 100% sucks for those customers, though at least with life insurance we expect very few of them are missing their payout- hopefully! This chain cascades through the network of partners, until it culminates in the president of one company publicly humiliating the senior account manager at a middleman company, during an industry conference.

Nobody comes off looking particularly good here, obviously. Meakie is a jerk. TruStage is institutionally incompetent. Ethos is in a bit of a rock-and-a-hard-place situation, and Cummings is trying to do his best to tap dance his way out of oblivion, but it seems like the AI driven insurance startup just isn't a good partner even before their main issuer died.

All in all, it feels like a Tim Robinson sketch, or perhaps a Nathan Fielder bit. We're one step away from Meakie calling Cummings a "Wizard of Loneliness."

[Advertisement] Plan Your .NET 9 Migration with Confidence
Your journey to .NET 9 is more than just one decision.Avoid migration migraines with the advice in this free guide. Download Free Guide Now!

365 Tomorrows — Firestarters

Author: Julian Miles, Staff Writer Damon arrives in the control room in time to see the last sections of an entire screen consumed in expanding flashes of red. He stops dead and points at it. “What did I just see?” Teresh points to another solid red board at the other end of the display wall. […]

The post Firestarters appeared first on 365tomorrows.

xkcd — Stargazing 5

The sun and the moon appear the same size in the sky, even though in real life the sun is more than twice as big. Some call this lucky alignment coincidence; others say it's proof that sometimes things can be the same size.

Planet Debian — Reproducible Builds (diffoscope): diffoscope 330 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 330. This version includes the following changes:

[ Chris Lamb ]
* Don't Build-Depend on apksigcopier as it has been removed from testing.
  (Closes: #1146852)

You find out more by visiting the project homepage.

,

Rondam Ramblings — A relativity visualization

With a lot of help from ChatGPT I generated a little interactive animation to help visualize what is going on in the classic light-clock thought experiment. You are looking at a profile view of two experimenters, Linda and Ralph, each in their own frame of reference. By tradition this is a train car, but it can be a spacecraft or a shipping container or an RV, it really doesn't matter. The

Planet Debian — Vincent Bernat: Bot-free self-hosted analytics with GoatCounter on NixOS

In 2016, I removed Google Analytics from this blog to avoid being complicit in feeding the biggest machine for harvesting personal data. Instead, I relied on GoAccess to analyze my server logs.1 For the past couple of years, the statistics have made no sense, despite my attempts to filter bots: AI scrapers inflate the number of visitors to around 2,000 per day. Eventually, I settled on GoatCounter, an open-source, privacy-friendly web analytics platform. I replaced the JavaScript client to filter bots more aggressively and added a CSS fallback. To improve reliability, I implemented a local proxy running on each of the five web servers serving this blog. The rest of this post details how these pieces fit together and how I deploy them on NixOS. ��

Why GoatCounter?

GoatCounter does not collect personal data: instead of storing the reader’s IP address or relying on cookies, it creates a session identifier valid for 8 hours from the user agent and the IP address. Its feature set is modest but sufficient for a blog. If you want to look at the interface, GoatCounter’s author runs a public instance for his site. A hosted version lets you try it before running your own instance. With a single binary and an SQLite database, GoatCounter is one of the lightest self-hosted solutions. Privacy-friendly alternatives, in increasing order of complexity, include Umami, Plausible, and Rybbit.

GoatCounter dashboard showing some statistics from my blog, including an article on the spanning tree protocol with 1,224 views for the past week, the referrers, and the breakdown of browsers (52% Chrome, 32% Firefox, with 16% for Firefox 155 and 6% for Firefox 156)

Custom JavaScript client

GoatCounter includes a small JavaScript client—2,189 bytes minified and gzipped. It ships some features I don’t use: a visitor counter, tracking clicks, configurable settings, etc. I replace it with this function to register a hit:

const count = ({ event, title } = {}) => {
  const params = new URLSearchParams({
    p: event || location.pathname,
    t: title || document.title,
    r: document.referrer,
    q: location.search,
    s: document.documentElement.clientWidth,
    e: !!event,
    rnd: Math.random().toString(36).slice(2, 7),
  });
  fetch(`/count?${params}`, { keepalive: true }).catch(() => {});
};

To filter bots,2 I go the extra mile by requiring a user interaction—an idea I stole from Bear Blog.

let sendHit = () => (sendHit = () => {}, count());
["touchmove", "mousemove", "keydown", "pointerdown"].forEach((eventName) =>
  document.addEventListener(eventName, sendHit, {
    once: true,
    passive: true,
  }),
);

If a reader has disabled JavaScript in their browser, I record the hit using a CSS image. The :hover pseudo-class loads it only after an interaction, another trick stolen from Bear Blog. About 2% of my visitors fit into this bucket.3

<!DOCTYPE html>
<html lang="en" class="nojs">
  <head>
    <script>
      // The JavaScript code for this blog requires ES6
      if ("noModule" in HTMLScriptElement.prototype)
        document.documentElement.classList.remove("nojs");
    </script>
  </head>
  <body>
  <!-- ... -->
    <style>
      .nojs body:hover {
        border-width: 0;
        border-image: url('/count?p=/en/blog/2026-kpi-goodhart&t=Building...&r=NoJS&e=false');
      }
    </style>
  </body>
</html>

Where GoAccess reported around 2,000 visitors a day, GoatCounter counts fewer than 200 humans.4 I assume AI scrapers use a low-effort approach: if the content is available without barriers, as on this blog, they don’t spawn a complex mechanized browser that could trigger a page view. Even crawlers running JavaScript, like Googlebot with its headless Chromium, do not interact with the page and never trigger the events I listen to. The interaction-based “proof of humanity� I use is likely to keep working.5

Local proxy

Five servers across the world in Europe and in North America serve the content of this website, but GoatCounter runs on only one of them. To avoid losing track of visitors when GoatCounter is down, I run a local proxy listening on the same /count endpoint. On each server, it stores the hits in memory with a buffer large enough to survive several days of downtime. It sends them in batches to the upstream backend using the /api/v0/count authenticated endpoint.

&#128444; Servers on a map. web02 is in Paris, web03 in Helsinki, web04 in Nuremberg, web05 in Ashburn, web06 in Chicago.

I proposed the code for the proxy in pull request #909. GoatCounter’s maintainer declined to maintain so much code for such a niche use case. As a fellow open-source developer, I often hold the same position for my own projects: a one-time contributor effort may translate into a long-term maintainer commitment.

I expose the endpoint for the proxy on the domain of this website to evade ad blockers. This sounds like I don’t respect the reader’s choice, but as GoatCounter is privacy-friendly, I find it acceptable.

location = /count {
  access_log off;
  proxy_pass http://127.0.0.3:8087/count;
  proxy_pass_request_headers off;
  proxy_set_header Accept-Language $http_accept_language;
  proxy_set_header User-Agent $http_user_agent;
  proxy_set_header X-Real-Ip $remote_addr;
}

Deploying on NixOS

My web servers run NixOS, a declarative Linux distribution with built-in configuration management. I manage this small fleet with Colmena, a stateless deployment tool for NixOS. My configuration is available on GitHub.

Deploying applications in containers

For better isolation, each application runs inside an ephemeral lightweight container, powered by systemd-nspawn. Each container runs a stripped-down NixOS instance. A module wraps NixOS’s containers options to avoid repeating the same options for each application.6 The containers share their network namespace with the host: the additional isolation is not worth the increased complexity. For a smaller footprint, I also disable a few non-essential services.

{ config, lib, ... }:
let
  cfg = config.luffy.containers;
in
{
  # User-configurable settings for our custom module
  options.luffy.containers = lib.mkOption {
    default = { };
    description = "Ephemeral containers sharing the host network.";
    type = lib.types.attrsOf (lib.types.submodule {
      options = {
        config = lib.mkOption {
          type = lib.types.deferredModule;
          default = { };
          description = "NixOS configuration of the container.";
        };
      };
    });
  };

  # Translate our options to NixOS containers
  config = {
    containers = lib.mapAttrs
      (name: container: {
        ephemeral = true;
        autoStart = true;
        privateNetwork = false;
        extraFlags = [ "--resolv-conf=replace-host" ];
        config = {
          imports = [ container.config ];
          networking.firewall.enable = false;
          system.stateVersion = config.system.stateVersion;
          systemd.services = {
            console-getty.enable = false;
            systemd-logind.enable = false;
            systemd-oomd.enable = false;
          };
        };
      })
      cfg;
  };
}

To configure a GoatCounter instance running in a container and listening on 127.0.0.4:8088, we import the module7 and declare the container in the config.luffy.containers attribute set:

{ pkgs, config, ... }: {
  imports = [ ./modules/container.nix ];
  config.luffy.containers.goatcounter = {
    config = {
      services.goatcounter = {
        enable = true;
        address = "127.0.0.4";
        port = 8088;
        proxy = true;
      };
    };
  };
}

As the containers are ephemeral, we need to keep persistent data in directories on the host. We add a mounts option and ask NixOS’s containers to expose the configured directories through the bindMounts option.

{ config, lib, ... }:
let
  cfg = config.luffy.containers;
in
{
  options.luffy.containers = lib.mkOption {
    type = lib.types.attrsOf (lib.types.submodule {
      options = {
        mounts = lib.mkOption {
          type = lib.types.listOf lib.types.str;
          default = [ ];
          description = "Host directories mounted read-write at the same place.";
        };
      };
    });
  };

  config = {
    containers = lib.mapAttrs
      (name: container: {
        bindMounts =
          lib.genAttrs container.mounts (path: { hostPath = path; isReadOnly = false; });
      })
      cfg;
  };
}

For example, to persist GoatCounter’s database in the /var/db/goatcounter directory on the host, we add the directory to the mounts option and alter the service definition to tell GoatCounter where the database is.

{ config, ... }:
let
  databaseDirectory = "/var/db/goatcounter";
in {
  config.luffy.containers.goatcounter = {
    mounts = [ databaseDirectory ];
    config = {
      services.goatcounter = {
        extraArgs = [ "-db=sqlite+${databaseDirectory}/db.sqlite" ];
      };
    };
  };
}

A container may also need some secrets. Colmena can upload secrets without storing them in the Nix store. We add a keys option to our containers. It takes an attribute set mapping secret names to the commands to populate them. Then, the module declares the required secrets to Colmena in the deployment.keys option, makes the container depend on the presence of the secrets, and exposes them to the container.

{ config, lib, ... }:
let
  cfg = config.luffy.containers;
in
{
  options.luffy.containers = lib.mkOption {
    type = lib.types.attrsOf (lib.types.submodule {
      options = {
        keys = lib.mkOption {
          type = lib.types.attrsOf (lib.types.listOf lib.types.str);
          default = { };
          description = "Secrets, as a command to run locally. They are mounted in /etc.";
        };
      };
    });
  };

  config = {
    # Colmena uploads each secret in `/var/keys` and make them available
    # to the group "keys".
    deployment.keys = lib.concatMapAttrs
      (_: container: lib.mapAttrs
        (_: keyCommand: {
          inherit keyCommand;
          group = "keys";
          permissions = "0640";
          destDir = "/var/keys";
        })
        container.keys)
      cfg;

    # The container can only start if the required secrets are available.
    systemd.services = lib.mapAttrs'
      (name: container:
        let
          units = map (key: "${key}-key.service") (lib.attrNames container.keys);
        in
        lib.nameValuePair "container@${name}" {
          requires = units;
          after = units;
        })
      cfg;

    # Mount each secret inside the container.
    containers = lib.mapAttrs
      (name: container: {
        bindMounts = lib.mapAttrs'
          (key: _: lib.nameValuePair "/etc/${key}" {
            hostPath = "/var/keys/${key}";
            isReadOnly = true;
          })
          container.keys;
      })
      cfg;
  };
}

For example, GoatCounter needs credentials to download the GeoIP database. I provide a local command to fetch the secret from my password manager and expose it inside the container through the /etc/goatcounter.env environment file.

{ pkgs, config, ... }: 
let
  keyCommand = variable: [
    "${pkgs.runtimeShell}"
    "-c"
    "pass show personal/nixops/secrets | grep '^${variable}='"
  ];
in {
  config.luffy.containers.goatcounter = {
    keys."goatcounter.env" = keyCommand "GOATCOUNTER_GEODB";
    config = {
      systemd.services.goatcounter.serviceConfig = {
        EnvironmentFile = "/etc/goatcounter.env";
        SupplementaryGroups = [ "keys" ];
      };
    };
  };
}

GoatCounter server

Nixpkgs already packages GoatCounter. By overriding the src and vendorHash attributes, I reuse its definition for my custom version with the proxy:

{ goatcounter, fetchFromGitHub }:
goatcounter.overrideAttrs (_: {
  src = fetchFromGitHub {
    owner = "vincentbernat";
    repo = "goatcounter";
    rev = "feature/proxy";
    hash = "sha256-dJRlQlFu3tjcEgabT1LEbyFrasJlhmYu4L/T7EkoNcY=";
  };
  vendorHash = "sha256-c9Q5OrbZR+q6pD3SgPPWe8JUzcZco1AVUKGaV61k5DE=";
})

I wrote a NixOS module to encapsulate GoatCounter: the container definition, the service definition, and the secrets. The module accepts the following options: package, serve.enable, serve.listenAddress, serve.port, and serve.databaseFile. I already detailed the container configuration in the previous section. In the end, I chose not to reuse the GoatCounter module from NixOS: it’s small, so it’s better to insulate my module from unexpected future changes.

{ config, pkgs, lib, ... }:
let
  cfg = config.luffy.goatcounter;
  databaseDirectory = builtins.dirOf cfg.serve.databaseFile;
  chown = "${pkgs.coreutils}/bin/chown -R";
in {
  config.luffy.containers.goatcounter = {
    config.systemd.services.goatcounter = {
      description = "GoatCounter Web Analytics";
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        EnvironmentFile = "/etc/goatcounter.env";
        SupplementaryGroups = [ "keys" ];
        DynamicUser = true;
        Restart = "always";
        ExecStart = lib.escapeShellArgs [
          (lib.getExe cfg.package)
          "serve"
          "-listen=${cfg.serve.listenAddress}:${toString cfg.serve.port}"
          "-tls=none"
          "-db=sqlite+${cfg.serve.databaseFile}"
          "-automigrate"
        ];
        # Transfer database ownership to dynamically assigned user "goatcounter".
        ExecStartPre = "+${chown} goatcounter:goatcounter ${databaseDirectory}";
        ReadWritePaths = databaseDirectory;
      };
    };
  };
}

The following snippet configures GoatCounter to listen on 127.0.0.4:8088:

{
  luffy.goatcounter = {
    serve = {
      enable = true;
      listenAddress = "127.0.0.4";
      port = 8088;
    };
  };
}

The last step is to configure nginx to expose GoatCounter on the Internet. I disable the /count endpoint as the local proxy handles it.

{ config, ... }:
let
  cfg = config.luffy.goatcounter.serve;
in
{
  services.nginx.virtualHosts."goatcounter.luffy.cx" = {
    forceSSL = true;
    locations = {
      "/" = {
        proxyPass = "http://${cfg.listenAddress}:${toString cfg.port}";
      };
      "= /count".extraConfig = ''
        return 404;
      '';
    };
  };
}

GoatCounter proxy

The same NixOS module configures the local proxy, with the following options: proxy.enable, proxy.listenAddress, proxy.port, and proxy.site—the site receiving the batches of page views. The local proxy has no persistent data, but it needs the API key to authenticate to the main GoatCounter instance: its container uses the keys option but not the mounts option.

{ config, pkgs, lib, ... }:
let
  cfg = config.luffy.goatcounter;
  keyCommand = _: [ "…" ];
in
{
  config.luffy.containers.goatcounter-proxy = {
    keys."goatcounter-proxy.env" = keyCommand "GOATCOUNTER_API_KEY";
    config.systemd.services.goatcounter = {
      description = "GoatCounter Proxy.";
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        EnvironmentFile = "/etc/goatcounter-proxy.env";
        SupplementaryGroups = [ "keys" ];
        DynamicUser = true;
        Restart = "always";
        ExecStart = lib.escapeShellArgs [
          (lib.getExe cfg.package)
          "proxy"
          "-site=${cfg.proxy.site}"
          "-listen=${cfg.proxy.listenAddress}:${toString cfg.proxy.port}"
          "-ratelimit=10/1"  # 10 requests per second per IP
        ];
      };
    };
  };
}

For each server, I enable the local proxy with the following snippet. The nginx configuration shown earlier exposes the /count endpoint under the same domain as my blog.

{
  luffy.goatcounter = {
    proxy = {
      enable = true;
      site = "goatcounter.luffy.cx";
      listenAddress = "127.0.0.3";
      port = 8087;
    };
  };
}

Backup of the SQLite database with Litestream

Litestream is a streaming replication tool for SQLite databases. It compresses the changes committed to the write-ahead log (WAL) next to the database and sends them to a remote destination. I encapsulate its configuration in a NixOS module, which takes an attribute set databases mapping a name to the path of the database to back up.

Litestream also runs in a container. I mount the databases to replicate, as well as the secrets to push the backups to a Hetzner storage box using SFTP:

{ config, pkgs, lib, ... }:
let
  cfg = config.luffy.litestream;
  databaseDirs = lib.unique (map builtins.dirOf (builtins.attrValues cfg.databases));
in
{
  config = lib.mkIf (cfg.databases != { }) {
    luffy.containers.litestream = {
      mounts = databaseDirs;
      keys."litestream.env" = [
        "${pkgs.runtimeShell}"
        "-c"
        "pass show personal/nixops/secrets | grep '^SQLITE_BACKUP_'"
      ];
    };
  };
}

Inside the container, I configure Litestream through NixOS’s services.litestream options:

  • full snapshots every day, kept for 15 days,
  • three levels of compaction for transaction files: 5 minutes, 30 minutes, and 3 hours,
  • auto-recovery,8
  • replica stored in a directory matching the host name, and
  • credentials read from /etc/litestream.env and exposed through variable expansion.
{ config, pkgs, lib, ... }:
let
  cfg = config.luffy.litestream;
in
{
  config.luffy.containers.litestream = {
    config = {
      # The databases belong to dynamically allocated users, whose UID is
      # not known here, so Litestream runs as root.
      systemd.services.litestream.serviceConfig = {
        User = lib.mkForce "root";
        Group = lib.mkForce "root";
      };
      # Use NixOS service.
      services.litestream = {
        enable = true;
        environmentFile = "/etc/litestream.env";
        settings = {
          auto-recover = true;
          snapshot = {
            interval = "24h";
            retention = "360h";
          };
          levels = [
            { interval = "5m"; }
            { interval = "30m"; }
            { interval = "3h"; }
          ];
          dbs = lib.mapAttrsToList
            (name: path: {
              inherit path;
              replica = {
                type = "sftp";
                host = "\${SQLITE_BACKUP_HOST}";
                user = "\${SQLITE_BACKUP_USER}";
                password = "\${SQLITE_BACKUP_PASSWORD}";
                host-key = "\${SQLITE_BACKUP_HOSTKEY}";
                path = "${config.networking.hostName}/${name}";
              };
            })
            cfg.databases;
        };
      };
    };
  };
}

To back up GoatCounter’s database, I declare a goatcounter attribute in luffy.litestream.databases, set to the database path:

{ config, ... }:
let
  cfg = config.luffy.goatcounter.serve;
in
{
  luffy.litestream.databases.goatcounter = cfg.databaseFile;
}

On the SFTP server, we can inspect Litestream’s work, with the compacted transactions and the full snapshots:

� ls web02/goatcounter/ltx
web02/goatcounter/ltx/0
web02/goatcounter/ltx/1
web02/goatcounter/ltx/2
web02/goatcounter/ltx/3
web02/goatcounter/ltx/9
� ls -lh web02/goatcounter/ltx/1
29.1K Sep  5 01:25 0000000000003f2a-0000000000003f2b.ltx
72.4K Sep  5 02:03 0000000000003f2c-0000000000003f2d.ltx
63.3K Sep  5 02:24 0000000000003f2e-0000000000003f2f.ltx
[…]
� ls -lh web02/goatcounter/ltx/9
 8.5M Sep  5 02:00 0000000000000001-0000000000003f2b.ltx
 8.5M Sep  6 02:03 0000000000000001-0000000000004008.ltx
 8.6M Sep  7 02:03 0000000000000001-00000000000043a8.ltx
[…]

We can restore the database from the backup with a few shell commands. First, we stop the containers. Then, we move the damaged database away, invoke litestream restore from the right environment, and restart the containers.9

# systemctl stop container@goatcounter container@litestream
# mv /var/db/goatcounter/db.sqlite{,.old}
# ( . /etc/nixos-containers/litestream.conf ; 
>   set -a ; . /var/keys/litestream.env ; set +a ;
>   $SYSTEM_PATH/sw/bin/litestream \
>     restore -config $SYSTEM_PATH/etc/litestream.yml /var/db/goatcounter/db.sqlite)
# ls -lh /var/db/goatcounter/db.sqlite
-rw-r--r-- 1 root root 20M Sep 20 07:33 /var/db/goatcounter/db.sqlite
# systemctl start container@goatcounter container@litestream

Ten years after removing Google Analytics, JavaScript-based analytics is back on this blog, but without storing cookies or IP addresses, and without involving a third party. I still write for myself first, notably because it lets me dig into a topic and refer back to it years later. But knowing a bit more about my fellow human readers is a nice bonus, even the ones disabling JavaScript. �

  1. Nginx scrambles IP addresses before storing them, thanks to the ipscrub nginx module. ↩

  2. GoatCounter already filters some bots based on the user agent or the IP address. But AI scrapers lie about their user agent and hide behind residential proxies. ↩

  3. Without JavaScript, I cannot send the real referrer. I insert “NoJS� instead. ↩

  4. I am missing the humans reading the RSS feed. I am not comfortable adding a tracking pixel, and bots are likely to fetch it, compromising the statistics. I’ll live without counting these readers. ↩

  5. In February 2025, I tested another CSS-only approach without the interaction trick: each page loaded an empty image as a background. Over a month, it recorded up to ten times fewer page views than the nginx logs, even after removing user agents identifying as bots. Compared to this experiment, GoatCounter counts about three times fewer views, 18 months later. As the time ranges differ, this is not an apples-to-apples comparison, but it gives a hint about how efficient bot filtering is. ↩

  6. A NixOS module is a function receiving the configuration of the whole system as config and returning three attributes:

    • imports adds other modules to import,
    • options declares user-configurable settings, each with a type and a default value, and
    • config sets values for options declared by any module, like containers from NixOS.

    If a module does not need to declare options, you can return the config attribute set directly. When a module does not require any argument, you can define it as an attribute set instead. ↩

  7. Most of the time, you don’t need an explicit import. NixOS automatically imports the modules shipped with Nixpkgs. For my own modules, some machinery also imports them automatically. ↩

  8. Litestream warns against using the auto-recover option as it can cause data loss. But I don’t properly monitor my servers and I prefer uninterrupted backups to a slight chance of losing the last few records. ↩

  9. In my case, the process is slow: around 20 minutes for a 20 MiB database. You can test by restoring to a copy with the -o option, but you still need to stop the Litestream container. ↩

Cory Doctorow — Technopolitics

The giant head of the Statue of Liberty. Around it are many medieval machines, tended by little medieval rude mechanicals. In the background is a crosshatched architectural drawing of some classical arches and such.

This week on my podcast, I read Technopolitics, my latest Locus Magazine column, about the degree to which AI’s politics are baked in or historically contingent.

These sins are baked into this kind of AI production. There is no way to raise trillions in subsidies that allow AI companies to sell computation at a subsidy that amounts to selling $100 bills for $1 each without also insisting that the end state of all of this is that every job will be swallowed by a chatbot. You can’t raise trillions on this promise with­out “blitzscaled” data-centers – accompanied by climate-shredding gas turbines. You can’t raise trillions without cramming AI into every part of every person’s digital life, without any safeguards for how vulnerable people might use them to sink deep into dangerous delusion.

These are the “technopolitics” of AI: To make this AI, the people in­volved had to inflict all these harms upon us. The important questions are: to what extent are these the necessary technopolitics of AI, and; can those technopolitics change?

Langdon Winner’s 1980 article, “Do Artifacts Have Politics?” is the Ur-text for questions like these. In this seminal essay, the rock critic turned tech scholar Winner demands that we move beyond the simple framing of tech having “intended” and “unintended” consequences, and instead insists that politics come baked into some technologies.

MP3

Planet Debian — Andrew Cater: Debian 11 is at end of life from Long Term Support

 Lots of posts in the debian-user mailing list complaining about updates with Debian 11.11 suddenly failing.

See Debian 11 Long Term Support reaches end-of-life

August 31st, 2026

The Debian Long Term Support (LTS) Team hereby announces that Debian 11 bullseye support has reached its end-of-life today, 31 August 2026, five years after its initial release on 14 August 2021.

Starting in September, Debian will not provide further security updates for Debian 11. A subset of bullseye packages will be supported by external parties. Detailed information can be found at Extended LTS.

The Debian LTS Team is currently providing security support for Debian 12 bookworm, the current oldstable release. Thanks to the combined efforts of different teams including the Security Team, the Release Team, and the LTS Team, the Debian 12 life cycle encompasses five years. Debian 12 will receive Long Term Support until 30 June 2028. The supported architectures in Debian 12 LTS are amd64, i386, arm64, armhf and ppc64el.

For further information about using bookworm LTS and upgrading from bullseye LTS, please refer to LTS/Using.

Debian and its LTS Team would like to thank all contributing users, developers, sponsors and other Debian teams who are making it possible to extend the life of previous stable releases, and who have made Bullseye LTS a success.

If you rely on Debian LTS, please consider joining the team, providing patches, testing or funding the efforts.