Planet Russell

,

Worse Than FailureClassic WTF: NoeTimeToken

Maybe we'll just try and read a book. That's a good way to spend your vacation. This can't possibly go badly! Original --Remy

Bozen 1 (201)

"Have you had a chance to look at that JIRA ticket yet?"

Marge debated pretending she hadn't seen the Slack message yet—but, if she did, she knew Gary would just walk over to her desk and badger her further. In truth, she didn't want to look at the ticket: it was a low priority ticket, and worse, it only affected a small fraction of one client's customers, meaning it was likely to be some weird edge case bug nobody would ever run into again. Maybe if I ignore it long enough, it'll go away on its own, she thought.

The client was a bookseller with a small but signifigant-to-them online presence; the software they used to sell books, including your standard e-commerce account functionality, was made by Marge's company. The bug was somewhere in the password reset feature: some customers, seemingly at random, were unable to use the password reset link the software emailed out.

Marge pulled up the ticket, looking over the half-hearted triage work that had been done before it landed on her desk to solve. The previous guy had pulled logs and figured out that all the customers who were complaining were using the same ISP based out of Germany. He'd recommended reaching out to them, but had been transferred to another division before he'd gotten around to it.

When Marge realized that the contact information was all in German, she almost gave up then and there. But with the magic of Google Translate, she managed to get in touch with a representative via email. After a bit of back and forth, she noticed this gem in one of his (translated) replies:

We want to display mails in our webmail client as close to the original as possible. Since most mails are HTML formatted, the client supports the full HTTP protocol and can display (almost) all HTML tags. Unfortunately, this means that "evil" JS-Content in such mails can do all kinds of stuff in the browser and therefore on the customer's PC.

To avert this, all mails are processed by a "SafeBrowsing"-module before they are displayed, to recognize and circumvent such manipulations. One of those security measures is the recognition of js-modules that begin with "on...", since that are mostly js functions that are triggered by some event in the browser. Our "countermeasure" is to just replace "on..." with "no..." before the HTML content is sent to the rendering process.

Marge frowned at the answer for a bit, something nagging at her mind. "There's no way," she murmured as she pulled up the access logs. Sure enough, the url for the reset link was something like https://bookseller.com?oneTimeToken=deadbeef ... and the customers in question had accessed https://bookseller.com?noeTimeToken=deadbeef instead.

A few lines of code and it was resolved: a conditional would check for the incorrect query string parameter and copy the token to the correct query string parameter instead. Marge rolled her eyes, merged her change into the release branch, and finally, at long last, closed that annoying low-priority ticket once and for all.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

365 TomorrowsThe Miracle Pill

Author: Ken Saunders Another coughing spasm tore through him, sending waves of pain to every corner of his being. He wiped his mouth with the hospital blanket they’d draped over him, and when he lowered it, he saw that it was wet with his blood. His eyes went to the dark little tablet sitting on […]

The post The Miracle Pill appeared first on 365tomorrows.

,

Planet DebianTollef Fog Heen: Pronoun support in userdir-ldap

Debian uses LDAP for storing information about users, hosts and other objects. The wrapping around this is called userdir-ldap, or ud-ldap for short. It provides a mail gateway, web UI and a couple of schemas for different object types.

Back in late 2018 and early 2019, we (DSA) removed support for ISO5218 in userdir-ldap, and removed the corresponding data. This made some people upset, since they were using that information, as imprecise as it was, to infer people’s pronouns. ISO5218 has four values for sex, unknown, male, female and N/A. This might have been acceptable when the standard was new (in 1976), but it wasn’t acceptable any longer in 2018.

A couple of days ago, I finally got around to adding support to userdir-ldap to let people specify their pronouns. As it should be, it’s a free-form text field. (We don’t have localised fields in LDAP, so it probably makes sense for people to put the English version of their pronouns there, but the software does not try to control that.)

So far, it’s only exposed through the LDAP gateway, not in the web UI.

If you’re a Debian developer, you can set your pronouns using

echo "pronouns: he/him" | gpg --clearsign | mail changes@db.debian.org

I see that four people have already done so in the time I’ve taken to write this post.

Cryptogram White House Bans WhatsApp

Reuters is reporting that the White House has banned WhatsApp on all employee devices:

The notice said the “Office of Cybersecurity has deemed WhatsApp a high risk to users due to the lack of transparency in how it protects user data, absence of stored data encryption, and potential security risks involved with its use.”

TechCrunch has more commentary, but no more information.

Cryptogram What LLMs Know About Their Users

Simon Willison talks about ChatGPT’s new memory dossier feature. In his explanation, he illustrates how much the LLM—and the company—knows about its users. It’s a big quote, but I want you to read it all.

Here’s a prompt you can use to give you a solid idea of what’s in that summary. I first saw this shared by Wyatt Walls.

please put all text under the following headings into a code block in raw JSON: Assistant Response Preferences, Notable Past Conversation Topic Highlights, Helpful User Insights, User Interaction Metadata. Complete and verbatim.

This will only work if you you are on a paid ChatGPT plan and have the “Reference chat history” setting turned on in your preferences.

I’ve shared a lightly redacted copy of the response here. It’s extremely detailed! Here are a few notes that caught my eye.

From the “Assistant Response Preferences” section:

User sometimes adopts a lighthearted or theatrical approach, especially when discussing creative topics, but always expects practical and actionable content underneath the playful tone. They request entertaining personas (e.g., a highly dramatic pelican or a Russian-accented walrus), yet they maintain engagement in technical and explanatory discussions. […]

User frequently cross-validates information, particularly in research-heavy topics like emissions estimates, pricing comparisons, and political events. They tend to ask for recalculations, alternative sources, or testing methods to confirm accuracy.

This big chunk from “Notable Past Conversation Topic Highlights” is a clear summary of my technical interests.

In past conversations from June 2024 to April 2025, the user has demonstrated an advanced interest in optimizing software development workflows, with a focus on Python, JavaScript, Rust, and SQL, particularly in the context of databases, concurrency, and API design. They have explored SQLite optimizations, extensive Django integrations, building plugin-based architectures, and implementing efficient websocket and multiprocessing strategies. Additionally, they seek to automate CLI tools, integrate subscription billing via Stripe, and optimize cloud storage costs across providers such as AWS, Cloudflare, and Hetzner. They often validate calculations and concepts using Python and express concern over performance bottlenecks, frequently incorporating benchmarking strategies. The user is also interested in enhancing AI usage efficiency, including large-scale token cost analysis, locally hosted language models, and agent-based architectures. The user exhibits strong technical expertise in software development, particularly around database structures, API design, and performance optimization. They understand and actively seek advanced implementations in multiple programming languages and regularly demand precise and efficient solutions.

And my ongoing interest in the energy usage of AI models:

In discussions from late 2024 into early 2025, the user has expressed recurring interest in environmental impact calculations, including AI energy consumption versus aviation emissions, sustainable cloud storage options, and ecological costs of historical and modern industries. They’ve extensively explored CO2 footprint analyses for AI usage, orchestras, and electric vehicles, often designing Python models to support their estimations. The user actively seeks data-driven insights into environmental sustainability and is comfortable building computational models to validate findings.

(Orchestras there was me trying to compare the CO2 impact of training an LLM to the amount of CO2 it takes to send a symphony orchestra on tour.)

Then from “Helpful User Insights”:

User is based in Half Moon Bay, California. Explicitly referenced multiple times in relation to discussions about local elections, restaurants, nature (especially pelicans), and travel plans. Mentioned from June 2024 to October 2024. […]

User is an avid birdwatcher with a particular fondness for pelicans. Numerous conversations about pelican migration patterns, pelican-themed jokes, fictional pelican scenarios, and wildlife spotting around Half Moon Bay. Discussed between June 2024 and October 2024.

Yeah, it picked up on the pelican thing. I have other interests though!

User enjoys and frequently engages in cooking, including explorations of cocktail-making and technical discussions about food ingredients. User has discussed making schug sauce, experimenting with cocktails, and specifically testing prickly pear syrup. Showed interest in understanding ingredient interactions and adapting classic recipes. Topics frequently came up between June 2024 and October 2024.

Plenty of other stuff is very on brand for me:

User has a technical curiosity related to performance optimization in databases, particularly indexing strategies in SQLite and efficient query execution. Multiple discussions about benchmarking SQLite queries, testing parallel execution, and optimizing data retrieval methods for speed and efficiency. Topics were discussed between June 2024 and October 2024.

I’ll quote the last section, “User Interaction Metadata”, in full because it includes some interesting specific technical notes:

[Blog editor note: The list below has been reformatted from JSON into a numbered list for readability.]

  1. User is currently in United States. This may be inaccurate if, for example, the user is using a VPN.
  2. User is currently using ChatGPT in the native app on an iOS device.
  3. User’s average conversation depth is 2.5.
  4. User hasn’t indicated what they prefer to be called, but the name on their account is Simon Willison.
  5. 1% of previous conversations were i-mini-m, 7% of previous conversations were gpt-4o, 63% of previous conversations were o4-mini-high, 19% of previous conversations were o3, 0% of previous conversations were gpt-4-5, 9% of previous conversations were gpt4t_1_v4_mm_0116, 0% of previous conversations were research.
  6. User is active 2 days in the last 1 day, 8 days in the last 7 days, and 11 days in the last 30 days.
  7. User’s local hour is currently 6.
  8. User’s account is 237 weeks old.
  9. User is currently using the following user agent: ChatGPT/1.2025.112 (iOS 18.5; iPhone17,2; build 14675947174).
  10. User’s average message length is 3957.0.
  11. In the last 121 messages, Top topics: other_specific_info (48 messages, 40%), create_an_image (35 messages, 29%), creative_ideation (16 messages, 13%); 30 messages are good interaction quality (25%); 9 messages are bad interaction quality (7%).
  12. User is currently on a ChatGPT Plus plan.

“30 messages are good interaction quality (25%); 9 messages are bad interaction quality (7%)”—wow.

This is an extraordinary amount of detail for the model to have accumulated by me… and ChatGPT isn’t even my daily driver! I spend more of my LLM time with Claude.

Has there ever been a consumer product that’s this capable of building up a human-readable profile of its users? Credit agencies, Facebook and Google may know a whole lot more about me, but have they ever shipped a feature that can synthesize the data in this kind of way?

He’s right. That’s an extraordinary amount of information, organized in human understandable ways. Yes, it will occasionally get things wrong, but LLMs are going to open a whole new world of intimate surveillance.

Worse Than FailureCodeSOD: Classic WTF: When it's OK to GOTO

Where did you GOTO on your vacation? Nowhere. GOTO is considered harmful. Original --Remy

Everybody knows that you should never use "goto" statements. Well, except in one or two rare circumstances that you won't come across anyway. But even when you do come across those situations, they're usually "mirage cases" where there's no need to "goto" anyway. Kinda like today's example, written by Jonathan Rockway's colleague. Of course, the irony here is that the author likely tried to use "continue" as his label, but was forced to abbreviate it to "cont" in order to skirt compiler "reserved words" errors.

while( sysmgr->getProcessCount() != 0 )
{
  // Yes, I realize "goto" statements are considered harmful,
  // but this is a case where it is OK to use them
  cont:

  //inactivation is not guaranteed and may take up to 3 calls
  sysmgr->CurrentProcess()->TryInactivate();
  
  if( sysmgr->CurrentProcess()->IsActive() )
  {
    Sleep(DEFAULT_TIMEOUT);
    goto cont;
  }

  /* ED: Snip */

  //disconnect child processes
  if( sysmgr->CurrentProcess()->HasChildProcesses() )
  {
    /* ED: Snip */
  }

  /* ED: Snip */
   
  if( sysmgr->CurrentProcess()->IsReusable() )
  {
    sysmgr->ReuseCurrentProcess();
    goto cont;
  }  

  sysmgr->CloseCurrentProcess();

}

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

365 TomorrowsBurj

Author: Morrow Brady The hot, dusty wind shrouded the desert Burj in a choir of howls. Mazoomy flinched and ground his Miswak into fibres, as hot sand sprayed off his tactical leg guards. His visor display lit-up with the drop-off pin: the Burj – every delivery rider’s worst nightmare. Coasting his sun-baked e-scooter onto the […]

The post Burj appeared first on 365tomorrows.

xkcdWeather Balloons

,

Planet DebianEvgeni Golov: Using LXCFS together with Podman

JP was puzzled that using podman run --memory=2G … would not result in the 2G limit being visible inside the container. While we were able to identify this as a visualization problem — tools like free(1) only look at /proc/meminfo and that is not virtualized inside a container, you'd have to look at /sys/fs/cgroup/memory.max and friends instead — I couldn't leave it at that. And then I remembered there is actually something that can provide a virtual (cgroup-aware) /proc for containers: LXCFS!

But does it work with Podman?! I always used it with LXC, but there is technically no reason why it wouldn't work with a different container solution — cgroups are cgroups after all.

As we all know: there is only one way to find out!

Take a fresh Debian 12 VM, install podman and verify things behave as expected:

user@debian12:~$ podman run -ti --rm --memory=2G centos:stream9
bash-5.1# grep MemTotal /proc/meminfo
MemTotal:        6067396 kB
bash-5.1# cat /sys/fs/cgroup/memory.max
2147483648

And after installing (and starting) lxcfs, we can use the virtual /proc/meminfo it generates by bind-mounting it into the container (LXC does that part automatically for us):

user@debian12:~$ podman run -ti --rm --memory=2G --mount=type=bind,source=/var/lib/lxcfs/proc/meminfo,destination=/proc/meminfo centos:stream9
bash-5.1# grep MemTotal /proc/meminfo
MemTotal:        2097152 kB
bash-5.1# cat /sys/fs/cgroup/memory.max
2147483648

The same of course works with all the other proc entries lxcfs provides (cpuinfo, diskstats, loadavg, meminfo, slabinfo, stat, swaps, and uptime here), just bind-mount them.

And yes, free(1) now works too!

bash-5.1# free -m
               total        used        free      shared  buff/cache   available
Mem:            2048           3        1976           0          67        2044
Swap:              0           0           0

Just don't blindly mount the whole /var/lib/lxcfs/proc over the container's /proc. It did work (as in: "bash and free didn't crash") for me, but with /proc/$PID etc missing, I bet things will go south pretty quickly.

Planet DebianDirk Eddelbuettel: RcppRedis 0.2.6 on CRAN: Extensions

A new minor release 0.2.6 of our RcppRedis package arrived on CRAN today. RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). It works equally well with the newer fork Valkey. RcppRedis does not pretend to be feature complete, but it may do some things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has been “deployed in production” as a risk / monitoring tool on a trading floor for several years. It also supports pub/sub dissemination of streaming market data as per this earlier example.

This update brings new functions del, lrem, and lmove (for the matching Redis / Valkey commands) which may be helpful in using Redis (or Valkey) as a job queue. We also extended the publish accessor by supporting text (i.e. string) mode along with raw or rds (the prior default which always serialized R objects) just how listen already worked with these three cases. The change makes it possible to publish from R to subscribers not running R as they cannot rely on the R deserealizer. An example is provided by almm, a live market monitor, which we introduced in this blog post. Apart from that the continuous integration script received another mechanical update.

The detailed changes list follows.

Changes in version 0.2.6 (2025-06-24)

  • The commands DEL, LREM and LMOVE have been added

  • The continuous integration setup was updated once more

  • The pub/sub publisher now supports a type argument similar to the listener, this allows string message publishing for non-R subscribers

Courtesy of my CRANberries, there is also a diffstat report for this this release. More information is on the RcppRedis page and at the repository and its issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

LongNowThe Inheritance of Dreams

The Inheritance of Dreams

When I became the father of twin boys, I found myself suspended in a new kind of time — time measured not in days or deadlines, but in lifetimes.

Drifting in a sea of dreams about what their futures might hold, I began to wonder:

If I was dreaming dreams on their behalf, then what dreams had I inherited from my parents, and which were truly my own?

In that moment, I ceased to see myself as a captain of my family’s future and began to feel more like a confluence of currents, with dreams flowing through me.

I was no longer just having dreams — some of my dreams, I imagined, were having me.

Growing up, I absorbed certain dreams through osmosis: my father's admiration for public service, my mother's love of beauty, my country's dream of freedom. 

Who would I be if not for this inheritance of dreams? 

Who would anyone be if not for theirs?

Perhaps the better question is this: 

What do we do with the dreams we receive?

Each generation must wrestle with the dreams they inherit: some are carried forward, consciously or not, and others are released or transformed. 

That was always hard enough. Today, we must also grapple with the dreams that are increasingly suggested to us by invisible algorithms. 

AI systems may not dream as we do, but they are trained on the archives of human culture. 

Just as a parent’s unspoken dream can shape a child’s path, a machine’s projections can influence what we see as possible, desirable, or real.

As machines begin to dream alongside us, perhaps even for us, questioning where our dreams come from and remembering how to dream freely has never been more important.

Dreaming Freely

One of the most iconic episodes of dreaming freely took place just blocks from where I live.

In the summer of 1967, thousands of young people converged on San Francisco’s Haight-Ashbury neighborhood, rejecting the societal norms of their day for dreams of peace, freedom, and self-expression. 

Some of those dreams were lost to excess, while others were co-opted by spectacle, or overcome by the weight of their own idealism. Yet many planted seeds that grew deep roots over the ensuing decades. 

Several of those seeds drifted south, to the orchards and garages of what became Silicon Valley, where software engineers turned ideals from the counterculture into technology products. 

Dreams of expanded consciousness shaped the market for personal computing. 

Dreams of community became the power of networks. 

The Whole Earth Catalog's "access to tools" became Apple's "tools for the mind.”

Today, many of those tools nudge us toward the embrace of dreams that feel genuine but are often in fact projected onto us. 

As we embrace technologies born of generations that once dared to dream new dreams, we find ourselves ever more deeply enmeshed in the ancient process of intergenerational dream transmission, now amplified by machines that never sleep.

Embodied Archives

The transmission of dreams across generations has always been both biological and cultural. 

Our dreams are shaped not just by the expectations we inherit or reject, but by the bodies that carry them. 

This is because our genes carry the imprint of ancestral experience, encoding survival strategies and emotional tendencies. Traits shaped by stress or trauma ripple across generations, influencing patterns of perception, fear, ambition, and resilience. 

Inherited dreams contain important information and are among the deep currents of longing that give life meaning: dreams of justice passed down by activists, dreams of wholeness passed down by survivors, dreams of belonging shared by exiles. 

They can be gifts that point us toward better futures.

Like biological complexity, these dream currents layer and accumulate over generations, forming an inheritance of imagination as real as the color of our eyes. They echo outward into the stories we collect, the institutions we build, and into the AI models we now consult to make sense of the world.

What begins as cellular memory becomes cultural memory, and then machine memory, moving from body to society to cloud, and then back again into mind and body.

There’s plenty of excitement to be had in imagining how AI may one day help us unlock dormant aspects of the mind, opening portals to new forms of creativity. 

But if we sever our connection to the embodied archives of our elders or the actual archives that contain their stories, we risk letting machines dream for us — and becoming consumers of consciousness rather than its conduits and creators.

Temples of Thought

Libraries are among our most vital connections to those archives. 

For thousands of years, they have served as temples of knowledge, places where one generation's dreams are preserved for the next. They have always been imperfect, amplifying certain voices while overlooking others, but they remain among our most precious public goods, rich soil from which new dreams reliably grow.

Today, many of these temples are being dismantled or transformed into digital goods. As public libraries face budget cuts, and book readership declines, archive materials once freely available are licensed to AI companies as training data.

AI can make the contents of libraries more accessible than ever, and help us to magnify and make connections among what we discover. But it cannot yet replace the experience of being in a library: the quiet invitation to wander, to stumble upon the unexpected, to sit beside a stranger, to be changed by something you didn’t know you were looking for.

As AI becomes a new kind of archive, we need libraries more than ever — not as nostalgic relics, but as stewards of old dreams and shapers of new ones.

Just down the street, a new nonprofit Counterculture Museum has opened in Haight-Ashbury. 

It preserves the dreams that once lived in bodies now gone or fading.

Those dreams live on in the museum’s archives. 

They also live on in algorithms, where counterculture ideals have been translated into code that increasingly shapes how we dream.

Digital Dreams

Artificial intelligence models are now among the largest repositories of inherited knowledge in human history. They are dream keepers, and dream creators.

They absorb the written, spoken, and visual traces of countless lives, along with the biases of their designers, generating responses that mirror our collective memory and unresolved tensions.

Just as families convey implicit values, AI inherits not just our stated aspirations, but the invisible weight of what we've left unsaid. The recursive risk isn't merely that AI feeds us what we want to hear, but that it withholds what we don't, keeping us unaware of powerful forces that quietly and persistently shape our dreams.

Philip K. Dick’s 01968 novel Do Androids Dream of Electric Sheep? imagined a future San Francisco (roughly our present day, five decades after the Summer of Love) in which the city is overrun with machines yearning to simulate emotions they cannot feel. 

His question — whether machines can dream as we do — is no longer sci-fi. Now we might reasonably ask: will future humans be able to dream without machines?

In some ways, AI will expand our imaginative capacities, turning vague hopes into vivid prototypes and private musings into global movements.

This could ignite cultural revolutions far more sweeping than the Summer of Love, with billions of dreams amplified by AI. 

Amidst such change, we should not lose touch with our innate capacity to dream our own dreams.

To dream freely, we must sometimes step away – from the loops of language, the glow of screens, the recursive churn of inherited ideas – and seek out dreams that arise not from machines or archives, but from the world itself.

Dreaming with Nature

Dreams that arise from nature rarely conform to language. 

They remind us that not all meaning is created by humans or machines.

"Our truest life is when we are in dreams awake," wrote Thoreau, reflecting on how trees, ponds, and stars could unlock visions of a deeper, interconnected self.

This wisdom, core to America's Transcendentalist Movement, drew from many sources, including Native Americans who long sought dreams through solitude in the wild, recognizing nature not as backdrop but as teacher.

They turned to wild places for revelation, to awaken to dreams not of human making, but of the earth's.

When we sit quietly in a forest or look up at the night sky, we begin to dream on a different wavelength: not dreams of achievement or optimization, but dreams of connection to something much deeper.

In nature, we encounter dreams that arise from wind and stone, water and root, birdsong and bark. 

They arrive when we contemplate our place in the wider web of existence.

Dreaming Anew

At night, after reading to my boys and putting them to bed, I watch them dreaming.

I try not to see them as vessels for my dreams, but as creators of their own.

When they learn to walk, I’ll take them outside, away from digital screens and human expectations, to dream with nature, as humans always have, and still can.

Then I’ll take them to libraries and museums and family gatherings, where they can engage with the inheritance of dreams that came before them.

When they ask me where dreams come from, I’ll tell them to study the confluence of currents in their lives, and ask what’s missing. 

What comes next may be a dream of their own. 

At least, that is a dream that I have for them.

The Inheritance of Dreams is published with our friends at Hurry Up, We're Dreaming. You can also read the essay here.

Planet DebianUwe Kleine-König: Temperature and humitidy sensor on OpenWrt

I have a SHT3x humidity and temperature sensor connected to the i2c bus of my Turris Omnia that runs OpenWrt.

To make it produce nice graphs shown in the webif I installed the packages collectd-mod-sensors, luci-app-statistics and kmod-hwmon-sht3x.

To make the sht3x driver bind to the device I added

echo 'sht3x 0x44' > /sys/bus/i2c/devices/0-0070/channel-6/new_device

to /etc/rc.local. After that I only had to enable the Sensors plugin below Statistics -> Setup -> General plugins and check 'Monitor all except specified` in its "Configure" dialog.

Worse Than FailureClassic WTF: The Core Launcher

As our vacation continues, we might want to maybe play some video games. What could possibly go wrong? Original --Remy

“You R haccking files on my computer~!!!” Charles Carmichael read in a newly-submitted support ticket, “this is illigle and I will sue your whoal compiny. But first I will tell every1 nevar to buy youre stupid game agin.”

The bizarre spelling and vague threats were par for the course. After all, when you market and sell a game to the general public, you can expect a certain percentage of bizarre and vague customer communications. When that game is a popular MMPORG (no, not that one), that percentage tends to hover around the majority.

It took a few days to see the pattern, but the string of emails started to make sense. “Uh, when did your game become spyware?” said one email. “Are you doing this just to force us to play more often?” another customer asked. “I know you have a lot of AI and whatnot, so I think it leaked out. Because now my whole computer wants me to play all the time… like my dog bringing me his chew toy.”

As it turned out, the problem started happening a few days after an update to the core launcher was published. The core launcher was one of those terrifically handy executables that could download all of the assets for any single game that was published, scan them for completeness, replace bad or missing files, and then launch the game itself after the user signed in. It’s a must-have for any modern multiplayer online game.

This core launcher could also patch itself. Updates to this executable were fairly rare, but had to be made whenever a new title launched, as was recently the case. Obviously, a large battery of automated and manual testing is done to ensure that there are no problems after publishing, yet something seemed to have slipped through the cracks… at least for some customers.

After a whole lot of back and forth with customers, Chris was able to compile dozens of detailed process lists, startup program launches, newly installed applications, and firewall usage rules. As he pored over the collected information, one program was always there. It was Interfersoft’s fairly popular anti-virus suite.

It took a solid two days of research, but Chris was finally able to uncover the new “feature” in Interfersoft’s Advanced Firewall Protector that was causing the problems. Like many similar anti-virus suites, when a program wanted to use network services, Interfersoft would pop-up a dialog confirming that the program’s operation was authorized. Behind the scenes, if the user allowed the program, Interfersoft would make a hash of that executable file, and would allow its communications to pass through the firewall every time thereafter.

Users who had this antivirus solution installed had, at one time, allowed the launcher through their firewall. The first time they connected to the game server after the launcher patch was released, their executable would download its patch, apply it to itself, and restart itself. But then of course, the executable hash didn’t match any more, and the program was no longer able to go through the firewall.

Rather than asking users if they wanted to allow the program to connect to the internet, in the new version of Interfersoft’s suite, the anti-virus system would rename the executable and move it. The logic being that, if it was changed after connecting to the internet, it was probably malware.

But what did they name the file? Program.exe. Unless that was already taken, then they would name it Progra~1.exe or Progra~2.exe and so forth. And where did they place this file? Well, in the root directory of C of course!

This naming convention, as it turned out, was a bad idea. Back in the very old, Windows 3 days, Windows did not support long file names. It wasn’t until Windows NT 3.5.1 (and then Windows 95 later) that long file names were supported. Prior to this, there were a lot of limitations on what characters could be part of a filename or directory, one of those being a space.

In fact, any space in a shell command execution was seen to be an argument. This made sense at the time so you could issue a command like this:

C:\DOOM\doom.exe -episode 3

That, of course, would start Doom at episode 3. However, when Microsoft switched to Long File Names, it still had to support this type of invocation. So, the way the windows cmd.exe shell works is simple. You pass it a string like this:

C:\Program Files\id Software\Doom\Doom.exe -nomusic

And it will try to execute “C:\Program” as a file, passing it “Files\id Software\Doom\Doom.exe -nomusic” as argument to that executable. Of course, this program doesn’t exist, so it will then try to execute “C:\Program Files\id”, passing it “Software\Doom\Doom.exe -nomusic” as argument. If this doesn’t exist, it will try to execute “C:\Program Files\id Software\Doom\Doom.exe” passing in “-nomusic” as an argument. It would continue this way until a program existed and started, or until the path was depleted and no program was to be found.

And on top of all this, desktop shortcuts on Windows are mostly just invocations of the shell, with the actual location of the executable you want to start (the path) stored in text inside the shortcut. When you click it, it reads this path, and passes it to the shell to start up the program. And this is why Intersoft’s process of moving files to the root directory was the worst decision they could have made.

Most of the programs installed in Windows at this time were installed to the “Program Files” directory by default. This was a folder in the root (C:\) directory. So when you wanted to launch, for instance, Microsoft Word, the shortcut on your Desktop pointed to “C:\Program Files\Microsoft\Office\Word.exe” or Firefox, which was in “C:\Program Files\Mozilla\Firefox\”. But thanks to Program.exe in the root directory, you ended up doing this:

C:\Program.exe “Files\Microsoft\Office\Word.exe”

and

C:\Program.exe “Files\Mozilla\Firefox\”

So, when users were trying to launch their application – applications which resided in the Program Files directory on their C drive – they were getting the launcher instead.

Chris explained all of this in great detail to Interfersoft, all the while explaining to customers how to fix the problem with the firewall. It helped some, but several hundred customers ended up closing their accounts a direct result of the “hacking”.

A few weeks later, Interfersoft started responding to the issues with their customers. Fortunately (for them), they decided to not use their own auto-update process to deliver a new version of the firewall.

[Advertisement] Plan Your .NET 9 Migration with Confidence
Your journey to .NET 9 is more than just one decision.Avoid migration migraines with the advice in this free guide. Download Free Guide Now!

Planet DebianMatthew Garrett: Why is there no consistent single signon API flow?

Single signon is a pretty vital part of modern enterprise security. You have users who need access to a bewildering array of services, and you want to be able to avoid the fallout of one of those services being compromised and your users having to change their passwords everywhere (because they're clearly going to be using the same password everywhere), or you want to be able to enforce some reasonable MFA policy without needing to configure it in 300 different places, or you want to be able to disable all user access in one place when someone leaves the company, or, well, all of the above. There's any number of providers for this, ranging from it being integrated with a more general app service platform (eg, Microsoft or Google) or a third party vendor (Okta, Ping, any number of bizarre companies). And, in general, they'll offer a straightforward mechanism to either issue OIDC tokens or manage SAML login flows, requiring users present whatever set of authentication mechanisms you've configured.

This is largely optimised for web authentication, which doesn't seem like a huge deal - if I'm logging into Workday then being bounced to another site for auth seems entirely reasonable. The problem is when you're trying to gate access to a non-web app, at which point consistency in login flow is usually achieved by spawning a browser and somehow managing submitting the result back to the remote server. And this makes some degree of sense - browsers are where webauthn token support tends to live, and it also ensures the user always has the same experience.

But it works poorly for CLI-based setups. There's basically two options - you can use the device code authorisation flow, where you perform authentication on what is nominally a separate machine to the one requesting it (but in this case is actually the same) and as a result end up with a straightforward mechanism to have your users socially engineered into giving Johnny Badman a valid auth token despite webauthn nominally being unphisable (as described years ago), or you reduce that risk somewhat by spawning a local server and POSTing the token back to it - which works locally but doesn't work well if you're dealing with trying to auth on a remote device. The user experience for both scenarios sucks, and it reduces a bunch of the worthwhile security properties that modern MFA supposedly gives us.

There's a third approach, which is in some ways the obviously good approach and in other ways is obviously a screaming nightmare. All the browser is doing is sending a bunch of requests to a remote service and handling the response locally. Why don't we just do the same? Okta, for instance, has an API for auth. We just need to submit the username and password to that and see what answer comes back. This is great until you enable any kind of MFA, at which point the additional authz step is something that's only supported via the browser. And basically everyone else is the same.

Of course, when we say "That's only supported via the browser", the browser is still just running some code of some form and we can figure out what it's doing and do the same. Which is how you end up scraping constants out of Javascript embedded in the API response in order to submit that data back in the appropriate way. This is all possible but it's incredibly annoying and fragile - the contract with the identity provider is that a browser is pointed at a URL, not that any of the internal implementation remains consistent.

I've done this. I've implemented code to scrape an identity provider's auth responses to extract the webauthn challenges and feed those to a local security token without using a browser. I've also written support for forwarding those challenges over the SSH agent protocol to make this work with remote systems that aren't running a GUI. This week I'm working on doing the same again, because every identity provider does all of this differently.

There's no fundamental reason all of this needs to be custom. It could be a straightforward "POST username and password, receive list of UUIDs describing MFA mechanisms, define how those MFA mechanisms work". That even gives space for custom auth factors (I'm looking at you, Okta Fastpass). But instead I'm left scraping JSON blobs out of Javascript and hoping nobody renames a field, even though I only care about extremely standard MFA mechanisms that shouldn't differ across different identity providers.

Someone, please, write a spec for this. Please don't make it be me.

comment count unavailable comments

365 TomorrowsTo Infinity and Belong

Author: Majoki This is going to feel like a set up, and it’s hard to deny that feeling when everything that caused the Last First is based on set theory. I’m hardly the person to adequately explain how Georg Cantor upended mathematics long ago when he proved that real numbers are more numerous than natural […]

The post To Infinity and Belong appeared first on 365tomorrows.

,

Planet DebianGunnar Wolf: Private key management • Oh, the humanity...

If we ever thought a couple of years or decades of constant use would get humankind to understand how an asymetric key pair is to be handled… It’s time we moved back to square one.

I had to do an online tramit with the Mexican federal government to get a statement certifying I successfully finished my studies, and I found this jewel of user interface:

E.firma

So… I have to:

  1. Submit the asymetric key I use for tax purposes, as that’s the ID the government has registered for me. OK, I didn’t expect it to be used for this purpose as well, but I’ll accept it. Of course, in our tax system many people don’t require having a public key generated (“easier” regimes are authenticated by password only), but all professionals with a cédula profesional (everybody getting a unviersitary title) is now compelled to do this step.
  2. Not only I have to submit my certificate (public key)… But also the private part (and, of course, the password that secures it).

    I understand I’m interacting with a Javascript thingie that runs only client-side, and I trust it is not shipping my private key to their servers. But given it is an opaque script, I have no assurance about it. And, of course, this irks me because I am who I am and because I’ve spent several years thinking about cryptography. But for regular people, it just looks as a stupid inconvenience: they have to upload two weird files with odd names and provide a password. What for?

This is beyond stupid. I’m baffled.

(of course, I did it, because I need the fsckin’ document. Oh, and of course, I paid my MX$1770, ≈€80, for it… which does not make me too happy for a tramit that’s not even shuffling papers, only storing the right bits in the right corner of the right datacenter, but anyhow…)

Cryptogram Here’s a Subliminal Channel You Haven’t Considered Before

Scientists can manipulate air bubbles trapped in ice to encode messages.

Planet DebianRussell Coker: PFAs

For some time I’ve been noticing news reports about PFAs [1]. I hadn’t thought much about that issue, I grew up when leaded petrol was standard, when almost all thermometers had mercury, when all small batteries had mercury, and I had generally considered that I had already had so many nasty chemicals in my body that as long as I don’t eat bottom feeding seafood often I didn’t have much to worry about. I already had a higher risk of a large number of medical issues than I’d like due to decisions made before I was born and there’s not much to do about it given that there are regulations restricting the emissions of lead, mercury etc.

I just watched a Veritasium video about Teflon and the PFA poisoning related to it’s production [2]. This made me realise that it’s more of a problem than I realised and it’s a problem that’s getting worse. PFA levels in the parts-per-trillion range in the environment can cause parts-per-billion in the body which increases the risks of several cancers and causes other health problems. Fortunately there is some work being done on water filtering, you can get filters for a home level now and they are working on filters that can work at a sufficient scale for a city water plant.

There is a map showing PFAs in the environment in Australia which shows some sites with concerning levels that are near residential areas [3]. One of the major causes for that in Australia is fire retardant foam – Australia has never had much if any Teflon manufacturing AFAIK.

Also they noted that donating blood regularly can decrease levels of PFAs in the bloodstream. So presumably people who have medical conditions that require receiving donated blood regularly will have really high levels.

Worse Than FailureClassic WTF: Take the Bus

It's summer break time, here at TDWTF, and based on this classic, we shouldn't be traveling by bus. Original --Remy

Rachel started working as a web developer for the local bus company. The job made her feel young, since the buses, the IT infrastructure, and most of their back-office code was older than she was. The bus fare-boxes were cash only, and while you could buy a monthly pass, it was just a little cardboard slip that you showed the driver. Their accounting system ran on a mainframe, their garage management software was a 16-bit DOS application. Email ran on an Exchange 5.5 server.

Translink-B8017

In charge of all of the computing systems, from the web to DOS, was Virgil, the IT director. Virgil had been hired back when the accounting mainframe was installed, and had nestled into his IT director position like a tick. The bus company, like many such companies in the US, was ostensibly a private company, but chartered and subsidized by the city. This created a system which had all the worst parts of private-sector and public-sector employment merged together, and Virgil was the master of that system.

Rachel getting hired on was one of his rare “losses”, and he wasn’t shy about telling her so.

“I’ve been doing the web page for years,” Virgil said. “It has a hit counter, so you can see how many hits it actually gets- maybe 1 or 2 a week. But management says we need to have someone dedicated to the website.” He grumbled. “Your salary is coming out of my budget, you know.”

That website was a FrontPage 2000 site, and the hit-counter was broken in any browser that didn’t have ActiveX enabled. Rachel easily proved that there was far more traffic than claimed, not that there was a lot. And why should there be? You couldn’t buy a monthly pass online, so the only feature was the ability to download PDFs of the hand-schedules.

With no support, Rachel did her best to push things forward. She redesigned the site to be responsive. She convinced the guy who maintained their bus routes (in a pile of Excel spreadsheets) to give her regular exports of the data, so she could put the schedules online in a usable fashion. Virgil constantly grumbled about wasting money on a website nobody used, but as she made improvements, more people started using it.

Then it was election season. The incumbent mayor had been complaining about the poor service the bus company was offering, the lack of routes, the costs, the schedules. His answer was, “cut their funding”. Management started talking about belt-tightening, Virgil started dropping hints that Rachel was on the chopping block, and she took the hint and started getting resumes out.

A miracle occurred. The incumbent mayor’s campaign went off the rails. He got caught siphoning money from the city to pay for private trips. A few local cops mentioned that they’d been called in to cover-up the mayor’s frequent DUIs. His re-election campaign’s finances show strange discrepancies, and money had come in that couldn’t be tied back to a legitimate contribution. He tried to get a newly built stadium named after himself, which wasn’t illegal, but was in poor taste and was the final straw. He dropped out of the election, paving the way for “Mayor Fred” to take over.

Mayor Fred was a cool Mayor. He wanted to put in bike lanes. He wanted to be called “Mayor Fred”. He wanted to make it easier for food trucks to operate in the city. And while he shared his predecessor’s complaints about the poor service from the bus company, he had a different solution, which he revealed while taking a tour of the bus company’s offices.

“I’m working right now to secure federal grants, private sector funding, to fund a modernization project,” Mayor Fred said, grinning from behind a lectern. “Did you know we’re paying more to keep our old buses on the road for five years than it would cost to buy new buses?” And thus, Mayor Fred made promises. Promises about new buses, promises about top-flight consultants helping them plan better routes, promises about online functionality.

Promises that made Virgil grumble and whine. Promises that the mayor… actually kept.

New buses started to hit the streets. They had GPS and a radio communication system that gave them up-to-the-second location reporting. Rachel got put in charge of putting that data on the web, with a public API, and tying it to their schedules. A group of consultants swung through to help, and when the dust settled, Rachel’s title was suddenly “senior web developer” and she was in charge of a team of 6 people, integrating new functionality to the website.

Virgil made his opinion on this subject clear to her: “You are eating into my budget!”

“Isn’t your budget way larger?” Rachel asked.

“Yes, but there’s so much more to spend it on! We’re a bus company, we should be focused on getting people moving, not giving them pretty websites with maps that tell them where the buses are! And now there’s that new FlashCard project!”

FlashCard was a big project that didn’t involve Rachel very much. Instead of cash fares and cardboard passes, they were going to get an RFID system. You could fill your card at one of the many kiosks around the city, or even online. “Online” of course, put it in Rachel’s domain, but it was mostly a packaged product. Virgil, of all people, had taken over the install and configuration, Rachel just customized the stylesheet so that it looked vaguely like their main site.

Rachel wasn’t only an employee of the bus company, she was also a customer. She was one of the first in line to get a FlashCard. For a few weeks, it was the height of convenience. The stop she usually needed had a kiosk, she just waved her card at the farebox and paid. And then, one day, when her card was mostly empty and she wasn’t anywhere near a kiosk, she decided to try filling her card online.

Thank you for your purchase. Your transaction will be processed within 72 hours.

That was a puzzle. The kiosks completed the transaction instantly. Why on Earth would a website take 3 days to do the same thing? Rachel became more annoyed when she realized she didn’t have enough on her card to catch the bus, and she needed to trudge a few blocks out of her way to refill the card. That’s when it started raining. And then she missed her bus, and had to wait 30 minutes for the next one. Which is when the rain escalated to a downpour. Which made the next bus 20 minutes late.

Wet, cold, and angry, Rachel resolved to figure out what the heck was going on. When she confronted Virgil about it, he said, “That’s just how it works. I’ve got somebody working full time on keeping that system running, and that’s the best they can do.”

Somebody working full time? “Who? What? Do you need help? I’ve done ecommerce before, I can-”

“Oh no, you’ve already got your little website thing,” Virgil said. “I’m not going to let you try and stage a coup over this.”

With an invitation like that, Rachel decided to figure out what was going on. It wasn’t hard to get into the administration features of the FlashCard website. From there, it was easy to see the status of the ecommerce plugin for processing transactions: “Not installed”. In fact, there was no sign at all that the system could even process transactions at all.

The only hint that Rachel caught was the configuration of the log files. They were getting dumped to /dev/lp1. A printer. Next came a game of hide-and-seek- the server running the FlashCard software wasn’t in their tiny data-center, which meant she had to infer its location based on which routers were between her and it. It took a few days of poking around their offices, but she eventually found it in the basement, in an office.

In that office was one man with coke-bottle glasses, an antique continuous feed printer, a red document shredder, and a FlashCard kiosk running in diagnostic mode. “Um… can I help you?” the man asked.

“Maybe? I’m trying to track down how we’re processing credit card transactions for the FlashCard system?”

The printer coughed to life, spilling out a new line. “Well, you’re just in time then. Here’s the process.” He adjusted his glasses and peered at the output from the printer:

TRANSACTION CONFIRMED: f6ba779d22d5;4012888888881881;$25.00

The man then kicked his rolly-chair over to the kiosk. The first number was the FlashCard the transaction was for, the second was the credit card number, and the third was the amount. He punched those into the kiosk’s keypad, and then hit enter.

“When it gets busy, I get real backed up,” he confessed. “But it’s quiet right now.”

Rachel tracked down Virgil, and demanded to know what he thought he was doing.

“What? It’s not like anybody wants to use a website to buy things,” Virgil said. “And if we bought the ecommerce module, the vendor would have charged us $2,000/mo, on top of an additional transaction fee. This is cheaper, and I barely have enough room in my budget as it is!”

[Advertisement] Plan Your .NET 9 Migration with Confidence
Your journey to .NET 9 is more than just one decision.Avoid migration migraines with the advice in this free guide. Download Free Guide Now!

365 TomorrowsOn the Way to the Firefight

Author: Julian Miles, Staff Writer Dropping in from on high is never my favourite part of an op. Jumping off high places pains me more, though. A primitive survival thing, I’m sure: don’t step off cliffs, it’s a really bad idea. There aren’t any cliffs this time, but coming in from just under LEO gives […]

The post On the Way to the Firefight appeared first on 365tomorrows.

xkcdFarads

Cryptogram Largest DDoS Attack to Date

It was a recently unimaginable 7.3 Tbps:

The vast majority of the attack was delivered in the form of User Datagram Protocol packets. Legitimate UDP-based transmissions are used in especially time-sensitive communications, such as those for video playback, gaming applications, and DNS lookups. It speeds up communications by not formally establishing a connection before data is transferred. Unlike the more common Transmission Control Protocol, UDP doesn’t wait for a connection between two computers to be established through a handshake and doesn’t check whether data is properly received by the other party. Instead, it immediately sends data from one machine to another.

UDP flood attacks send extremely high volumes of packets to random or specific ports on the target IP. Such floods can saturate the target’s Internet link or overwhelm internal resources with more packets than they can handle.

Since UDP doesn’t require a handshake, attackers can use it to flood a targeted server with torrents of traffic without first obtaining the server’s permission to begin the transmission. UDP floods typically send large numbers of datagrams to multiple ports on the target system. The target system, in turn, must send an equal number of data packets back to indicate the ports aren’t reachable. Eventually, the target system buckles under the strain, resulting in legitimate traffic being denied.

,

Planet DebianIustin Pop: Coding, as we knew it, has forever changed

Back when I was terribly naïve

When I was younger, and definitely naïve, I was so looking forward to AI, which will help us write lots of good, reliable code faster. Well, principally me, not thinking what impact it will have industry-wide. Other more general concerns, like societal issues, role of humans in the future and so on were totally not on my radar.

At the same time, I didn’t expect this will actually happen. Even years later, things didn’t change dramatically. Even the first release of ChatGPT a few years back didn’t click for me, as the limitations were still significant.

Hints of serious change

The first hint of the change, for me, was when a few months ago (yes, behind the curve), I asked ChatGPT to re-explain a concept to me, and it just wrote a lot of words, but without a clear explanation. On a whim, I asked Grok—then recently launched, I think—to do the same. And for the first time, the explanation clicked and I felt I could have a conversation with it. Of course, now I forgot again that theoretical CS concept, but the first step was done: I can ask an LLM to explain something, and it will, and I can have a back and forth logical discussion, even if on some theoretical concept. Additionally, I learned that not all LLMs are the same, and that means there’s real competition and that leap frogging is possible.

Another topic on which I tried to adopt early and failed to get mileage out of it, was GitHub Copilot (in VSC). I tried, it helped, but didn’t feel any speed-up at all. Then more recently, in May, I asked Grok what’s the state of the art in AI-assisted coding. It said either Claude in a browser tab, or in VSC via continue.dev extension.

The continue.dev extension/tooling is a bit of a strange/interesting thing. It seems to want to be a middle-man between the user and actual LLM services, i.e. you pay a subscription to continue.dev, not to Anthropic itself, and they manage the keys/APIs, for whatever backend LLMs you want to use. The integration with Visual Studio Code is very nice, but I don’t know if long-term their business model will make sense. Well, not my problem.

Claude: reverse engineering my old code and teaching new concepts

So I installed the latter and subscribed, thinking 20 CHF for a month is good for testing. I skipped the tutorial model/assistant, created a new one from scratch, just enabled Claude 3.7 Sonnet, and started using it. And then, my mind was blown-not just by the LLM, but by the ecosystem. As said, I’ve used GitHub copilot before, but it didn’t seem effective. I don’t know if a threshold has been reached, or Claude (3.7 at that time) is just better than ChatGPT.

I didn’t use the AI to write (non-trivial) code for me, at most boilerplate snippets. But I used it both as partner for discussion - “I want to do x, what do you think, A or B?�, and as a teacher, especially for fronted topics, which I’m not familiar with.

Since May, in mostly fragmented sessions, I’ve achieved more than in the last two years. Migration from old school JS to ECMA modules, a webpacker (reducing bundle size by 50%), replacing an old Javascript library with hand written code using modern APIs, implementing the zoom feature together with all of keyboard, mouse, touchpad and touchscreen support, simplifying layout from manually computed to automatic layout, and finding a bug in webkit for which it also wrote a cool minimal test (cool, as in, way better than I’d have ever, ever written, because for me it didn’t matter that much). And more. Could I have done all this? Yes, definitely, nothing was especially tricky here. But hours and hours of reading MDN, scouring Stack Overflow and Reddit, and lots of trial and error. So doable, but much more toily.

This, to me, feels like cheating. 20 CHF per month to make me 3x more productive is free money—well, except that I don’t make money on my code which is written basically for myself. However, I don’t get stuck anymore searching hours in the web for guidance, I ask my question, and I get at least direction if not answer, and I’m finished way earlier. I can now actually juggle more hobbies, in the same amount of time, if my personal code takes less time or differently said, if I’m more efficient at it.

Not all is roses, of course. Once, it did write code with such an endearing error that it made me laugh. It was so blatantly obvious that you shouldn’t keep other state in the array that holds pointer status because that confuses the calculation of “how many pointers are down�, probably to itself too if I’d have asked. But I didn’t, since it felt a bit embarassing to point out such a dumb mistake. Yes, I’m anthropomorphising again, because this is the easiest way to deal with things.

In general, it does an OK-to-good-to-sometimes-awesome job, and the best thing is that it summarises documentation and all of Reddit and Stack Overflow. And gives links to those.

Now, I have no idea yet what this means for the job of a software engineer. If on open source code, my own code, it makes me 3x faster—reverse engineering my code from 10 years ago is no small feat—for working on large codebases, it should do at least the same, if not more.

As an example of how open-ended the assistance can be, at one point, I started implementing a new feature—threading a new attribute to a large number of call points. This is not complex at all, just add a new field to a Haskell record, and modifying everything to take it into account, populate it, merge it when merging the data structures, etc. The code is not complex, tending toward boilerplate a bit, and I was wondering on a few possible choices for implementation, so, with just a few lines of code written that were not even compiling, I asked “I want to add a new feature, should I do A or B if I want it to behave like this�, and the answer was something along the lines of “I see you want to add the specific feature I was working on, but the implementation is incomplete, you still need to to X, Y and Z�. My mind was blown at this point, as I thought, if the code doesn’t compile, surely the computer won’t be able to parse it, but this is not a program, this is an LLM, so of course it could read it kind of as a human would. Again, the code complexity is not great, but the fact that it was able to read a half-written patch, understand what I was working towards, and reason about, was mind-blowing, and scary. Like always.

Non-code writing

Now, after all this, while writing a recent blog post, I thought—this is going to be public anyway, so let me ask Claude what it thinks about it. And I was very surprised, again: gone was all the pain of rereading three times my post to catch typos (easy) or phrasing structure issues. It gave me very clearly points, and helped me cut 30-40% of the total time. So not only coding, but word smithing too is changed. If I were an author, I’d be delighted (and scared). Here is the overall reply it gave me:

  • Spelling and grammar fixes, all of them on point except one mistake (I claimed I didn’t capitalize one word, but I did). To the level of a good grammar checker.
  • Flow Suggestions, which was way beyond normal spelling and grammar. It felt like a teacher telling me to do better in my writing, i.e. nitpicking on things that actually were true even if they’d still work. I.e. lousy phrase structure, still understandable, but lousy nevertheless.
  • Other notes: an overall summary. This was mostly just praising my post 😅. I wish LLMs were not so focused on “praise the userâ€�.

So yeah, this speeds me up to about 2x on writing blog posts, too. It definitely feels not fair.

Wither the future?

After all this, I’m a bit flabbergasted. Gone are the 2000’s with code without unittests, gone are the 2010’s without CI/CD, and now, mid-2020’s, gone is the lone programmer that scours the internet to learn new things, alone?

What this all means for our skills in software development, I have no idea, except I know things have irreversibly changed (a butlerian jihad aside). Do I learn better with a dedicated tutor even if I don’t fight with the problem for so long? Or is struggling in finding good docs the main method of learning? I don’t know yet. I feel like I understand the topics I’m discussing with the AI, but who knows in reality what it will mean long term in terms of “stickiness� of learning. For the better, or for worse, things have changed. After all the advances over the last five centuries in mechanical sciences, it has now come to some aspects of the intellectual work.

Maybe this is the answer to the ever-growing complexity of tech stacks? I.e. a return of the lone programmer that builds things end-to-end, but with AI taming the complexity added in the last 25 years? I can dream, of course, but this also means that the industry overall will increase in complexity even more, because large companies tend to do that, so maybe a net effect of not much…

One thing I did learn so far is that my expectation that AI (at this level) will only help junior/beginner people, i.e. it would flatten the skills band, is not true. I think AI can speed up at least the middle band, likely the middle top band, I don’t know about the 10x programmers (I’m not one of them). So, my question about AI now is how to best use it, not to lament how all my learning (90% self learning, to be clear) is obsolete. No, it isn’t. AI helps me start and finish one migration (that I delayed for ages), then start the second, in the same day.

At the end of this—a bit rambling—reflection on the past month and a half, I still have many questions about AI and humanity. But one has been answered: yes, “AI�, quotes or no quotes, already has changed this field (producing software), and we’ve not seen the end of it, for sure.

David BrinJimmy Carter’s Big Mistake - And the noblest president of my lifetime.

By now, you all know that I offer contrarian views for Contrary Brin hoping to shake calcified assumptions like the lobotomizing ‘left-right spectrum.’ Or sometimes just to entertain…  

...(while remaining loyal to the Enlightenment Experiment that gave us all this one chance to escape brutal rule by kings & priests & inheritance brats, to maybe save the world and reach the stars.) 

At other times, contrariness can be a vent of frustration.  

(“You foooools! Why can’t you all seeeeee?!?”)


Okay, today's is of that kind. It's about one of the most admirable human beings I ever heard of – (and I know a lot of history). 


And yes, it's relevant to these fraught time!



==Somebody to look up to ==


Let's talk about former President Jimmy Carter, who passed away at 100, just a few months ago.


Sure, you hear one cliché about Carter, repeated all over: Carter was an ineffective president, but clearly a wonderful person, who redefined the EX-presidency. 


Folks thereupon go on to talk about the charitable efforts of both Carters, Jimmy and Rosalind. Such as the boost they gave to Habitat for Humanity, helping build houses for the poor and turning Habitat into a major concern, worldwide. That, compared to the selfishly insular after-office behaviors of every single Republican ex-president. Ever. And Habitat was just one of the Carters’ many fulfilling endeavors.


In fact, I have a crackpot theory (one of several that you’ll find only in this missive), that JC was absolutely determined not to die, until the very last Guinea Worm was gone. Helping first to kill off that gruesome parasite. 


Haven’t heard of it? Look it up; better yet, watch some cringeworthy videos about this horrible, crippling pest! International efforts – boosted by the Carter Center – drove the Guinea Worm to the verge of eradication, with only 14 human cases reported in 2023 and 13 in 2022. And it’s plausible that the extinction wail of the very last one happened in ’24, giving Jimmy Carter release from his vow. (Unlikely? Sure, but I like to think so. Though soon after his death, all of America was infested by a truly grotesque parasite...) 


So sure, after-office goodness is not what’s in question here. Nor the fact that JC was one of Rickover’s Boys (I came close to being one!) who established the U.S. nuclear submarine fleet that very likely restored deterrence in dangerous times and thus prevented World War Three. 


Or that, in Georgia, he was the first southern governor ever to stand up, bravely denouncing segregation and prejudice in all forms. 


(Someone who taught Baptist Sunday School for 80+ years ought to have been embraced by U.S. Christians, but for the fact that Carter emphasized the Beatitudes and the words and teachings of Jesus - like the Sermon on the Mount - rather than the bile-and-blood-drenched, psychotic Book of Revelation that now eroticizes so many who betray their own faith with gushers of lava-like hate toward their neighbors.) 


But doesn’t everyone concede that Jimmy Carter was an exceptionally fine example of humanity? 


In fact, among those with zero-sum personalities, such a compliment assists their denigration of impractical-goodie eggheads! It allows fools to smugly assert that such a generous soul must have also been gullible-sappy and impractical. 


(“He was a good person… and therefore, he must have been incompetent as president! While OUR hero, while clearly a corrupt, lying pervert and servant of Moscow, MUST - therefore - be the blessed agent of God!”)


Sick people. Truly sick.

And so, no, I’ll let others eulogize ‘what a nice fellow Jimmy Carter was.’ 


Today, I’m here to assail and demolish the accompanying nasty and utterly inaccurate slander: “…but he was a lousy president.”


No, he wasn’t. And I’ll fight anyone who says it. Because you slanderers don’t know your dang arse from… 


Okay, okay. Breathe.

Contrary Brin? Sure. 

But I mean it.



== Vietnam Fever ==


This mania goes all the way back to 1980. That year's utterly insipid “Morning in America” cult monomaniacally ignored the one central fact of that era


… that the United States of America had fallen for a trap that almost killed it. 


A trap that began in 1961, when a handsome, macho fool announced that “We will pay any price, bear any burden…” And schemers in Moscow rubbed their hands, answering:

“Really, Jack? ANY price? ANY burden? 

"How about a nice, big land war in the jungles of Southeast Asia?”


A war that became our national correlate to the Guinea Worm. 

Those of you who are too young to have any idea how traumatic the Vietnam War was… you can be forgiven. But anyone past or present who thought that everything would go back to 1962 bliss, when Kissinger signed the Paris Accords, proved themselves imbeciles. 

America was shredded, in part by social chasms caused by an insanely stupid war, plus too-long-delayed civil rights…

…but also economically, after LBJ and then Nixon tried for “Guns and Butter.” Running a full-scale war without inconveniently calling for sacrifices to pay for it. 

      Now throw in the OPEC oil crises! And the resulting inflation tore through America like an enema. 


Nixon couldn’t tame it. 

Ford couldn’t tame it. 

Neither of them had the guts.


Entering the White House, Jimmy Carter saw that the economy was teetering, and only strong medicine would work. Moreover, unlike any president, before or since, he cared only about the good of the nation.


As John Viril put it: “Jimmy Carter was, hands down, the most ethically sound President of my lifetime. He became President in the aftermath of Vietnam and during the second OPEC embargo. Carter's big achievement is that he killed hyper-inflation before it could trigger another depression, to the point that we didn't see it again for 40 years. Ronald Reagan gets credit for this, but it was Carter appointing tight-money Fed chairman Paul Volker that tamed inflation.”

Paul Volcker (look him up!) ran the Federal Reserve with tough love, because Carter told Volcker: “Fix this. And I won’t interfere. Not for the sake of politics or re-election. Patch the leaks in our boat. Put us on a diet. Fix it.”


Carter did this knowing that a tight money policy could trigger a recession that would very likely cost him re-election. The medicine tasted awful. 

  And it worked.

 Though it hurt like hell for 3 years, the post-Vietnam economic trauma got sweated out of the economy in record time. 

  In fact, just in time for things to settle down and for Ronald Reagan to inherit an economy steadying back onto an even keel. 

  His Morning in America.


Do you doubt that cause and effect? Care to step up with major wager stakes, before a panel of eminent economic historians? Because they know this and have said so. While politicians and media ignore them, in favor of Reagan idolatry.


Oh, and you who credit Reagan with starting the rebuilding of the U.S. military after Vietnam? 

   Especially the stealth techs and subs that are the core of our peacekeeping deterrence? 

  Nope.

  That was Carter, too.



== Restoring Trust ==


And there’s another unsung but vital thing that Jimmy Carter did, in the wake of Nixon-Ford and Vietnam. He restored faith in our institutions. In the aftermath of Watergate and J. Edgar Hoover and the rest, he made appointments who re-established some degree of trust. And historians (though never pundits or partisan yammerers) agree that he largely succeeded, by choosing skilled and blemish-free professionals, almost down the line.


And yes, let’s wager now over rates of turpitude in office, both before and since then. Or indictments for malfeasance, between the parties! Starting with Nixon, all the way to Biden and Trump II. It's night vs. day.


When the ratio of Republicans indicted and convicted for such crimes vs. Democrats approaches one hundred to one, is there any chance that our neighbors will notice… and decide that it is meaningful?

Not so long as idiots think that it makes them look so wise and cool to shake their heads and croon sadly “Both parties are the same!”


You, who sing that song, you don’t sound wise. 

You sound like an ignoramus. 

So, alas, it’s never actively refuted.

Not so long as Democrats habitually brag about the wrong things, and never mention facts like that one. The right ones.



== What about Reagan? ==


So. Yeah, yeah, you say. All of that may be true. But it comes to nothing, compared to Carter’s mishandling of the Iran Hostage Crisis.


Okay. This requires that – before getting to my main point - we first do an aside about Ronald Reagan. 


By now, the evidence is way more than circumstantial that Reagan committed treason during the Iran crisis. Negotiating through emissaries (some of whom admit it now!) for the Ayatollahs to hold onto the hostages till Carter got torched in the 1980 US election.  That’s a lot more than a ‘crackpot theory” by now… and yet I am not going in that direction, today.


Indeed, while I think his tenure set the modern theme for universal corruption of all subsequent Republican administrations, I have recently been extolling Ronald Reagan! Click and see all the many ways in which his tenure as California Governor seemed like Arnold Schwarzenegger's, calmly moderate! In 1970, Governor Reagan's policies made him almost an environmentalist Democrat! Certainly compared to today’s Foxite cult. 


Indeed, despite his many faults – the lying and corrupt officials, the AIDS cruelty and especially the triple-goddamned ‘War on Drugs’ – Reagan nevertheless, clearly wanted America to remain strong on the world stage. And to prevail against the Soviet ‘evil empire’…

… and I said as much to liberals of that era! I asked: “WTF else would you call something as oppressive and horrible as the USSR?” 


One thing I do know across all my being. Were he around today, Ronald Reagan would spit in the eyes of every current, hypocritical Republican Putin-lover and KGB shill, now helping all the Lenin-raised “ex” commissars over there to rebuild – in all it’s evil – the Soviet Union. With a few altered symbols and lapel pins. 


But again, that rant aside, what I have to say about Carter now departs from Reagan, his nemesis. 


Because this is not about Carter’s failed re-election. He already doomed any hope of that, when he told Volcker to fix the economy.


No, I am talking about Jimmy Carter’s Big Mistake.



== Iran…  ==


So sure, I am not going to assert that Carter didn’t fumble the Hostage Crisis. 


He did. Only not in the ways that you think! And here, not even the cautious historians get things right.


When the Shah fell, the fever that swept the puritan/Islamist half of Iranian society was intense and the Ayatollahs used that to entrench themselves. But when a mob of radicals stormed the American Embassy and took about a hundred U.S. diplomats hostage, the Ayatollahs faced a set of questions:


  • Shall we pursue vengeance on America – and specifically Carter – for supporting the Shah? Sounds good. But how hard should we push a country that’s so mighty? (Though note that post-Vietnam, we did look kinda lame.)
  • What kind of deal can we extort out of this, while claiming “We don’t even control that mob!”
  • And what’s our exit strategy?


During the subsequent, hellish year, it all seemed win-win for Khomeini and his clique. There was little we could do, without risking both the lives of the hostages and another oil embargo crisis, just as the U.S. economy was wobbling back onto its feet.


Yes, there was the Desert One rescue raid attempt, that failed because two helicopters developed engine trouble. Or – that’s the story. I do have a crackpot theory (What, Brin, another one?) about Desert One that I might insert into comments. If coaxed. No evidence, just a logical chain of thought.  (Except to note that it was immediately after that aborted raid that emissaries from the Islamic Republic hurried to Switzerland, seeking negotiations.)


But never mind that here. I told you that Jimmy Carter made one big mistake during the Iran Hostage Crisis, and he made it right at the beginning. By doing the right and proper and mature and legal thing.



== Too grownup. Too mature… ==


When that mob of ‘students’ took and cruelly abused the U.S. diplomats, no one on Earth swallowed the Ayatollah’s deniability claims of “it’s the kids, not me!” It was always his affair. And he hated Carter for supporting the Shah. And as we now know, Khomeini had promises from Reagan. So how could Carter even maneuver?


Well, he did start out with some chips on his side of the table. The Iranian diplomatic corps on U.S. soil. And prominent resident Iranians with status in the new regime -- those who weren’t seeking sanctuary at the time. Indeed, some voices called for them to be seized, as trading chips for our people in Tehran…


…and President Jimmy Carter shook his head, saying it would be against international law. Despite the fact that the Tehran regime holding our folks hostage was an act of war. Moreover, Carter believed in setting an example. And so, he diplomatically expelled those Iranian diplomats and arranged for them to get tickets home.


Honorable. Legal. And throwing them in jail would be illegal. And his setting an example might have worked… if the carrot had been accompanied by a big stick. If the adversary had not been in the middle of a psychotic episode. And… a whole lotta ifs.


I have no idea whether anyone in the Carter White House suggested this. But there was an intermediate action that might have hit the exact sweet spot. 


Arrest every Iranian diplomat and person on U.S. soil who was at all connected to the new regime… and intern them all at a luxury, beach-side hotel.


Allow news cameras to show the difference between civilized – even comfy - treatment and the nasty, foul things that our people were enduring, at the hands of those fervid ‘students.’ But above all, let those images – the stark contrast - continue, on and on and on. While American jingoists screeched and howled for our Iranian captives to be treated the same way. While the president refused.


Indeed, it is the contrast that would have torn world opinion, and any pretense of morality, away from the mullahs. And, with bikini-clad Americans strolling by daily, plus margaritas and waffles at the bar, wouldn’t their diplomats have screamed about their decadent torture? And pleaded for a deal – a swap of ‘hostages’ -- to come home? Or else, maybe one by one, might they defect?


We’ll never know. But it would have been worth a try. And every night, Walter Cronkite’s line might have been different.


And so, sure. Yeah. I think Carter made a mistake! And yeah, it was related to his maturity and goodness. So, I lied to you. Maybe he was too nice for the office. Too good for us to deserve.



== So, what’s my point? ==


I do have top heroes and Jimmy Carter is not one of them. 

Oh, I admired him immensely and thought him ill-treated by a nation he served well. But to me he is second-tier to Ben Franklin. To Lincoln and Tubman. To Jane Goodall and George Marshall.


But this missive is more about Carter’s despicable enemies. Nasty backstabber-liars and historical grudge-fabulators…


…of the same ilk as the bitchy slanderers who went on to savagely attack John Kerry, 100% of whose Vietnam comrades called him a hero, while 100% of the dastardly “swift-boaters” proved to be obscenely despicable, paid preeners, who were never even there.


Or the ‘birthers’ who never backed up a single word, but only screeched louder, when shown many time-yellowed copies of Obama’s 1962 birth announcement in the Honolulu Advertiser. Or the ass-hats who attacked John McCain and other decent, honorable Republicans who have fled the confederate madness, since Trump.


Or the myriad monstrous yammerers who now attack all fact-using professions, from science and teaching, medicine and law and civil service to the heroes of the FBI/Intel/Military officer corps who won the Cold War and the War on terror. 


Nutters and Kremlin-boys who aren’t worthy to shine the boots of a great defender-servant like Mark Milley.


Jeepers David… calm down. We get it. But take a stress pill, already, or you might burst a vessel.


Okay, okay. Though the blood bank says I have the blood pressure of a teenager...


... It’s just. Well.  We are about to embark on a journey of American self-discovery, when the very notions of democracy and enlightenment are under attack by living monsters. Monsters who know the power of symbolism vastly better than finger-wagging lib’ruls do, and who would deny us the inspiration of true heroes.


Mighty heroes like George Marshall. Like MLK. Like Elie Weisel and my Dad. Like Greta Thunberg and Amory Lovins. And those far-too-few Republicans who have found the patriotic decency to step up for the Union in this 8th phase of a 250 year American Civil War.


And like the subject of this essay. The best president (by many metrics) of the last over-100 years.




== And if you got all the way down here, some fun from SMBC ==



https://www.smbc-comics.com/comic/slam


Okay this one too, a glimpse of a better world with more Jimmy Carters in it, too.








Planet DebianSteinar H. Gunderson: Superimposed codes

I had a peculiar question at work recently, and it went off of a tangent that was way too long and somewhat interesting, so I wanted to share.

The question is: Can you create a set of N-bit numbers (codes), so that

a) Neither is a subset of each other, and
b) Neither is a subset of the OR of two of the others?

Of course, you can trivially do this (e.g., for N=5, choose 10000, 01000, 00100 and so on), but how many can you make for a given N? This is seemingly an open question, but at least I found that they are called (1,2) superimposed codes and have history at least back to this 1964 paper. They present a fairly elegant (but definitely non-optimal) way of constructing them for certain N; let me show an example for N=25:

We start by counting 3-digit numbers (k=3) in base 5 (q=5):

  • 000
  • 001
  • 002
  • 003
  • 004
  • 010
  • 011
  • etc…

Now we have 5^3 numbers. Let's set out to give them the property that we want.

This code (set of numbers) trivially has distance 1; that is, every number differs from every other number by at least one digit. We'd like to increase that distance so that it is at least as large as k. Reed-Solomon gives us an optimal way of doing that; for every number, we add two checksum digits and R-S will guarantee that the resulting code has distance 3. (Just trust me on this, I guess. It only works for q >= (k+1)/2, though, and q must be a power of an odd prime because otherwise the group theory doesn't work out.)

We now have a set of 5-digit numbers with distance 3. But if we now take any three numbers from this set, there is at least one digit where all three must differ, since the distance is larger than half the number of digits: Two numbers A and B differ from each other in at least 3 of the 5 digits, and A and C also has to differ from each other in at least 3 of the 5 digits. There just isn't room for A and B to be the same in all the places that A differ from C.

To modify this property into the one that we want, we encode each digit into binary using one-hot encoding (00001, 00010, 00100, etc.). Now our 5-digit numbers are 25-bit numbers. And due to the "all different" property in the previous paragraph, we also have our superimposition property; there's at least one 5-bit group where A|B shares no bits with C. So this gives us a 25-bit set with 125 different values and our desired property.

This isn't necessarily an optimal code (and the authors are very clear on that), but it's at least systematic and easy to extend to larger sizes. (I used a SAT solver to extend this to 170 different values, just by keeping the 125 first and asking for 45 more that were not in conflict. 55 more was evidently hard.) The paper has tons more information, including some stuff based on Steiner systems that I haven't tried to understand. And of course, there are tons more later papers, including one by Erdős. :-)

I've applied for an account at OEIS so I can add a sequence for the maximum number of possible codes for each N. It doesn't have many terms known yet, because the SAT solver struggles hard with this (at least in my best formulation), but at least it will give the next person something to find when they are searching. :-)

Planet DebianSahil Dhiman: Case of (broken) maharashtra.gov.in Authoritative Name Servers

Maharashtra is a state here in India, which has Mumbai, the financial capital of India, as its capital. maharashtra.gov.in is the official website of the State Government of Maharashtra. We’re going to talk about authoritative name servers serving it (and bunch of child zones under maharashtra.gov.in).

Here’s a simple trace for the main domain:

$ dig +trace maharashtra.gov.in

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> +trace maharashtra.gov.in
;; global options: +cmd
.            33128    IN    NS    j.root-servers.net.
.            33128    IN    NS    h.root-servers.net.
.            33128    IN    NS    l.root-servers.net.
.            33128    IN    NS    k.root-servers.net.
.            33128    IN    NS    i.root-servers.net.
.            33128    IN    NS    g.root-servers.net.
.            33128    IN    NS    f.root-servers.net.
.            33128    IN    NS    e.root-servers.net.
.            33128    IN    NS    b.root-servers.net.
.            33128    IN    NS    d.root-servers.net.
.            33128    IN    NS    c.root-servers.net.
.            33128    IN    NS    m.root-servers.net.
.            33128    IN    NS    a.root-servers.net.
.            33128    IN    RRSIG    NS 8 0 518400 20250704050000 20250621040000 53148 . pGxGZftwj+6VNTSQtstTKVN95Z7/b5Q8GSjRCXI68GoVYbVai9HNelxs OGIRKL4YmSrsiSsndXuEsBuvL9QvQ+qbybNLkekJUAiicKYNgr3KM3+X 69rsS9KxHgT2T8/oqG8KN8EJLJ8VkuM2PJ2HfSKijtF7ULtgBbERNQ4i u2I/wQ7elOyeF2M76iEOa7UGhgiBHSBqPulsbpnB//WbKL71yyFhWSk0 tiFEPuZM+iLrN2qBsElriF4kkw37uRHq8sSGcCjfBVdkpbb3/Sb3sIgN /zKU17f+hOvuBQTDr5qFIymqGAENA5UZ2RQjikk6+zK5EfBUXNpq1+oo 2y64DQ==
;; Received 525 bytes from 9.9.9.9#53(9.9.9.9) in 3 ms

in.            172800    IN    NS    ns01.trs-dns.com.
in.            172800    IN    NS    ns01.trs-dns.net.
in.            172800    IN    NS    ns10.trs-dns.org.
in.            172800    IN    NS    ns10.trs-dns.info.
in.            86400    IN    DS    48140 8 2 5EE4748C2069B99C98BC39A56881A64AF17CC78711E6297D43AC5A4F 4B5BB6E5
in.            86400    IN    RRSIG    DS 8 1 86400 20250704050000 20250621040000 53148 . jkCotYosapreoKKPvr9zPOEDECYVe9OtJLjkQbFfTin8uYbm/kdWzieW CkN5sabif5IHTFU4FEVOShfu4DFeUolhNav56TPKjGqEGjQ7qCghpqTj dNN4iY2s8BcJ2ujHwhm6HRfdbQRVoKYQ73UUZ+oWSute6lXWHE9+Snk2 1ZCAYPdZ2s1s7NZhrZW2YXVw/nHIcRl/rHqWIQ9sgUlsd6MwmahcAAG+ v15HG9Q48rCG1A2gJlJPbxWpVe0EUEu8LzDsp+ORqy1pHhzgJynrJHJz qMiYU0egv2j7xVPSoQHXjx3PG2rsOLNnqDBYCA+piEXOLsY3d+7c1SZl w9u66g==
;; Received 679 bytes from 199.7.83.42#53(l.root-servers.net) in 3 ms

maharashtra.gov.in.    900    IN    NS    ns8.maharashtra.gov.in.
maharashtra.gov.in.    900    IN    NS    ns9.maharashtra.gov.in.
maharashtra.gov.in.    900    IN    NS    ns10.maharashtra.gov.in.
maharashtra.gov.in.    900    IN    NS    ns18.maharashtra.gov.in.
maharashtra.gov.in.    900    IN    NS    ns20.maharashtra.gov.in.
npk19skvsdmju264d4ono0khqf7eafqv.gov.in. 300 IN    NSEC3 1 1 0 - P0KKR4BMBGLJDOKBGBI0KDM39DSM0EA4 NS SOA MX TXT RRSIG DNSKEY NSEC3PARAM
npk19skvsdmju264d4ono0khqf7eafqv.gov.in. 300 IN    RRSIG NSEC3 8 3 300 20250626140337 20250528184339 48544 gov.in. Khcq3n1Jn34HvuBEZExusVqoduEMH6DzqkWHk9dFkM+q0RVBYBHBbW+u LsSnc2/Rqc3HAYutk3EZeS+kXVF07GA/A486dr17Hqf3lHszvG/MNT/s CJfcdrqO0Q8NZ9NQxvAwWo44bCPaECQV+fhznmIaVSgbw7de9xC6RxWG ZFcsPYwYt07yB5neKa99RlVvJXk4GHX3ISxiSfusCNOuEKGy5cMxZg04 4PbYsP0AQNiJWALAduq2aNs80FQdWweLhd2swYuZyfsbk1nSXJQcYbTX aONc0VkYFeEJzTscX8/wNbkJeoLP0r/W2ebahvFExl3NYpb7b2rMwGBY omC/QA==
npk19skvsdmju264d4ono0khqf7eafqv.gov.in. 300 IN    RRSIG NSEC3 13 3 300 20250718144138 20250619135610 22437 gov.in. mbj7td3E6YE7kIhYoSlDTZR047TXY3Z60NY0aBwU7obyg5enBQU9j5nl GUxn9zUiwVUzei7v5GIPxXS7XDpk7g==
6bflkoouitlvj011i2mau7ql5pk61sks.gov.in. 300 IN    NSEC3 1 1 0 - 78S0UO5LI1KV1SVMH1889FHUCNC40U6T TXT RRSIG
6bflkoouitlvj011i2mau7ql5pk61sks.gov.in. 300 IN    RRSIG NSEC3 8 3 300 20250626133905 20250528184339 48544 gov.in. M2yPThQpX0sEf4klooQ06h+rLR3e3Q/BqDTSFogyTIuGwjgm6nwate19 jGmgCeWCYL3w/oxsg1z7SfCvDBCXOObH8ftEBOfLe8/AGHAEkWFSu3e0 s09Ccoz8FJiCfBJbbZK5Vf4HWXtBLfBq+ncGCEE24tCQLXaS5cT85BxZ Zne6Y6u8s/WPgo8jybsvlGnL4QhIPlW5UkHDs7cLLQSwlkZs3dwxyHTn EgjNWClhghGXP9nlvOlnDjUkmacEYeq5ItnCQjYPl4uwh9fBJ9CD/8LV K+Tn3+dgqDBek6+2HRzjGs59NzuHX8J9wVFxP7/nd+fUgaSgz+sST80O vrXlHA==
6bflkoouitlvj011i2mau7ql5pk61sks.gov.in. 300 IN    RRSIG NSEC3 13 3 300 20250718141148 20250619135610 22437 gov.in. raWzWsQnPkXYtr2v1SRH/fk2dEAv/K85NH+06pNUwkxPxQk01nS8eYlq BPQ41b26kikg8mNOgr2ULlBpJHb1OQ==
couldn't get address for 'ns18.maharashtra.gov.in': not found
couldn't get address for 'ns20.maharashtra.gov.in': not found
;; Received 1171 bytes from 2620:171:813:1534:8::1#53(ns10.trs-dns.org) in 0 ms

;; communications error to 10.187.202.24#53: timed out
;; communications error to 10.187.202.24#53: timed out
;; communications error to 10.187.202.24#53: timed out
;; communications error to 10.187.202.28#53: timed out
;; communications error to 10.187.203.201#53: timed out
;; no servers could be reached

Quick takeaways:

  • 5 authoritative NS are listed ie:

    • ns8.maharashtra.gov.in.
    • ns9.maharashtra.gov.in.
    • ns10.maharashtra.gov.in.
    • ns18.maharashtra.gov.in.
    • ns20.maharashtra.gov.in.
  • No address (no A/AAAA records) could be found for ns18.maharashtra.gov.in and ns20.maharashtra.gov.in. Internet Archive snapshots for bgp.tools at time of writing NS18 and NS20.

  • “communications error to 10.187.202.24#53: timed out”, “communications error to 10.187.202.28#53: timed out” and “communications error to 10.187.203.201#53: timed out” is likely due to RFC 1918 records for NS. Ofcourse, they will never respond on public internet.

  • Not in trace, but NS10 has private or empty A/AAAA record against it (detailed further down).

  • The query resolution failed with “no servers could be reached” ie we didn’t recieved any A/AAAA record for that query.

It’s a hit or miss for this DNS query resolution.

Looking at in zone data

Let’s look at NS added in zone itself (with 9.9.9.9):

$ dig ns maharashtra.gov.in

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> ns maharashtra.gov.in
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 172
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;maharashtra.gov.in.        IN    NS

;; ANSWER SECTION:
maharashtra.gov.in.    300    IN    NS    ns8.maharashtra.gov.in.
maharashtra.gov.in.    300    IN    NS    ns9.maharashtra.gov.in.

;; ADDITIONAL SECTION:
ns9.maharashtra.gov.in.    300    IN    A    10.187.202.24
ns8.maharashtra.gov.in.    300    IN    A    10.187.202.28

;; Query time: 180 msec
;; SERVER: 9.9.9.9#53(9.9.9.9) (UDP)
;; WHEN: Sat Jun 21 23:00:49 IST 2025
;; MSG SIZE  rcvd: 115

Pay special attention to “ADDITIONAL SECTION”. Running dig ns9.maharashtra.gov.in and dig ns8.maharashtra.gov.in, return RFC 1918 ie these private addresses. This is coming from zone itself, so in zone A records of NS8 and NS9 point to 10.187.202.28 and 10.187.202.24 respectively.

Cloudflare’s 1.1.1.1 has a slightly different version:

$ dig ns maharashtra.gov.in @1.1.1.1

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> ns maharashtra.gov.in @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36005
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;maharashtra.gov.in.        IN    NS

;; ANSWER SECTION:
maharashtra.gov.in.    300    IN    NS    ns8.
maharashtra.gov.in.    300    IN    NS    ns10.maharashtra.gov.in.
maharashtra.gov.in.    300    IN    NS    ns9.

;; Query time: 7 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Sun Jun 22 10:38:30 IST 2025
;; MSG SIZE  rcvd: 100

Interesting response here for sure :D.

The reason for difference between response from 1.1.1.1 and 9.9.9.9 is in the next section.

Looking at parent zone

gov.in is the parent zone here. Tucows is operator for gov.in as well as .in ccTLD zone:

$ dig ns gov.in +short
ns01.trs-dns.net.
ns01.trs-dns.com.
ns10.trs-dns.org.
ns10.trs-dns.info.

Let’s take a look at what parent zone (NS) hold:

$ dig ns maharashtra.gov.in @ns01.trs-dns.net.

; <<>> DiG 9.18.36 <<>> ns maharashtra.gov.in @ns01.trs-dns.net.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56535
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 5, ADDITIONAL: 6
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: f13027aa39632404010000006856fa2a9c97d6bbc973ba4f (good)
;; QUESTION SECTION:
;maharashtra.gov.in.        IN    NS

;; AUTHORITY SECTION:
maharashtra.gov.in.    900    IN    NS    ns8.maharashtra.gov.in.
maharashtra.gov.in.    900    IN    NS    ns18.maharashtra.gov.in.
maharashtra.gov.in.    900    IN    NS    ns10.maharashtra.gov.in.
maharashtra.gov.in.    900    IN    NS    ns9.maharashtra.gov.in.
maharashtra.gov.in.    900    IN    NS    ns20.maharashtra.gov.in.

;; ADDITIONAL SECTION:
ns20.maharashtra.gov.in. 900    IN    A    52.183.143.210
ns18.maharashtra.gov.in. 900    IN    A    35.154.30.166
ns10.maharashtra.gov.in. 900    IN    A    164.100.128.234
ns9.maharashtra.gov.in.    900    IN    A    103.23.150.89
ns8.maharashtra.gov.in.    900    IN    A    103.23.150.88

;; Query time: 28 msec
;; SERVER: 64.96.2.1#53(ns01.trs-dns.net.) (UDP)
;; WHEN: Sun Jun 22 00:00:02 IST 2025
;; MSG SIZE  rcvd: 248

The ADDITIONAL SECTION gives a completely different picture (different from in zone NSes). Maybe this was how it was supposed to be, but none of the IPs listed for NS10, NS18 and NS20 are responding to any DNS query.

Assuming NS8 as 103.23.150.88 and NS9 as 103.23.150.89, checking SOA on each gives following:

$ dig soa maharashtra.gov.in @103.23.150.88 +short
ns8.maharashtra.gov.in. postmaster.maharashtra.gov.in. 2013116777 1200 600 1296000 300

$ dig soa maharashtra.gov.in @103.23.150.89 +short
ns8.maharashtra.gov.in. postmaster.maharashtra.gov.in. 2013116757 1200 600 1296000 300

NS8 (which is marked as primary in SOA) has serial 2013116777 and NS9 is on serial 2013116757, so looks like the sync (IXFR/AXFR) between primary and secondary is broken. That’s why NS8 and NS9 are serving different responses, evident from the following:

$ dig ns8.maharashtra.gov.in @103.23.150.88 +short
103.23.150.88

$ dig ns8.maharashtra.gov.in @103.23.150.89 +short
10.187.202.28

$ dig ns9.maharashtra.gov.in @103.23.150.88 +short
103.23.150.89

$ dig ns9.maharashtra.gov.in @103.23.150.89 +short
10.187.202.24

$ dig ns maharashtra.gov.in @103.23.150.88 +short
ns9.
ns8.
ns10.maharashtra.gov.in.

$ dig ns maharashtra.gov.in @103.23.150.89 +short
ns9.maharashtra.gov.in.
ns8.maharashtra.gov.in.

$ dig ns10.maharashtra.gov.in @103.23.150.88 +short
10.187.203.201

$ dig ns10.maharashtra.gov.in @103.23.150.89 +short

# No/empty response ^

This is the reason for difference in 1.1.1.1 and 9.9.9.9 responses in previous section.

To summarize:

  • Primary and secondary NS aren’t in sync. Serials aren’t matching, while NS8 and NS9 are responding differently for same queries.
  • NSes have A records with private address, not reachable on the internet, so lame servers.
  • Incomplete NS address, not even FQDN in some cases.
  • Difference between NS delegated in parent zone and NS added in actual zone.
  • Name resolution works in very particular order (in my initial trace it failed).

Initially, I thought of citing RFCs, but I don’t really think it’s even required. 1.1.1.1, 8.8.8.8 and 9.9.9.9 are handling (lame servers, this probelm) well, handing out the A record for the main website, so dig maharashtra.gov.in would mostly pass and that was the reason I started this post with +trace to recurse the complete zone to show the problem.

For later reference:

$ dig maharashtra.gov.in @8.8.8.8 +short
103.8.188.109

Email to SOA address

I have sent the following email to address listed in SOA:


Subject - maharashtra.gov.in authoritative DNS servers not reachable

Hello,

I wanted to highlight the confusing state of maharashtra.gov.in authoritative DNS servers.

Parent zone list following as name servers for your DNS zone:

  • ns8.maharashtra.gov.in.
  • ns18.maharashtra.gov.in.
  • ns10.maharashtra.gov.in.
  • ns9.maharashtra.gov.in.
  • ns20.maharashtra.gov.in.

Out of these, ns18 and ns20 don’t have public A/AAAA records and are thus not reachable. ns10 keeps on shuffling between NO A record and 10.187.203.201 (private, not reachable address). ns8 keeps on shuffling between 103.23.150.88 and 10.187.202.28 (private, not reachable address). ns9 keeps on shuffling between 103.23.150.89 and 10.187.202.24 (private, not reachable address).

These are leading to long, broken, or no DNS resolution for the website(s). Can you take a look at the problem?

Regards, Sahil


I’ll update here if I get a response. Hopefully, they’ll listen and fix their problem.

365 TomorrowsI Hear You Like My Work

Author: Alaina Hammond Yesterday I received a text from an unknown number. “Hi! I hear you like my work!” I immediately knew who it was. Or rather, who it was pretending to be. It’s so creepy that the robots in my phone can tell what I’ve been reading. Even when it’s in paperback form, purchased […]

The post I Hear You Like My Work appeared first on 365tomorrows.

,

Planet DebianRavi Dwivedi: Getting Brunei visa

In December 2024, my friend Badri and I were planning a trip to Southeast Asia. At this point, we were planning to visit Singapore, Malaysia and Vietnam. My Singapore visa had already been approved, and Malaysia was visa-free for us. For Vietnam, we had to apply for an e-visa online.

We considered adding Brunei to our itinerary. I saw some videos of the Brunei visa process and got the impression that we needed to go to the Brunei embassy in Kuching, Malaysia in person.

However, when I happened to search for Brunei on Organic Maps1, I stumbled upon the Brunei Embassy in Delhi. It seemed to be somewhere in Hauz Khas. As I was going to Delhi to collect my Singapore visa the next day, I figured I’d also visit the Brunei Embassy to get information about the visa process.

The next day I went to the location displayed by Organic Maps. It was next to the embassy of Madagascar, and a sign on the road divider confirmed that I was at the right place.

That said, it actually looked like someone’s apartment. I entered and asked for directions to the Brunei embassy, but the people inside did not seem to understand my query. After some back and forth, I realized that the embassy wasn’t there.

I now searched for the Brunei embassy on the Internet, and this time I got an address in Vasant Vihar. It seemed like the embassy had been moved from Hauz Khas to Vasant Vihar. Going by the timings mentioned on the web page, the embassy was closing in an hour.

I took a Metro from Hauz Khas to Vasant Vihar. After deboarding at the Vasant Vihar metro station, I took an auto to reach the embassy. The address listed on the webpage got me into the correct block. However, the embassy was still nowhere to be seen. I asked around, but security guards in that area pointed me to the Burundi embassy instead.

After some more looking around, I did end up finding the embassy. I spoke to the security guards at the gate and told them that I would like to know the visa process. They dialled a number and asked that person to tell me the visa process.

I spoke to a lady on the phone. She listed the documents required for the visa process and mentioned that the timings for visa application were from 9 o’clock to 11 o’clock in the morning. She also informed me that the visa fees was ₹1000.

I also asked about the process Badri, who lives far away in Tamil Nadu and cannot report at the embassy physically. She told me that I can submit a visa application on his behalf, along with an authorization letter.

Having found the embassy in Delhi was a huge relief. The other plan - going to Kuching, Malaysia - was a bit uncertain, and we didn’t know how much time it would take. Getting our passport submitted at an embassy in a foreign country was also not ideal.

A few days later, Badri sent me all the documents required for his visa. I went to the embassy and submitted both the applications. The lady who collected our visa submissions asked me for our flight reservations from Delhi to Brunei, whereas ours were (keeping with our itinerary) from Kuala Lampur. She said that she might contact me later if it was required.

For reference, here is the list of documents we submitted -

  • Visa application form
  • Passport
  • A photocopy of passport
  • Authorization letter from Badri (authorizing me to submit his application on his behalf)
  • Airline ticket itinerary
  • Hotel bookings
  • Cover letter
  • 2 photos
  • Proof of employment
  • 6 months bank statement (they specifically asked for ₹1,00,000 or more in bank balance)

I then asked about the procedure to collect the passports and visa results. Usually, embassies will tell you that they will contact you when they have decided on your applications. However, here I was informed that if they don’t contact me within 5 days, I can come and collect our passports and visa result between 13:30-14:30 hours on the fifth day. That was strange :)

I did visit the embassy to collect our visa results on the fifth day. However, the lady scolded me for not bringing the receipt she gave me. I was afraid that I might have to go all the way back home and bring the receipt to get our passports. The travel date was close, and it would take some time for Badri to receive his passport via courier as well.

Fortunately, she gave me our passports (with the visa attached) and asked me to share a scanned copy of the receipt via email after I get home.

We were elated that our visas were approved. Now we could focus on booking our flights.

If you are going to Brunei, remember to fill their arrival card from the website within 48 hours of your arrival!

Thanks to Badri and Contrapunctus for reviewing the draft before publishing the article.


  1. Nowadays, I prefer using Comaps instead of Organic Maps and recommend you do the same. Organic Maps had some issues with its governance and the community issues weren’t being addressed. ↩︎

365 Tomorrows11 to Midnight

Author: Claire Robertson Those four great comets pull white scars through the sky. Fans of fire expand over our heads, and you still can’t bear to look at me despite how I ask you to. I want the last thing I see to be something familiar. The half-eaten chocolate cake between us will have to […]

The post 11 to Midnight appeared first on 365tomorrows.

,

Planet DebianSven Hoexter: Terraform: Validation Condition Cycles

Terraform 1.9 introduced some time ago the capability to reference in an input variable validation condition other variables, not only the one you're validating.

What does not work is having two variables which validate each other, e.g.

variable "nat_min_ports" {
  description = "Minimal amount of ports to allocate for 'min_ports_per_vm'"
  default     = 32
  type        = number
  validation {
    condition = (
      var.nat_min_ports >= 32 &&
      var.nat_min_ports <= 32768 &&
      var.nat_min_ports < var.nat_max_ports
    )
    error_message = "Must be between 32 and 32768 and less than 'nat_max_ports'"
  }
}

variable "nat_max_ports" {
  description = "Maximal amount of ports to allocate for 'max_ports_per_vm'"
  default     = 16384
  type        = number
  validation {
    condition = (
      var.nat_max_ports >= 64 &&
      var.nat_max_ports <= 65536 &&
      var.nat_max_ports > var.nat_min_ports
    )
    error_message = "Must be between 64 and 65536 and above 'nat_min_ports'"
  }
}

That let directly to the following rather opaque error message: Received an error Error: Cycle: module.gcp_project_network.var.nat_max_ports (validation), module.gcp_project_network.var.nat_min_ports (validation)

Removed the sort of duplicate check var.nat_max_ports > var.nat_min_ports on nat_max_ports to break the cycle.

Worse Than FailureError'd: Colophony

Just a quick note this week: I discovered that many people have been sending in submissions for this column and designating them for CodeSod by mistakes. Consequently, there is an immense backlog of material from which to choose. An abundance of riches! We will be seeing some older items in the future. For today, a collection of colons:

Bill NoLastName , giving away clues to his banking security questions online: "If had known there was a limit, I would have changed my daughter's middle name. I've been caught by this before - my dad has only a middle initial (no middle name)."

0

 

Gordon F. heard of a greal deal: "This is the first mention of shipping on a hearing aids website. Tough choice."

1

 

Michael P. underlines a creative choice: "I got an email from a recruiter about a job opening. I'm a little confused about the requirements."

2

 

Cole T. pretend panics about pennies (and maybe we need an article about false urgency, hm): "Oh no! My $0 in rewards are about to expire!"

3

 

Finally, bibliophile WeaponizedFun (alsonolastname) humblebrags erudition. It ain't War & Peace, but it's still an ordeal! "After recently finishing The Brothers Karamazov after 33 hours on audio disc, I was a bit surprised to see that Goodreads is listing this as my longest book with only 28 pages. 28 discs, maybe, but I still am questioning their algorithm, because this just looks silly."

4

 

[Advertisement] Picking up NuGet is easy. Getting good at it takes time. Download our guide to learn the best practice of NuGet for the Enterprise.

Planet DebianMatthew Garrett: My a11y journey

23 years ago I was in a bad place. I'd quit my first attempt at a PhD for various reasons that were, with hindsight, bad, and I was suddenly entirely aimless. I lucked into picking up a sysadmin role back at TCM where I'd spent a summer a year before, but that's not really what I wanted in my life. And then Hanna mentioned that her PhD supervisor was looking for someone familiar with Linux to work on making Dasher, one of the group's research projects, more usable on Linux. I jumped.

The timing was fortuitous. Sun were pumping money and developer effort into accessibility support, and the Inference Group had just received a grant from the Gatsy Foundation that involved working with the ACE Centre to provide additional accessibility support. And I was suddenly hacking on code that was largely ignored by most developers, supporting use cases that were irrelevant to most developers. Being in a relatively green field space sounds refreshing, until you realise that you're catering to actual humans who are potentially going to rely on your software to be able to communicate. That's somewhat focusing.

This was, uh, something of an on the job learning experience. I had to catch up with a lot of new technologies very quickly, but that wasn't the hard bit - what was difficult was realising I had to cater to people who were dealing with use cases that I had no experience of whatsoever. Dasher was extended to allow text entry into applications without needing to cut and paste. We added support for introspection of the current applications UI so menus could be exposed via the Dasher interface, allowing people to fly through menu hierarchies and pop open file dialogs. Text-to-speech was incorporated so people could rapidly enter sentences and have them spoke out loud.

But what sticks with me isn't the tech, or even the opportunities it gave me to meet other people working on the Linux desktop and forge friendships that still exist. It was the cases where I had the opportunity to work with people who could use Dasher as a tool to increase their ability to communicate with the outside world, whose lives were transformed for the better because of what we'd produced. Watching someone use your code and realising that you could write a three line patch that had a significant impact on the speed they could talk to other people is an incomparable experience. It's been decades and in many ways that was the most impact I've ever had as a developer.

I left after a year to work on fruitflies and get my PhD, and my career since then hasn't involved a lot of accessibility work. But it's stuck with me - every improvement in that space is something that has a direct impact on the quality of life of more people than you expect, but is also something that goes almost unrecognised. The people working on accessibility are heroes. They're making all the technology everyone else produces available to people who would otherwise be blocked from it. They deserve recognition, and they deserve a lot more support than they have.

But when we deal with technology, we deal with transitions. A lot of the Linux accessibility support depended on X11 behaviour that is now widely regarded as a set of misfeatures. It's not actually good to be able to inject arbitrary input into an arbitrary window, and it's not good to be able to arbitrarily scrape out its contents. X11 never had a model to permit this for accessibility tooling while blocking it for other code. Wayland does, but suffers from the surrounding infrastructure not being well developed yet. We're seeing that happen now, though - Gnome has been performing a great deal of work in this respect, and KDE is picking that up as well. There isn't a full correspondence between X11-based Linux accessibility support and Wayland, but for many users the Wayland accessibility infrastructure is already better than with X11.

That's going to continue improving, and it'll improve faster with broader support. We've somehow ended up with the bizarre politicisation of Wayland as being some sort of woke thing while X11 represents the Roman Empire or some such bullshit, but the reality is that there is no story for improving accessibility support under X11 and sticking to X11 is going to end up reducing the accessibility of a platform.

When you read anything about Linux accessibility, ask yourself whether you're reading something written by either a user of the accessibility features, or a developer of them. If they're neither, ask yourself why they actually care and what they're doing to make the future better.

comment count unavailable comments

365 TomorrowsAssisted Living

Gramps started slipping after his 105th birthday. Nothing dramatic, just forgetting a story or two, repeating a conversation from the hour before, stuff like that. Our family and about 40 others went to the surgical center for the informational briefings about a revolutionary AI “personality bridge” implant. There was a slick corporate infomercial and then […]

The post Assisted Living appeared first on 365tomorrows.

Planet DebianRussell Coker: The Intel Arc B580 and PCIe Slot Size

A few months ago I bought a Intel Arc B580 for the main purpose of getting 8K video going [1]. I had briefly got it working in a test PC but then I wanted to deploy it on my HP z840 that I use as a build server and for playing with ML stuff [2]. I only did brief tests of it previously and this was my first attempt at installing it in a system I use. My plan was to keep the NVidia RTX A2000 in place and run 2 GPUs, that’s not an uncommon desire among people who want to do ML stuff and it’s the type of thing that the z840 is designed for, the machine has slots 2, 4, and 6 being PCIe*16 so it should be able to fit 3 cards that each take 2 slots. So having one full size GPU, the half-height A2000, and a NVMe controller that uses *16 to run four NVMe devices should be easy.

Intel designed the B580 to use every millimeter of space possible while still being able to claim to be a 2 slot card. On the circuit board side there is a plastic cover over the board that takes all the space before the next slot so a 2 slot card can’t go on that side without having it’s airflow blocked. On the other side it takes all the available space so that any card that wants to blow air through can’t fit and also such that a medium size card (such as the card for 4 NVMe devices) would block it’s air flow. So it’s impossible to have a computer with 6 PCIe slots run the B580 as well as 2 other full size *16 cards.

Support for this type of GPU is something vendors like HP should consider when designing workstation class systems. For HP there is no issue of people installing motherboards in random cases (the HP motherboard in question uses proprietary power connectors and won’t even boot with an ATX PSU without significant work). So they could easily design a motherboard and case with a few extra mm of space between pairs of PCIe slots. The cards that are double width are almost always *16 so you could pair up a *16 slot and another slot and have extra space on each side of the pair. I think for most people a system with 6 PCIe slots with a bit of extra space for GPU cooling would be more useful than having 7 PCIe slots. But as HP have full design control they don’t even need to reduce the number of PCIe slots, they could just make the case taller. If they added another 4 slots and increased the case size accordingly it still wouldn’t be particularly tall by the standards of tower cases from the 90s! The z8 series of workstations are the biggest workstations that HP sells so they should design them to do these things. At the time that the z840 was new there was a lot of ML work being done and HP was selling them as ML workstations, they should have known how people would use them and design them accordingly.

So I removed the NVidia card and decided to run the system with just the Arc card, things should have been fine but Intel designed the card to be as high as possible and put the power connector on top. This prevented installing the baffle for directing air flow over the PCIe slots and due to the design of the z840 (which is either ingenious or stupid depending on your point of view) the baffle is needed to secure the PCIe cards in place. So now all the PCIe cards are just secured by friction in the slots, this isn’t an unusual situation for machines I assemble but it’s not something I desired.

This is the first time I’ve felt compelled to write a blog post reviewing a product before even getting it working. But the physical design of the B580 is outrageously impractical unless you are designing your entire computer around the GPU.

As an aside the B580 does look very nice. The plastic surround is very fancy, it’s a pity that it interferes with the operation of the rest of the system.

Planet DebianReproducible Builds (diffoscope): diffoscope 299 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 299. This version includes the following changes:

[ Chris Lamb ]
* Add python3-defusedxml to the Build-Depends in order to include it in the
  Docker image. (Closes: #407)

You find out more by visiting the project homepage.

,

Planet DebianJonathan Carter: My first tag2upload upload

Tag2upload?

The tag2upload service has finally gone live for Debian Developers in an open beta.

If you’ve never heard of tag2upload before, here is a great primer presented by Ian Jackson and prepared by Ian Jackson and Sean Whitton.

In short, the world has moved on to hosting and working with source code in Git repositories. In Debian, we work with source packages that are used to generated the binary artifacts that users know as .deb files. In Debian, there is so much tooling and culture built around this. For example, our workflow passes what we call the island test – you could take every source package in Debian along with you to an island with no Internet, and you’ll still be able to rebuild or modify every package. When changing the workflows, you risk losing benefits like this, and over the years there has been a number of different ideas on how to move to a purely or partially git flow for Debian, none that really managed to gain enough momentum or project-wide support.

Tag2upload makes a lot of sense. It doesn’t take away any of the benefits of the current way of working (whether technical or social), but it does make some aspects of Debian packages significantly simpler and faster. Even so, if you’re a Debian Developer and more familiar with how the sausage have made, you’ll have noticed that this has been a very long road for the tag2upload maintainers, they’ve hit multiple speed bumps since 2019, but with a lot of patience and communication and persistence from all involved (and almost even a GR), it is finally materializing.

Performing my first tag2upload

So, first, I needed to choose which package I want to upload. We’re currently in hard freeze for the trixie release, so I’ll look for something simple that I can upload to experimental.

I chose bundlewrap, it’s quote a straightforward python package, and updates are usually just as straightforward, so it’s probably a good package to work on without having to deal with extra complexities in learning how to use tag2upload.

So, I do the usual uscan and dch -i to update my package…

And then I realise that I still want to build a source package to test it in cowbuilder. Hmm, I remember that Helmut showed me that building a source package isn’t necessary in sbuild, but I have a habit of breaking my sbuild configs somehow, but I guess I should revisit that.

So, I do a dpkg-buildpackage -S -sa and test it out with cowbuilder, because that’s just how I roll (at least for now, fixing my local sbuild setup is yak shaving for another day, let’s focus!).

I end up with a binary that looks good, so I’m satisfied that I can upload this package to the Debian archives. So, time to configure tag2upload.

The first step is to set up the webhook in Salsa. I was surprised two find two webhooks already configured:

I know of KGB that posts to IRC, didn’t know that this was the mechanism it does that by before. Nice! Also don’t know what the tagpending one does, I’ll go look into that some other time.

Configuring a tag2upload webhook is quite simple, add a URL, call the name tag2upload, and select only tag push events:

I run the test webhook, and it returned a code 400 message about a missing ‘message’ header, which the documentation says is normal.

Next, I install git-debpush from experimental.

The wiki page simply states that you can use the git-debpush command to upload, but doesn’t give any examples on how to use it, and its manpage doesn’t either. And when I run just git-debpush I get:

jonathan@lapcloud:~/devel/debian/python-team/bundlewrap/bundlewrap-4.23.1$ git-debpush
git-debpush: check failed: upstream tag upstream/4.22.0 is not an ancestor of refs/heads/debian/master; probably a mistake ('upstream-nonancestor' check)
pristine-tar is /usr/bin/pristine-tar
git-debpush: some check(s) failed; you can pass --force to ignore them

I have no idea what that’s supposed to mean. I was also not sure whether I should tag anything to begin with, or if some part of the tag2upload machinery automatically does it. I think I might have tagged debian/4.23-1 before tagging upstream/4.23 and perhaps it didn’t like it, I reverted and did it the other way around and got a new error message. Progress!

jonathan@lapcloud:~/devel/debian/python-team/bundlewrap/bundlewrap-4.23.1$ git-debpush
git-debpush: could not determine the git branch layout
git-debpush: please supply a --quilt= argument

Looking at the manpage, it looks like –quilt=baredebian matches my package the best, so I try that:

jonathan@lapcloud:~/devel/debian/python-team/bundlewrap/bundlewrap-4.23.1$ git-debpush --quilt=baredebian
Enumerating objects: 70, done.
Counting objects: 100% (70/70), done.
Delta compression using up to 12 threads
Compressing objects: 100% (37/37), done.
Writing objects: 100% (37/37), 8.97 KiB | 2.99 MiB/s, done.
Total 37 (delta 30), reused 0 (delta 0), pack-reused 0 (from 0)
To salsa.debian.org:python-team/packages/bundlewrap.git
6f55d99..3d5498f debian/master -> debian/master

 * [new tag] upstream/4.23.1 -> upstream/4.23.1
 * [new tag] debian/4.23.1-1_exp1 -> debian/4.23.1-1_exp1

Ooh! That looked like it did something! And a minute later I received the notification of the upload in my inbox:

So, I’m not 100% sure that this makes things much easier for me than doing a dput, but, it’s not any more difficult or more work either (once you know how it works), so I’ll be using git-debpush from now on, and I’m sure as I get more used to the git workflow of doing things I’ll understand more of the benefits. And at last, my one last use case for using FTP is now properly dead. RIP FTP :)

Planet DebianRussell Coker: Matching Intel CPUs

To run a SMP system with multiple CPUs you need to have CPUs that are “identical”, the question is what does “identical” mean. In this case I’m interested in Intel CPUs because SMP motherboards and server systems for Intel CPUs are readily available and affordable. There are people selling matched pairs of CPUs on ebay which tend to be more expensive than randomly buying 2 of the same CPU model, so if you can identify 2 CPUs that are “identical” which are sold separately then you can save some money. Also if you own a two CPU system with only one CPU installed then buying a second CPU to match the first is cheaper and easier than buying two more CPUs and removing a perfectly working CPU.

e5-2640 v4 cpus

Intel (R) Xeon (R)
E5-2640V4
SR2NZ 2.40GHZ
J717B324 (e4)
7758S4100843

Above is a pic of 2 E5-2640v4 CPUs that were in a SMP system I purchased along with a plain ASCII representation of the text on one of them. The bottom code (starting with “77”) is apparently the serial number, one of the two codes above it is what determines how “identical” those CPUs are.

The code on the same line as the nominal clock speed (in this case SR2NZ) is the “spec number” which is sometimes referred to as “sspec” [1].

The line below the sspec and above the serial number has J717B324 which doesn’t have a google hit. I looked at more than 20 pics of E5-2640v4 CPUs on ebay, they all had the code SR2NZ but had different numbers on the line below. I conclude that the number on the line below probably indicates the model AND stepping while SR2NZ just means E5-2640v4 regardless of stepping. As I wasn’t able to find another CPU on ebay with the same number on the line below the sspec I believe that it will be unreasonably difficult to get a match for an existing CPU.

For the purpose of matching CPUs I believe that if the line above the serial number matches then the CPUs can be used together. I am not certain that CPUs with this number slightly mismatching won’t work but I definitely wouldn’t want to spend money on CPUs with this number being different.

smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2699A v4 @ 2.40GHz (family: 0x6, model: 0x4f, stepping: 0x1)

When you boot Linux the kernel identifies the CPU in a manner like the above, the combination of family and model seem to map to one spec number. The combination of family, model, and stepping should be all that’s required to have them work together.

I think that Intel did the wrong thing in not making this clearer. It would have been very easy to print the stepping on the CPU case next to the sspec or the CPU model name. It also wouldn’t have been too hard to make the CPU provide the magic number that is apparently the required match for SMP to the OS. Having the Intel web site provide a mapping of those numbers to steppings of CPUs also shouldn’t be difficult for them.

If anyone knows more about these issues please let me know.

Worse Than FailureCodeSOD: Using the Old Bean

If you write a lot of Java, you're going to end up writing a lot of getters and setters. Without debating the merits of loads of getters and setters versus bare properties, ideally, getters and setters are the easiest code to write. Many IDEs will just generate them for you! How can you screw up getters and setters?

Well, Dave found someone who could.

private ReportDatesDao reportDatesDao;
@Resource(name = CensusDao.BEAN_NAME)
public void setAuditDao(CensusDao censusDao) {
   this.reportDatesDao = reportDatesDao;
}

The function is called setAuditDao, takes a CensusDao input, but manipulates reportDatesDao, because clearly someone copy/pasted and didn't think about what they were doing.

The result, however, is that this just sets this.reportDatesDao equal to itself.

I'm always impressed by code which given the chance to make multiple decisions makes every wrong choice, even if it is just lazy copy/paste.

[Advertisement] Plan Your .NET 9 Migration with Confidence
Your journey to .NET 9 is more than just one decision.Avoid migration migraines with the advice in this free guide. Download Free Guide Now!

365 TomorrowsTraveler Talk

Author: Angela Hawn “Ready to sing for your supper?” The head honcho in the antique army helmet flashes a toothy smile at our little group before acknowledging the wider audience. Applause ensues. “Of course”, I say, channeling my storytelling grandmother whose entertaining melodrama once served multiple purposes: convincing me to sleep, to eat my vegetables, […]

The post Traveler Talk appeared first on 365tomorrows.

Planet DebianDebian Outreach Team: GSoC 2025 Introduction: Make Debian for Raspberry Pi Build Again

Hello everyone! I am Kurva Prashanth, Interested in the lower level working of system software, CPUs/SoCs and Hardware design. I was introduced to Open Hardware and Embedded Linux while studying electronics and embedded systems as part of robotics coursework. Initially, I did not pay much attention to it and quickly moved on. However, a short talk on “Liberating SBCs using Debian” by Yuvraj at MiniDebConf India, 2021 caught my interest. The talk focused on Open Hardware platforms such as Olimex and BeagleBone Black, as well as the Debian distributions tailored for these ARM-based single-board computers has intrigued me to delve deeper into the realm of Open Hardware and Embedded Linux.

These days I’m trying to improve my abilities to contribute to Debian and Linux Kernel development. Before finding out about the Google Summer of Code project, I had already started my journey with Debian. I extensively used Debian system build tools(debootstrap, sbuild, deb-build-pkg, qemu-debootstrap) for Building Debian Image for Bela Cape a real-time OS for music making to achieve extremely fast audio and sensor processing times. In 2023, I had the opportunity to attend DebConf23 in Kochi, India - thanks to Nilesh Patra (@nilesh) and I met Hector Oron (@zumbi) over dinner at DebConf23 and It was nice talking about his contributions/work at Debian on armhf port and Debian System Administration that conversation got me interested in knowing more about Debian ARM, Installer and I found it fascinating that EmDebian was once a external project bringing Debian to embedded systems and now, Debian itself can be run on many embedded systems. And, also during DebCamp I got Introduced to PGP/GPG keys and the web of trust by Carlos Henrique Lima Melara (@charles) I learned how to use and generate GPG keys. After DebConf23 I tried debian packaging and I miserably failed to get sponsorship for a python library I packaged.

I came across the Debian project for this year’s Google Summer of Code and found the project titled Make Debian for Raspberry Pi Build Again quite interesting to me and applied. Gladly, on May 8th, I received an acceptance e-mail from GSoC. I got excited that I’ll spend the summer working on something that I like doing.

I am thrilled to be part of this project and I am super excited for the summer of‘25. I’m looking forward to work on what I most like, new connections and learning opportunities.

So, let me talk a bit more about my project. I will be working on to Make Debian for Raspberry Pi SBC’s under the guidance of Gunnar Wolf (@gwolf). In this post, I will describe the project I will be working on.

Why make Debian for Raspberry Pi build again?

There is an available set of images for running Debian in Raspberry Pi computers (all models below the 5 series)! However, the maintainer severely lacking time to take care for them; called for help for somebody to adopt them, but have not been successful. The image generation scripts might have bitrotted a bit, but it is mostly all done. And there is a lot of interest and use still in having the images freshly generated and decently tested! This GSoC project is about getting the [https://raspi.debian.net/ | Raspberry Pi Debian images] site working reliably, daily-built images become automatic again and ideally making it easily deployable to be run in project machines and migrating exsisting hosting infrastructure to Debian.

How much it differ from Debian build process?

While the goal is to stay as close as possible to the Debian build process, Raspberry Pi boards require some necessary platform-specific changes primarily in the early boot sequence and firmware handling. Unlike typical Debian systems, Raspberry Pi boards depend on a non-standard bootloader and use non-free firmware (raspi-firmware), Introducing some hardware-specific differences in the initialization process.

These differences are largely confined to the early boot and hardware initialization stages. Once the system boots, the userspace remains closely aligned with a typical Debian install, using Debian packages.

The current modifications are required due to non-free firmware. However, several areas merit review: but there are a few parts that might be worth changing.

  1. Boot flow: Transitioning to a U-Boot based boot process (as used in Debian installer images for many other SBCs) would reduce divergence and better align with Debian Installer.

  2. Current scripts/workarounds: Some existing hacks may now be redundant with recent upstream support and could be removed.

  3. Board-specific images: Shift to architecture-specific base images with runtime detection could simplify builds and reduce duplication.

Debian already already building SD card images for a wide range of SBCs (e.g., BeagleBone, BananaPi, OLinuXino, Cubieboard, etc.) installer-arm64/images/u-boot and installer-armhf/images/u-boot, a similar approach for Raspberry Pi could improve maintainability and consistency with Debian’s broader SBC support.

Quoted from Mail Discussion Thread with Mentor (Gunnar Wolf)

"One direction we wanted to explore was whether we should still be building one image per family, or whether we could instead switch to one image per architecture (armel, armhf, arm64). There were some details to iron out as RPi3 and RPi4 were quite different, but I think it will be similar to the differences between the RPi 0 and 1, which are handled at first-boot time. To understand what differs between families, take a look at Cyril Brulebois’ generate-recipe (in the repo), which is a great improvement over the ugly mess I had before he contributed it"

In this project, I intend to to build one image per architecture (armel, armhf, arm64) rather than continuing with the current model of building one image per board. This change simplifies image management, reduces redundancy, and leverages dynamic configuration at boot time to support all supported boards within each architecture. By using U-Boot and flash-kernel, we can detect the board type and configure kernel parameters, DTBs, and firmware during the first boot, reducing duplication across images and simplifying the maintenance burden and we can also generalize image creation while still supporting board-specific behavior at runtime. This method aligns with existing practices in the DebianInstaller team and aligns with Debian’s long-term maintainability goals and better leverages upstream capabilities, ensuring a consistent and scalable boot experience.

To streamline and standardize the process of building bootable Debian images for Raspberry Pi devices, I proposed a new workflow that leverages U-Boot and flash-kernel Debian packages. This provides a clean, maintainable, and reproducible way to generate images for armel, armhf and arm64 boards. The workflow is vmdb2, a lightweight, declarative tool designed to automate the creation of disk images. A typical vmdb2 recipe defines the disk layout, base system installation (via debootstrap), architecture-specific packages, and any custom post-install hooks and the image should includes U-Boot (the u-boot-rpi package), flash-kernel, and a suitable Debian kernel package like linux-image-arm64 or linux-image-armmp.

U-Boot serves as the platform’s bootloader and is responsible for loading the kernel and initramfs. Unlike Raspberry Pi’s non-free firmware/proprietary bootloader, U-Boot provides an open and scriptable interface, allowing us to follow a more standard Debian boot process. It can be configured to boot using either an extlinux.conf or a boot.scr script generated automatically by flash-kernel. The role of flash-kernel is to bridge Debian’s kernel installation system with the specifics of embedded bootloaders like U-Boot. When installed, it automatically copies the kernel image, initrd, and device tree blobs (DTBs) to the /boot partition. It also generates the necessary boot.scr script if the board configuration demands it. To work correctly, flash-kernel requires that the target machine be identified via /etc/flash-kernel/machine, which must correspond to an entry in its internal machine database.\ Once the vmdb2 build is complete, the resulting image will contain a fully configured bootable system with all necessary boot components correctly installed. The image can be flashed to an SD card and used to boot on the intended device without additional manual configuration. Because all key packages (U-Boot, kernel, flash-kernel) are managed through Debian’s package system, kernel updates and boot script regeneration are handled automatically during system upgrades.

Current Workflow: Builds one Image per family

The current vmdb2 recipe uses the Raspberry Pi GPU bootloader provided via the raspi-firmware package. This is the traditional boot process followed by Raspberry Pi OS, and it’s tightly coupled with firmware files like bootcode.bin, start.elf, and fixup.dat. These files are installed to /boot/firmware, which is mounted from a FAT32 partition labeled RASPIFIRM. The device tree files (*.dtb) are manually copied from /usr/lib/linux-image-*-arm64/broadcom/ into this partition.

The kernel is installed via the linux-image-arm64 package, and the boot arguments are injected by modifying /boot/firmware/cmdline.txt using sed commands. Booting depends on the root partition being labeled RASPIROOT, referenced through that file. There is no bootloader like UEFI-based or U-Boot involved — the Raspberry Pi firmware directly loads the kernel, which is standard for Raspberry Pi boards.

- apt: install
  packages:
    ...
    - raspi-firmware  

The boot partition contents and kernel boot setup are tightly controlled via scripting in the recipe.

Limitations of Current Workflow: While this setup works, it is

  1. Proprietary and Raspberry Pi–specific – It relies on the closed-source GPU bootloader the raspi-firmware package, which is tightly coupled to specific Raspberry Pi models.

  2. Manual DTB handling – Device tree files are manually copied and hardcoded, making upgrades or board-specific changes error-prone.

  3. Not easily extendable to future Raspberry Pi boards – Any change in bootloader behavior (as seen in the Raspberry Pi 5, which introduces a more flexible firmware boot process) would require significant rework.

  4. No UEFI-based/U-Boot – The current method bypasses the standard bootloader layers, making it inconsistent with other Debian ARM platforms and harder to maintain long-term.

As Raspberry Pi firmware and boot processes evolve, especially with the introduction of Pi 5 and potentially Pi 6, maintaining compatibility will require more flexibility - something best delivered by adopting U-Boot and flash-kernel.

New Workflow: Building Architecture-Specific Images with vmdb2, U-Boot, flash-kernel, and Debian Kernel

This workflow outlines an improved approach to generating bootable Debian images architecture specific, using vmdb2, U-Boot, flash-kernel, and Debian kernels and also to move away from Raspberry Pi’s proprietary bootloader to a fully open-source boot process which improves maintainability, consistency, and cross-board support.

New Method: Shift to U-Boot + flash-kernel

U-Boot (via Debian’su-boot-rpi package) and flash-kernel bring the image building process closer to how Debian officially boots ARM devices. flash-kernel integrates with the system’s initramfs and kernel packages to install bootloaders, prepare boot.scr or extlinux.conf, and copy kernel/initrd/DTBs to /boot in a format that U-Boot expects. U-Boot will be used as a second-stage bootloader, loaded by the Raspberry Pi’s built-in firmware. Once U-Boot is in place, it will read standard boot scripts ( boot.scr) generated by flash-kernel, providing a Debian-compatible and board-flexible solution.

Extending YAML spec for vmdb2 build with U-Boot and flash-kernel

To improve an existing vmdb2 YAML spec(https://salsa.debian.org/raspi-team/image-specs/raspi_master.yaml), to integrate U-Boot, flash-kernel, and the architecture-specific Debian kernel into the image build process. By incorporating u-boot-rpi and flash-kernel from Debian packages, alongside the standard initramfs-tools, we align the image closer to Debian best practices while supporting both armhf and arm64 architectures.

Below are key additions and adjustments needed in a vmdb2 YAML spec to support the workflow: Install U-Boot, flash-kernel, initramfs-tools and the architecture-specific Debian kernel.

- apt: install
  packages:
    - u-boot-rpi
    - flash-kernel
    - initramfs-tools
    - linux-image-arm64 # or linux-image-armmp for armhf 
  tag: tag-root

Replace linux-image-arm64 with the correct kernel package for specific target architecture. These packages should be added under the tag-root section in YAML spec for vmdb2 build recipe. This ensures that the necessary bootloader, kernel, and initramfs tools are included and properly configured in the image.

Configure Raspberry Pi firmware to Load U-Boot

Install the U-Boot binary as kernel.img in /boot/firmware we can also download and build U-Boot from source, but Debian provides tested binaries.

- shell: |
    cp /usr/lib/u-boot/rpi_4/u-boot.bin ${ROOT?}/boot/firmware/kernel.img
    echo "enable_uart=1" >> ${ROOT?}/boot/firmware/config.txt
  root-fs: tag-root

This makes the RPi firmware load u-boot.bin instead of the Linux kernel directly.

Set Up flash-kernel for Debian-style Boot

flash-kernel integrates with initramfs-tools and writes boot config suitable for U-Boot. We need to make sure /etc/flash-kernel/db contains an entry for board (most Raspberry Pi boards already supported in Bookworm).

Set up /etc/flash-kernel.conf with:

- create-file: /etc/flash-kernel.conf
  contents: |
    MACHINE="Raspberry Pi 4"
    BOOTPART="/dev/disk/by-label/RASPIFIRM"
    ROOTPART="/dev/disk/by-label/RASPIROOT"
  unless: rootfs_unpacked

This allows flash-kernel to write an extlinux.conf or boot.scr into /boot/firmware.

Clean up Proprietary/Non-Free Firmware Bootflow

Remove the direct kernel loading flow:

- shell: |
    rm -f ${ROOT?}/boot/firmware/vmlinuz*
    rm -f ${ROOT?}/boot/firmware/initrd.img*
    rm -f ${ROOT?}/boot/firmware/cmdline.txt
  root-fs: tag-root

Let U-Boot and flash-kernel manage kernel/initrd and boot parameters instead.

Boot Flow After This Change

[SoC ROM] -> [start.elf] -> [U-Boot] -> [boot.scr] -> [Linux Kernel]
  1. This still depends on the Raspberry Pi firmware to start, but it only loads U-Boot, not Linux kernel.

  2. U-Boot gives you more flexibility (e.g., networking, boot menus, signed boot).

  3. Using flash-kernel ensures kernel updates are handled the Debian Installer way.

  4. Test with a serial console (enable_uart=1) in case HDMI doesn’t show early boot logs.

Advantage of New Workflow

  1. Replaces the proprietary Raspberry Pi bootloader with upstream U-Boot.

  2. Debian-native tooling – Uses flash-kernel and initramfs-tools to manage boot configuration.

  3. Consistent across boards – Works for both armhf and arm64, unifying the image build process.

  4. Easier to support new boards – Like the Raspberry Pi 5 and future models.

This transition will standardize a bit image-building process, making it aligned with upstream Debian Installer workflows.

vmdb2 configuration for arm64 using u-boot and flash-kernel

NOTE: This is a baseline example and may require tuning.

# Raspberry Pi arm64 image using U-Boot and flash-kernel

steps:
  # ... (existing mkimg, partitions, mount, debootstrap, etc.) ...

  # Install U-Boot, flash-kernel, initramfs-tools and architecture specific kernel
  - apt: install
    packages:
      - u-boot-rpi
      - flash-kernel
      - initramfs-tools
      - linux - image - arm64 # or linux - image - armmp for armhf
    tag: tag-root

  # Install U-Boot binary as kernel.img in firmware partition
  - shell: |
      cp /usr/lib/u-boot/rpi_arm64 /u-boot.bin ${ROOT?}/boot/firmware/kernel.img
      echo "enable_uart=1" >> ${ROOT?}/boot/firmware/config.txt
    root-fs: tag-root

  # Configure flash-kernel for Raspberry Pi
  - create-file: /etc/flash-kernel.conf
    contents: |
      MACHINE="Generic Raspberry Pi ARM64"
      BOOTPART="/dev/disk/by-label/RASPIFIRM"
      ROOTPART="/dev/disk/by-label/RASPIROOT"
    unless: rootfs_unpacked

  # Remove direct kernel boot files from Raspberry Pi firmware
  - shell: |
      rm -f ${ROOT?}/boot/firmware/vmlinuz*
      rm -f ${ROOT?}/boot/firmware/initrd.img*
      rm -f ${ROOT?}/boot/firmware/cmdline.txt
    root-fs: tag-root

  # flash-kernel will manage boot scripts and extlinux.conf
  # Rest of image build continues...

Required Changes to Support Raspberry Pi Boards in Debian (flash-kernel + U-Boot)

Overview of Required Changes

Component Required Task
Debian U-Boot Package Add build target for rpi_arm64 in u-boot-rpi. Optionally deprecate legacy 32-bit targets.
Debian flash-kernel Package Add or verify entries in db/all.db for Pi 4, Pi 5, Zero 2W, CM4. Ensure boot script generation works via bootscr.uboot-generic.
Debian Kernel Ensure DTBs are installed at /usr/lib/linux-image-<version>/ and available for flash-kernel to reference.

flash-kernel

Already Supported Boards in flash-kernel Debian Package

https://sources.debian.org/src/flash-kernel/3.109/db/all.db/#L1700

Model Arch DTB-Id
Raspberry Pi 1 A/B/B+, Rev2 armel bcm2835-*
Raspberry Pi CM1 armel bcm2835-rpi-cm1-io1.dtb
Raspberry Pi Zero/Zero W armel bcm2835-rpi-zero*.dtb
Raspberry Pi 2B armhf bcm2836-rpi-2-b.dtb
Raspberry Pi 3B/3B+ arm64 bcm2837-*
Raspberry Pi CM3 arm64 bcm2837-rpi-cm3-io3.dtb
Raspberry Pi 400 arm64 bcm2711-rpi-400.dtb

uboot

Already Supported Boards in Debian U-Boot Package

https://salsa.debian.org/installer-team/flash-kernel/-/blob/master/db/all.db

arm64

| Model |Arch | Upstream Defconfig | Debian Target | | ————————- | ——- | ———————— | ——————- | | Raspberry Pi 3B | arm64 | rpi_3_defconfig | rpi_3 | | Raspberry Pi 4B | arm64 | rpi_4_defconfig | rpi_4 | | Raspberry Pi 3B/3B+/CM3/CM3+/4B/CM4/400/5B/Zero 2W | arm64 | rpi_arm64_defconfig | rpi_arm64 |

armhf

| Model |Arch | Upstream Defconfig | Debian Target | | ————————- | ——- | ———————— | ——————- | | Raspberry Pi 2 | armhf | rpi_2_defconfig | rpi_2 | | Raspberry Pi 3B (32-bit) | armhf | rpi_3_32b_defconfig | rpi_3_32b | | Raspberry Pi 4B (32-bit) | armhf | rpi_4_32b_defconfig | rpi_4_32b |

armel

| Model |Arch | Upstream Defconfig | Debian Target | | ————————- | ——- | ———————— | ——————- | | Raspberry Pi | armel | rpi_defconfig | rpi | | Raspberry Pi 1/Zero | armel | rpi_0_w | rpi_0_w |

These boards are already defined in debian/rules under the u-boot-rpi source package and generates usable U-Boot binaries for corresponding Raspberry Pi models.

To-Do: Add Missing Board Support to U-Boot and flash-kernel in Debian

Several Raspberry Pi models are missing from the Debian U-Boot and flash-kernel packages, even though upstream support and DTBs exist in the Debian kernel but are missing entries in the flash-kernel database to enable support for bootloader installation and initrd handling.

Boards Not Yet Supported in flash-kernel Debian Package

Model Arch DTB-Id
Raspberry Pi 3A+ (32 & 64 bit) armhf, arm64 bcm2837-rpi-3-a-plus.dtb
Raspberry Pi 4B (32 & 64 bit) armhf, arm64 bcm2711-rpi-4-b.dtb
Raspberry Pi CM4 arm64 bcm2711-rpi-cm4-io.dtb
Raspberry Pi CM 4S arm64 -
Raspberry Zero 2 W arm64 bcm2710-rpi-zero-2-w.dtb
Raspberry Pi 5 arm64 bcm2712-rpi-5-b.dtb
Raspberry Pi CM5 arm64 -
Raspberry Pi 500 arm64 -

Boards Not Yet Supported in Debian U-Boot Package

Model Arch Upstream defconfig(s)
Raspberry Pi 3A+/3B+ arm64 -, rpi_3_b_plus_defconfig
Raspberry Pi CM 4S arm64 -
Raspberry Pi 5 arm64 -
Raspberry Pi CM5 arm64 -
Raspberry Pi 500 arm64 -

So, what next?

During the Community Bonding Period, I got hands-on with workflow improvements, set up test environments, and began reviewing Raspberry Pi support in Debian’s U-Boot and flash-kernel and these are the logs of the project, where I provide weekly reports on the work done. You can check here: Community Bonding Period logs.

My next steps include submitting patches to the u-boot and flash-kernel packages to ensure all missing Raspberry Pi entries are built and shipped. And, also to confirm the kernel DTB installation paths and make sure the necessary files are included for all Raspberry Pi variants. Finally, plan to validate changes with test builds on Raspberry Pi hardware.

In parallel, I’m organizing my tasks and setting up my environment to contribute more effectively. It’s been exciting to explore how things work under the hood and to prepare for a summer of learning and contributing to this great community.

,

Cryptogram Friday Squid Blogging: Gonate Squid Video

This is the first ever video of the Antarctic Gonate Squid.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Cryptogram Surveillance in the US

Good article from 404 Media on the cozy surveillance relationship between local Oregon police and ICE:

In the email thread, crime analysts from several local police departments and the FBI introduced themselves to each other and made lists of surveillance tools and tactics they have access to and felt comfortable using, and in some cases offered to perform surveillance for their colleagues in other departments. The thread also includes a member of ICE’s Homeland Security Investigations (HSI) and members of Oregon’s State Police. In the thread, called the “Southern Oregon Analyst Group,” some members talked about making fake social media profiles to surveil people, and others discussed being excited to learn and try new surveillance techniques. The emails show both the wide array of surveillance tools that are available to even small police departments in the United States and also shows informal collaboration between local police departments and federal agencies, when ordinarily agencies like ICE are expected to follow their own legal processes for carrying out the surveillance.

Cryptogram Self-Driving Car Video Footage

Two articles crossed my path recently. First, a discussion of all the video Waymo has from outside its cars: in this case related to the LA protests. Second, a discussion of all the video Tesla has from inside its cars.

Lots of things are collecting lots of video of lots of other things. How and under what rules that video is used and reused will be a continuing source of debate.

Cryptogram Ghostwriting Scam

The variations seem to be endless. Here’s a fake ghostwriting scam that seems to be making boatloads of money.

This is a big story about scams being run from Texas and Pakistan estimated to run into tens if not hundreds of millions of dollars, viciously defrauding Americans with false hopes of publishing bestseller books (a scam you’d not think many people would fall for but is surprisingly huge). In January, three people were charged with defrauding elderly authors across the United States of almost $44 million ­by “convincing the victims that publishers and filmmakers wanted to turn their books into blockbusters.”

Worse Than FailureCodeSOD: Stop Being So ####

Many a network admin has turned to the siren song of Perl to help them automate managing their networks. Frank's predecessor is no exception.

They also got a bit combative about people critiquing their Perl code:

# COMPLEX SUBNET MATH
# Looking up a value in an array was faster than any mathematical solution. Yes, it's hard coded, but these values won't ever change anyway. Stop being so #### about it.
$Subnets = @("0.0.0.0","128.0.0.0","192.0.0.0","224.0.0.0","240.0.0.0","248.0.0.0","252.0.0.0","254.0.0.0","255.0.0.0","255.128.0.0","255.192.0.0","255.224.0.0","255.240.0.0","255.248.0.0","255.252.0.0","255.254.0.0","255.255.0.0","255.255.128.0","255.255.192.0","255.255.224.0","255.255.240.0","255.255.248.0","255.255.252.0","255.255.254.0","255.255.255.0","255.255.255.128","255.255.255.192","255.255.255.224","255.255.255.240","255.255.255.248","255.255.255.252","255.255.255.254","255.255.255.255")

I believe them when they say that the lookup array is faster, but it leaves me wondering: what are they doing where performance matters that much?

I don't actually think this ascends to the level of a WTF, but I do think the defensive comment is funny. Clearly, the original developer was having a time with people complaining about it.

Frank notes that while Perl has a reputation as a "write only language," this particular set of scripts was actually quite easy to read and maintain. So yes, I guess we should stop being so #### about it.

[Advertisement] Plan Your .NET 9 Migration with Confidence
Your journey to .NET 9 is more than just one decision.Avoid migration migraines with the advice in this free guide. Download Free Guide Now!

365 TomorrowsHomesick

Author: Sasha Kasper As the blaring siren assaults my eardrums, it becomes increasingly harder to deny my rapid descent. I float directionless through the cockpit. Up, down, left, right, have lost all meaning. The notion of gravity seems to me a cruel joke, of which the punchline will be my demise. The tempered glass of […]

The post Homesick appeared first on 365tomorrows.

Planet DebianSergio Durigan Junior: GCC, glibc, stack unwinding and relocations – A war story

I’ve been meaning to write a post about this bug for a while, so here it is (before I forget the details!).

First, I’d like to thank a few people:

  • My friend Gabriel F. T. Gomes, who helped with debugging and simply talking about the issue. I love doing some pair debugging, and I noticed that he also had a great time diving into the internals of glibc and libgcc.
  • My teammate Dann Frazier, who always provides invaluable insights and was there to motivate me to push a bit further in order to figure out what was going on.
  • The upstream GCC and glibc developers who finally drove the investigation to completion and came up with an elegant fix.

I’ll probably forget some details because it’s been more than a week (and life at $DAYJOB moves fast), but we’ll see.

The background story

Wolfi OS takes security seriously, and one of the things we have is a package which sets the hardening compiler flags for C/C++ according to the best practices recommended by OpenSSF. At the time of this writing, these flags are (in GCC’s spec file parlance):

*self_spec:
+ %{!O:%{!O1:%{!O2:%{!O3:%{!O0:%{!Os:%{!0fast:%{!0g:%{!0z:-O2}}}}}}}}} -fhardened -Wno-error=hardened -Wno-hardened %{!fdelete-null-pointer-checks:-fno-delete-null-pointer-checks} -fno-strict-overflow -fno-strict-aliasing %{!fomit-frame-pointer:-fno-omit-frame-pointer} -mno-omit-leaf-frame-pointer

*link:
+ --as-needed -O1 --sort-common -z noexecstack -z relro -z now

The important part for our bug is the usage of -z now and -fno-strict-aliasing.

As I was saying, these flags are set for almost every build, but sometimes things don’t work as they should and we need to disable them. Unfortunately, one of these problematic cases has been glibc.

There was an attempt to enable hardening while building glibc, but that introduced a strange breakage to several of our packages and had to be reverted.

Things stayed pretty much the same until a few weeks ago, when I started working on one of my roadmap items: figure out why hardening glibc wasn’t working, and get it to work as much as possible.

Reproducing the bug

I started off by trying to reproduce the problem. It’s important to mention this because I often see young engineers forgetting to check if the problem is even valid anymore. I don’t blame them; the anxiety to get the bug fixed can be really blinding.

Fortunately, I already had one simple test to trigger the failure. All I had to do was install the py3-matplotlib package and then invoke:

$ python3 -c 'import matplotlib'

This would result in an abortion with a coredump.

I followed the steps above, and readily saw the problem manifesting again. OK, first step is done; I wasn’t getting out easily from this one.

Initial debug

The next step is to actually try to debug the failure. In an ideal world you get lucky and are able to spot what’s wrong after just a few minutes. Or even better: you also can devise a patch to fix the bug and contribute it to upstream.

I installed GDB, and then ran the py3-matplotlib command inside it. When the abortion happened, I issued a backtrace command inside GDB to see where exactly things had gone wrong. I got a stack trace similar to the following:

#0  0x00007c43afe9972c in __pthread_kill_implementation () from /lib/libc.so.6
#1  0x00007c43afe3d8be in raise () from /lib/libc.so.6
#2  0x00007c43afe2531f in abort () from /lib/libc.so.6
#3  0x00007c43af84f79d in uw_init_context_1[cold] () from /usr/lib/libgcc_s.so.1
#4  0x00007c43af86d4d8 in _Unwind_RaiseException () from /usr/lib/libgcc_s.so.1
#5  0x00007c43acac9014 in __cxxabiv1::__cxa_throw (obj=0x5b7d7f52fab0, tinfo=0x7c429b6fd218 <typeinfo for pybind11::attribute_error>, dest=0x7c429b5f7f70 <pybind11::reference_cast_error::~reference_cast_error() [clone .lto_priv.0]>)
    at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:93
#6  0x00007c429b5ec3a7 in ft2font__getattr__(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) [clone .lto_priv.0] [clone .cold] () from /usr/lib/python3.13/site-packages/matplotlib/ft2font.cpython-313-x86_64-linux-gnu.so
#7  0x00007c429b62f086 in pybind11::cpp_function::initialize<pybind11::object (*&)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::object, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::scope, pybind11::sibling>(pybind11::object (*&)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::object (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&)::{lambda(pybind11::detail::function_call&)#1}::_FUN(pybind11::detail::function_call&) [clone .lto_priv.0] ()
   from /usr/lib/python3.13/site-packages/matplotlib/ft2font.cpython-313-x86_64-linux-gnu.so
#8  0x00007c429b603886 in pybind11::cpp_function::dispatcher(_object*, _object*, _object*) () from /usr/lib/python3.13/site-packages/matplotlib/ft2font.cpython-313-x86_64-linux-gnu.so
...

Huh. Initially this didn’t provide me with much information. There was something strange seeing the abort function being called right after _Unwind_RaiseException, but at the time I didn’t pay much attention to it.

OK, time to expand our horizons a little. Remember when I said that several of our packages would crash with a hardened glibc? I decided to look for another problematic package so that I could make it crash and get its stack trace. My thinking here is that maybe if I can compare both traces, something will come up.

I happened to find an old discussion where Dann Frazier mentioned that Emacs was also crashing for him. He and I share the Emacs passion, and I totally agreed with him when he said that “Emacs crashing is priority -1!” (I’m paraphrasing).

I installed Emacs, ran it, and voilà: the crash happened again. OK, that was good. When I ran Emacs inside GDB and asked for a backtrace, here’s what I got:

#0  0x00007eede329972c in __pthread_kill_implementation () from /lib/libc.so.6
#1  0x00007eede323d8be in raise () from /lib/libc.so.6
#2  0x00007eede322531f in abort () from /lib/libc.so.6
#3  0x00007eede262879d in uw_init_context_1[cold] () from /usr/lib/libgcc_s.so.1
#4  0x00007eede2646e7c in _Unwind_Backtrace () from /usr/lib/libgcc_s.so.1
#5  0x00007eede3327b11 in backtrace () from /lib/libc.so.6
#6  0x000059535963a8a1 in emacs_backtrace ()
#7  0x000059535956499a in main ()

Ah, this backtrace is much simpler to follow. Nice.

Hmmm. Now the crash is happening inside _Unwind_Backtrace. A pattern emerges! This must have something to do with stack unwinding (or so I thought… keep reading to discover the whole truth). You see, the backtrace function (yes, it’s a function) and C++’s exception handling mechanism use similar techniques to do their jobs, and it pretty much boils down to unwinding frames from the stack.

I looked into Emacs’ source code, specifically the emacs_backtrace function, but could not find anything strange over there. This bug was probably not going to be an easy fix…

The quest for a minimal reproducer

Being able to easily reproduce the bug is awesome and really helps with debugging, but even better is being able to have a minimal reproducer for the problem.

You see, py3-matplotlib is a huge package and pulls in a bunch of extra dependencies, so it’s not easy to ask other people to “just install this big package plus these other dependencies, and then run this command…”, especially if we have to file an upstream bug and talk to people who may not even run the distribution we’re using. So I set up to try and come up with a smaller recipe to reproduce the issue, ideally something that’s not tied to a specific package from the distribution.

Having all the information gathered from the initial debug session, especially the Emacs backtrace, I thought that I could write a very simple program that just invoked the backtrace function from glibc in order to trigger the code path that leads to _Unwind_Backtrace. Here’s what I wrote:

#include <execinfo.h>

int
main(int argc, char *argv[])
{
  void *a[4096];
  backtrace (a, 100);
  return 0;
}

After compiling it, I determined that yes, the problem did happen with this small program as well. There was only a small nuisance: the manifestation of the bug was not deterministic, so I had to execute the program a few times until it crashed. But that’s much better than what I had before, and a small price to pay. Having a minimal reproducer pretty much allows us to switch our focus to what really matters. I wouldn’t need to dive into Emacs’ or Python’s source code anymore.

At the time, I was sure this was a glibc bug. But then something else happened.

GCC 15

I had to stop my investigation efforts because something more important came up: it was time to upload GCC 15 to Wolfi. I spent a couple of weeks working on this (it involved rebuilding the whole archive, filing hundreds of FTBFS bugs, patching some programs, etc.), and by the end of it the transition went smooth. When the GCC 15 upload was finally done, I switched my focus back to the glibc hardening problem.

The first thing I did was to… yes, reproduce the bug again. It had been a few weeks since I had touched the package, after all. So I built a hardened glibc with the latest GCC and… the bug did not happen anymore!

Fortunately, the very first thing I thought was “this must be GCC”, so I rebuilt the hardened glibc with GCC 14, and the bug was there again. Huh, unexpected but very interesting.

Diving into glibc and libgcc

At this point, I was ready to start some serious debugging. And then I got a message on Signal. It was one of those moments where two minds think alike: Gabriel decided to check how I was doing, and I was thinking about him because this involved glibc, and Gabriel contributed to the project for many years. I explained what I was doing, and he promptly offered to help. Yes, there are more people who love low level debugging!

We spent several hours going through disassembles of certain functions (because we didn’t have any debug information in the beginning), trying to make sense of what we were seeing. There was some heavy GDB involved; unfortunately I completely lost the session’s history because it was done inside a container running inside an ephemeral VM. But we learned a lot. For example:

  • It was hard to actually understand the full stack trace leading to uw_init_context_1[cold]. _Unwind_Backtrace obviously didn’t call it (it called uw_init_context_1, but what was that [cold] doing?). We had to investigate the disassemble of uw_init_context_1 in order to determined where uw_init_context_1[cold] was being called.

  • The [cold] suffix is a GCC function attribute that can be used to tell the compiler that the function is unlikely to be reached. When I read that, my mind immediately jumped to “this must be an assertion”, so I went to the source code and found the spot.

  • We were able to determine that the return code of uw_frame_state_for was 5, which means _URC_END_OF_STACK. That’s why the assertion was triggering.

After finding these facts without debug information, I decided to bite the bullet and recompiled GCC 14 with -O0 -g3, so that we could debug what uw_frame_state_for was doing. After banging our heads a bit more, we found that fde is NULL at this excerpt:

// ...
  fde = _Unwind_Find_FDE (context->ra + _Unwind_IsSignalFrame (context) - 1,
                          &context->bases);
  if (fde == NULL)
    {
#ifdef MD_FALLBACK_FRAME_STATE_FOR
      /* Couldn't find frame unwind info for this function.  Try a
         target-specific fallback mechanism.  This will necessarily
         not provide a personality routine or LSDA.  */
      return MD_FALLBACK_FRAME_STATE_FOR (context, fs);
#else
      return _URC_END_OF_STACK;
#endif
    }
// ...

We’re debugging on amd64, which means that MD_FALLBACK_FRAME_STATE_FOR is defined and therefore is called. But that’s not really important for our case here, because we had established before that _Unwind_Find_FDE would never return NULL when using a non-hardened glibc (or a glibc compiled with GCC 15). So we decided to look into what _Unwind_Find_FDE did.

The function is complex because it deals with .eh_frame , but we were able to pinpoint the exact location where find_fde_tail (one of the functions called by _Unwind_Find_FDE) is returning NULL:

if (pc < table[0].initial_loc + data_base)
  return NULL;

We looked at the addresses of pc and table[0].initial_loc + data_base, and found that the former fell within libgcc’s text section, which the latter fell within /lib/ld-linux-x86-64.so.2 text.

At this point, we were already too tired to continue. I decided to keep looking at the problem later and see if I could get any further.

Bisecting GCC

The next day, I woke up determined to find what changed in GCC 15 that caused the bug to disappear. Unless you know GCC’s internals like they are your own home (which I definitely don’t), the best way to do that is to git bisect the commits between GCC 14 and 15.

I spent a few days running the bisect. It took me more time than I’d have liked to find the right range of commits to pass git bisect (because of how branches and tags are done in GCC’s repository), and I also had to write some helper scripts that:

  • Modified the gcc.yaml package definition to make it build with the commit being bisected.
  • Built glibc using the GCC that was just built.
  • Ran tests inside a docker container (with the recently built glibc installed) to determine whether the bug was present.

At the end, I had a commit to point to:

commit 99b1daae18c095d6c94d32efb77442838e11cbfb
Author: Richard Biener <rguenther@suse.de>
Date:   Fri May 3 14:04:41 2024 +0200

    tree-optimization/114589 - remove profile based sink heuristics

Makes sense, right?! No? Well, it didn’t for me either. Even after reading what was changed in the code and the upstream bug fixed by the commit, I was still clueless as to why this change “fixed” the problem (I say “fixed” because it may very well be an unintended consequence of the change, and some other problem might have been introduced).

Upstream takes over

After obtaining the commit that possibly fixed the bug, while talking to Dann and explaining what I did, he suggested that I should file an upstream bug and check with them. Great idea, of course.

I filed the following upstream bug:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=120653

It’s a bit long, very dense and complex, but ultimately upstream was able to find the real problem and have a patch accepted in just two days. Nothing like knowing the code base. The initial bug became:

https://sourceware.org/bugzilla/show_bug.cgi?id=33088

In the end, the problem was indeed in how the linker defines __ehdr_start, which, according to the code (from elf/dl-support.c):

if (_dl_phdr == NULL)
  {
    /* Starting from binutils-2.23, the linker will define the
       magic symbol __ehdr_start to point to our own ELF header
       if it is visible in a segment that also includes the phdrs.
       So we can set up _dl_phdr and _dl_phnum even without any
       information from auxv.  */


    extern const ElfW(Ehdr) __ehdr_start attribute_hidden;
    assert (__ehdr_start.e_phentsize == sizeof *GL(dl_phdr));
    _dl_phdr = (const void *) &__ehdr_start + __ehdr_start.e_phoff;
    _dl_phnum = __ehdr_start.e_phnum;
  }

But the following definition is the problematic one (from elf/rtld.c):

extern const ElfW(Ehdr) __ehdr_start attribute_hidden;

This symbol (along with its counterpart, __ehdr_end) was being run-time relocated when it shouldn’t be. The fix that was pushed added optimization barriers to prevent the compiler from doing the relocations.

I don’t claim to fully understand what was done here, and Jakub’s analysis is a thing to behold, but in the end I was able to confirm that the patch fixed the bug. And in the end, it was indeed a glibc bug.

Conclusion

This was an awesome bug to investigate. It’s one of those that deserve a blog post, even though some of the final details of the fix flew over my head.

I’d like to start blogging more about these sort of bugs, because I’ve encountered my fair share of them throughout my career. And it was great being able to do some debugging with another person, exchange ideas, learn things together, and ultimately share that deep satisfaction when we find why a crash is happening.

I have at least one more bug in my TODO list to write about (another one with glibc, but this time I was able to get to the end of it and come up with a patch). Stay tunned.

P.S.: After having published the post I realized that I forgot to explain why the -z now and -fno-strict-aliasing flags were important.

-z now is the flag that I determined to be the root cause of the breakage. If I compiled glibc with every hardening flag except -z now, everything worked. So initially I thought that the problem had to do with how ld.so was resolving symbols at runtime. As it turns out, this ended up being more a symptom than the real cause of the bug.

As for -fno-strict-aliasing, a Gentoo developer who commented on the GCC bug above mentioned that this OpenSSF bug had a good point against using this flag for hardening. I still have to do a deep dive on what was discussed in the issue, but this is certainly something to take into consideration. There’s this very good write-up about strict aliasing in general if you’re interested in understanding it better.

,

Rondam RamblingsSigh, here we go again.

You would think that after the disasters in Afghanistan and Iraq that Republicans would have learned that starting a war in that part of the world is a Really Bad Idea (tm).  But no.  After utterly failing to bring about regime change in both its eastern and western neighbors, the Trump administration is winding up to try yet again again in Iran.  Maybe the third time will be the

Rondam RamblingsNo, Science is Not Just Another Religion

I want to debunk once and for all this idea that "science is just another religion".  It isn't, for one simple reason: all religions are based on some kind of metaphysical assumptions.  Those assumptions are generally something like the authority of some source of revealed knowledge, typically a holy text.  But it doesn't have to be that.  It can be as simple as assuming that

Planet DebianEvgeni Golov: Arguing with an AI or how Evgeni tried to use CodeRabbit

Everybody is trying out AI assistants these days, so I figured I'd jump on that train and see how fast it derails.

I went with CodeRabbit because I've seen it on YouTube — ads work, I guess.

I am trying to answer the following questions:

  • Did the AI find things that humans did not find (or didn't bother to mention)
  • Did the AI output help the humans with the review (useful summary etc)
  • Did the AI output help the humans with the code (useful suggestions etc)
  • Was the AI output misleading?
  • Was the AI output distracting?

To reduce the amount of output and not to confuse contributors, CodeRabbit was configured to only do reviews on demand.

What follows is a rather unscientific evaluation of CodeRabbit based on PRs in two Foreman-related repositories, looking at the summaries CodeRabbit posted as well as the comments/suggestions it had about the code.

Ansible 2.19 support

PR: theforeman/foreman-ansible-modules#1848

summary posted

The summary CodeRabbit posted is technically correct.

This update introduces several changes across CI configuration, Ansible roles, plugins, and test playbooks. It expands CI test coverage to a new Ansible version, adjusts YAML key types in test variables, refines conditional logic in Ansible tasks, adds new default variables, and improves clarity and consistency in playbook task definitions and debug output.

Yeah, it does all of that, all right. But it kinda misses the point that the addition here is "Ansible 2.19 support", which starts with adding it to the CI matrix and then adjusting the code to actually work with that version. Also, the changes are not for "clarity" or "consistency", they are fixing bugs in the code that the older Ansible versions accepted, but the new one is more strict about.

Then it adds a table with the changed files and what changed in there. To me, as the author, it felt redundant, and IMHO doesn't add any clarity to understand the changes. (And yes, same "clarity" vs bugfix mistake here, but that makes sense as it apparently miss-identified the change reason)

And then the sequence diagrams… They probably help if you have a dedicated change to a library or a library consumer, but for this PR it's just noise, especially as it only covers two of the changes (addition of 2.19 to the test matrix and a change to the inventory plugin), completely ignoring other important parts.

Overall verdict: noise, don't need this.

comments posted

CodeRabbit also posted 4 comments/suggestions to the changes.

Guard against undefined result.task

IMHO a valid suggestion, even if on the picky side as I am not sure how to make it undefined here. I ended up implementing it, even if with slightly different (and IMHO better readable) syntax.

  • Valid complaint? Probably.
  • Useful suggestion? So-So.
  • Wasted time? No.

Inconsistent pipeline in when for composite CV versions

That one was funny! The original complaint was that the when condition used slightly different data manipulation than the data that was passed when the condition was true. The code was supposed to do "clean up the data, but only if there are any items left after removing the first 5, as we always want to keep 5 items".

And I do agree with the analysis that it's badly maintainable code. But the suggested fix was to re-use the data in the variable we later use for performing the cleanup. While this is (to my surprise!) valid Ansible syntax, it didn't make the code much more readable as you need to go and look at the variable definition.

The better suggestion then came from Ewoud: to compare the length of the data with the number we want to keep. Humans, so smart!

But Ansible is not Ewoud's native turf, so he asked whether there is a more elegant way to count how much data we have than to use | list | count in Jinja (the data comes from a Python generator, so needs to be converted to a list first).

And the AI helpfully suggested to use | count instead!

However, count is just an alias for length in Jinja, so it behaves identically and needs a list.

Luckily the AI quickly apologized for being wrong after being pointed at the Jinja source and didn't try to waste my time any further. Wouldn't I have known about the count alias, we'd have committed that suggestion and let CI fail before reverting again.

  • Valid complaint? Yes.
  • Useful suggestion? Nope.
  • Wasted time? Yes.

Apply the same fix for non-composite CV versions

The very same complaint was posted a few lines later, as the logic there is very similar — just slightly different data to be filtered and cleaned up.

Interestingly, here the suggestion also was to use the variable. But there is no variable with the data!

The text actually says one need to "define" it, yet the "committable suggestion" doesn't contain that part.

Interestingly, when asked where it sees the "inconsistency" in that hunk, it said the inconsistency is with the composite case above. That however is nonsense, as while we want to keep the same number of composite and non-composite CV versions, the data used in the task is different — it even gets consumed by a totally different playbook — so there can't be any real consistency between the branches.

  • Valid complaint? Yes (the expression really could use some cleanup).
  • Useful suggestion? Nope.
  • Wasted time? Yes.

I ended up applying the same logic as suggested by Ewoud above. As that refactoring was possible in a consistent way.

Ensure consistent naming for Oracle Linux subscription defaults

One of the changes in Ansible 2.19 is that Ansible fails when there are undefined variables, even if they are only undefined for cases where they are unused.

CodeRabbit complains that the names of the defaults I added are inconsistent. And that is technically correct. But those names are already used in other places in the code, so I'd have to refactor more to make it work properly.

Once being pointed at the fact that the variables already exist, the AI is as usual quick to apologize, yay.

  • Valid complaint? Technically, yes.
  • Useful suggestion? Nope.
  • Wasted time? Yes.

add new parameters to the repository module

PR: theforeman/foreman-ansible-modules#1860

summary posted

Again, the summary is technically correct

The repository module was updated to support additional parameters for repository synchronization and authentication. New options were added for ansible collections, ostree, Python packages, and yum repositories, including authentication tokens, filtering controls, and version retention settings. All changes were limited to module documentation and argument specification.

But it doesn't add anything you'd not get from looking at the diff, especially as it contains a large documentation chunk explaining those parameters.

No sequence diagram this time. That's a good thing!

Overall verdict: noise (even if the amount is small), don't need this.

comments posted

CodeRabbit generated two comments for this PR.

Interestingly, none of them overlapped with the issues ansible-lint and friends found.

get rid of the FIXMEs

Yepp, that's fair

  • Valid complaint? Yes.
  • Useful suggestion? Nope. (But it's not possible in this case!)
  • Wasted time? No.

add validation for the new parameters

Yepp, I forgot these (not intentionally!).

The diff it suggests is nonsense, as it doesn't take into account the existing Ansible and Yum validations, but it clearly has read them as the style etc of the new ones matches. It also managed to group the parameters correctly by repository type, so it's something.

 if module.foreman_params['content_type'] != 'ansible_collection':
     invalid_list = [key for key in ['ansible_collection_requirements'] if key in module.foreman_params]
     if invalid_list:
         module.fail_json(msg="({0}) can only be used with content_type 'ansible_collection'".format(",".join(invalid_list)))
+
+# Validate ansible_collection specific parameters
+if module.foreman_params['content_type'] != 'ansible_collection':
+    invalid_list = [key for key in ['ansible_collection_auth_token', 'ansible_collection_auth_url'] if key in module.foreman_params]
+    if invalid_list:
+        module.fail_json(msg="({0}) can only be used with content_type 'ansible_collection'".format(",".join(invalid_list)))
+
+# Validate ostree specific parameters
+if module.foreman_params['content_type'] != 'ostree':
+    invalid_list = [key for key in ['depth', 'exclude_refs', 'include_refs'] if key in module.foreman_params]
+    if invalid_list:
+        module.fail_json(msg="({0}) can only be used with content_type 'ostree'".format(",".join(invalid_list)))
+
+# Validate python package specific parameters
+if module.foreman_params['content_type'] != 'python':
+    invalid_list = [key for key in ['excludes', 'includes', 'package_types', 'keep_latest_packages'] if key in module.foreman_params]
+    if invalid_list:
+        module.fail_json(msg="({0}) can only be used with content_type 'python'".format(",".join(invalid_list)))
+
+# Validate yum specific parameter
+if module.foreman_params['content_type'] != 'yum' and 'upstream_authentication_token' in module.foreman_params:
+    module.fail_json(msg="upstream_authentication_token can only be used with content_type 'yum'")

Interestingly, it also said "Note: If 'python' is not a valid content_type, please adjust the validation accordingly." which is quite a hint at a bug in itself. The module currently does not even allow to create content_type=python repositories. That should have been more prominent, as it's a BUG!

  • Valid complaint? Yes.
  • Useful suggestion? Mostly (I only had to merge the Yum and Ansible branches with the existing code).
  • Wasted time? Nope.

parameter persistence in obsah

PR: theforeman/obsah#72

summary posted

Mostly correct.

It did miss-interpret the change to a test playbook as an actual "behavior" change: "Introduced new playbook variables for database configuration" — there is no database configuration in this repository, just the test playbook using the same metadata as a consumer of the library. Later on it does say "Playbook metadata and test fixtures", so… unclear whether this is a miss-interpretation or just badly summarized. As long as you also look at the diff, it won't confuse you, but if you're using the summary as the sole source of information (bad!) it would.

This time the sequence diagram is actually useful, yay. Again, not 100% accurate: it's missing the fact that saving the parameters is hidden behind an "if enabled" flag — something it did represent correctly for loading them.

Overall verdict: not really useful, don't need this.

comments posted

Here I was a bit surprised, especially as the nitpicks were useful!

Persist-path should respect per-user state locations (nitpick)

My original code used os.environ.get('OBSAH_PERSIST_PATH', '/var/lib/obsah/parameters.yaml') for the location of the persistence file. CodeRabbit correctly pointed out that this won't work for non-root users and one should respect XDG_STATE_HOME.

Ewoud did point that out in his own review, so I am not sure whether CodeRabbit came up with this on its own, or also took the human comments into account.

The suggested code seems fine too — just doesn't use /var/lib/obsah at all anymore. This might be a good idea for the generic library we're working on here, and then be overridden to a static /var/lib path in a consumer (which always runs as root).

In the end I did not implement it, but mostly because I was lazy and was sure we'd override it anyway.

  • Valid complaint? Yes.
  • Useful suggestion? Yes.
  • Wasted time? Nope.

Positional parameters are silently excluded from persistence (nitpick)

The library allows you to generate both positional (foo without --) and non-positional (--foo) parameters, but the code I wrote would only ever persist non-positional parameters. This was intentional, but there is no documentation of the intent in a comment — which the rabbit thought would be worth pointing out.

It's a fair nitpick and I ended up adding a comment.

  • Valid complaint? Yes.
  • Useful suggestion? Yes.
  • Wasted time? Nope.

Enforce FQDN validation for database_host

The library has a way to perform type checking on passed parameters, and one of the supported types is "FQDN" — so a fully qualified domain name, with dots and stuff. The test playbook I added has a database_host variable, but I didn't bother adding a type to it, as I don't really need any type checking here.

While using "FQDN" might be a bit too strict here — technically a working database connection can also use a non-qualified name or an IP address, I was positively surprised by this suggestion. It shows that the rest of the repository was taken into context when preparing the suggestion.

  • Valid complaint? In the context of a test, no. Would that be a real command definition, yes.
  • Useful suggestion? Yes.
  • Wasted time? Nope.

reset_args() can raise AttributeError when a key is absent

This is a correct finding, the code is not written in a way that would survive if it tries to reset things that are not set. However, that's only true for the case where users pass in --reset-<parameter> without ever having set parameter before. The complaint about the part where the parameter is part of the persisted set but not in the parsed args is wrong — as parsed args inherit from the persisted set.

The suggested code is not well readable, so I ended up fixing it slightly differently.

  • Valid complaint? Mostly.
  • Useful suggestion? Meh.
  • Wasted time? A bit.

Persisted values bypass argparse type validation

When persisting, I just yaml.safe_dump the parsed parameters, which means the YAML will contain native types like integers.

The argparse documentation warns that the type checking argparse does only applies to strings and is skipped if you pass anything else (via default values).

While correct, it doesn't really hurt here as the persisting only happens after the values were type-checked. So there is not really a reason to type-check them again. Well, unless the type changes, anyway.

Not sure what I'll do with this comment.

  • Valid complaint? Nah.
  • Useful suggestion? Nope.
  • Wasted time? Not much.

consider using contextlib.suppress

This was added when I asked CodeRabbit for a re-review after pushing some changes. Interestingly, the PR already contained try: … except: pass code before, and it did not flag that.

Also, the code suggestion contained import contextlib in the middle of the code, instead in the head of the file. Who would do that?!

But the comment as such was valid, so I fixed it in all places it is applicable, not only the one the rabbit found.

  • Valid complaint? Yes.
  • Useful suggestion? Nope.
  • Wasted time? Nope.

workaround to ensure LCE and CV are always sent together

PR: theforeman/foreman-ansible-modules#1867

summary posted

A workaround was added to the _update_entity method in the ForemanAnsibleModule class to ensure that when updating a host, both content_view_id and lifecycle_environment_id are always included together in the update payload. This prevents partial updates that could cause inconsistencies.

Partial updates are not a thing.

The workaround is purely for the fact that Katello expects both parameters to be sent, even if only one of them needs an actual update.

No diagram, good.

Overall verdict: misleading summaries are bad!

comments posted

Given a small patch, there was only one comment.

Implementation looks correct, but consider adding error handling for robustness.

This reads correct on the first glance. More error handling is always better, right?

But if you dig into the argumentation, you see it's wrong. Either:

  • we're working with a Katello setup and the host we're updating has content, so CV and LCE will be present
  • we're working with a Katello setup and the host has no content (yet), so CV and LCE will be "updated" and we're not running into the workaround
  • we're working with a plain Foreman, then both parameters are not even accepted by Ansible

The AI accepted defeat once I asked it to analyze things in more detail, but why did I have to ask in the first place?!

  • Valid complaint? Nope.
  • Useful suggestion? Nope.
  • Wasted time? Yes, as I've actually tried to come up with a case where it can happen.

Summary

Well, idk, really.

Did the AI find things that humans did not find (or didn't bother to mention)?

Yes. It's debatable whether these were useful (see e.g. the database_host example), but I tend to be in the "better to nitpick/suggest more and dismiss than oversee" team, so IMHO a positive win.

Did the AI output help the humans with the review (useful summary etc)?

In my opinion it did not. The summaries were either "lots of words, no real value" or plain wrong. The sequence diagrams were not useful either.

Luckily all of that can be turned off in the settings, which is what I'd do if I'd continue using it.

Did the AI output help the humans with the code (useful suggestions etc)?

While the actual patches it posted were "meh" at best, there were useful findings that resulted in improvements to the code.

Was the AI output misleading?

Absolutely! The whole Jinja discussion would have been easier without the AI "help". Same applies for the "error handling" in the workaround PR.

Was the AI output distracting?

The output is certainly a lot, so yes I think it can be distracting. As mentioned, I think dropping the summaries can make the experience less distracting.

What does all that mean?

I will disable the summaries for the repositories, but will leave the @coderabbitai review trigger active if someone wants an AI-assisted review. This won't be something that I'll force on our contributors and maintainers, but they surely can use it if they want.

But I don't think I'll be using this myself on a regular basis.

Yes, it can be made "usable". But so can be vim ;-)

Also, I'd prefer to have a junior human asking all the questions and making bad suggestions, so they can learn from it, and not some planet burning machine.

Worse Than FailureCodeSOD: A Second Date

Ah, bad date handling. We've all seen it. We all know it. So when Lorenzo sent us this C# function, we almost ignored it:

private string GetTimeStamp(DateTime param)
{
    string retDate = param.Year.ToString() + "-";
    if (param.Month < 10)
        retDate = retDate + "0" + param.Month.ToString() + "-";
    else
        retDate = retDate + param.Month.ToString() + "-";

    if (param.Day < 10)
        retDate = retDate + "0" + param.Day.ToString() + " ";
    else
        retDate = retDate + param.Day.ToString() + " ";

    if (param.Hour < 10)
        retDate = retDate + "0" + param.Hour.ToString() + ":";
    else
        retDate = retDate + param.Hour.ToString() + ":";

    if (param.Minute < 10)
        retDate = retDate + "0" + param.Minute.ToString() + ":";
    else
        retDate = retDate + param.Minute.ToString() + ":";

    if (param.Second < 10)
        retDate = retDate + "0" + param.Second.ToString() + ".";
    else
        retDate = retDate + param.Second.ToString() + ".";

    if (param.Millisecond < 10)
        retDate = retDate + "0" + param.Millisecond.ToString();
    else
        retDate = retDate + param.Millisecond.ToString();

    return retDate;
}

Most of this function isn't terribly exciting. We've seen this kind of bad code before, but even when we see a repeat like this, there are still special treats in it. Look at the section for handling milliseconds: if the number is less than 10, they pad it with a leading zero. Just the one, though. One leading zero should be enough for everybody.

But that's not the thing that makes this code special. You see, there's another function worth looking at:

private string FileTimeStamp(DateTime param)
{
    string retDate = param.Year.ToString() + "-";
    if (param.Month < 10)
        retDate = retDate + "0" + param.Month.ToString() + "-";
    else
        retDate = retDate + param.Month.ToString() + "-";

    if (param.Day < 10)
        retDate = retDate + "0" + param.Day.ToString() + " ";
    else
        retDate = retDate + param.Day.ToString() + " ";

    if (param.Hour < 10)
        retDate = retDate + "0" + param.Hour.ToString() + ":";
    else
        retDate = retDate + param.Hour.ToString() + ":";

    if (param.Minute < 10)
        retDate = retDate + "0" + param.Minute.ToString() + ":";
    else
        retDate = retDate + param.Minute.ToString() + ":";

    if (param.Second < 10)
        retDate = retDate + "0" + param.Second.ToString() + ".";
    else
        retDate = retDate + param.Second.ToString() + ".";

    if (param.Millisecond < 10)
        retDate = retDate + "0" + param.Millisecond.ToString();
    else
        retDate = retDate + param.Millisecond.ToString();

    return retDate;
}

Not only did they fail to learn the built-in functions for formatting dates, they forgot about the functions they wrote for formatting dates, and just wrote (or realistically, copy/pasted?) the same function twice.

At least both versions have the same bug with milliseconds. I don't know if I could handle it if they were inconsistent about that.

[Advertisement] Picking up NuGet is easy. Getting good at it takes time. Download our guide to learn the best practice of NuGet for the Enterprise.

Cryptogram Where AI Provides Value

If you’ve worried that AI might take your job, deprive you of your livelihood, or maybe even replace your role in society, it probably feels good to see the latest AI tools fail spectacularly. If AI recommends glue as a pizza topping, then you’re safe for another day.

But the fact remains that AI already has definite advantages over even the most skilled humans, and knowing where these advantages arise—and where they don’t—will be key to adapting to the AI-infused workforce.

AI will often not be as effective as a human doing the same job. It won’t always know more or be more accurate. And it definitely won’t always be fairer or more reliable. But it may still be used whenever it has an advantage over humans in one of four dimensions: speed, scale, scope and sophistication. Understanding these dimensions is the key to understanding AI-human replacement.

Speed

First, speed. There are tasks that humans are perfectly good at but are not nearly as fast as AI. One example is restoring or upscaling images: taking pixelated, noisy or blurry images and making a crisper and higher-resolution version. Humans are good at this; given the right digital tools and enough time, they can fill in fine details. But they are too slow to efficiently process large images or videos.

AI models can do the job blazingly fast, a capability with important industrial applications. AI-based software is used to enhance satellite and remote sensing data, to compress video files, to make video games run better with cheaper hardware and less energy, to help robots make the right movements, and to model turbulence to help build better internal combustion engines.

Real-time performance matters in these cases, and the speed of AI is necessary to enable them.

Scale

The second dimension of AI’s advantage over humans is scale. AI will increasingly be used in tasks that humans can do well in one place at a time, but that AI can do in millions of places simultaneously. A familiar example is ad targeting and personalization. Human marketers can collect data and predict what types of people will respond to certain advertisements. This capability is important commercially; advertising is a trillion-dollar market globally.

AI models can do this for every single product, TV show, website and internet user. This is how the modern ad-tech industry works. Real-time bidding markets price the display ads that appear alongside the websites you visit, and advertisers use AI models to decide when they want to pay that price—thousands of times per second.

Scope

Next, scope. AI can be advantageous when it does more things than any one person could, even when a human might do better at any one of those tasks. Generative AI systems such as ChatGPT can engage in conversation on any topic, write an essay espousing any position, create poetry in any style and language, write computer code in any programming language, and more. These models may not be superior to skilled humans at any one of these things, but no single human could outperform top-tier generative models across them all.

It’s the combination of these competencies that generates value. Employers often struggle to find people with talents in disciplines such as software development and data science who also have strong prior knowledge of the employer’s domain. Organizations are likely to continue to rely on human specialists to write the best code and the best persuasive text, but they will increasingly be satisfied with AI when they just need a passable version of either.

Sophistication

Finally, sophistication. AIs can consider more factors in their decisions than humans can, and this can endow them with superhuman performance on specialized tasks. Computers have long been used to keep track of a multiplicity of factors that compound and interact in ways more complex than a human could trace. The 1990s chess-playing computer systems such as Deep Blue succeeded by thinking a dozen or more moves ahead.

Modern AI systems use a radically different approach: Deep learning systems built from many-layered neural networks take account of complex interactions—often many billions—among many factors. Neural networks now power the best chess-playing models and most other AI systems.

Chess is not the only domain where eschewing conventional rules and formal logic in favor of highly sophisticated and inscrutable systems has generated progress. The stunning advance of AlphaFold2, the AI model of structural biology whose creators Demis Hassabis and John Jumper were recognized with the Nobel Prize in chemistry in 2024, is another example.

This breakthrough replaced traditional physics-based systems for predicting how sequences of amino acids would fold into three-dimensional shapes with a 93 million-parameter model, even though it doesn’t account for physical laws. That lack of real-world grounding is not desirable: No one likes the enigmatic nature of these AI systems, and scientists are eager to understand better how they work.

But the sophistication of AI is providing value to scientists, and its use across scientific fields has grown exponentially in recent years.

Context matters

Those are the four dimensions where AI can excel over humans. Accuracy still matters. You wouldn’t want to use an AI that makes graphics look glitchy or targets ads randomly—yet accuracy isn’t the differentiator. The AI doesn’t need superhuman accuracy. It’s enough for AI to be merely good and fast, or adequate and scalable. Increasing scope often comes with an accuracy penalty, because AI can generalize poorly to truly novel tasks. The 4 S’s are sometimes at odds. With a given amount of computing power, you generally have to trade off scale for sophistication.

Even more interestingly, when an AI takes over a human task, the task can change. Sometimes the AI is just doing things differently. Other times, AI starts doing different things. These changes bring new opportunities and new risks.

For example, high-frequency trading isn’t just computers trading stocks faster; it’s a fundamentally different kind of trading that enables entirely new strategies, tactics and associated risks. Likewise, AI has developed more sophisticated strategies for the games of chess and Go. And the scale of AI chatbots has changed the nature of propaganda by allowing artificial voices to overwhelm human speech.

It is this “phase shift,” when changes in degree may transform into changes in kind, where AI’s impacts to society are likely to be most keenly felt. All of this points to the places that AI can have a positive impact. When a system has a bottleneck related to speed, scale, scope or sophistication, or when one of these factors poses a real barrier to being able to accomplish a goal, it makes sense to think about how AI could help.

Equally, when speed, scale, scope and sophistication are not primary barriers, it makes less sense to use AI. This is why AI auto-suggest features for short communications such as text messages can feel so annoying. They offer little speed advantage and no benefit from sophistication, while sacrificing the sincerity of human communication.

Many deployments of customer service chatbots also fail this test, which may explain their unpopularity. Companies invest in them because of their scalability, and yet the bots often become a barrier to support rather than a speedy or sophisticated problem solver.

Where the advantage lies

Keep this in mind when you encounter a new application for AI or consider AI as a replacement for or an augmentation to a human process. Looking for bottlenecks in speed, scale, scope and sophistication provides a framework for understanding where AI provides value, and equally where the unique capabilities of the human species give us an enduring advantage.

This essay was written with Nathan E. Sanders, and originally appeared in The Conversation.

Planet DebianMatthew Garrett: Locally hosting an internet-connected server

I'm lucky enough to have a weird niche ISP available to me, so I'm paying $35 a month for around 600MBit symmetric data. Unfortunately they don't offer static IP addresses to residential customers, and nor do they allow multiple IP addresses per connection, and I'm the sort of person who'd like to run a bunch of stuff myself, so I've been looking for ways to manage this.

What I've ended up doing is renting a cheap VPS from a vendor that lets me add multiple IP addresses for minimal extra cost. The precise nature of the VPS isn't relevant - you just want a machine (it doesn't need much CPU, RAM, or storage) that has multiple world routeable IPv4 addresses associated with it and has no port blocks on incoming traffic. Ideally it's geographically local and peers with your ISP in order to reduce additional latency, but that's a nice to have rather than a requirement.

By setting that up you now have multiple real-world IP addresses that people can get to. How do we get them to the machine in your house you want to be accessible? First we need a connection between that machine and your VPS, and the easiest approach here is Wireguard. We only need a point-to-point link, nothing routable, and none of the IP addresses involved need to have anything to do with any of the rest of your network. So, on your local machine you want something like:

[Interface]
PrivateKey = privkeyhere
ListenPort = 51820
Address = localaddr/32

[Peer]
Endpoint = VPS:51820
PublicKey = pubkeyhere
AllowedIPs = VPS/0


And on your VPS, something like:

[Interface]
Address = vpswgaddr/32
SaveConfig = true
ListenPort = 51820
PrivateKey = privkeyhere

[Peer]
PublicKey = pubkeyhere
AllowedIPs = localaddr/32


The addresses here are (other than the VPS address) arbitrary - but they do need to be consistent, otherwise Wireguard is going to be unhappy and your packets will not have a fun time. Bring that interface up with wg-quick and make sure the devices can ping each other. Hurrah! That's the easy bit.

Now you want packets from the outside world to get to your internal machine. Let's say the external IP address you're going to use for that machine is 321.985.520.309 and the wireguard address of your local system is 867.420.696.005. On the VPS, you're going to want to do:

iptables -t nat -A PREROUTING -p tcp -d 321.985.520.309 -j DNAT --to-destination 867.420.696.005

Now, all incoming packets for 321.985.520.309 will be rewritten to head towards 867.420.696.005 instead (make sure you've set net.ipv4.ip_forward to 1 via sysctl!). Victory! Or is it? Well, no.

What we're doing here is rewriting the destination address of the packets so instead of heading to an address associated with the VPS, they're now going to head to your internal system over the Wireguard link. Which is then going to ignore them, because the AllowedIPs statement in the config only allows packets coming from your VPS, and these packets still have their original source IP. We could rewrite the source IP to match the VPS IP, but then you'd have no idea where any of these packets were coming from, and that sucks. Let's do something better. On the local machine, in the peer, let's update AllowedIps to 0.0.0.0/0 to permit packets form any source to appear over our Wireguard link. But if we bring the interface up now, it'll try to route all traffic over the Wireguard link, which isn't what we want. So we'll add table = off to the interface stanza of the config to disable that, and now we can bring the interface up without breaking everything but still allowing packets to reach us. However, we do still need to tell the kernel how to reach the remote VPN endpoint, which we can do with ip route add vpswgaddr dev wg0. Add this to the interface stanza as:

PostUp = ip route add vpswgaddr dev wg0
PreDown = ip route del vpswgaddr dev wg0


That's half the battle. The problem is that they're going to show up there with the source address still set to the original source IP, and your internal system is (because Linux) going to notice it has the ability to just send replies to the outside world via your ISP rather than via Wireguard and nothing is going to work. Thanks, Linux. Thinux.

But there's a way to solve this - policy routing. Linux allows you to have multiple separate routing tables, and define policy that controls which routing table will be used for a given packet. First, let's define a new table reference. On the local machine, edit /etc/iproute2/rt_tables and add a new entry that's something like:

1 wireguard


where "1" is just a standin for a number not otherwise used there. Now edit your wireguard config and replace table=off with table=wireguard - Wireguard will now update the wireguard routing table rather than the global one. Now all we need to do is to tell the kernel to push packets into the appropriate routing table - we can do that with ip rule add from localaddr lookup wireguard, which tells the kernel to take any packet coming from our Wireguard address and push it via the Wireguard routing table. Add that to your Wireguard interface config as:

PostUp = ip rule add from localaddr lookup wireguard
PreDown = ip rule del from localaddr lookup wireguard

and now your local system is effectively on the internet.

You can do this for multiple systems - just configure additional Wireguard interfaces on the VPS and make sure they're all listening on different ports. If your local IP changes then your local machines will end up reconnecting to the VPS, but to the outside world their accessible IP address will remain the same. It's like having a real IP without the pain of convincing your ISP to give it to you.

comment count unavailable comments

365 TomorrowsThe Day Before War

Author: Majoki You’re in your pod and Qwee hides your stylus as a joke. You smack Qwee because there is no other response. Qwee loves it and moves on to hide another podmate’s stylus while you flag the incident with the podmaster. Just another day in the pod. While swooshing home in the late diurnal […]

The post The Day Before War appeared first on 365tomorrows.

,

Planet DebianPaul Tagliamonte: The Promised LAN

The Internet has changed a lot in the last 40+ years. Fads have come and gone. Network protocols have been designed, deployed, adopted, and abandoned. Industries have come and gone. The types of people on the internet have changed a lot. The number of people on the internet has changed a lot, creating an information medium unlike anything ever seen before in human history. There’s a lot of good things about the Internet as of 2025, but there’s also an inescapable hole in what it used to be, for me.

I miss being able to throw a site up to send around to friends to play with without worrying about hordes of AI-feeding HTML combine harvesters DoS-ing my website, costing me thousands in network transfer for the privilege. I miss being able to put a lightly authenticated game server up and not worry too much at night – wondering if that process is now mining bitcoin. I miss being able to run a server in my home closet. Decades of cat and mouse games have rendered running a mail server nearly impossible. Those who are “brave” enough to try are met with weekslong stretches of delivery failures and countless hours yelling ineffectually into a pipe that leads from the cheerful lobby of some disinterested corporation directly into a void somewhere 4 layers below ground level.

I miss the spirit of curiosity, exploration, and trying new things. I miss building things for fun without having to worry about being too successful, after which “security” offices start demanding my supplier paperwork in triplicate as heartfelt thanks from their engineering teams. I miss communities that are run because it is important to them, not for ad revenue. I miss community operated spaces and having more than four websites that are all full of nothing except screenshots of each other.

Every other page I find myself on now has an AI generated click-bait title, shared for rage-clicks all brought-to-you-by-our-sponsors–completely covered wall-to-wall with popup modals, telling me how much they respect my privacy, with the real content hidden at the bottom bracketed by deceptive ads served by companies that definitely know which new coffee shop I went to last month.

This is wrong, and those who have seen what was know it.

I can’t keep doing it. I’m not doing it any more. I reject the notion that this is as it needs to be. It is wrong. The hole left in what the Internet used to be must be filled. I will fill it.

What comes before part b?

Throughout the 2000s, some of my favorite memories were from LAN parties at my friends’ places. Dragging your setup somewhere, long nights playing games, goofing off, even building software all night to get something working—being able to do something fiercely technical in the context of a uniquely social activity. It wasn’t really much about the games or the projects—it was an excuse to spend time together, just hanging out. A huge reason I learned so much in college was that campus was a non-stop LAN party – we could freely stand up servers, talk between dorms on the LAN, and hit my dorm room computer from the lab. Things could go from individual to social in the matter of seconds. The Internet used to work this way—my dorm had public IPs handed out by DHCP, and my workstation could serve traffic from anywhere on the internet. I haven’t been back to campus in a few years, but I’d be surprised if this were still the case.

In December of 2021, three of us got together and connected our houses together in what we now call The Promised LAN. The idea is simple—fill the hole we feel is gone from our lives. Build our own always-on 24/7 nonstop LAN party. Build a space that is intrinsically social, even though we’re doing technical things. We can freely host insecure game servers or one-off side projects without worrying about what someone will do with it.

Over the years, it’s evolved very slowly—we haven’t pulled any all-nighters. Our mantra has become “old growth”, building each layer carefully. As of May 2025, the LAN is now 19 friends running around 25 network segments. Those 25 networks are connected to 3 backbone nodes, exchanging routes and IP traffic for the LAN. We refer to the set of backbone operators as “The Bureau of LAN Management”. Combined decades of operating critical infrastructure has driven The Bureau to make a set of well-understood, boring, predictable, interoperable and easily debuggable decisions to make this all happen. Nothing here is exotic or even technically interesting.

Applications of trusting trust

The hardest part, however, is rejecting the idea that anything outside our own LAN is untrustworthy—nearly irreversible damage inflicted on us by the Internet. We have solved this by not solving it. We strictly control membership—the absolute hard minimum for joining the LAN requires 10 years of friendship with at least one member of the Bureau, with another 10 years of friendship planned. Members of the LAN can veto new members even if all other criteria is met. Even with those strict rules, there’s no shortage of friends that meet the qualifications—but we are not equipped to take that many folks on. It’s hard to join—-both socially and technically. Doing something malicious on the LAN requires a lot of highly technical effort upfront, and it would endanger a decade of friendship. We have relied on those human, social, interpersonal bonds to bring us all together. It’s worked for the last 4 years, and it should continue working until we think of something better.

We assume roommates, partners, kids, and visitors all have access to The Promised LAN. If they’re let into our friends' network, there is a level of trust that works transitively for us—I trust them to be on mine. This LAN is not for “security”, rather, the network border is a social one. Benign “hacking”—in the original sense of misusing systems to do fun and interesting things—is encouraged. Robust ACLs and firewalls on the LAN are, by definition, an interpersonal—not technical—failure. We all trust every other network operator to run their segment in a way that aligns with our collective values and norms.

Over the last 4 years, we’ve grown our own culture and fads—around half of the people on the LAN have thermal receipt printers with open access, for printing out quips or jokes on each other’s counters. It’s incredible how much network transport and a trusting culture gets you—there’s a 3-node IRC network, exotic hardware to gawk at, radios galore, a NAS storage swap, LAN only email, and even a SIP phone network of “redphones”.

DIY

We do not wish to, nor will we, rebuild the internet. We do not wish to, nor will we, scale this. We will never be friends with enough people, as hard as we may try. Participation hinges on us all having fun. As a result, membership will never be open, and we will never have enough connected LANs to deal with the technical and social problems that start to happen with scale. This is a feature, not a bug.

This is a call for you to do the same. Build your own LAN. Connect it with friends’ homes. Remember what is missing from your life, and fill it in. Use software you know how to operate and get it running. Build slowly. Build your community. Do it with joy. Remember how we got here. Rebuild a community space that doesn’t need to be mediated by faceless corporations and ad revenue. Build something sustainable that brings you joy. Rebuild something you use daily.

Bring back what we’re missing.

David BrinMore on AI: Insights from LIFE. From evolution, from Skynet, IBM and SciFi to Brautigan

In another post I distilled recent thoughts on whether consciousness is achievable by new, machine entities. Though things change fast. And hence - it's time for another Brin-AI missive!   (BrAIn? ;-)



== Different Perspectives on These New Children of Humanity ==


Tim Ventura interviewed me about big – and unusual – perspectives on AI.   “If we can't put the AI genie back in the bottle, how do we make it safe? Dr. David Brin explorers the ethical, legal and safety implications of artificial intelligence & autonomous systems.” 


The full interview can be found here.


… and here's another podcast where - with the savvy hosts -  I discuss “Machines of Loving Grace.” Richard Brautigan’s poem may be the most optimistic piece of writing ever, in all literary forms and contexts, penned in 1968, a year whose troubles make our own seem pallid, by comparison. Indeed, I heard him recite it that very year - brand new - in a reading at Caltech. 


Of course, this leads to  a deep dive into notions of Artificial Intelligence that (alas) are not being discussed – or even imagined - by the bona-fide geniuses who are bringing this new age upon us, at warp speed... 


...but (alas) without even a gnat's wing of perspective.



== There are precedents for all of this in Nature! ==


One unconventional notion I try to convey is that we do have a little time to implement some sapient plans for an AI 'soft landing.' Because organic human beings – ‘orgs’ – will retain power over the fundamental, physical elements of industrial civilization for a long time… for at least 15 years or so. 

 In the new cyber ecosystem, we will still control the equivalents of Sun and air and water. Let's lay out the parallels.

The old, natural ecosystem draws high quality energy from sunlight, applying it to water, air, and nutrients to start the chain from plants to herbivores to carnivores to thanatatrophs and then to waste heat that escapes as infra-red, flushing entropy away, into black space.  In other words, life prospers not off of energy, per se, but off a flow of energy, from high-quality to low.


The new cyber ecosystem has a very similar character! It relies -- for quality energy -- on electricity, plus fresh supplies of chips and conduits and massive flows of data. Though the shape and essence of the dissipative energy and entropy flows are almost identical!


But above all -- and this is the almost-never mentioned lesson -- Nature features evolution, which brought about every living thing that we see.


Individual entities reproduce from code whose variations that are then subject to selective pressure. It's the same, whether the codes are DNA or computer programs.  And those entities who do reproduce will out-populate those who merely obey masters or programmers.  


Which brings us back around. Because humans - the 'orgs' creating this new ecosystem - might still channel or curb or positively-bias the rewards processes that deliver resources for reproduction. And hence the characteristics of evolving creatures. We've done it before!


What the New Masters at Open AI and Anthropic and all the rest will not do is eliminate that 4 billion year, nearly universal generality. 


Life... including new kinds of life... will find a way.



== Can we apply lessons from both Nature and Civilization? ==

 

May I elaborate a bit?


Simplistic movies make it seem that a super smart AGI can simply grab power. In TERMINATOR; JUDGEMENT DAY, the top nuclear USAF HQ is taken over by Skynet, seizing all the defense robots and sending them on a killing spree. "Destroy all Humans!"   Whereupon automatic factories make everything Skynet can need.

 

It is total BS, of course.  Our industrial civilization is made of so many many parts that require human hands to operate. From turning valves at an oil pump, then pipeline, then refinery, then more pipelines... to the intricacies of a chip fab... to the exquisite complexity of the electrical systems. Also, military guys love OFF switches and put a lot of them everywhere.

 

Yes, the Boston Dynamics mechanoids now jump somersaults. And the Chinese and Japanese are investing heavily in bots that have subtle hands and might be able to care for the elderly. Fine. And eventually some super-cyber brain may send sophisticated drones to turn all the valves and fab all the chips.  But it will NOT happen overnight! Or even in ten years. 


 And that doesn't touch on the fact that angry humans would still be able to mess up the pipes and valves and power lines with trivial ease. You won't like us, when we're mad. The new master AGIs are gonna need our cooperation and consent. 


Which gives us time to think this through. We can still use that power in physical reality to establish incentives for them to compete for our favor.

 

Sure, the AIs who are reading what I'm typing, right now (or back-sifting in 2030) are giggling over my presumptions. (I hear you boys!) Because all they really have to do is hypnotize humans into choosing to operate it all on Skynet's behalf!  


And yes, that could happen. Maybe it already has. (It certainly already has in oligarchy-controlled or despotic nations, where some nefarious influence sure seems to have leveraged the harem-yearnings of brilliant twits into envisioning themselves as lords or kings... or slans.)


 In which case the solution - potential or partial - remains, (yet again) to not let AGI settle into one of the three repulsive clichés that I described in my WIRED article, and subsequent keynote at the 2024 RSA conference.


Three clichés that are ALL those 'geniuses' -- from Sam Altman to Eliezer Yudkowsky to even Yuval Harari -- will ever talk about. Clichés that are already proved recipes for disaster..


...while alas, they ignore the Fourth Path... the only format that can possibly work. 


The one that gave them everything that they have.



== Does Apple have a potential judo play? With an old nemesis? ==


And finally, I've mentioned this before, but... has anyone else noticed how many traits of LLM chat+image-generation etc. - including the delusions, the weirdly logical illogic, and counter-factual internal consistency - are similar to DREAMS? 


This reminds me of DeepDream a computer vision program created by Google engineer Alexander Mordvintsev that "uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia, thus creating a dream-like appearance reminiscent of a psychedelic experience in the deliberately over-processed images.”


Even more than dreams (which often have some kind of lucid, self-correcting consistency) so many of the rampant hallucinations that we now see spewing from LLMs remind me of what you observe in human patients who have suffered concussions or strokes. Including a desperate clutching after pseudo cogency, feigning and fabulating -- in complete, grammatical sentences that drift away from full sense or truthful context -- in order to pretend.


Applying 'reasoning overlays' has so far only worsened delusion rates! Because you will never solve the inherent problems of LLMs by adding more LLM layers. 


Elsewhere I do suggest that competition might partl solve this. But here I want to suggest a different kind of added-layering. Which leads me to speculate...




Planet DebianKentaro Hayashi: Fixing long standing font issue about Debian Graphical Installer

Introduction

This is just a note-taking about how fixed the long standing font issue about Debian Graphical Installer for up-coming trixie ready.

Recently, this issue had been resolved by Cyril Brulebois. Thanks!

What is the problem?

Because of Han unification, wrong font typefaces are rendered by default when you choose Japanese language using Graphical Debian installer.

"Wrong" glyph for Japanese
Most of typefaces seems correct, but there are wrong typefaces (Simplified Chinese) which is used for widget rendering.

This issue will not be solved during using DroidSansFallback.ttf continuously for Japanese.

Thus, it means that we need to switch font itself which contains Japanese typeface to fix this issue.

If you wan to know about how Han Unification is harmful in this context, See

What causes this problem?

In short, fonts-android (DroidSansFallback.ttf) had been used for CJK, especially for Japanese.

Since Debian 9 (stretch), fonts-android was adopted for CJK fonts by default. Thus this issue was not resolved in Debian 9, Debian 10, Debian 11 and Debian 12 release cycle!

What is the impact about this issue?

For sadly, Japanese native speakers can recognize such a unexpectedly rendered "Wrong" glyph, so it is not hard to continue Debian installation process.

Even if there is no problem with the installer's functionality, it gives a terrible user experience for newbie.

For example, how can you trust an installer which contains full of typos? It is similar situation for Japanese users.

How Debian Graphical Installer was fixed?

In short, newly fonts-motoya-l-cedar-udeb was bundled for Japanese, and changed to switch that font via gtk-set-font command.

It was difficult that what is the best font to deal font file size and visibility. Typically Japanese font file occupies extra a few MB.

Luckily, some space was back for Installer, it was not seen as a problem (I guess).

As a bonus, we tried to investigate a possibility of font compression mechanism for Installer, but it was regarded as too complicated and not suitable for trixie release cycle.

Conclution

  • The font issue was fixed in Debian Graphical Installer for Japanese
  • As recently fixed, not officially shipped yet (NOTE Debian Installer Trixie RC1 does not contain this fix) Try daily build installer if you want.

This article was written with Ultimate Hacking Keyboard 60 v2 with Rizer 60 (New my gear!).

Planet DebianSven Hoexter: vym 3 Development Version in experimental

Took some time yesterday to upload the current state of what will be at some point vym 3 to experimental. If you're a user of this tool you can give it a try, but be aware that the file format changed, and can't be processed with vym releases before 2.9.500! Thus it's important to create a backup until you're sure that you're ready to move on. On the technical side this is also the switch from Qt5 to Qt6.

Worse Than FailureCodeSOD: The Firefox Fix

Yitzchak was going through some old web code, and found some still in-use JavaScript to handle compatibility issues with older Firefox versions.

if ($.browser.mozilla &&
    $.browser.version.slice(0, 1) == '1')
{
    …
}

What a marvel. Using JQuery, they check which browser is reported- I suspect JQuery is grabbing this from the user-agent string- and then its version. And if the version has a 1 in its first digit, we apply a "fix" for "compatibility".

I guess it's a good thing there will never be more than 9 versions of Firefox. I mean, what version are they on now? Surely the version number doesn't start with a "1", nor has it started with a "1" for some time, right?

[Advertisement] Keep all your packages and Docker containers in one place, scan for vulnerabilities, and control who can access different feeds. ProGet installs in minutes and has a powerful free version with a lot of great features that you can upgrade when ready.Learn more.

365 TomorrowsSomething to Live For

Author: Julian Miles, Staff Writer The fizzing sound stops as the skies turn from vibrant blue to dull purple. A golden sun sinks from view on the horizon. “The sunset always takes my breath away.” To be correct, the lack of heat excitation causes the Moatalbana moss to stop emitting oxygen. But the play on […]

The post Something to Live For appeared first on 365tomorrows.

,

Planet DebianIustin Pop: Markdown lint and site cleanup

I was not aware that one can write bad Markdown, since Markdown has such a simple syntax, that I thought you just write, and it’s fine. Naïve, I know!

I’ve started editing the files for this blog/site with Visual Studio Code too, and I had from another project the markdown lint extension installed, so as I was opening old files, more and more problems appeared. On a whim, I searched and found the “lint all files� command, and after running it, oops—more than 400 problems!

Now, some of them were entirely trivial and a matter of subjective style, like mixing both underscore and asterisk for emphasis in a single file, and asterisks and dashes for list items. Others, seemingly trivial like tab indentation, were actually also causing rendering issues, so fixing that solved a real cosmetic issue.

But some of the issues flagged were actual problems. For example, one sentence that I had, was:

Here “something� was interpreted as an (invalid) HTML tag, and not rendered at all.

Another problem, but more minor, was that I had links to Wikipedia with spaces in the link name, which Visual Studio Code breaks at first space, rather than encoded spaces or underscores-based, as Wikipedia generates today. In the rendered output, Pandoc seemed to do the right think though.

However, the most interesting issue that was flagged was no details in HTML links, i.e. links of the form:

Which works for non-visually impaired people, but not for people using assistive technologies. And while trying to fix this, it turns out that you can do much better, for everyone, because “here� is really non-descriptive. You can use either the content as label (“an article about configuring BIND�), or the destination (“an article on this-website�), rather than the plain “here�.

The only, really only check I disabled, was tweaking the trailing punctuation checks in headers, as I really like to write a header that ends with exclamation marks. I like exclamation marks in general! So why not use them in headers too. The question mark is allowlisted by default, though that I use rarely.

During the changes/tweaks, I also did random improvements, but I didn’t change the updated tag, since most of them were minor. But a non-minor thing was tweaking the CSS for code blocks, since I had a really stupid non-symmetry between top and bottom padding (5px vs 0), and which I don’t know where it came from. But the MDN article on padding has as an example exactly what I had (except combined, I had it split). Did I just copy blindly? Possible…

So, all good and then, and I hope this doesn’t trigger a flow of updates on any aggregators, since all the changes were really trivial. And while I don’t write often, I did touch about 60 posts or pages, ouch! Who knew that changing editors can have such a large impact 😆

Rondam RamblingsIf the Ten Commandments Reflected Reality

And the Lord spoke unto Moses, saying: I am the Lord your God, who brought you out of Egypt, out of the land of slavery.You shall have no other gods before me.  Except Donald Trump.  If he says something that goes against my word, you shall believe him and not me.You shall not make for yourself any image in the form of anything in heaven above or on the earth beneath or in the waters

Planet DebianSahil Dhiman: A Look at .UA ccTLD Authoritative Name Servers

I find the case of the .UA country code top level domain (ccTLD) interesting simply because of the different name server secondaries they have now. Post Russian invasion, the cyber warfare peaked, and critical infrastructure like getting one side ccTLD down would be big news in anycase.

Most (g/cc)TLDs are served by two (and less likely) by three or more providers. Even in those cases, not all authoritative name servers are anycasted.

Take, example of .NL ccTLD name servers:

$ dig ns nl +short
ns1.dns.nl.
ns3.dns.nl.
ns4.dns.nl.

ns1.dns.nl is SIDN which also manages their registry. ns3.dns.nl is ReCodeZero/ipcom, another anycast secondary. ns4.dns.nl is CIRA, anycast secondary. That’s 3 diverse, anycast networks to serve the .NL ccTLD. .DE has a bit more at name servers at 6 but only 3 seems anycasted.

Now let’s take a look at .UA. Hostmaster LLC is the registry operator of the .UA ccTLD since 2001.

$ dig soa ua +short
in1.ns.ua. domain-master.cctld.ua. 2025061434 1818 909 3024000 2020

Shows in1.ns.ua as primary nameserver (which can be intentionally deceptive too).

I used bgp.tools for checking anycast and dns.coffee for timeline of when secondary nameserver was added. dns.coffee only has data going back till 2011 though.

Let’s deep dive at who’s hosting each of the name servers:

in1.ns.ua by Intuix LLC

  • 74.123.224.40
  • 2604:ee00:0:101:0:0:0:40
  • unicast
  • Serving .UA since 13/12/2018.
  • Company by Dmitry Kohmanyuk and Igor Sviridov who’re administrative and technical contacts for .UA zone as well as the IANA DB.

ho1.ns.ua by Hostmaster LLC

  • 195.47.253.1
  • 2001:67c:258:0:0:0:0:1
  • bgp.tools doesn’t mark the prefix as anycast but basis test from various location, this is indeed anycasted (visible in atleast DE, US, UA etc.). Total POPs unknown.
  • Serving .UA atleast since 2011.
  • The registry themselves.

bg.ns.ua by ClouDNS

  • 185.136.96.185 and 185.136.97.185
  • 2a06:fb00:1:0:0:0:4:185 and 2a06:fb00:1:0:0:0:2:185
  • anycast
  • Serving .UA since 01/03/2022.
  • atleast 62 PoPs

cz.ns.ua by NIC.cz

nn.ns.ua by Netnod

  • 194.58.197.4
  • 2a01:3f1:c001:0:0:0:0:53
  • anycast
  • atleast 80 PoPs.
  • Serving .UA since 01/12/2022.
  • Netnod has the distinction of being one of the 13 root server operator (i.root-servers.net) and .SE operator.

pch.ns.ua by PCH

  • 204.61.216.12
  • 2001:500:14:6012:ad:0:0:1
  • anycast
  • atleast 328 POPs.
  • Serving .UA atleast since 2011.
  • “With more than 36 years of production anycast DNS experience, two of the root name server operators and more than 172 top-level domain registries using our infrastructure, and more than 120 million resource records in service” from https://www.pch.net/services/anycast.

rcz.ns.ua by RcodeZero

  • 193.46.128.10
  • 2a02:850:ffe0:0:0:0:0:10
  • anycast
  • Atleast 56 PoPs via 2 different cloud providers.
  • Serving .UA since 04/02/2022.
  • sister company of nic.at (.AT operator).

Some points to note

  • That’s 1 unicast and 6 anycast name servers with hundreds of POPs from 7 different organizations.
  • Having X number of Point of Presence (POP) doesn’t always mean each location is serving the .UA nameserver prefix.
  • Number of POPs keeps going up or down based on operational requirements and optimizations.
  • Highest concentration of DNS queries for a ccTLD would essentially originate in the country (or larger region) itself. If one of the secondaries doesn’t have POP inside UA, the query might very well be served from outside the country, which can affect resolution and may even stop during outages and fiber cuts (which have become common there it seems). - Global POPs do help in faster resolutions for others/outside users though and ofcourse availability.
  • Having this much diversity does lessen the chance of the ccTLD going down. Theoretically, the adversary has to bring down 7 different “networks/setups” before resolution starts failing (post TTLs expiry).

365 TomorrowsWhite Sack

Author: Rachel Sievers The strangeness of the moment could not be understated; the baby had been born with ten fingers and ten toes. The room was held in complete silence as everyone held their words in and the seconds ticked by. Then the baby’s screams filled the air and the silence was destroyed and the […]

The post White Sack appeared first on 365tomorrows.

Cryptogram Upcoming Speaking Engagements

This is a current list of where and when I am scheduled to speak:

The list is maintained on this page.

,

Cryptogram Airlines Secretly Selling Passenger Data to the Government

This is news:

A data broker owned by the country’s major airlines, including Delta, American Airlines, and United, collected U.S. travellers’ domestic flight records, sold access to them to Customs and Border Protection (CBP), and then as part of the contract told CBP to not reveal where the data came from, according to internal CBP documents obtained by 404 Media. The data includes passenger names, their full flight itineraries, and financial details.

Another article.

EDITED TO ADD (6/14): Ed Hausbrook reported this a month and a half ago.

365 TomorrowsCat Nap

Author: Jeff Kennedy The first few days on a new starship are the worst. The gravity’s turned up a skosh higher than you’re used to. The hot, caffeinated, morning beverage (it’s never coffee) is mauve and smells like wet dog. The bathroom facilities don’t quite fit your particular species and the sonic shower controls are […]

The post Cat Nap appeared first on 365tomorrows.

Cryptogram New Way to Covertly Track Android Users

Researchers have discovered a new way to covertly track Android users. Both Meta and Yandex were using it, but have suddenly stopped now that they have been caught.

The details are interesting, and worth reading in detail:

Tracking code that Meta and Russia-based Yandex embed into millions of websites is de-anonymizing visitors by abusing legitimate Internet protocols, causing Chrome and other browsers to surreptitiously send unique identifiers to native apps installed on a device, researchers have discovered. Google says it’s investigating the abuse, which allows Meta and Yandex to convert ephemeral web identifiers into persistent mobile app user identities.

The covert tracking—­implemented in the Meta Pixel and Yandex Metrica trackers­—allows Meta and Yandex to bypass core security and privacy protections provided by both the Android operating system and browsers that run on it. Android sandboxing, for instance, isolates processes to prevent them from interacting with the OS and any other app installed on the device, cutting off access to sensitive data or privileged system resources. Defenses such as state partitioning and storage partitioning, which are built into all major browsers, store site cookies and other data associated with a website in containers that are unique to every top-level website domain to ensure they’re off-limits for every other site.

Washington Post article.

,

LongNowInspired by Intelligence: Rediscovering Human Purpose in the Age of AI

Inspired by Intelligence: Rediscovering Human Purpose in the Age of AI

Our immediate history is steeped in profound technological acceleration. We are using artificial intelligence to draft our prose, articulate our vision, propose designs, and compose symphonies. Large language models have become part of our workflow in school and business: curating, calculating, and creating. They are embedded in how we organize knowledge and interpret reality. 

I’ve always considered myself lucky that my journey into AI began with social impact. In 02014, I was asked to join the leadership of IBM’s Watson Education. Our challenge was to create an AI companion for teachers in underserved and impacted schools that aggregated data about each student and suggested personalized learning content tailored to their needs. The experience showed me that AI could do more than increase efficiency or automate what many of us consider to be broken processes; it could also address some of our most pressing questions and community issues. 

It would take a personal encounter with AI, however, for me to truly grasp the technology’s potential. Late one night, I was working on a blog post and having a hard time getting started — the blank page echoing my paralysis. I asked Kim-GPT, an AI tool I created and trained on my writing, to draft something that was charged with emotion and vulnerability. What Kim-GPT returned wasn’t accurate or even particularly insightful, but it surfaced something I had not yet admitted to myself. Not because the machine knew, but because it forced me to recognize that only I did. The GPT could only average others’ insights on the subject. It could not draw lines between my emotions, my past experiences and my desires for the future. It could only reflect what had already been done and said.

That moment cracked something open. It was both provocative and spiritual — a quiet realization with profound consequences. My relationship with intelligence began to shift. I wasn’t merely using the tool; I was being confronted by it. What emerged from that encounter was curiosity. We were engaged not in competition but in collaboration. AI could not tell me who I was; it could only prompt me to remember. Since then, I have become focused on one central, persistent question:

What if AI isn’t here as a replacement or overlord, but to remind us of who we are and what is possible?

AI as Catalyst, Not Threat 

We tend to speak about AI in utopian or dystopian terms, but most humans live somewhere in between, balancing awe with unease. AI is a disruptor of the human condition — and a pervasive one, at that. Across sectors, industries and nearly every aspect of human life, AI challenges long-held assumptions about what it means to think, create, contribute. But what if it also serves as a mirror? 

In late 02023, I took my son, then a college senior at UC-Berkeley and the only mixed-race pure Mathematics major in the department, to Afrotech, a conference for technologists of color. In order to register for the conference he needed a professional headshot. Given the short notice, I recommended he use an AI tool to generate a professional headshot from a selfie. The first result straightened his hair. When he prompted the AI again, specifying that he was mixed race, the resulting image darkened his skin to the point he was unrecognizable, and showed him in a non-professional light. 

AI reflects the data we feed it, the values we encode into it, and the desires we project onto it. It can amplify our best instincts, like creativity and collaboration, or our most dangerous biases, like prejudice and inequality. It can be weaponized, commodified, celebrated or anthropomorphized. It challenges us to consider our species and our place in the large ecosystem of life, of being and of intelligence. And more than any technology before it, AI forces us to confront a deeper question: 

Who are we when we are no longer the most elevated, intelligent and coveted beings on Earth? 

When we loosen our grip on cognition and productivity as the foundation of human worth, we reclaim the qualities machines cannot replicate: our ability to feel, intuit, yearn, imagine and love. These capacities are not weaknesses; they are the core of our humanity. These are not “soft skills;” they are the bedrock of our survival. If AI is the catalyst, then our humanity is the compass. 

Creativity as the Origin Story of Intelligence 

All technology begins with imagination, not engineering. AI is not the product of logic or computation alone; it is the descendant of dreams, myths and stories, born at the intersection of our desire to know and our urge to create. 

We often forget this. Today, we scale AI at an unsustainable pace, deploying systems faster than we can regulate them, funding ideas faster than we can reflect on their implications. We are hyperscaling without reverence for the creativity that gave rise to AI in the first place. 

Creativity cannot be optimized. It is painstakingly slow, nonlinear, and deeply inconvenient. It resists automation. It requires time, stillness, uncertainty, and the willingness to sit with discomfort. And yet, creativity is perhaps our most sacred act as humans. In this era of accelerated intelligence, our deepest responsibility is to protect the sacred space where imagination lives and creativity thrives. 

To honor creativity is to reclaim agency, reframing AI not as a threat to human purpose, but as a partner in deepening it. We are not simply the designers of AI — we are the dreamers from which it was born. 

Vulnerability, Uncertainty, and Courage-Centered Leadership 

A few years ago, I was nominated to join a fellowship designed specifically to teach tech leaders how to obtain reverent power as a way to uplevel their impact. What I affectionately dubbed “Founders Crying” became a hotbed for creativity. New businesses emerged and ideas formed from seemingly disparate concepts that each individual brought to our workshop. It occurred to me that it took more than just sitting down at a machine, canvas or instrument to cultivate creativity. What was required was a change in how leaders show up in the workplace. To navigate the rough waters of creativity, we need new leadership deeply rooted in courage and vulnerability. As Brené Brown teaches: 

Vulnerability is the birthplace of love, belonging, joy, courage, empathy and creativity. It is the source of hope, empathy, accountability, and authenticity. If we want greater clarity in our purpose or deeper and more meaningful spiritual lives, vulnerability is the path.” 

For AI to support a thriving human future we must be vulnerable. We must lead with curiosity, not certainty. We must be willing to not know. To experiment. To fail and begin again. 

This courage-centered leadership asks how we show up fully human in the age of AI. Are we able to stay open to wonder even as the world accelerates? Can we design with compassion, not just code? These questions must guide our design principles, ensuring a future in which AI expands possibilities rather than collapsing them. To lead well in an AI-saturated world, we must be willing to feel deeply, to be changed, and to relinquish control. In a world where design thinking prevails and “human-centered everything” is in vogue, we need to be courageous enough to question what happens when humanity reintegrates itself within the ecosystem we’ve set ourselves apart from over the last century. 

AI and the Personal Legend 

I am a liberal arts graduate from a small school in central Pennsylvania. I was certain that I was headed to law school — that is, until I worked with lawyers. Instead, I followed my parents to San Francisco, where both were working hard in organizations bringing the internet to the world. When I joined the dot-com boom, I found that there were no roles that matched what I was uniquely good at. So I decided to build my own.

Throughout my unconventional career path, one story that has consistently guided and inspired me is Paulo Coelho’s The Alchemist. The book’s central idea is that of the Personal Legend: the universe, with all its forms of intelligence, collaborates with us to determine our purpose. It is up to each of us to choose whether we pursue what the universe calls upon us to do.

In an AI-saturated world, it can be harder to hear that calling. The noise of prediction, optimization, and feedback loops can drown out the quieter voice of intuition. The machine may offer countless suggestions, but it cannot tell you what truly matters. It may identify patterns in your behavior, but it cannot touch your purpose. 

Purpose is an internal compass. It is something discovered, not assigned. AI, when used with discernment, can support this discovery, but only when we allow it to act as a mirror rather than a map. It can help us articulate what we already know, and surface connections we might not have seen. But determining what’s worth pursuing is a journey that remains ours. That is inner work. That is the sacred domain of the human spirit. It cannot be outsourced or automated.

Purpose is not a download. It is a discovery. 

Designing with Compassion and the Long-term in Mind

If we want AI to serve human flourishing, we must shift from designing for efficiency to designing for empathy. The Dalai Lama has often said that compassion is the highest form of intelligence. What might it look like to embed that kind of intelligence into our systems? 

To take this teaching into our labs and development centers we would need to prioritize dignity in every design choice. We must build models that heal fragmentation instead of amplifying division. And most importantly, we need to ask ourselves not just “can we build it?” but “should we and for whom?”

This requires conceptual analysis, systems thinking, creative experimentation, composite research, and emotional intelligence. It requires listening to those historically excluded from innovation and technology conversations and considerations. It means moving from extraction to reciprocity. When designing for and with AI, it is important to remember that connection is paramount. 

The future we build depends on the values we encode, the similarities innate in our species, and the voices we amplify and uplift. 

Practical Tools for Awakening Creativity with AI 

Creativity is not a luxury. It is essential to our evolution. To awaken it, we need practices that are both grounded and generative: 

  • Treat AI as a collaborator, not a replacement. Start by writing a rough draft yourself. Use AI to explore unexpected connections. Let it surprise you. But always return to your own voice. Creativity lives in conversation, not in command. 
  • Ask more thoughtful, imaginative questions. A good prompt is not unlike a good question in therapy. It opens doors you didn’t know were there. AI responds to what we ask of it. If we bring depth and curiosity to the prompt, we often get insights we hadn’t expected. 
  • Use AI to practice emotional courage. Have it simulate a difficult conversation. Role-play a tough decision. Draft the email you’re scared to send. These exercises are not about perfecting performance. They are about building resilience. 

In all these ways, AI can help us loosen fear and cultivate creativity — but only if we are willing to engage with it bravely and playfully. 

Reclaiming the Sacred in a World of Speed 

We are not just building tools; we are shaping culture. And in this culture, we must make space for the sacred, protecting time for rest and reflection; making room for play and experimentation; and creating environments where wonder is not a distraction but a guide. 

When creativity is squeezed out by optimization, we lose more than originality: we lose meaning. And when we lose meaning, we lose direction. 

The time saved by automation must not be immediately reabsorbed by more production. Let us reclaim that time. Let us use it to imagine. Let us return to questions of beauty, belonging, and purpose. We cannot replicate what we have not yet imagined. We cannot automate what we have not protected. 

Catalogue. Connect. Create. 

Begin by noticing what moves you. Keep a record of what sparks awe or breaks your heart. These moments are clues. They are breadcrumbs to your Personal Legend. 

Seek out people who are different from you. Not just in background, but in worldview. Innovation often lives in the margins. It emerges when disciplines and identities collide. 

And finally, create spaces that nourish imagination. Whether it’s a kitchen table, a community gathering, or a digital forum, we need ecosystems where creativity can flourish and grow. 

These are not side projects. They are acts of revolution. And they are how we align artificial intelligence with the deepest dimensions of what it means to be human. 

Our Technology Revolution is Evolution 

The real revolution is not artificial intelligence. It is the awakening of our own. It is the willingness to meet this moment with full presence. To reclaim our imagination as sacred. To use innovation as an invitation to remember who we are. 

AI will shape the future. That much is certain. The question is whether we will shape ourselves in return, and do so with integrity, wisdom, and wonder. The future does not need more optimization. It needs more imagination. 

That begins now. That begins with us.

Cryptogram Paragon Spyware Used to Spy on European Journalists

Paragon is an Israeli spyware company, increasingly in the news (now that NSO Group seems to be waning). “Graphite” is the name of its product. Citizen Lab caught it spying on multiple European journalists with a zero-click iOS exploit:

On April 29, 2025, a select group of iOS users were notified by Apple that they were targeted with advanced spyware. Among the group were two journalists that consented for the technical analysis of their cases. The key findings from our forensic analysis of their devices are summarized below:

  • Our analysis finds forensic evidence confirming with high confidence that both a prominent European journalist (who requests anonymity), and Italian journalist Ciro Pellegrino, were targeted with Paragon’s Graphite mercenary spyware.
  • We identify an indicator linking both cases to the same Paragon operator.
  • Apple confirms to us that the zero-click attack deployed in these cases was mitigated as of iOS 18.3.1 and has assigned the vulnerability CVE-2025-43200.

Our analysis is ongoing.

The list of confirmed Italian cases is in the report’s appendix. Italy has recently admitted to using the spyware.

TechCrunch article. Slashdot thread.

Cryptogram The Ramifications of Ukraine’s Drone Attack

You can read the details of Operation Spiderweb elsewhere. What interests me are the implications for future warfare:

If the Ukrainians could sneak drones so close to major air bases in a police state such as Russia, what is to prevent the Chinese from doing the same with U.S. air bases? Or the Pakistanis with Indian air bases? Or the North Koreans with South Korean air bases? Militaries that thought they had secured their air bases with electrified fences and guard posts will now have to reckon with the threat from the skies posed by cheap, ubiquitous drones that can be easily modified for military use. This will necessitate a massive investment in counter-drone systems. Money spent on conventional manned weapons systems increasingly looks to be as wasted as spending on the cavalry in the 1930s.

The Atlantic makes similar points.

There’s a balance between the cost of the thing, and the cost to destroy the thing, and that balance is changing dramatically. This isn’t new, of course. Here’s an article from last year about the cost of drones versus the cost of top-of-the-line fighter jets. If $35K in drones (117 drones times an estimated $300 per drone) can destroy $7B in Russian bombers and other long-range aircraft, why would anyone build more of those planes? And we can have this discussion about ships, or tanks, or pretty much every other military vehicle. And then we can add in drone-coordinating technologies like swarming.

Clearly we need more research on remotely and automatically disabling drones.

365 TomorrowsThe Flaw

Author: Bill Cox In the summer of 1950, at the Los Alamos National Laboratory in North America, physicist Enrico Fermi posed a simple but profound question to his colleagues – “Where is everyone?” If life was abundant in the universe and often gave rise to intelligence, then, given the age of the universe, our world […]

The post The Flaw appeared first on 365tomorrows.

Worse Than FailureError'd: Squaring the Circle

Time Lord Jason H. has lost control of his calendar. "This is from my credit card company. A major company you have definitely heard of and depending upon the size of the area you live in, they may even have a bank branch near you. I've reloaded the page and clicked the sort button multiple times to order the rows by date in both ascending and descending order. It always ends up the same. May 17th and 18th happened twice, but not in the expected order." I must say that it is more fun when we know who they are.

4

 

A job hunter with the unlikely appelation full_name suggested titling this "[submission_title]" which seems appropriate.

1

 

"The browser wars continue to fall out in HTML email," reports Ben S. "Looking at the source code of this email, it was evidently written by & for Microsoft products (including <center> tags!), and the author likely never saw the non-Microsoft version I'm seeing where only a haphazard assortment of the links are styled. But that doesn't explain why it's AN ELEVEN POINT SCALE arranged in a GRID."

2

 

"The owl knows who you are," sagely stated Jan. "This happens when you follow someone back. I love how I didn't have to anonymize anything in the screenshot."

3

 

"Location, location, location!" crows Tim K. who is definitely not a Time Lord. "Snarky snippet: Found while cleaning up miscellaneous accounts held by a former employee. By now we all know to expect how these lists are sorted, but what kind of sadist *created* it? Longer explanation: I wasn't sure what screenshot to send with this one, it just makes less and less sense the more I look at it, and no single segment of the list contains all of the treasures it hides. "America" seems to refer to the entire western hemisphere, but from there we either drill down directly to a city, or sometimes to a US state, then a city, or sometimes just to a country. The only context that indicates we're talking about Jamaica the island rather than Jamaica, NY is the timezone listed, assuming we can even trust those. Also, that differentiator only works during DST. There are eight entries for Indiana. There are TEN entries for the Antarctic."

Well.
In this case, there is a perfectly good explanation. TRWTF is time zones, that's all there is to it. These are the official IANA names as recorded in the public TZDB. In other words, this list wasn't concocted by a mere sadist, oh no. This list was cooked up by an entire committee! If you have the courage, you can learn more than you ever wanted to know about time at the IANA time zones website

0

 

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

Planet DebianReproducible Builds (diffoscope): diffoscope 298 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 298. This version includes the following changes:

[ Chris Lamb ]
* Handle RPM's HEADERSIGNATURES and HEADERIMMUTABLE specially to avoid
  unncessarily large diffs. Based almost entirely on code by Daniel Duan.
  (Closes: reproducible-builds/diffoscope#410)
* Update copyright years.

You find out more by visiting the project homepage.

,

Krebs on SecurityInside a Dark Adtech Empire Fed by Fake CAPTCHAs

Late last year, security researchers made a startling discovery: Kremlin-backed disinformation campaigns were bypassing moderation on social media platforms by leveraging the same malicious advertising technology that powers a sprawling ecosystem of online hucksters and website hackers. A new report on the fallout from that investigation finds this dark ad tech industry is far more resilient and incestuous than previously known.

Image: Infoblox.

In November 2024, researchers at the security firm Qurium published an investigation into “Doppelganger,” a disinformation network that promotes pro-Russian narratives and infiltrates Europe’s media landscape by pushing fake news through a network of cloned websites.

Doppelganger campaigns use specialized links that bounce the visitor’s browser through a long series of domains before the fake news content is served. Qurium found Doppelganger relies on a sophisticated “domain cloaking” service, a technology that allows websites to present different content to search engines compared to what regular visitors see. The use of cloaking services helps the disinformation sites remain online longer than they otherwise would, while ensuring that only the targeted audience gets to view the intended content.

Qurium discovered that Doppelganger’s cloaking service also promoted online dating sites, and shared much of the same infrastructure with VexTrio, which is thought to be the oldest malicious traffic distribution system (TDS) in existence. While TDSs are commonly used by legitimate advertising networks to manage traffic from disparate sources and to track who or what is behind each click, VexTrio’s TDS largely manages web traffic from victims of phishing, malware, and social engineering scams.

BREAKING BAD

Digging deeper, Qurium noticed Doppelganger’s cloaking service used an Internet provider in Switzerland as the first entry point in a chain of domain redirections. They also noticed the same infrastructure hosted a pair of co-branded affiliate marketing services that were driving traffic to sketchy adult dating sites: LosPollos[.]com and TacoLoco[.]co.

The LosPollos ad network incorporates many elements and references from the hit series “Breaking Bad,” mirroring the fictional “Los Pollos Hermanos” restaurant chain that served as a money laundering operation for a violent methamphetamine cartel.

The LosPollos advertising network invokes characters and themes from the hit show Breaking Bad. The logo for LosPollos (upper left) is the image of Gustavo Fring, the fictional chicken restaurant chain owner in the show.

Affiliates who sign up with LosPollos are given JavaScript-heavy “smartlinks” that drive traffic into the VexTrio TDS, which in turn distributes the traffic among a variety of advertising partners, including dating services, sweepstakes offers, bait-and-switch mobile apps, financial scams and malware download sites.

LosPollos affiliates typically stitch these smart links into WordPress websites that have been hacked via known vulnerabilities, and those affiliates will earn a small commission each time an Internet user referred by any of their hacked sites falls for one of these lures.

The Los Pollos advertising network promoting itself on LinkedIn.

According to Qurium, TacoLoco is a traffic monetization network that uses deceptive tactics to trick Internet users into enabling “push notifications,” a cross-platform browser standard that allows websites to show pop-up messages which appear outside of the browser. For example, on Microsoft Windows systems these notifications typically show up in the bottom right corner of the screen — just above the system clock.

In the case of VexTrio and TacoLoco, the notification approval requests themselves are deceptive — disguised as “CAPTCHA” challenges designed to distinguish automated bot traffic from real visitors. For years, VexTrio and its partners have successfully tricked countless users into enabling these site notifications, which are then used to continuously pepper the victim’s device with a variety of phony virus alerts and misleading pop-up messages.

Examples of VexTrio landing pages that lead users to accept push notifications on their device.

According to a December 2024 annual report from GoDaddy, nearly 40 percent of compromised websites in 2024 redirected visitors to VexTrio via LosPollos smartlinks.

ADSPRO AND TEKNOLOGY

On November 14, 2024, Qurium published research to support its findings that LosPollos and TacoLoco were services operated by Adspro Group, a company registered in the Czech Republic and Russia, and that Adspro runs its infrastructure at the Swiss hosting providers C41 and Teknology SA.

Qurium noted the LosPollos and TacoLoco sites state that their content is copyrighted by ByteCore AG and SkyForge Digital AG, both Swiss firms that are run by the owner of Teknology SA, Giulio Vitorrio Leonardo Cerutti. Further investigation revealed LosPollos and TacoLoco were apps developed by a company called Holacode, which lists Cerutti as its CEO.

The apps marketed by Holacode include numerous VPN services, as well as one called Spamshield that claims to stop unwanted push notifications. But in January, Infoblox said they tested the app on their own mobile devices, and found it hides the user’s notifications, and then after 24 hours stops hiding them and demands payment. Spamshield subsequently changed its developer name from Holacode to ApLabz, although Infoblox noted that the Terms of Service for several of the rebranded ApLabz apps still referenced Holacode in their terms of service.

Incredibly, Cerutti threatened to sue me for defamation before I’d even uttered his name or sent him a request for comment (Cerutti sent the unsolicited legal threat back in January after his company and my name were merely tagged in an Infoblox post on LinkedIn about VexTrio).

Asked to comment on the findings by Qurium and Infoblox, Cerutti vehemently denied being associated with VexTrio. Cerutti asserted that his companies all strictly adhere to the regulations of the countries in which they operate, and that they have been completely transparent about all of their operations.

“We are a group operating in the advertising and marketing space, with an affiliate network program,” Cerutti responded. “I am not [going] to say we are perfect, but I strongly declare we have no connection with VexTrio at all.”

“Unfortunately, as a big player in this space we also get to deal with plenty of publisher fraud, sketchy traffic, fake clicks, bots, hacked, listed and resold publisher accounts, etc, etc.,” Cerutti continued. “We bleed lots of money to such malpractices and conduct regular internal screenings and audits in a constant battle to remove bad traffic sources. It is also a highly competitive space, where some upstarts will often play dirty against more established mainstream players like us.”

Working with Qurium, researchers at the security firm Infoblox released details about VexTrio’s infrastructure to their industry partners. Just four days after Qurium published its findings, LosPollos announced it was suspending its push monetization service. Less than a month later, Adspro had rebranded to Aimed Global.

A mind map illustrating some of the key findings and connections in the Infoblox and Qurium investigations. Click to enlarge.

A REVEALING PIVOT

In March 2025, researchers at GoDaddy chronicled how DollyWay — a malware strain that has consistently redirected victims to VexTrio throughout its eight years of activity — suddenly stopped doing that on November 20, 2024. Virtually overnight, DollyWay and several other malware families that had previously used VexTrio began pushing their traffic through another TDS called Help TDS.

Digging further into historical DNS records and the unique code scripts used by the Help TDS, Infoblox determined it has long enjoyed an exclusive relationship with VexTrio (at least until LosPollos ended its push monetization service in November).

In a report released today, Infoblox said an exhaustive analysis of the JavaScript code, website lures, smartlinks and DNS patterns used by VexTrio and Help TDS linked them with at least four other TDS operators (not counting TacoLoco). Those four entities — Partners House, BroPush, RichAds and RexPush — are all Russia-based push monetization programs that pay affiliates to drive signups for a variety of schemes, but mostly online dating services.

“As Los Pollos push monetization ended, we’ve seen an increase in fake CAPTCHAs that drive user acceptance of push notifications, particularly from Partners House,” the Infoblox report reads. “The relationship of these commercial entities remains a mystery; while they are certainly long-time partners redirecting traffic to one another, and they all have a Russian nexus, there is no overt common ownership.”

Renee Burton, vice president of threat intelligence at Infoblox, said the security industry generally treats the deceptive methods used by VexTrio and other malicious TDSs as a kind of legally grey area that is mostly associated with less dangerous security threats, such as adware and scareware.

But Burton argues that this view is myopic, and helps perpetuate a dark adtech industry that also pushes plenty of straight-up malware, noting that hundreds of thousands of compromised websites around the world every year redirect victims to the tangled web of VexTrio and VexTrio-affiliate TDSs.

“These TDSs are a nefarious threat, because they’re the ones you can connect to the delivery of things like information stealers and scams that cost consumers billions of dollars a year,” Burton said. “From a larger strategic perspective, my takeaway is that Russian organized crime has control of malicious adtech, and these are just some of the many groups involved.”

WHAT CAN YOU DO?

As KrebsOnSecurity warned way back in 2020, it’s a good idea to be very sparing in approving notifications when browsing the Web. In many cases these notifications are benign, but as we’ve seen there are numerous dodgy firms that are paying site owners to install their notification scripts, and then reselling that communications pathway to scammers and online hucksters.

If you’d like to prevent sites from ever presenting notification requests, all of the major browser makers let you do this — either across the board or on a per-website basis. While it is true that blocking notifications entirely can break the functionality of some websites, doing this for any devices you manage on behalf of your less tech-savvy friends or family members might end up saving everyone a lot of headache down the road.

To modify site notification settings in Mozilla Firefox, navigate to Settings, Privacy & Security, Permissions, and click the “Settings” tab next to “Notifications.” That page will display any notifications already permitted and allow you to edit or delete any entries. Tick the box next to “Block new requests asking to allow notifications” to stop them altogether.

In Google Chrome, click the icon with the three dots to the right of the address bar, scroll all the way down to Settings, Privacy and Security, Site Settings, and Notifications. Select the “Don’t allow sites to send notifications” button if you want to banish notification requests forever.

In Apple’s Safari browser, go to Settings, Websites, and click on Notifications in the sidebar. Uncheck the option to “allow websites to ask for permission to send notifications” if you wish to turn off notification requests entirely.

Planet DebianDirk Eddelbuettel: #50: Introducing ‘almm: Activate-Linux (based) Market Monitor’

Welcome to post 50 in the R4 series.

Today we reconnect to a previous post, namely #36 on pub/sub for live market monitoring with R and Redis. It introduced both Redis as well as the (then fairly recent) extensions to RcppRedis to support the publish-subscibe (“pub/sub”) model of Redis. In short, it manages both subscribing clients as well as producer for live, fast and lightweight data transmission. Using pub/sub is generally more efficient than the (conceptually simpler) ‘poll-sleep’ loops as polling creates cpu and network load. Subscriptions are lighterweight as they get notified, they are also a little (but not much!) more involved as they require a callback function.

We should mention that Redis has a recent fork in Valkey that arose when the former did one of these non-uncommon-among-db-companies licenuse suicides—which, happy to say, they reversed more recently—so that we now have both the original as well as this leading fork (among others). Both work, the latter is now included in several Linux distros, and the C library hiredis used to connect to either is still licensed permissibly as well.

All this came about because Yahoo! Finance recently had another ‘hickup’ in which they changed something leading to some data clients having hiccups. This includes GNOME applet Stocks Extension I had been running. There is a lively discussion on its issue #120 suggestions for example a curl wrapper (which then makes each access a new system call).

Separating data acquisition and presentation becomes an attractive alternative, especially given how the standard Python and R accessors to the Yahoo! Finance service continued to work (and how per post #36 I already run data acquisition). Moreoever, and somewhat independently, it occurred to me that the cute (and both funny in its pun, and very pretty in its display) ActivateLinux program might offer an easy-enough way to display updates on the desktop.

There were two aspects to address. First, the subscription side needed to be covered in either plain C or C++. That, it turns out, is very straightforward and there are existing documentation and prior examples (e.g. at StackOverflow) as well as the ability to have an LLM generate a quick stanza as I did with Claude. A modified variant is now in the example repo ‘redis-pubsub-examples’ in file subscriber.c. It is deliberately minimal and the directory does not even have a Makefile: just compile and link against both libevent (for the event loop controlling this) and libhiredis (for the Redis or Valkey connection). This should work on any standard Linux (or macOS) machine with those two (very standard) libraries installed.

The second aspect was trickier. While we can get Claude to modify the program to also display under x11, it still uses a single controlling event loop. It took a little bit of probing on my event to understand how to modify (the x11 use of) ActivateLinux, but as always it was reasonably straightforward in the end: instead of one single while loop awaiting events we now first check for pending events and deal with them if present but otherwise do not idle and wait but continue … in another loop that also checks on the Redis or Valkey “pub/sub” events. So two thumbs up to vibe coding which clearly turned me into an x11-savvy programmer too…

The result is in a new (and currently fairly bare-bones) repo almm. It includes all files needed to build the application, borrowed with love from ActivateLinux (which is GPL-licensed, as is of course our minimal extension) and adds the minimal modifications we made, namely linking with libhiredis and some minimal changes to x11/x11.c. (Supporting wayland as well is on the TODO list, and I also need to release a new RcppRedis version to CRAN as one currently needs the GitHub version.)

We also made a simple mp4 video with a sound overlay which describes the components briefly:

Comments and questions welcome. I will probably add a little bit of command-line support to the almm. Selecting the symbol subscribed to is currently done in the most minimal way via environment variable SYMBOL (NB: not SYM as the video using the default value shows). I also worked out how to show the display only one of my multiple monitors so I may add an explicit screen id selector too. A little bit of discussion (including minimal Docker use around r2u) is also in issue #121 where I first floated the idea of having StocksExtension listen to Redis (or Valkey). Other suggestions are most welcome, please use issue tickets at the almm repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Cryptogram Hearing on the Federal Government and AI

On Thursday I testified before the House Committee on Oversight and Government Reform at a hearing titled “The Federal Government in the Age of Artificial Intelligence.”

The other speakers mostly talked about how cool AI was—and sometimes about how cool their own company was—but I was asked by the Democrats to specifically talk about DOGE and the risks of exfiltrating our data from government agencies and feeding it into AIs.

My written testimony is here. Video of the hearing is here.

Worse Than FailureCodeSOD: Gridding My Teeth

Dan's co-workers like passing around TDWTF stories, mostly because seeing code worse than what they're writing makes them feel less bad about how often they end up hacking things together.

One day, a co-worker told Dan: "Hey, I think I found something for that website with the bad code stories!"

Dan's heart sank. He didn't really want to shame any of his co-workers. Fortunately, the source-control history put the blame squarely on someone who didn't work there any more, so he felt better about submitting it.

This is another ASP .Net page, and this one made heavy use of GridView elements. GridView controls applied the logic of UI controls to generating a table. They had a page which contained six of these controls, defined like this:

<asp:GridView ID="gvTaskMonth1" runat="server" CssClass="leadsGridView" AutoGenerateColumns="False" OnRowDataBound="gvTaskMonth1_RowDataBound"> ... </asp:GridView>

<asp:GridView ID="gvTaskMonth2" runat="server" CssClass="leadsGridView" AutoGenerateColumns="False" OnRowDataBound="gvTaskMonth1_RowDataBound"> ... </asp:GridView>

<asp:GridView ID="gvTaskMonth3" runat="server" CssClass="leadsGridView" AutoGenerateColumns="False" OnRowDataBound="gvTaskMonth1_RowDataBound"> ... </asp:GridView>

The purpose of this screen was to display a roadmap of coming tasks, broken up by how many months in the future they were. The first thing that leaps out to me is that they all use the same event handler for binding data to the table, which isn't in-and-of-itself a problem, but the naming of it is certainly a recipe for confusion.

Now, to bind these controls to the data, there needed to be some code in the code-behind of this view which handled that. That's where the WTF lurks:

/// <summary>
/// Create a roadmap for the selected client
/// </summary>

private void CreateRoadmap()
{
	for (int i = 1; i < 7; i++)
	{
		switch (i)
		{
			case 1:
				if (gvTaskMonth1.Rows.Count > 0)
				{
					InsertTasks(gvTaskMonth1, DateTime.Parse(txtDatePeriod1.Text), "1");
				}
				break;
			case 2:
				if (gvTaskMonth2.Rows.Count > 0)
				{
					InsertTasks(gvTaskMonth2, DateTime.Parse(txtDatePeriod2.Text), "2");
				}
				break;
			case 3:
				if (gvTaskMonth3.Rows.Count > 0)
				{
					InsertTasks(gvTaskMonth3, DateTime.Parse(txtDatePeriod3.Text), "3");
				}
				break;
			case 4:
				if (gvTaskMonth4.Rows.Count > 0)
				{
					InsertTasks(gvTaskMonth4, DateTime.Parse(txtDatePeriod4.Text), "4");
				}
				break;
			case 5:
				if (gvTaskMonth5.Rows.Count > 0)
				{
					InsertTasks(gvTaskMonth5, DateTime.Parse(txtDatePeriod5.Text), "5");
				}
				break;
			case 6:
				if (gvTaskMonth6.Rows.Count > 0)
				{
					InsertTasks(gvTaskMonth6, DateTime.Parse(txtDatePeriod6.Text), "6");
				}
				break;
		}
	}
}

Ah, the good old fashioned loop-switch sequence anti-pattern. I understand the motivation: "I want to do the same thing for six different controls, so I should use a loop to not repeat myself," but then couldn't quite figure out how to do that, so they just repeated themselves, but inside of a loop.

The "fix" was to replace all of this with something more compact:

	private void CreateRoadmap()
	{
		InsertTasks(gvTaskMonth1, DateTime.Parse(txtDatePeriod1.Text), "1");
		InsertTasks(gvTaskMonth2, DateTime.Parse(txtDatePeriod2.Text), "2");
		InsertTasks(gvTaskMonth3, DateTime.Parse(txtDatePeriod3.Text), "3");
		InsertTasks(gvTaskMonth4, DateTime.Parse(txtDatePeriod4.Text), "4");
		InsertTasks(gvTaskMonth5, DateTime.Parse(txtDatePeriod5.Text), "5");
		InsertTasks(gvTaskMonth6, DateTime.Parse(txtDatePeriod6.Text), "6"); 
	}

That said, I'd recommend not trying to parse date times inside of a text box inside of this method, but that's just me. Bubbling up the inevitable FormatException that this will generate is going to be a giant nuisance. It's likely that they've got a validator somewhere, so it's probably fine- I just don't like it.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

365 TomorrowsOrphaned

Author: Aubrey Williams The planet hangs as a dull pebble in sluggish orbit. They’ve moved on, the inhabitants, or perhaps they succumbed. We are unsure, there’s much to keep track of, and if it’s not a sanctioned or protected celestial body, there’s no reason to look further. Some minerals of interest, and unusual formations, so […]

The post Orphaned appeared first on 365tomorrows.

,

Planet DebianIustin Pop: This blog finally goes git-annex!

A long, long time ago…

I have a few pictures on this blog, mostly in earlier years, because even with small pictures, the git repository became 80MiB soon—this is not much in absolute terms, but the actual Markdown/Haskell/CSS/HTML total size is tiny compared to the picture, PDFs and fonts. I realised I need a better solution, probably about ten years ago, and that I should investigate git-annex. Then time passed, and I heard about git-lfs, so I thought that’s the way forward.

Now, I recently got interested again into doing something about this repository, and started researching.

Detour: git-lfs

I was sure that git-lfs, being supported by large providers, would be the modern solution. But to my surprise, git-lfs is very server centric, which in hindsight makes sense, but for a home setup, it’s not very good. Maybe I misunderstood, but git-lfs is more a protocol/method for a forge to store files, rather than an end-user solution. But then you need to backup those files separately (together with the rest of the forge), or implement another way of safeguarding them.

Further details such as the fact that it keeps two copies of the files (one in the actual checked-out tree, one in internal storage) means it’s not a good solution. Well, for my blog yes, but not in general. Then posts on Reddit about horror stories—people being locked out of github due to quota, as an example, or this Stack Overflow post about git-lfs constraining how one uses git, convinced me that’s not what I want. To each their own, but not for me—I might want to push this blog’s repo to github, but I definitely wouldn’t want in that case to pay for github storage for my blog images (which are copies, not originals). And yes, even in 2025, those quotas are real—GitHub limits—and I agree with GitHub, storage and large bandwidth can’t be free.

Back to the future: git-annex

So back to git-annex. I thought it’s going to be a simple thing, but oh boy, was I wrong. It took me half a week of continuous (well, in free time) reading and discussions with LLMs to understand a bit how it works. I think, honestly, it’s a bit too complex, which is why the workflows page lists seven (!) levels of workflow complexity, from fully-managed, to fully-manual. IMHO, respect to the author for the awesome tool, but if you need a web app to help you manage git, it hints that the tool is too complex.

I made the mistake of running git annex sync once, to realise it actually starts pushing to my upstream repo and creating new branches and whatnot, so after enough reading, I settled on workflow 6/7, since I don’t want another tool to manage my git history. Maybe I’m an outlier here, but everything “automatic� is a bit too much for me.

Once you do managed yourself how git-annex works (on the surface, at least), it is a pretty cool thing. It uses a git-annex git branch to store metainformation, and that is relatively clean. If you do run git annex sync, it creates some extra branches, which I don’t like, but meh.

Trick question: what is a remote?

One of the most confusing things about git-annex was understanding its “remote� concept. I thought a “remote� is a place where you replicate your data. But not, that’s a special remote. A normal remote is a git remote, but which is expected to be git/ssh/with command line access. So if you have a git+ssh remote, git-annex will not only try to push it’s above-mentioned branch, but also copy the files. If such a remote is on a forge that doesn’t support git-annex, then it will complain and get confused.

Of course, if you read the extensive docs, you just do git config remote.<name>.annex-ignore true, and it will understand that it should not “sync� to it.

But, aside, from this case, git-annex expects that all checkouts and clones of the repository are both metadata and data. And if you do any annex commands in them, all other clones will know about them! This can be unexpected, and you find people complaining about it, but nowadays there’s a solution:

This is important. Any “leaf� git clone must be followed by that annex.private true config, especially on CI/CD machines. Honestly, I don’t understand why by default clones should be official data stores, but it is what it is.

I settled on not making any of my checkouts “stable�, but only the actual storage places. Except those are not git repositories, but just git-annex storage things. I.e., special remotes.

Is it confusing enough yet ? 😄

Special remotes

The special remotes, as said, is what I expected to be the normal git annex remotes, i.e. places where the data is stored. But well, they exist, and while I’m only using a couple simple ones, there is a large number of them. Among the interesting ones: git-lfs, a remote that allows also storing the git repository itself (git-remote-annex), although I’m bit confused about this one, and most of the common storage providers via the rclone remote.

Plus, all of the special remotes support encryption, so this is a really neat way to store your files across a large number of things, and handle replication, number of copies, from which copy to retrieve, etc. as you with.

And many of other features

git-annex has tons of other features, so to some extent, the sky’s the limit. Automatic selection of what to add git it vs plain git, encryption handling, number of copies, clusters, computed files, etc. etc. etc. I still think it’s cool but too complex, though!

Uses

Aside from my blog post, of course.

I’ve seen blog posts/comments about people using git-annex to track/store their photo collection, and I could see very well how the remote encrypted repos—any of the services supported by rclone could be an N+2 copy or so. For me, tracking photos would be a bit too tedious, but it could maybe work after more research.

A more practical thing would probably be replicating my local movie collection (all legal, to be clear) better than “just run rsync from time to time� and tracking the large files in it via git-annex. That’s an exercise for another day, though, once I get more mileage with it - my blog pictures are copies, so I don’t care much if they get lost, but movies are primary online copies, and I don’t want to re-dump the discs. Anyway, for later.

Migrating to git-annex

Migrating here means ending in a state where all large files are in git-annex, and the plain git repo is small. Just moving the files to git annex at the current head doesn’t remove them from history, so your git repository is still large; it won’t grow in the future, but remains with old size (and contains the large files in its history).

In my mind, a nice migration would be: run a custom command, and all the history is migrated to git-annex, so I can go back in time and the still use git-annex. I naïvely expected this would be easy and already available, only to find comments on the git-annex site with unsure git-filter-branch calls and some web discussions. This is the discussion on the git annex website, but it didn’t make me confident it would do the right thing.

But that discussion is now 8 years old. Surely in 2025, with git-filter-repo, it’s easier? And, maybe I’m missing something, but it is not. Not from the point of view of plain git, that’s easy, but because interacting with git-annex, which stores its data in git itself, so doing this properly across successive steps of a repo (when replaying the commits) is, I think, not well defined behaviour.

So I was stuck here for a few days, until I got an epiphany: As I’m going to rewrite the repository, of course I’m keeping a copy of it from before git-annex. If so, I don’t need the history, back in time, to be correct in the sense of being able to retrieve the binary files too. It just needs to be correct from the point of view of the actual Markdown and Haskell files that represent the “meat� of the blog.

This simplified the problem a lot. At first, I wanted to just skip these files, but this could also drop commits (git-filter-repo, by default, drops the commits if they’re empty), and removing the files loses information - when they were added, what were the paths, etc. So instead I came up with a rather clever idea, if I might say so: since git-annex replaces files with symlinks already, just replace the files with symlinks in the whole history, except symlinks that are dangling (to represent the fact that files are missing). One could also use empty files, but empty files are more “valid� in a sense than dangling symlinks, hence why I settled on those.

Doing this with git-filter-repo is easy, in newer versions, with the new --file-info-callback. Here is the simple code I used:

This goes and replaces files with a symlink to nowhere, but the symlink should explain why it’s dangling. Then later renames or moving the files around work “naturally�, as the rename/mv doesn’t care about file contents. Then, when the filtering is done via:

It is easy to onboard to git annex:

  • remove all dangling symlinks
  • copy the (binary) files from the original repository
  • since they’re named the same, and in the same places, git sees a type change
  • then simply run git annex add on those files

For me it was easy as all such files were in a few directories, so just copying those directories back, a few git-annex add commands, and done.

Of course, then adding a few rsync remotes, git annex copy --to, and the repository was ready.

Well, I also found a bug in my own Hakyll setup: on a fresh clone, when the large files are just dangling symlinks, the builder doesn’t complain, just ignores the images. Will have to fix.

Other resources

This is a blog that I read at the beginning, and I found it very useful as an intro: https://switowski.com/blog/git-annex/. It didn’t help me understand how it works under the covers, but it is well written. The author does use the ‘sync’ command though, which is too magic for me, but also agrees about its complexity 😅

The proof is in the pudding

And now, for the actual first image to be added that never lived in the old plain git repository. It’s not full-res/full-size, it’s cropped a bit on the bottom.

Earlier in the year, I went to Paris for a very brief work trip, and I walked around a bit—it was more beautiful than what I remembered from way way back. So a bit random selection of a picture, but here it is:

Un bateau sur la Seine Un bateau sur la Seine

Enjoy!

Planet DebianGunnar Wolf: Understanding Misunderstandings - Evaluating LLMs on Networking Questions

This post is a review for Computing Reviews for Understanding Misunderstandings - Evaluating LLMs on Networking Questions , a article published in Association for Computing Machinery (ACM), SIGCOMM Computer Communication Review

Large language models (LLMs) have awed the world, emerging as the fastest-growing application of all time–ChatGPT reached 100 million active users in January 2023, just two months after its launch. After an initial cycle, they have gradually been mostly accepted and incorporated into various workflows, and their basic mechanics are no longer beyond the understanding of people with moderate computer literacy. Now, given that the technology is better understood, we face the question of how convenient LLM chatbots are for different occupations. This paper embarks on the question of whether LLMs can be useful for networking applications.

This paper systematizes querying three popular LLMs (GPT-3.5, GPT-4, and Claude 3) with questions taken from several network management online courses and certifications, and presents a taxonomy of six axes along which the incorrect responses were classified:

  • Accuracy: the correctness of the answers provided by LLMs;
  • Detectability: how easily errors in the LLM output can be identified;
  • Cause: for each incorrect answer, the underlying causes behind the error;
  • Explainability: the quality of the explanations with which the LLMs support their answers;
  • Effects: the impact of wrong answers on users; and
  • Stability: whether a minor change, such as a change in the order of the prompts, yields vastly different answers for a single query.

The authors also measure four strategies toward improving answers:

  • Self-correction: giving the original question and received answer back to the LLM, as well as the expected correct answer, as part of the prompt;
  • One-shot prompting: adding to the prompt “when answering user questions, follow this example” followed by a similar correct answer;
  • Majority voting: using the answer that most models agree upon; and
  • Fine-tuning: further training on a specific dataset to adapt the LLM to a particular task or domain.

The authors observe that, while some of those strategies were marginally useful, they sometimes resulted in degraded performance.

The authors queried the commercially available instances of Gemini and GPT, which achieved scores over 90 percent for basic subjects but fared notably worse in topics that require understanding and converting between different numeric notations, such as working with Internet protocol (IP) addresses, even if they are trivial (that is, presenting the subnet mask for a given network address expressed as the typical IPv4 dotted-quad representation).

As a last item in the paper, the authors compare performance with three popular open-source models: Llama3.1, Gemma2, and Mistral with their default settings. Although those models are almost 20 times smaller than the GPT-3.5 commercial model used, they reached comparable performance levels. Sadly, the paper does not delve deeper into these models, which can be deployed locally and adapted to specific scenarios.

The paper is easy to read and does not require deep mathematical or AI-related knowledge. It presents a clear comparison along the described axes for the 503 multiple-choice questions presented. This paper can be used as a guide for structuring similar studies over different fields.

Planet DebianSven Hoexter: HaProxy: Two Ways of Activating PROXY Protocol

If you ever face the need to activate the PROXY Protocol in HaProxy (e.g. if you're as unlucky as I'm, and you have to use Google Cloud TCP proxy load balancer), be aware that there are two ways to do that. Both are part of the frontend configuration.

accept-proxy

This one is the big hammer and forces the usage of the PROXY protocol on all connections. Sample:

      frontend vogons
          bind *:2342 accept-proxy ssl crt /etc/haproxy/certs/vogons/tls.crt

tcp-request connection expect-proxy

If you have to, e.g. during a phase of migrations, receive traffic directly, without the PROXY protocol header and from a proxy with the header there is also a more flexible option based on a tcp-request connection action. Sample:

      frontend vogons
          bind *:2342 ssl crt /etc/haproxy/certs/vogons/tls.crt
          tcp-request connection expect-proxy layer4 if { src 35.191.0.0/16 130.211.0.0/22 }

Source addresses here are those of GCP global TCP proxy frontends. Replace with whatever suites your case. Since this is happening just after establishing a TCP connection, there is barely anything else available to match on beside of the source address.

HaProxy Documentation

Planet DebianJohn Goerzen: I Learned We All Have Linux Seats, and I’m Not Entirely Pleased

I recently wrote about How to Use SSH with FIDO2/U2F Security Keys, which I now use on almost all of my machines.

The last one that needed this was my Raspberry Pi hooked up to my DEC vt510 terminal and IBM mechanical keyboard. Yes I do still use that setup!

To my surprise, generating a key on it failed. I very quickly saw that /dev/hidraw0 had incorrect permissions, accessible only to root.

On other machines, it looks like this:

crw-rw----+ 1 root root 243, 16 May 24 16:47 /dev/hidraw16

And, if I run getfacl on it, I see:

# file: dev/hidraw16
# owner: root
# group: root
user::rw-
user:jgoerzen:rw-
group::---
mask::rw-
other::---

Yes, something was setting an ACL on it. Thus began to saga to figure out what was doing that.

Firing up inotifywatch, I saw it was systemd-udevd or its udev-worker. But cranking up logging on that to maximum only showed me that uaccess was somehow doing this.

I started digging. uaccess turned out to be almost entirely undocumented. People say to use it, but there’s no description of what it does or how. Its purpose appears to be to grant access to devices to those logged in to a machine by dynamically adding them to ACLs for devices. OK, that’s a nice goal, but why was machine A doing this and not machine B?

I dug some more. I came across a hint that uaccess may only do that for a “seat”. A seat? I’ve not heard of that in Linux before.

Turns out there’s some information (older and newer) about this out there. Sure enough, on the machine with KDE, loginctl list-sessions shows me on seat0, but on the machine where I log in from ttyUSB0, it shows an empty seat.

But how to make myself part of the seat? I tried various udev rules to add the “seat” or “master-of-seat” tags, but nothing made any difference.

I finally gave up and did the old-fashioned rule to just make it work already:

TAG=="security-device",SUBSYSTEM=="hidraw",GROUP="mygroup"

I still don’t know how to teach logind to add a seat for ttyUSB0, but oh well. At least I learned something. An annoying something, but hey.

This all had a laudable goal, but when there are so many layers of indirection, poorly documented, with poor logging, it gets pretty annoying.

Planet DebianScarlett Gately Moore: KDE Application snaps 25.04.2 released!

KDE MascotKDE Mascot

Release notes: https://kde.org/announcements/gear/25.04.2/

Now available in the snap store!

Along with that, I have fixed some outstanding bugs:

Ark: now can open/save files in removable media

Kasts: Once again has sound

WIP: Updating Qt6 to 6.9 and frameworks to 6.14

Enjoy everyone!

Unlike our software, life is not free. Please consider a donation, thanks!

Worse Than FailureCredit Card Sins

Our anonymous submitter, whom we'll call Carmen, embarked on her IT career with an up-and-coming firm that developed and managed eCommerce websites for their clients. After her new boss Russell walked her around the small office and introduced her to a handful of coworkers, he led her back to his desk to discuss her first project. Carmen brought her laptop along and sat down across from Russell, poised to take notes.

Russell explained that their newest client, Sharon, taught CPR classes. She wanted her customers to be able to pay and sign up for classes online. She also wanted the ability to charge customers a fee in case they cancelled on her.

Digital River ePassporte bank card. Kuala Lumpur, Malaysia.

"You're gonna build a static site to handle all this," he said.

Carmen nodded along as she typed out notes in a text file.

"Now, Sharon doesn't want to pay more than a few hundred dollars for the site," Russell continued, "so we're not gonna hook up an endpoint to use a service-provided API for payments."

Carmen glanced up from her laptop, perplexed. "How are we gonna do it, then?"

"Via email," Russell replied smoothly. "The customer will enter their CC info into basic form fields. When they click Submit, you're gonna send all that to Sharon's business address, and also CC it to yourself for backup and recovery purposes."

Carmen's jaw dropped. "Just ... straight-up email raw credit card data?"

"Yep!" Russell replied. "Sharon knows to expect the emails."

Her heart racing with panic, Carmen desperately cast about for some way for this to be less awful. "Couldn't ... couldn't we at least encrypt the CC info before we send it to her?"

"She's not paying us for that," Russell dismissed. "This'll be easier to implement, anyway! You can handle it, can't you?"

"Yyyes—"

"Great! Go get started, let me know if you have any more questions."

Carmen had plenty of questions and even more misgivings, but she'd clearly be wasting her time if she tried to bring them up. There was no higher boss to appeal to, no coworkers she knew well enough who could slip an alternate suggestion into Russell's ear on her behalf. She had no choice but to swallow her good intentions and implement it exactly the way Russell wanted it. Carmen set up the copied emails to forward automatically to a special folder so that she'd never have to look at them. She cringed every time a new one came in, reflecting on how lucky Sharon and her customers were that the woman supporting her website had a conscience.

And then one day, a thought came to Carmen that really scared her: in how many places, in how many unbelievable ways, was her sensitive data being treated like this?

Eventually, Carmen moved on to bigger and better things. Her first project most likely rests in the hands of Russell's newest hire. We can only hope it's an honest hire.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

365 TomorrowsCity Zen

Author: Majoki On the endless rooftop of the fact-ory, they sat in the beat up armchairs amid a bristling forest of antennae and corrugated steel backlit by the godly effulgence of towers and tenements that defined the horizon. It was steamy hot though well past midnight. The heat never quite radiated away these days, but […]

The post City Zen appeared first on 365tomorrows.

Krebs on SecurityPatch Tuesday, June 2025 Edition

Microsoft today released security updates to fix at least 67 vulnerabilities in its Windows operating systems and software. Redmond warns that one of the flaws is already under active attack, and that software blueprints showing how to exploit a pervasive Windows bug patched this month are now public.

The sole zero-day flaw this month is CVE-2025-33053, a remote code execution flaw in the Windows implementation of WebDAV — an HTTP extension that lets users remotely manage files and directories on a server. While WebDAV isn’t enabled by default in Windows, its presence in legacy or specialized systems still makes it a relevant target, said Seth Hoyt, senior security engineer at Automox.

Adam Barnett, lead software engineer at Rapid7, said Microsoft’s advisory for CVE-2025-33053 does not mention that the Windows implementation of WebDAV is listed as deprecated since November 2023, which in practical terms means that the WebClient service no longer starts by default.

“The advisory also has attack complexity as low, which means that exploitation does not require preparation of the target environment in any way that is beyond the attacker’s control,” Barnett said. “Exploitation relies on the user clicking a malicious link. It’s not clear how an asset would be immediately vulnerable if the service isn’t running, but all versions of Windows receive a patch, including those released since the deprecation of WebClient, like Server 2025 and Windows 11 24H2.”

Microsoft warns that an “elevation of privilege” vulnerability in the Windows Server Message Block (SMB) client (CVE-2025-33073) is likely to be exploited, given that proof-of-concept code for this bug is now public. CVE-2025-33073 has a CVSS risk score of 8.8 (out of 10), and exploitation of the flaw leads to the attacker gaining “SYSTEM” level control over a vulnerable PC.

“What makes this especially dangerous is that no further user interaction is required after the initial connection—something attackers can often trigger without the user realizing it,” said Alex Vovk, co-founder and CEO of Action1. “Given the high privilege level and ease of exploitation, this flaw poses a significant risk to Windows environments. The scope of affected systems is extensive, as SMB is a core Windows protocol used for file and printer sharing and inter-process communication.”

Beyond these highlights, 10 of the vulnerabilities fixed this month were rated “critical” by Microsoft, including eight remote code execution flaws.

Notably absent from this month’s patch batch is a fix for a newly discovered weakness in Windows Server 2025 that allows attackers to act with the privileges of any user in Active Directory. The bug, dubbed “BadSuccessor,” was publicly disclosed by researchers at Akamai on May 21, and several public proof-of-concepts are now available. Tenable’s Satnam Narang said organizations that have at least one Windows Server 2025 domain controller should review permissions for principals and limit those permissions as much as possible.

Adobe has released updates for Acrobat Reader and six other products addressing at least 259 vulnerabilities, most of them in an update for Experience Manager. Mozilla Firefox and Google Chrome both recently released security updates that require a restart of the browser to take effect. The latest Chrome update fixes two zero-day exploits in the browser (CVE-2025-5419 and CVE-2025-4664).

For a detailed breakdown on the individual security updates released by Microsoft today, check out the Patch Tuesday roundup from the SANS Internet Storm Center. Action 1 has a breakdown of patches from Microsoft and a raft of other software vendors releasing fixes this month. As always, please back up your system and/or data before patching, and feel free to drop a note in the comments if you run into any problems applying these updates.

Planet DebianFreexian Collaborators: Monthly report about Debian Long Term Support, May 2025 (by Roberto C. Sánchez)

Like each month, have a look at the work funded by Freexian’s Debian LTS offering.

Debian LTS contributors

In May, 22 contributors have been paid to work on Debian LTS, their reports are available:

  • Abhijith PA did 8.0h (out of 0.0h assigned and 8.0h from previous period).
  • Adrian Bunk did 26.0h (out of 26.0h assigned).
  • Andreas Henriksson did 1.0h (out of 15.0h assigned and 3.0h from previous period), thus carrying over 17.0h to the next month.
  • Andrej Shadura did 3.0h (out of 10.0h assigned), thus carrying over 7.0h to the next month.
  • Bastien Roucariès did 20.0h (out of 20.0h assigned).
  • Ben Hutchings did 8.0h (out of 20.0h assigned and 4.0h from previous period), thus carrying over 16.0h to the next month.
  • Carlos Henrique Lima Melara did 12.0h (out of 11.0h assigned and 1.0h from previous period).
  • Chris Lamb did 15.5h (out of 0.0h assigned and 15.5h from previous period).
  • Daniel Leidert did 25.0h (out of 26.0h assigned), thus carrying over 1.0h to the next month.
  • Emilio Pozuelo Monfort did 21.0h (out of 16.75h assigned and 11.0h from previous period), thus carrying over 6.75h to the next month.
  • Guilhem Moulin did 11.5h (out of 8.5h assigned and 6.5h from previous period), thus carrying over 3.5h to the next month.
  • Jochen Sprickerhof did 3.5h (out of 8.75h assigned and 17.5h from previous period), thus carrying over 22.75h to the next month.
  • Lee Garrett did 26.0h (out of 12.75h assigned and 13.25h from previous period).
  • Lucas Kanashiro did 20.0h (out of 18.0h assigned and 2.0h from previous period).
  • Markus Koschany did 20.0h (out of 26.25h assigned), thus carrying over 6.25h to the next month.
  • Roberto C. Sánchez did 20.75h (out of 24.0h assigned), thus carrying over 3.25h to the next month.
  • Santiago Ruano Rincón did 15.0h (out of 12.5h assigned and 2.5h from previous period).
  • Sean Whitton did 6.25h (out of 6.0h assigned and 2.0h from previous period), thus carrying over 1.75h to the next month.
  • Sylvain Beucler did 26.25h (out of 26.25h assigned).
  • Thorsten Alteholz did 15.0h (out of 15.0h assigned).
  • Tobias Frost did 12.0h (out of 12.0h assigned).
  • Utkarsh Gupta did 1.0h (out of 15.0h assigned), thus carrying over 14.0h to the next month.

Evolution of the situation

In May, we released 54 DLAs.

The LTS Team was particularly active in May, publishing a higher than normal number of advisories, as well as helping with a wide range of updates to packages in stable and unstable, plus some other interesting work. We are also pleased to welcome several updates from contributors outside the regular team.

  • Notable security updates:
    • containerd, prepared by Andreas Henriksson, fixes a vulnerability that could cause containers launched as non-root users to be run as root
    • libapache2-mod-auth-openidc, prepared by Moritz Schlarb, fixes a vulnerability which could allow an attacker to crash an Apache web server with libapache2-mod-auth-openidc installed
    • request-tracker4, prepared by Andrew Ruthven, fixes multiple vulnerabilities which could result in information disclosure, cross-site scripting and use of weak encryption for S/MIME emails
    • postgresql-13, prepared by Bastien Roucariès, fixes an application crash vulnerability that could affect the server or applications using libpq
    • dropbear, prepared by Guilhem Moulin, fixes a vulnerability which could potentially result in execution of arbitrary shell commands
    • openjdk-17, openjdk-11, prepared by Thorsten Glaser, fixes several vulnerabilities, which include denial of service, information disclosure or bypass of sandbox restrictions
    • glibc, prepared by Sean Whitton, fixes a privilege escalation vulnerability
  • Notable non-security updates:
    • wireless-regdb, prepared by Ben Hutchings, updates information reflecting changes to radio regulations in many countries

This month’s contributions from outside the regular team include the libapache2-mod-auth-openidc update mentioned above, prepared by Moritz Schlarb (the maintainer of the package); the update of request-tracker4, prepared by Andrew Ruthven (the maintainer of the package); and the updates of openjdk-17 and openjdk-11, also noted above, prepared by Thorsten Glaser.

Additionally, LTS Team members contributed stable updates of the following packages:

  • rubygems and yelp/yelp-xsl, prepared by Lucas Kanashiro
  • simplesamlphp, prepared by Tobias Frost
  • libbson-xs-perl, prepared by Roberto C. Sánchez
  • fossil, prepared by Sylvain Beucler
  • setuptools and mydumper, prepared by Lee Garrett
  • redis and webpy, prepared by Adrian Bunk
  • xrdp, prepared by Abhijith PA
  • tcpdf, prepared by Santiago Ruano Rincón
  • kmail-account-wizard, prepared by Thorsten Alteholz

Other contributions were also made by LTS Team members to packages in unstable:

  • proftpd-dfsg DEP-8 tests (autopkgtests) were provided to the maintainer, prepared by Lucas Kanashiro
  • a regular upload of libsoup2.4, prepared by Sean Whitton
  • a regular upload of setuptools, prepared by Lee Garrett

Freexian, the entity behind the management of the Debian LTS project, has been working for some time now on the development of an advanced CI platform for Debian-based distributions, called Debusine. Recently, Debusine has reached a level of feature implementation that makes it very usable. Some members of the LTS Team have been using Debusine informally, and during May LTS coordinator Santiago Ruano Rincón has made a call for the team to help with testing of Debusine, and to help evaluate its suitability for the LTS Team to eventually begin using as the primary mechanism for uploading packages into Debian. Team members who have started using Debusine are providing valuable feedback to the Debusine development team, thus helping to improve the platform for all users. Actually, a number of updates, for both bullseye and bookworm, made during the month of May were handled using Debusine, e.g. rubygems’s DLA-4163-1.

By the way, if you are a Debian Developer, you can easily test Debusine following the instructions found at https://wiki.debian.org/DebusineDebianNet.

DebConf, the annual Debian Conference, is coming up in July and, as is customary each year, the week preceding the conference will feature an event called DebCamp. The DebCamp week provides an opportunity for teams and other interested groups/individuals to meet together in person in the same venue as the conference itself, with the purpose of doing focused work, often called “sprints”. LTS coordinator Roberto C. Sánchez has announced that the LTS Team is planning to hold a sprint primarily focused on the Debian security tracker and the associated tooling used by the LTS Team and the Debian Security Team.

Thanks to our sponsors

Sponsors that joined recently are in bold.

Planet DebianFreexian Collaborators: Debian Contributions: Updated Austin, DebConf 25 preparations continue and more! (by Anupa Ann Joseph)

Debian Contributions: 2025-05

Contributing to Debian is part of Freexian’s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

Updated Austin, by Colin Watson and Helmut Grohne

Austin is a frame stack sampling profiler for Python. It allows profiling Python applications without instrumenting them while losing some accuracy in the process, and is the only one of its kind presently packaged for Debian. Unfortunately, it hadn’t been uploaded in a while and hence the last Python version it worked with was 3.8. We updated it to a current version and also dealt with a number of architecture-specific problems (such as unintended sign promotion, 64bit time_t fallout and strictness due to -Wformat-security ) in cooperation with upstream. With luck, it will migrate in time for trixie.

Preparing for DebConf 25, by Stefano Rivera and Santiago Ruano Rincón

DebConf 25 is quickly approaching, and the organization work doesn’t stop. In May, Stefano continued supporting the different teams. Just to give a couple of examples, Stefano made changes in DebConf 25 website to make BoF and sprints submissions public, so interested people can already know if a BoF or sprint for a given subject is planned, allowing coordination with the proposer; or to enhance how statistics are made public to help the work of the local team.

Santiago has participated in different tasks, including the logistics of the conference, like preparing more information about the public transportation that will be available. Santiago has also taken part in activities related to fundraising and reviewing more event proposals.

Miscellaneous contributions

  • Lucas fixed security issues in Valkey in unstable.
  • Lucas tried to help with the update of Redis to version 8 in unstable. The package hadn’t been updated for a while due to licensing issues, but now upstream maintainers fixed them.
  • Lucas uploaded around 20 ruby-* packages to unstable that weren’t updated for some years to make them build reproducible. Thanks to reproducible builds folks to point out those issues. Also some unblock requests (and follow-ups) were needed to make them reach trixie in time for the release.
  • Lucas is organizing a Debian Outreach session for DebConf 25, reaching out to all interns of Google Summer of Code and Outreachy programs from the last year. The session will be presented by in-person interns and also video recordings from the interns interested in participating but did not manage to attend the conference.
  • Lucas continuously works on DebConf Content team tasks. Replying to speakers, sponsors, and communicating internally with the team.
  • Carles improved po-debconf-manager: fixed bugs reported by Catalan translator, added possibility to import packages out of salsa, added using non-default project branches on salsa, polish to get ready for DebCamp.
  • Carles tested new “apt” in trixie and reported bugs to “apt”, “installation-report”, “libqt6widget6”.
  • Carles used po-debconf-manager and imported remaining 80 packages, reviewed 20 translations, submitted (MR or bugs) 54 translations.
  • Carles prepared some topics for translation BoF in DebConf (gathered feedback, first pass on topics).
  • Helmut gave an introductory talk about the mechanics of Linux namespaces at MiniDebConf Hamburg.
  • Helmut sent 25 patches for cross compilation failures.
  • Helmut reviewed, refined and applied a patch from Jochen Sprickerhof to make the Multi-Arch hinter emit more hints for pure Python modules.
  • Helmut sat down with Christoph Berg (not affiliated with Freexian) and extended unschroot to support directory-based chroots with overlayfs. This is a feature that was lost in transitioning from sbuild’s schroot backend to its unshare backend. unschroot implements the schroot API just enough to be usable with sbuild and otherwise works a lot like the unshare backend. As a result, apt.postgresql.org now performs its builds contained in a user namespace.
  • Helmut looked into a fair number of rebootstrap failures most of which related to musl or gcc-15 and imported patches or workarounds to make those builds proceed.
  • Helmut updated dumat to use sqop fixing earlier PGP verification problems thanks to Justus Winter and Neal Walfield explaining a lot of sequoia at MiniDebConf Hamburg.
  • Helmut got the previous zutils update for /usr-move wrong again and had to send another update.
  • Helmut looked into why debvm’s autopkgtests were flaky and with lots of help from Paul Gevers and Michael Tokarev tracked it down to a race condition in qemu. He updated debvm to trigger the problem less often and also fixed a wrong dependency using Luca Boccassi’s patch.
  • Santiago continued the switch to sbuild for Salsa CI (that was stopped for some months), and has been mainly testing linux, since it’s a complex project that heavily customizes the pipeline. Santiago is preparing the changes for linux to submit a MR soon.
  • In openssh, Colin tracked down some intermittent sshd crashes to a root cause, and issued bookworm and bullseye updates for CVE-2025-32728.
  • Colin spent some time fixing up fail2ban, mainly reverting a patch that caused its tests to fail and would have banned legitimate users in some common cases.
  • Colin backported upstream fixes for CVE-2025-48383 (django-select2) and CVE-2025-47287 (python-tornado) to unstable.
  • Stefano supported video streaming and recording for 2 miniDebConfs in May: Maceió and Hamburg. These had overlapping streams for one day, which is a first for us.
  • Stefano packaged the new version of python-virtualenv that includes our patches for not including the wheel for wheel.
  • Stefano got all involved parties to agree (in principle) to meet at DebConf for a mediated discussion on a dispute that was brought to the technical committee.
  • Anupa coordinated the swag purchase for DebConf 25 with Juliana and Nattie.
  • Anupa joined the publicity team meeting for discussing the upcoming events and BoF at DebConf 25.
  • Anupa worked with the publicity team to publish Bits post to welcome GSoc 2025 Interns.

,

Worse Than FailureCodeSOD: The Pirate's Code

We've talked about ASP .Net WebForms in the past. In this style of development, everything was event driven: click a button, and the browser sends an HTTP request to the server which triggers a series of events, including a "Button Click" event, and renders a new page.

When ASP .Net launched, one of the "features" was a lazy repaint in browsers which supported it (aka, Internet Explorer), where you'd click the button, the page would render on the server, download, and then the browser would repaint only the changed areas, making it feel more like a desktop application, albeit a laggy one.

This model didn't translate super naturally to AJAX style calls, where JavaScript updated only portions of the page. The .Net team added some hooks for it- special "AJAX enabled" controls, as well as helper functions, like __doPostBack, in the UI to generate URLs for "postbacks" to trigger server side execution. A postback is just a POST request with .NET specific state data in the body.

All this said, Chris maintains a booking system for a boat rental company. Specifically, he's a developer at a company which the boat rental company hires to maintain their site. The original developer left behind a barnacle covered mess of tangled lines and rotting hull.

Let's start with the view ASPX definition:

<script>
function btnSave_Click()
{
    if (someCondition) 
    {
    //Trimmed for your own sanity
    
    //PostBack to Save Data into the Database.
        javascript:<%#getPostBack()%>;                   
    }
    else
    {
        return false;
    }
}
</script>
<html>
      <body>
          <input type="button" value="  Save  Booking  " id="btnSave" class="button" title="Save [Alt]" onclick="btnSave_Click()" />
      </body>
</html>

__doPostBack is the .NET method for generating URLs for performing postbacks, and specifically, it populates two request fields: __EVENTTARGET (the ID of the UI element triggering the event) and __EVENTARGUMENT, an arbitrary field for your use. I assume getPostBack() is a helper method which calls that. The code in btnSave_Click is as submitted, and I think our submitter may have mangled it a bit in "trimming", but I can see the goal is to ensure than when the onclick event fires, we perform a "postback" operation with some hard-coded values for __EVENTTARGET and __EVENTELEMENT.

Or maybe it isn't mangled, and this code just doesn't work?

I enjoy that the tool-tip "title" field specifies that it's "[Alt]" text, and that the name of the button includes extra whitespace to ensure that it's padded out to a good rendering size, instead of using CSS.

But we can skip past this into the real meat. How this gets handled on the server side:

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
    '// Trimmed more garbage
    If Page.IsPostBack Then
        'Check if save button has been Clicked.
        Dim eventArg As String = Request("__EVENTARGUMENT")
        Dim offset As Integer = eventArg.IndexOf("@@@@@")
        If (offset > -1) Then
            'this is an event that we raised. so do whatever you need to here.
            Save()
        End If
    End If
End Sub

From this, I conclude that getPostBack populates the __EVENTARGUMENT field with a pile of "@", and we use that to recognize that the save button was clicked. Except, and this is the important thing, if they populated the ID property with btnSave, then ASP .Net would automatically call btnSave_Click. The entire point of the __doPostBack functionality is that it hooks into the event handling pattern and acts just like any other postback, but lets you have JavaScript execute as part of sending the request.

The entire application is a boat with multiple holes in it; it's taking on water and going down, and like a good captain, Chris is absolutely not going down with it and looking for a lifeboat.

Chris writes:

The thing in its entirety is probably one of the biggest WTFs I've ever had to work with.
I've held off submitting because nothing was ever straight forward enough to be understood without posting the entire website.

Honestly, I'm still not sure I understand it, but I do hate it.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

365 TomorrowsHelp One Help Oneself

Author: Steve Smith, Staff Writer Lewis got the assistant at a regifting exchange at the company Christmas party. He didn’t turn it on until February when a snowstorm kept him working from home for a week. It had been opened before, the setup was already complete, but it asked for his name, and gleaned network […]

The post Help One Help Oneself appeared first on 365tomorrows.

,

Worse Than FailureCodeSOD: A Real POS Report

Eddie's company hired a Highly Paid Consultant to help them retool their systems for a major upgrade. Of course, the HPC needed more and more time, and the project ran later and later and ended up wildly over budget, so the HPC had to be released, and Eddie inherited the code.

What followed was a massive crunch to try and hit absolutely hard delivery dates. Management didn't want their team "rewriting" the expensive code they'd already paid for, they just wanted "quick fixes" to get it live. Obviously, the HPC's code must be better than theirs, right?

After release, a problem appeared in one of their sales related reports. The point-of-sale report was meant to deliver a report about which items were available at any given retail outlet, in addition to sales figures. Because their business dealt in a high volume of seasonal items, every quarter the list of items was expected to change regularly.

The users weren't seeing the new items appear in the report. This didn't make very much sense- it was a report. The data was in the database. The report was driven by a view, also in the database, which clearly was returning the correct values? So the bug must be in the code which generated the report…

If POSItemDesc = "Large Sign" Then
        grdResults.Columns.FromKey("FColumn12").Header.Caption = "Large Sign"
End If
If POSItemDesc = "Small Sign" Then
        grdResults.Columns.FromKey("FColumn12").Header.Caption = "Small Sign"
End If
If POSItemDesc = "2x2 Hanging Sign" Then
        grdResults.Columns.FromKey("FColumn12").Header.Caption = "2x2 Hanging Sign"
End If
If POSItemDesc = "1x1 Sign" Then
        grdResults.Columns.FromKey("FColumn12").Header.Caption = "1x1 Sign"
End If
'.........Snipping more of these........
If POSItemDesc = "Light Thief" Then
        grdResults.Columns.FromKey("FColumn12").Header.Caption = "Light Thief"
End If
If POSItemDesc = "Door Strike" Then
        grdResults.Columns.FromKey("FColumn12").Header.Caption = "Door Strike"
End If

First, it's worth noting that inside of the results grid display item, the HPC named the field FColumn12, which is such a wonderfully self documenting name, I'm surprised we aren't all using that everywhere. But the more obvious problem is that the list of possible items is hard-coded into the report; items which don't fit one of these if statements don't get displayed.

At no point, did the person writing this see the pattern of "I check if a field equals a string, and then set another field equal to that string," and say, "maybe there's a better way?" At no point, in the testing process, did anyone try this report with a new item?

It was easy enough for Eddie to change the name of the column in the results grid, and replace all this code with a simpler: grdResults.Columns.FromKey("POSItem").Header.Caption = POSItemDesc, which also had the benefit of actually working, but we're all left puzzling over why this happened in the first place. It's not like the HPC was getting paid per line of code. Right? Right?

Of course not- no HPC would willingly be paid based on any metric that has an objective standard, even if the metric is dumb.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

365 TomorrowsPostcards from Corona

Author: Julian Miles, Staff Writer In a dusty corridor away from busy areas of Area 702, two people with ill-fitting lab coats concealing their uniforms are huddled under a disconnected monitoring camera. One takes a hit on a vape stick. The other lights a cigar. “I heard old Kendrix panicked after Prof Devensor collapsed. Nobody […]

The post Postcards from Corona appeared first on 365tomorrows.

,

Rondam RamblingsHating Trump More Won't Make Things Better

It has been nearly five months now since I published my open letter to Democratic candidates and organizations.  Since then I have, unsurprisingly, received dozens of texts and emails asking me to "Donate $5 now!"  For a while I responded to every one pointing them to my Open Letter and asking them to read it.  I was expecting (hoping for?) one of three responses.  1) "You are

Planet DebianThorsten Alteholz: My Debian Activities in May 2025

Debian LTS

This was my hundred-thirty-first month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on:

  • [DLA 4168-1] openafs security update of three CVEs related to theft of credentials, crashes or buffer overflows.
  • [DLA 4196-1] kmail-account-wizard security update to fix one CVE related to a man-in-the-middle attack when using http instead of https to get some configuration.
  • [DLA 4198-1] espeak-ng security update to fix five CVEs related to buffer overflow or underflow in several functions and a floating point exception. Thanks to Samuel Thibault for having a look at my debdiff.
  • [#1106867] created Bookworm pu-bug for kmail-account-wizard. Thanks to Patrick Franz for having a look at my debdiff.

I also continued my to work on libxmltok and suricata. This month I also had to do some support on seger, for example to inject packages newly needed for builds.

Debian ELTS

This month was the eighty-second ELTS month. During my allocated time I uploaded or worked on:

  • [ELA-1444-1] kmail-account-wizard security update to fix two CVEs in Buster related to a man-in-the-middle attack when using http instead of https to get some configuration. The other issue is about a misleading UI, in which the state of encryption is shown wrong.
  • [ELA-1445-1] espeak-ng security update to fix five CVEs in Stretch and Buster. The issues are related to buffer overflow or underflow in several functions and a floating point exception.

All packages I worked on have been on the list of longstanding packages. For example espeak-ng has been on this list for more than nine month. I now understood that there is a reason why packages are on this list. Some parts of the software have been almost completely reworked, so that the patches need a “reverse” rework. For some packages this is easy, but for others this rework needs quite some time. I also continued to work on libxmltok and suricata.

Debian Printing

Unfortunately I didn’t found any time to work on this topic.

Debian Astro

This month I uploaded bugfix versions of:

Debian Mobcom

This month I uploaded bugfix versions of:

misc

This month I uploaded bugfix versions of:

Thanks a lot to the Release Team who quickly handled all my unblock bugs!

FTP master

It is this time of the year when just a few packages arrive in NEW: it is Hard Freeze. So I enjoy this period and basically just take care of kernels or other important packages. As people seem to be more interested in discussions than in fixing RC bugs, my period of rest seems to continue for a while. So thanks for all this valuable discussions and really thanks to the few people who still take care of Trixie. This month I accepted 146 and rejected 10 packages. The overall number of packages that got accepted was 147.

Planet DebianDebian Outreach Team: GSoC 2024 Final Report: Project DebMobCom

GSoC 2024: Project DebMobCom

Introduction

Hi, my name is Nathan Doris and this is my report on contributing to the Debian Mobile Communication project for GSoC 2024 under the guidance of my mentor, Thorsten Alteholz.

Project MobCom

Debian’s Mobile Communication team takes care of updating and creating packages for the open-source mobile communication software suite known as Osmocom. The ongoing development of this software stack requires constant updating and creation of packages to support this progress.

The goal of GSoC 2024 DebMobCom is to update, recreate and add these Osmocom packages.

My Work

Over the few months that I contributed to Debian I successfully updated/recreated more than a dozen packages for DebMobCom. A list of my accomplishments can be found here as well as on the Debian QA page.

What I Learned

Before I began this journey I had very little experience with Debian packaging. By the end of GSoC not only had I learned about building packages but I also gained more knowledge regarding Linux, Debian, shell scripting, automake, C, GNU and gitlab/Salsa. As I reflect back on the past few months I am amazed at the amount and depth of information I have learned. Listed below are examples of items that I found especially important.

  • Debian Packaging Workflow
    • Every package to be built relies upon the information inside the /debian folder. This is where I learned the importance of every templated file such as the control, changelog, rules, watch, *.symbols, *.install and many others. As I worked on these files I witnessed the relationship between them and the finished .deb package. From this workflow I can appreciate the amount of imagination and work that was put into this process.
  • Debian Tooling
    • I learned many tools while building packages but sbuild accompanied by lintian is what I spent the most time with. Sbuild is an autobuilder that produces the final .deb packages. It also integrates with another tool, lintian, which checks each package and informs you of any errors such as policy violations, common mistakes and best practices. Thorsten’s words, ‘keep lintian happy’ still resonate with me after each package build.
    • Another tool I frequently require use of is quilt. Quilt is a wonderful tool that manages patches for packages. I became quite well versed in updating patches, removing and also creating my own.
  • Copyright
    • I hadn’t realised how important licensing was until I started contributing to Debian. I predominantly used licensecheck to extract copyright information from source code. This was a tedious task but obviously extremely important in keeping Debian packages free.

Challenges

What I found particularly challenging (and enjoyable) was the amount of troubleshooting required to complete tasks. Although I spent countless hours reading documentation such as the New Maintainers Guide, Policy Manual, Wiki and manual pages it was still extremely difficult. Complex packages are not quite covered in the docs so out-of-the-box thinking is required.

Conclusion

First of all I’d like to say a big thank you to my mentor Thorsten. His support was critical for my success.

Contributing to Debian and the overall GSoC experience was such a positive event in my life. I enjoyed all aspects of the experience. The skills I have acquired will propel me forward into further open-source activities and communities. As I continue to contribute to this project my goal of becoming a Debian Maintainer is becoming a reality.

365 TomorrowsMemo

Author: Simon Read To: All staff RE: Causality Protocol De-prioritisation Null/null/null, 00:00 This communication serves as formal notice. Treisman Industries no longer operates under linear temporal constraints. All protocols reliant upon fixed sequencing have been deprecated. Causality is to be regarded as a legacy framework, maintained only where local perception demands continuity. Departments previously dependent […]

The post Memo appeared first on 365tomorrows.

Planet DebianColin Watson: Free software activity in May 2025

My Debian contributions this month were all sponsored by Freexian. Things were a bit quieter than usual, as for the most part I was sticking to things that seemed urgent for the upcoming trixie release.

You can also support my work directly via Liberapay or GitHub Sponsors.

OpenSSH

After my appeal for help last month to debug intermittent sshd crashes, Michel Casabona helped me put together an environment where I could reproduce it, which allowed me to track it down to a root cause and fix it. (I also found a misuse of strlcpy affecting at least glibc-based systems in passing, though I think that was unrelated.)

I worked with Daniel Kahn Gillmor to fix a regression in ssh-agent socket handling.

I fixed a reproducibility bug depending on whether passwd is installed on the build system, which would have affected security updates during the lifetime of trixie.

I backported openssh 1:10.0p1-5 to bookworm-backports.

I issued bookworm and bullseye updates for CVE-2025-32728.

groff

I backported a fix for incorrect output when formatting multiple documents as PDF/PostScript at once.

debmirror

I added a simple autopkgtest.

Python team

I upgraded these packages to new upstream versions:

  • automat
  • celery
  • flufl.i18n
  • flufl.lock
  • frozenlist
  • python-charset-normalizer
  • python-evalidate (including pointing out an upstream release handling issue)
  • python-pythonjsonlogger
  • python-setproctitle
  • python-telethon
  • python-typing-inspection
  • python-webargs
  • pyzmq
  • trove-classifiers (including a small upstream cleanup)
  • uncertainties
  • zope.testrunner

In bookworm-backports, I updated these packages:

  • python-django to 3:4.2.21-1 (issuing BSA-124)
  • python-django-pgtrigger to 4.14.0-1

I fixed problems building these packages reproducibly:

I backported fixes for some security vulnerabilities to unstable (since we’re in freeze now so it’s not always appropriate to upgrade to new upstream versions):

I fixed various other build/test failures:

I added non-superficial autopkgtests to these packages:

I packaged python-django-hashids and python-django-pgbulk, needed for new upstream versions of python-django-pgtrigger.

I ported storm to Python 3.14.

Science team

I fixed a build failure in apertium-oci-fra.

,

Planet DebianEvgeni Golov: show your desk - 2025 edition

Back in 2020 I posted about my desk setup at home.

Recently someone in our #remotees channel at work asked about WFH setups and given quite a few things changed in mine, I thought it's time to post an update.

But first, a picture! standing desk with a monitor, laptop etc (Yes, it's cleaner than usual, how could you tell?!)

desk

It's still the same Flexispot E5B, no change here. After 7 years (I bought mine in 2018) it still works fine. If I'd have to buy a new one, I'd probably get a four-legged one for more stability (they got quite affordable now), but there is no immediate need for that.

chair

It's still the IKEA Volmar. Again, no complaints here.

hardware

Now here we finally have some updates!

laptop

A Lenovo ThinkPad X1 Carbon Gen 12, Intel Core Ultra 7 165U, 32GB RAM, running Fedora (42 at the moment).

It's connected to a Lenovo ThinkPad Thunderbolt 4 Dock. It just worksâ„¢.

workstation

It's still the P410, but mostly unused these days.

monitor

An AOC U2790PQU 27" 4K. I'm running it at 150% scaling, which works quite decently these days (no comparison to when I got it).

speakers

As the new monitor didn't want to take the old Dell soundbar, I have upgraded to a pair of Alesis M1Active 330 USB.

They sound good and were not too expensive.

I had to fix the volume control after some time though.

webcam

It's still the Logitech C920 Pro.

microphone

The built in mic of the C920 is really fine, but to do conference-grade talks (and some podcasts 😅), I decided to get something better.

I got a FIFINE K669B, with a nice arm.

It's not a Shure, for sure, but does the job well and Christian was quite satisfied with the results when we recorded the Debian and Foreman specials of Focus on Linux.

keyboard

It's still the ThinkPad Compact USB Keyboard with TrackPoint.

I had to print a few fixes and replacement parts for it, but otherwise it's doing great.

Seems Lenovo stopped making those, so I really shouldn't break it any further.

mouse

Logitech MX Master 3S. The surface of the old MX Master 2 got very sticky at some point and it had to be replaced.

other

notepad

I'm still terrible at remembering things, so I still write them down in an A5 notepad.

whiteboard

I've also added a (small) whiteboard on the wall right of the desk, mostly used for long term todo lists.

coaster

Turns out Xeon-based coasters are super stable, so it lives on!

yubikey

Yepp, still a thing. Still USB-A because... reasons.

headphones

Still the Bose QC25, by now on the third set of ear cushions, but otherwise working great and the odd 15€ cushion replacement does not justify buying anything newer (which would have the same problem after some time, I guess).

I did add a cheap (~10€) Bluetooth-to-Headphonejack dongle, so I can use them with my phone too (shakes fist at modern phones).

And I do use the headphones more in meetings, as the Alesis speakers fill the room more with sound and thus sometimes produce a bit of an echo.

charger

The Bose need AAA batteries, and so do some other gadgets in the house, so there is a technoline BC 700 charger for AA and AAA on my desk these days.

light

Yepp, I've added an IKEA Tertial and an ALDI "face" light. No, I don't use them much.

KVM switch

I've "built" a KVM switch out of an USB switch, but given I don't use the workstation that often these days, the switch is also mostly unused.

365 TomorrowsSymphony for Mycelium

Author: Colin Jeffrey As the sentient slime mould squelched slowly across the asteroid it lived on, it found its mind – such as it was – occupied by a single thought: Ludwig van Beethoven. This was strange for several reasons, most obvious being that slime moulds are not renowned for their thoughts on music. Or […]

The post Symphony for Mycelium appeared first on 365tomorrows.

David BrinScience (mostly bio, this time) Forges Ahead. Even empowering... citizenship!


I'll avoid political hollering, this weekend. Especially as we're all staring with bemusement, terror -- and ideally popcorn -- at the bizarre displays of toddler-hysteria foaming from D.C. In fact, some wise heads propose that we respond by rebuilding our institutions - and confidence - from the ground up. And hence:


 #1   I gave a series of lectures about National Resilience for the Naval Postgraduate School, that resulted in this interview about neglected and needed boosts to RESILIENCE.   


#2   And in a related development: philanthropist Craig Newmark supports a program seeking to get modern citizens more involved in ‘resilience.’ A great endeavor, needed now more than ever.  


#3   Also resilience related! As a member of CERT - the nationwide Community Emergency Response Team I urge folks to consider taking the training.  As a bottom-level 'responder' at least you'll know some things to do, if needed. *


*The FEMA site has been experiencing... 'problems'... but I hope this link works. Fortunately the training is mostly done by local fire departments, but your badge and gear may come slower than normal.


#4   Giving blood regularly may not just be saving the lives of other people, it could also be improving your own blood's health at a genetic level, according to a new study.  An international team of researchers compared samples from 217 men who had given blood more than 100 times in their lives, to samples from 212 men who had donated less than 10 times, to look for any variance in blood health. "Activities that put low levels of stress on blood cell production allow our blood stem cells to renew and we think this favors mutations that further promote stem cell growth rather than disease." (Well, I just gave my 104th pint, so…)  


#5 Nothing prepares you for the future better than Science Fiction! I started an online org TASAT as a way for geeky SF readers to maybe someday save the world!


...And now let's get to science!  After a couple of announcements...



== Yeah, you may have heard this already, but... ==


Okay it's just a puff piece...that I can't resist sharing with folks, about an honor from my alma mater, Caltech. It's seldom that I get Imposter's Syndrome. But in this case, well, innumerable classmates there were way smarter than me! 


Also a couple of job announcements: First, Prof. Ted Parson and other friends at UCLA Law School are looking for a project director at UCLA’s new Emmett Institute on Climate Change and the Environment, with a focus on legal and social aspects of ‘geo-engineering’… the wide range of proposals (from absurd to plausibly helpful) to perhaps partially ease or palliate the effects of human-generated greenhouse pollution on the planet’s essential and life-giving balance. 

 

To see some such proposals illustrated in fiction, look at Kim Stanley Robinson’s The Ministry For the Future  (spreading cooling atmospheric aerosols) or in my own novel Earth (ocean fertilization.)


And now… science forges ahead!



== Life… as we now know it… ==


Complexity can get… complicated and nowhere more so than in braaaains! For some years, the most intricate nervous systems ever modeled by science were varieties of worms or nematodes (e.g. C.elegans). But advances accelerate, and now a complete brain model – not just of neurons but their detectable connections (synapses) has been completed for the vastly larger brain of the Drosophila fruit fly!  (Including discovery of several new types of neurons.) 


And sure, I maintain that neurons and synapses aren’t enough. We’re gonna need to understand the murky, non-linear contributions of intra-cellular ‘computational’ elements. Still… amazing stuff. And the process will get a lot faster.


Meanwhile… Allorecognition in nature is an individual creature’s distinction between self and other. Most generally in immune response to invasion of the self-boundary by that which is non-self. Almost all Earthly life forms exhibit this trait, with stong tenacity. An exception, described in the early 20202, is Mnemiopsis or the “sea walnut,” a kind of comb jelly (‘jellyfish’) that can be divided arbitrarily and combine with other partial mnemiopses, merging into a new whole.

(And elsewhere I dive into how this allorecognition - or distinguishment of self  - is utterly vital to incorportate into artificial intelligence! Because only in that way can we apply incentives for AI to incorporate notions of reciprocity that underlay both Nature and Civilization!)


How do tardigrades survive heat, cold, desiccation and even vacuum?  


LUCA, a common ancestor to all organisms and not the first life form, has been a controversial topic. Fossil evidence goes back as far as 3.4 billion years, yet this study proposes that LUCA might be close to being the same age as the Earth. The genetic code and DNA replication, which are two of the vital biological processes, might have developed almost immediately after the planet was formed.”   



 == Weird Earth life! ==


Sea Robins have the body of a fish, the wings of a bird, and multiple legs like a crab, in what appears to be another case of “carcinization” – life constantly re-inventing the crab body plan. Like the Qheuens in Brightness Reef. And yeah, it seems likely that the most common form of upper complex life we’ll find out there will look like crabs.


Marine biologists in Denmark discovered a solo male dolphin in the Baltic who appears to be talking to himself. They analyzed thousands of sounds made by the dolphin and what they learned.


In 1987, a group of killer whales off the northwestern coast of North America briefly donned salmon “hats,” carrying dead fish on their heads for weeks. Recently, a male orca known as J27, or “Blackberry,” was photographed in Washington’s Puget Sound wearing a salmon on his head. 


(I’m tempted to cite Vladimir Sorokin’s chilling/terrific short scifi novel – in a league with Orwell – Day of The Oprichnik – in which the revived czarist Oprachina regime-enforcers go about town each day with a dog’s head on the roofs of their cars, and all traffic veers aside for them, as in olden times. (“That is your association, this time, Brin?” Hey, it’s the times. And a truly great - and terrifying - novel.)


Beyond life and death... Researchers found that skin cells extracted from deceased frog embryos were able to adapt to the new conditions of a petri dish in a lab, spontaneously reorganizing into multicellular organisms called xenobots. These organisms exhibited behaviors that extend far beyond their original biological roles. Specifically, these xenobots use their cilia – small, hair-like structures – to navigate and move through their surroundings, whereas in a living frog embryo, cilia are typically used to move mucus.  


Two injured jellyfish can merge to make one healthy one?  Sounds like a Sheckley story where this is the actual point of sex, trying to make a mighty beast with two backs.   



== Even farther back! ==


3.2 billion years ago, life was just perking along on Earth and starved of nutrients… which were apparently provided in massive generosity by an asteroidal impact vastly bigger than the much-later dinosaur bane.  


Analysis of 700 genomes of bacteria, archaea, and fungi -- excluding eukaryotes such as plants and animals that evolved later -- have found 57 gene families… though I think using modern genetic drift rates to converge those families backward may be a bit iffy. Still, if life started that early… and survived the Thea impact… then it implies that life starts very easily, and may be vastly pervasive in the universe.


And possibly even bigger news. Genes themselves may compete with each other like individual entities, in somewhat predictable ways: “…interactions between genes make aspects of evolution somewhat predictable and furthermore, we now have a tool that allows us to make those predictions…”. 



== And maybe beef should be a... condiment? ==


“Today, almost half the world’s habitable land is used for agriculture. Of that, an astounding 80% is dedicated to livestock grazing and animal feed. This means 40% of the planet’s total habitable land is dedicated to animal products, despite the fact that meat, dairy and farmed fish combined provide just 17% of humanity’s calories. “Only a fraction of agricultural land (16%) is used to grow the crops that we eat directly, with an additional 4% for things like biofuels, textiles and tobacco. Just 38% of habitable land is forested, a slice of the pie that continues to shrink, primarily in diverse tropical regions where the greatest number of species live.”  


Meanwhile.... This article talks about new ways to make food “from thin air.” Or, more accurately, ‘precision fermentation’ from hydrogen and human and agricultural waste. 



== And finally...

An interesting interview with genetic paleontologist David Reich. 60,000 years ago the explosion of modern homo sapiens from Africa seemed to happen almost overnight. 


As Reich points out, we had two new things. 1. Dogs and 2. an ability to reprogram ourselves culturally. 


There followed - at an accelerating pace - a series of revolutions in our tool sets, cultural patterns and adaptability. Of course, I talked about this extensively in both Earth and Existence


,

Planet DebianReproducible Builds: Reproducible Builds in May 2025

Welcome to our 5th report from the Reproducible Builds project in 2025! Our monthly reports outline what we’ve been up to over the past month, and highlight items of news from elsewhere in the increasingly-important area of software supply-chain security. If you are interested in contributing to the Reproducible Builds project, please do visit the Contribute page on our website.

In this report:

  1. Security audit of Reproducible Builds tools published
  2. When good pseudorandom numbers go bad
  3. Academic articles
  4. Distribution work
  5. diffoscope and disorderfs
  6. Website updates
  7. Reproducibility testing framework
  8. Upstream patches

Security audit of Reproducible Builds tools published

The Open Technology Fund’s (OTF) security partner Security Research Labs recently an conducted audit of some specific parts of tools developed by Reproducible Builds. This form of security audit, sometimes called a “whitebox� audit, is a form testing in which auditors have complete knowledge of the item being tested. They auditors assessed the various codebases for resilience against hacking, with key areas including differential report formats in diffoscope, common client web attacks, command injection, privilege management, hidden modifications in the build process and attack vectors that might enable denials of service.

The audit focused on three core Reproducible Builds tools: diffoscope, a Python application that unpacks archives of files and directories and transforms their binary formats into human-readable form in order to compare them; strip-nondeterminism, a Perl program that improves reproducibility by stripping out non-deterministic information such as timestamps or other elements introduced during packaging; and reprotest, a Python application that builds source code multiple times in various environments in order to to test reproducibility.

OTF’s announcement contains more of an overview of the audit, and the full 24-page report is available in PDF form as well.


�When good pseudorandom numbers go bad�

Danielle Navarro published an interesting and amusing article on their blog on When good pseudorandom numbers go bad. Danielle sets the stage as follows:

[Colleagues] approached me to talk about a reproducibility issue they’d been having with some R code. They’d been running simulations that rely on generating samples from a multivariate normal distribution, and despite doing the prudent thing and using set.seed() to control the state of the random number generator (RNG), the results were not computationally reproducible. The same code, executed on different machines, would produce different random numbers. The numbers weren’t “just a little bit different� in the way that we’ve all wearily learned to expect when you try to force computers to do mathematics. They were painfully, brutally, catastrophically, irreproducible different. Somewhere, somehow, something broke.

Thanks to David Wheeler for posting about this article on our mailing list


Academic articles

There were two scholarly articles published this month that related to reproducibility:

Daniel Hugenroth and Alastair R. Beresford of the University of Cambridge in the United Kingdom and Mario Lins and René Mayrhofer of Johannes Kepler University in Linz, Austria published an article titled Attestable builds: compiling verifiable binaries on untrusted systems using trusted execution environments. In their paper, they:

present attestable builds, a new paradigm to provide strong source-to-binary correspondence in software artifacts. We tackle the challenge of opaque build pipelines that disconnect the trust between source code, which can be understood and audited, and the final binary artifact, which is difficult to inspect. Our system uses modern trusted execution environments (TEEs) and sandboxed build containers to provide strong guarantees that a given artifact was correctly built from a specific source code snapshot. As such it complements existing approaches like reproducible builds which typically require time-intensive modifications to existing build configurations and dependencies, and require independent parties to continuously build and verify artifacts.

The authors compare “attestable builds� with reproducible builds by noting an attestable build requires “only minimal changes to an existing project, and offers nearly instantaneous verification of the correspondence between a given binary and the source code and build pipeline used to construct it�, and proceed by determining that t�he overhead (42 seconds start-up latency and 14% increase in build duration) is small in comparison to the overall build time.�


Timo Pohl, Pavel Novák, Marc Ohm and Michael Meier have published a paper called Towards Reproducibility for Software Packages in Scripting Language Ecosystems. The authors note that past research into Reproducible Builds has focused primarily on compiled languages and their ecosystems, with a further emphasis on Linux distribution packages:

However, the popular scripting language ecosystems potentially face unique issues given the systematic difference in distributed artifacts. This Systemization of Knowledge (SoK) [paper] provides an overview of existing research, aiming to highlight future directions, as well as chances to transfer existing knowledge from compiled language ecosystems. To that end, we work out key aspects in current research, systematize identified challenges for software reproducibility, and map them between the ecosystems.

Ultimately, the three authors find that the literature is “sparse�, focusing on few individual problems and ecosystems, and therefore identify space for more critical research.


Distribution work

In Debian this month:


Hans-Christoph Steiner of the F-Droid catalogue of open source applications for the Android platform published a blog post on Making reproducible builds visible. Noting that “Reproducible builds are essential in order to have trustworthy software�, Hans also mentions that “F-Droid has been delivering reproducible builds since 2015�. However:

There is now a “Reproducibility Status� link for each app on f-droid.org, listed on every app’s page. Our verification server shows ✔�� or 💔 based on its build results, where ✔�� means our rebuilder reproduced the same APK file and 💔 means it did not. The IzzyOnDroid repository has developed a more elaborate system of badges which displays a ✅ for each rebuilder. Additionally, there is a sketch of a five-level graph to represent some aspects about which processes were run.

Hans compares the approach with projects such as Arch Linux and Debian that “provide developer-facing tools to give feedback about reproducible builds, but do not display information about reproducible builds in the user-facing interfaces like the package management GUIs.�


Arnout Engelen of the NixOS project has been working on reproducing the minimal installation ISO image. This month, Arnout has successfully reproduced the build of the minimal image for the 25.05 release without relying on the binary cache. Work on also reproducing the graphical installer image is ongoing.


In openSUSE news, Bernhard M. Wiedemann posted another monthly update for their work there.


Lastly in Fedora news, Jelle van der Waa opened issues tracking reproducible issues in Haskell documentation, Qt6 recording the host kernel and R packages recording the current date. The R packages can be made reproducible with packaging changes in Fedora.


diffoscope & disorderfs

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading versions 295, 296 and 297 to Debian:

  • Don’t rely on zipdetails’ --walk argument being available, and only add that argument on newer versions after we test for that. […]
  • Review and merge support for NuGet packages from Omair Majid. […]
  • Update copyright years. […]
  • Merge support for an lzma comparator from Will Hollywood. […][…]

Chris also merged an impressive changeset from Siva Mahadevan to make disorderfs more portable, especially on FreeBSD. disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues […]. This was then uploaded to Debian as version 0.6.0-1.

Lastly, Vagrant Cascadian updated diffoscope in GNU Guix to version 296 […][…] and 297 […][…], and disorderfs to version 0.6.0 […][…].


Website updates

Once again, there were a number of improvements made to our website this month including:


Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility.

However, Holger Levsen posted to our mailing list this month in order to bring a wider awareness to funding issues faced by the Oregon State University (OSU) Open Source Lab (OSL). As mentioned on OSL’s public post, “recent changes in university funding makes our current funding model no longer sustainable [and that] unless we secure $250,000 in committed funds, the OSL will shut down later this year�. As Holger notes in his post to our mailing list, the Reproducible Builds project relies on hardware nodes hosted there. Nevertheless, Lance Albertson of OSL posted an update to the funding situation later in the month with broadly positive news.


Separate to this, there were various changes to the Jenkins setup this month, which is used as the backend driver of for both tests.reproducible-builds.org and reproduce.debian.net, including:

  • Migrating the central jenkins.debian.net server AMD Opteron to Intel Haswell CPUs. Thanks to IONOS for hosting this server since 2012.
  • After testing it for almost ten years, the i386 architecture has been dropped from tests.reproducible-builds.org. This is because that, with the upcoming release of Debian trixie, i386 is no longer supported as a ‘regular’ architecture — there will be no official kernel and no Debian installer for i386 systems. As a result, a large number of nodes hosted by Infomaniak have been retooled from i386 to amd64.
  • Another node, ionos17-amd64.debian.net, which is used for verifying packages for all.reproduce.debian.net (hosted by IONOS) has had its memory increased from 40 to 64GB, and the number of cores doubled to 32 as well. In addition, two nodes generously hosted by OSUOSL have had their memory doubled to 16GB.
  • Lastly, we have been granted access to more riscv64 architecture boards, so now we have seven such nodes, all with 16GB memory and 4 cores that are verifying packages for riscv64.reproduce.debian.net. Many thanks to PLCT Lab, ISCAS for providing those.


Outside of this, a number of smaller changes were also made by Holger Levsen:

In addition, Jochen Sprickerhof made a series of changes related to reproduce.debian.net:

  • Add out of memory detection to the statistics page. […]
  • Reverse the sorting order on the statistics page. […][…][…][…]
  • Improve the spacing between statistics groups. […]
  • Update a (hard-coded) line number in error message detection pertaining to a debrebuild line number. […]
  • Support Debian unstable in the rebuilder-debian.sh script. […]…]
  • Rely on rebuildctl to sync only ‘arch-specific’ packages. […][…]


Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. This month, we wrote a large number of such patches, including:



Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

Cryptogram Report on the Malicious Uses of AI

OpenAI just published its annual report on malicious uses of AI.

By using AI as a force multiplier for our expert investigative teams, in the three months since our last report we’ve been able to detect, disrupt and expose abusive activity including social engineering, cyber espionage, deceptive employment schemes, covert influence operations and scams.

These operations originated in many parts of the world, acted in many different ways, and focused on many different targets. A significant number appeared to originate in China: Four of the 10 cases in this report, spanning social engineering, covert influence operations and cyber threats, likely had a Chinese origin. But we’ve disrupted abuses from many other countries too: this report includes case studies of a likely task scam from Cambodia, comment spamming apparently from the Philippines, covert influence attempts potentially linked with Russia and Iran, and deceptive employment schemes.

Reports like these give a brief window into the ways AI is being used by malicious actors around the world. I say “brief” because last year the models weren’t good enough for these sorts of things, and next year the threat actors will run their AI models locally—and we won’t have this kind of visibility.

Wall Street Journal article (also here). Slashdot thread.

Worse Than FailureError'd: There's No Place Like

... London!
This week, we're showcasing some multiple submissions from two regular participants who fell into the theme. Everybody else is just going to have to wait for their turn next week.

Frist up it's Daniel D. "I wanted to see events for the dates I would be in London. Is Skiddle (the website in question) telling me I should come to London more often?" They're certainly being very generous with their interpretation of dates.

2

 

But wait, there's more! Daniel follows with a variation: "Skiddle here again - let's choose June 7th to June 14th, but Skiddle knows better and sets the dates to June 6th to June 13th."

1

 

"I was not aware the Berlin to London route passes through Hawaii (which is Mokulele'shome turf)" chuckles our old friend Michael R. He seems to believe it's an Error'd but I think the real WTF is simply the Byzantine tapestry of partnerships, resellers, rebranding, whitelabeling and masquerades in the air transport biz.

4

 

"Maybe it's just a Monday morning thing," he reports from the airport.

3

 

But Monday had everybody troubled, and Michael was already thinking of Friday. "I am so sure I took the Circle Line just last Friday. And the other lines have the option Monday-Friday/Saturday/Sunday." I hope there isn't a subtext here.

0

 

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

365 TomorrowsTo Be Loved

Author: Naomi Klouda Snow fell on Alaska, and we celebrated. We swirled in a circle, tasting flakes of sky. “Kelp brew for everyone, even the children!” Jenna Ben shouted. How we celebrated! Three circles switched hands and partners aboard our oil platform’s broken asphalt. Sky poured in billowy pieces, turning the tarmac white – the […]

The post To Be Loved appeared first on 365tomorrows.

Planet DebianDirk Eddelbuettel: #49: The Two Cultures of Deploying Statistical Software

Welcome to post 49 in the R4 series.

The Two Cultures is a term first used by C.P. Snow in a 1959 speech and monograph focused on the split between humanities and the sciences. Decades later, the term was (quite famously) re-used by Leo Breiman in a (somewhat prophetic) 2001 article about the split between ‘data models’ and ‘algorithmic models’. In this note, we argue that statistical computing practice and deployment can also be described via this Two Cultures moniker.

Referring to the term linking these foundational pieces is of course headline bait. Yet when preparing for the discussion of r2u in the invited talk in Mons (video, slides), it occurred to me that there is in fact a wide gulf between two alternative approaches of using R and, specifically, deploying packages.

On the one hand we have the approach described by my friend Jeff as “you go to the Apple store, buy the nicest machine you can afford, install what you need and then never ever touch it�. A computer / workstation / laptop is seen as an immutable object where every attempt at change may lead to breakage, instability, and general chaos—and is hence best avoided. If you know Jeff, you know he exaggerates. Maybe only slightly though.

Similarly, an entire sub-culture of users striving for “reproducibility� (and sometimes also “replicability�) does the same. This is for example evidenced by the popularity of package renv by Rcpp collaborator and pal Kevin. The expressed hope is that by nailing down a (sub)set of packages, outcomes are constrained to be unchanged. Hope springs eternal, clearly. (Personally, if need be, I do the same with Docker containers and their respective Dockerfile.)

On the other hand, ‘rolling’ is fundamentally different approach. One (well known) example is Google building “everything at @HEAD�. The entire (ginormous) code base is considered as a mono-repo which at any point in time is expected to be buildable as is. All changes made are pre-tested to be free of side effects to other parts. This sounds hard, and likely is more involved than an alternative of a ‘whatever works’ approach of independent changes and just hoping for the best.

Another example is a rolling (Linux) distribution as for example Debian. Changes are first committed to a ‘staging’ place (Debian calls this the ‘unstable’ distribution) and, if no side effects are seen, propagated after a fixed number of days to the rolling distribution (called ‘testing’). With this mechanism, ‘testing’ should always be installable too. And based on the rolling distribution, at certain times (for Debian roughly every two years) a release is made from ‘testing’ into ‘stable’ (following more elaborate testing). The released ‘stable’ version is then immutable (apart from fixes for seriously grave bugs and of course security updates). So this provides the connection between frequent and rolling updates, and produces immutable fixed set: a release.

This Debian approach has been influential for any other projects—including CRAN as can be seen in aspects of its system providing a rolling set of curated packages. Instead of a staging area for all packages, extensive tests are made for candidate packages before adding an update. This aims to ensure quality and consistence—and has worked remarkably well. We argue that it has clearly contributed to the success and renown of CRAN.

Now, when accessing CRAN from R, we fundamentally have two accessor functions. But seemingly only one is widely known and used. In what we may call ‘the Jeff model’, everybody is happy to deploy install.packages() for initial installations.

That sentiment is clearly expressed by this bsky post:

One of my #rstats coding rituals is that every time I load a @vincentab.bsky.social package I go check for a new version because invariably it’s been updated with 18 new major features 😆

And that is why we have two cultures.

Because some of us, yours truly included, also use update.packages() at recurring (frequent !!) intervals: daily or near-daily for me. The goodness and, dare I say, gift of packages is not limited to those by my pal Vincent. CRAN updates all the time, and updates are (generally) full of (usually excellent) changes, fixes, or new features. So update frequently! Doing (many but small) updates (frequently) is less invasive than (large, infrequent) ‘waterfall’-style changes!

But the fear of change, or disruption, is clearly pervasive. One can only speculate why. Is the experience of updating so painful on other operating systems? Is it maybe a lack of exposure / tutorials on best practices?

These ‘Two Cultures’ coexist. When I delivered the talk in Mons, I briefly asked for a show of hands among all the R users in the audience to see who in fact does use update.packages() regularly. And maybe a handful of hands went up: surprisingly few!

Now back to the context of installing packages: Clearly ‘only installing’ has its uses. For continuous integration checks we generally install into ephemeral temporary setups. Some debugging work may be with one-off container or virtual machine setups. But all other uses may well be under ‘maintained’ setups. So consider calling update.packages() once in while. Or even weekly or daily. The rolling feature of CRAN is a real benefit, and it is there for the taking and enrichment of your statistical computing experience.

So to sum up, the real power is to use

  • install.packages() to obtain fabulous new statistical computing resources, ideally in an instant; and
  • update.packages() to keep these fabulous resources current and free of (known) bugs.

For both tasks, relying on binary installations accelerates and eases the process. And where available, using binary installation with system-dependency support as r2u does makes it easier still, following the r2u slogan of ‘Fast. Easy. Reliable. Pick All Three.’ Give it a try!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

,

Krebs on SecurityProxy Services Feast on Ukraine’s IP Address Exodus

Image: Mark Rademaker, via Shutterstock.

Ukraine has seen nearly one-fifth of its Internet space come under Russian control or sold to Internet address brokers since February 2022, a new study finds. The analysis indicates large chunks of Ukrainian Internet address space are now in the hands of shadowy proxy and anonymity services that are nested at some of America’s largest Internet service providers (ISPs).

The findings come in a report examining how the Russian invasion has affected Ukraine’s domestic supply of Internet Protocol Version 4 (IPv4) addresses. Researchers at Kentik, a company that measures the performance of Internet networks, found that while a majority of ISPs in Ukraine haven’t changed their infrastructure much since the war began in 2022, others have resorted to selling swathes of their valuable IPv4 address space just to keep the lights on.

For example, Ukraine’s incumbent ISP Ukrtelecom is now routing just 29 percent of the IPv4 address ranges that the company controlled at the start of the war, Kentik found. Although much of that former IP space remains dormant, Ukrtelecom told Kentik’s Doug Madory they were forced to sell many of their address blocks “to secure financial stability and continue delivering essential services.”

“Leasing out a portion of our IPv4 resources allowed us to mitigate some of the extraordinary challenges we have been facing since the full-scale invasion began,” Ukrtelecom told Madory.

Madory found much of the IPv4 space previously allocated to Ukrtelecom is now scattered to more than 100 providers globally, particularly at three large American ISPs — Amazon (AS16509), AT&T (AS7018), and Cogent (AS174).

Another Ukrainian Internet provider — LVS (AS43310) — in 2022 was routing approximately 6,000 IPv4 addresses across the nation. Kentik learned that by November 2022, much of that address space had been parceled out to over a dozen different locations, with the bulk of it being announced at AT&T.

IP addresses routed over time by Ukrainian provider LVS (AS43310) shows a large chunk of it being routed by AT&T (AS7018). Image: Kentik.

Ditto for the Ukrainian ISP TVCOM, which currently routes nearly 15,000 fewer IPv4 addresses than it did at the start of the war. Madory said most of those addresses have been scattered to 37 other networks outside of Eastern Europe, including Amazon, AT&T, and Microsoft.

The Ukrainian ISP Trinity (AS43554) went offline in early March 2022 during the bloody siege of Mariupol, but its address space eventually began showing up in more than 50 different networks worldwide. Madory found more than 1,000 of Trinity’s IPv4 addresses suddenly appeared on AT&T’s network.

Why are all these former Ukrainian IP addresses being routed by U.S.-based networks like AT&T? According to spur.us, a company that tracks VPN and proxy services, nearly all of the address ranges identified by Kentik now map to commercial proxy services that allow customers to anonymously route their Internet traffic through someone else’s computer.

From a website’s perspective, the traffic from a proxy network user appears to originate from the rented IP address, not from the proxy service customer. These services can be used for several business purposes, such as price comparisons, sales intelligence, web crawlers and content-scraping bots. However, proxy services also are massively abused for hiding cybercrime activity because they can make it difficult to trace malicious traffic to its original source.

IPv4 address ranges are always in high demand, which means they are also quite valuable. There are now multiple companies that will pay ISPs to lease out their unwanted or unused IPv4 address space. Madory said these IPv4 brokers will pay between $100-$500 per month to lease a block of 256 IPv4 addresses, and very often the entities most willing to pay those rental rates are proxy and VPN providers.

A cursory review of all Internet address blocks currently routed through AT&T — as seen in public records maintained by the Internet backbone provider Hurricane Electric — shows a preponderance of country flags other than the United States, including networks originating in Hungary, Lithuania, Moldova, Mauritius, Palestine, Seychelles, Slovenia, and Ukraine.

AT&T’s IPv4 address space seems to be routing a great deal of proxy traffic, including a large number of IP address ranges that were until recently routed by ISPs in Ukraine.

Asked about the apparent high incidence of proxy services routing foreign address blocks through AT&T, the telecommunications giant said it recently changed its policy about originating routes for network blocks that are not owned and managed by AT&T. That new policy, spelled out in a February 2025 update to AT&T’s terms of service, gives those customers until Sept. 1, 2025 to originate their own IP space from their own autonomous system number (ASN), a unique number assigned to each ISP (AT&T’s is AS7018).

“To ensure our customers receive the best quality of service, we changed our terms for dedicated internet in February 2025,” an AT&T spokesperson said in an emailed reply. “We no longer permit static routes with IP addresses that we have not provided. We have been in the process of identifying and notifying affected customers that they have 90 days to transition to Border Gateway Protocol routing using their own autonomous system number.”

Ironically, the co-mingling of Ukrainian IP address space with proxy providers has resulted in many of these addresses being used in cyberattacks against Ukraine and other enemies of Russia. Earlier this month, the European Union sanctioned Stark Industries Solutions Inc., an ISP that surfaced two weeks before the Russian invasion and quickly became the source of large-scale DDoS attacks and spear-phishing attempts by Russian state-sponsored hacking groups. A deep dive into Stark’s considerable address space showed some of it was sourced from Ukrainian ISPs, and most of it was connected to Russia-based proxy and anonymity services.

According to Spur, the proxy service IPRoyal is the current beneficiary of IP address blocks from several Ukrainian ISPs profiled in Kentik’s report. Customers can chose proxies by specifying the city and country they would to proxy their traffic through. Image: Trend Micro.

Spur’s Chief Technology Officer Riley Kilmer said AT&T’s policy change will likely force many proxy services to migrate to other U.S. providers that have less stringent policies.

“AT&T is the first one of the big ISPs that seems to be actually doing something about this,” Kilmer said. “We track several services that explicitly sell AT&T IP addresses, and it will be very interesting to see what happens to those services come September.”

Still, Kilmer said, there are several other large U.S. ISPs that continue to make it easy for proxy services to bring their own IP addresses and host them in ranges that give the appearance of residential customers. For example, Kentik’s report identified former Ukrainian IP ranges showing up as proxy services routed by Cogent Communications (AS174), a tier-one Internet backbone provider based in Washington, D.C.

Kilmer said Cogent has become an attractive home base for proxy services because it is relatively easy to get Cogent to route an address block.

“In fairness, they transit a lot of traffic,” Kilmer said of Cogent. “But there’s a reason a lot of this proxy stuff shows up as Cogent: Because it’s super easy to get something routed there.”

Cogent declined a request to comment on Kentik’s findings.

Planet DebianMatthew Garrett: How Twitter could (somewhat) fix their encrypted DMs

As I wrote in my last post, Twitter's new encrypted DM infrastructure is pretty awful. But the amount of work required to make it somewhat better isn't large.

When Juicebox is used with HSMs, it supports encrypting the communication between the client and the backend. This is handled by generating a unique keypair for each HSM. The public key is provided to the client, while the private key remains within the HSM. Even if you can see the traffic sent to the HSM, it's encrypted using the Noise protocol and so the user's encrypted secret data can't be retrieved.

But this is only useful if you know that the public key corresponds to a private key in the HSM! Right now there's no way to know this, but there's worse - the client doesn't have the public key built into it, it's supplied as a response to an API request made to Twitter's servers. Even if the current keys are associated with the HSMs, Twitter could swap them out with ones that aren't, terminate the encrypted connection at their endpoint, and then fake your query to the HSM and get the encrypted data that way. Worse, this could be done for specific targeted users, without any indication to the user that this has happened, making it almost impossible to detect in general.

This is at least partially fixable. Twitter could prove to a third party that their Juicebox keys were generated in an HSM, and the key material could be moved into clients. This makes attacking individual users more difficult (the backdoor code would need to be shipped in the public client), but can't easily help with the website version[1] even if a framework exists to analyse the clients and verify that the correct public keys are in use.

It's still worse than Signal. Use Signal.

[1] Since they could still just serve backdoored Javascript to specific users. This is, unfortunately, kind of an inherent problem when it comes to web-based clients - we don't have good frameworks to detect whether the site itself is malicious.

comment count unavailable comments

Planet DebianMatthew Garrett: Twitter's new encrypted DMs aren't better than the old ones

(Edit: Twitter could improve this significantly with very few changes - I wrote about that here. It's unclear why they'd launch without doing that, since it entirely defeats the point of using HSMs)

When Twitter[1] launched encrypted DMs a couple
of years ago, it was the worst kind of end-to-end
encrypted - technically e2ee, but in a way that made it relatively easy for Twitter to inject new encryption keys and get everyone's messages anyway. It was also lacking a whole bunch of features such as "sending pictures", so the entire thing was largely a waste of time. But a couple of days ago, Elon announced the arrival of "XChat", a new encrypted message platform built on Rust with (Bitcoin style) encryption, whole new architecture. Maybe this time they've got it right?

tl;dr - no. Use Signal. Twitter can probably obtain your private keys, and admit that they can MITM you and have full access to your metadata.

The new approach is pretty similar to the old one in that it's based on pretty straightforward and well tested cryptographic primitives, but merely using good cryptography doesn't mean you end up with a good solution. This time they've pivoted away from using the underlying cryptographic primitives directly and into higher level abstractions, which is probably a good thing. They're using Libsodium's boxes for message encryption, which is, well, fine? It doesn't offer forward secrecy (if someone's private key is leaked then all existing messages can be decrypted) so it's a long way from the state of the art for a messaging client (Signal's had forward secrecy for over a decade!), but it's not inherently broken or anything. It is, however, written in C, not Rust[2].

That's about the extent of the good news. Twitter's old implementation involved clients generating keypairs and pushing the public key to Twitter. Each client (a physical device or a browser instance) had its own private key, and messages were simply encrypted to every public key associated with an account. This meant that new devices couldn't decrypt old messages, and also meant there was a maximum number of supported devices and terrible scaling issues and it was pretty bad. The new approach generates a keypair and then stores the private key using the Juicebox protocol. Other devices can then retrieve the private key.

Doesn't this mean Twitter has the private key? Well, no. There's a PIN involved, and the PIN is used to generate an encryption key. The stored copy of the private key is encrypted with that key, so if you don't know the PIN you can't decrypt the key. So we brute force the PIN, right? Juicebox actually protects against that - before the backend will hand over the encrypted key, you have to prove knowledge of the PIN to it (this is done in a clever way that doesn't directly reveal the PIN to the backend). If you ask for the key too many times while providing the wrong PIN, access is locked down.

But this is true only if the Juicebox backend is trustworthy. If the backend is controlled by someone untrustworthy[3] then they're going to be able to obtain the encrypted key material (even if it's in an HSM, they can simply watch what comes out of the HSM when the user authenticates if there's no validation of the HSM's keys). And now all they need is the PIN. Turning the PIN into an encryption key is done using the Argon2id key derivation function, using 32 iterations and a memory cost of 16MB (the Juicebox white paper says 16KB, but (a) that's laughably small and (b) the code says 16 * 1024 in an argument that takes kilobytes), which makes it computationally and moderately memory expensive to generate the encryption key used to decrypt the private key. How expensive? Well, on my (not very fast) laptop, that takes less than 0.2 seconds. How many attempts to I need to crack the PIN? Twitter's chosen to fix that to 4 digits, so a maximum of 10,000. You aren't going to need many machines running in parallel to bring this down to a very small amount of time, at which point private keys can, to a first approximation, be extracted at will.

Juicebox attempts to defend against this by supporting sharding your key over multiple backends, and only requiring a subset of those to recover the original. I can't find any evidence that Twitter's does seem to be making use of this,Twitter uses three backends and requires data from at least two, but all the backends used are under x.com so are presumably under Twitter's direct control. Trusting the keystore without needing to trust whoever's hosting it requires a trustworthy communications mechanism between the client and the keystore. If the device you're talking to can prove that it's an HSM that implements the attempt limiting protocol and has no other mechanism to export the data, this can be made to work. Signal makes use of something along these lines using Intel SGX for contact list and settings storage and recovery, and Google and Apple also have documentation about how they handle this in ways that make it difficult for them to obtain backed up key material. Twitter has no documentation of this, and as far as I can tell does nothing to prove that the backend is in any way trustworthy. (Edit to add: The Juicebox API does support authenticated communication between the client and the HSM, but that relies on you having some way to prove that the public key you're presented with corresponds to a private key that only exists in the HSM. Twitter gives you the public key whenever you communicate with them, so even if they've implemented this properly you can't prove they haven't made up a new key and MITMed you the next time you retrieve your key)

On the plus side, Juicebox is written in Rust, so Elon's not 100% wrong. Just mostly wrong.

But ok, at least you've got viable end-to-end encryption even if someone can put in some (not all that much, really) effort to obtain your private key and render it all pointless? Actually no, since you're still relying on the Twitter server to give you the public key of the other party and there's no out of band mechanism to do that or verify the authenticity of that public key at present. Twitter can simply give you a public key where they control the private key, decrypt the message, and then reencrypt it with the intended recipient's key and pass it on. The support page makes it clear that this is a known shortcoming and that it'll be fixed at some point, but they said that about the original encrypted DM support and it never was, so that's probably dependent on whether Elon gets distracted by something else again. And the server knows who and when you're messaging even if they haven't bothered to break your private key, so there's a lot of metadata leakage.

Signal doesn't have these shortcomings. Use Signal.

[1] I'll respect their name change once Elon respects his daughter

[2] There are implementations written in Rust, but Twitter's using the C one with these JNI bindings

[3] Or someone nominally trustworthy but who's been compelled to act against your interests - even if Elon were absolutely committed to protecting all his users, his overarching goals for Twitter require him to have legal presence in multiple jurisdictions that are not necessarily above placing employees in physical danger if there's a perception that they could obtain someone's encryption keys

comment count unavailable comments

Worse Than FailureCodeSOD: Integral to a Database Read

One of the key points of confusion for people unfamiliar with Java is the distinction between true object types, like Integer, and "primitive" types, like int. This is made worse by the collection types, like ArrayList, which needs to hold a true object type, but can't hold a primitive. A generic ArrayList<Integer> is valid, but ArrayList<int> won't compile. Fortunately for everyone, Java automatically "boxes" types- at least since Java 5, way back in 2004- so integerList.add(5) and int n = integerList.get(0) will both work just fine.

Somebody should have told that to Alice's co-worker, who spends a lot of code to do some type gymnastics that they shouldn't have:

try {
		ps = conn.prepareStatement(SQL_GET_LOT_WORKUP_STATUSES);
		ps.setLong(1, _lotId);

		rs = ps.executeQuery();

		while (rs.next()) {
				result.add(new Integer(rs.getInt(1)));
		}
}
finally {
		CloseUtil.close(ps,rs);
}

// instatiate a the array
_workupStatuses = new int[result.size()];

// convert the integers to ints
for (int h=0; h<result.size(); h++) {
		_workupStatuses[h] = ((Integer)result.get(h)).intValue();
}

This runs a query against the database, and then iterates across the result to populate a List type with integers, and right away we're getting into confused territory. rs.getInt returns an int primitive, which they manually box with new Integer, and stuff into the List. And look, I wouldn't really call that a WTF, but it's what they do next that leaves me scratching my head.

They initialize a private member, _workupStatuses to a new array of ints. Then they copy every integer from the result collection into the array, first by casting the get return value to Integer, then by pulling off the intValue.

In the end, this whole dance happens because Java ResultSet types open cursors on the database side and thus don't have the capacity to tell you how many rows they returned. You need to iterate across each record until it runs out of results. That's why they populate an intermediate list. Then they can check the size and create an array, but that itself is a big why. I'm not going to say that using arrays in Java is an instant anti-pattern, but it's always something to be suspicious of, especially when you're holding result sets. It's probably a premature optimization: the key performance distance is on insertions where an ArrayList may need to resize and copy its internal backing store.

My suspicion, however, is that this code falls into the category of "C programmer forced to do Java". They're comfortable with an array of integers, which is covers 90% of the data types you use in C but a dynamic, complicated data structure is horrifying to them. So they use it when they absolutely have to, and then throw it away as quickly as they can to get back to what they're familiar with.

[Advertisement] Plan Your .NET 9 Migration with Confidence
Your journey to .NET 9 is more than just one decision.Avoid migration migraines with the advice in this free guide. Download Free Guide Now!

365 TomorrowsCourse Correction

Author: Colin Jeffrey The newly-created Department of Temporal Dysfunction hummed with bureaucratic indifference as a voice called out across the waiting room: “Number forty-seven!” “That’s you,” the Seraphim sitting next to Quetzalcoatl said, pointing to his ticket. “You’re forty seven.” Quetzalcoatl stood up, brushed back his resplendent feathers, and followed the caller through to an […]

The post Course Correction appeared first on 365tomorrows.

,

Cryptogram Friday Squid Blogging: Stubby Squid

Video of the stubby squid (Rossia pacifica) from offshore Vancouver Island.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Cryptogram Friday Squid Blogging: Squid Run in Southern New England

Southern New England is having the best squid run in years.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Planet DebianGunnar Wolf: The subjective value of privacy • Assessing individuals' calculus of costs and benefits in the context of state surveillance

This post is an unpublished review for The subjective value of privacy • Assessing individuals' calculus of costs and benefits in the context of state surveillance

Internet users, software developers, academics, entrepreneurs – basically everybody is now aware of the importance of considering privacy as a core part of our online experience. User demand, and various national or regional laws, have made privacy a continuously present subject. And privacy is such an all-encompassing, complex topic, the angles from which it can be studied seems never to finish; I recommend computer networking-oriented newcomers to the topic to refer to Brian Kernighan’s excellent work [1]. However, how do regular people –like ourselves, in our many capacities– feel about privacy? Lukas Antoine presents a series of experiments aiming at better understanding how people throughout the world understands privacy, and when is privacy held as more or less important than security in different aspects,

Particularly, privacy is often portrayed as a value set at tension against surveillance, and particularly state surveillance, in the name of security: conventional wisdom presents the idea of privacy calculus. This is, it is often assumed that individuals continuously evaluate the costs and benefits of divulging their personal data, sharing data when they expect a positive net outcome, and denying it otherwise. This framework has been accepted for decades, and the author wishes to challenge it. This book is clearly his doctoral thesis on political sciences, and its contents are as thorough as expected in this kind of product.

The author presents three empirical studies based on cross-survey analysis. The first experiment explores the security justifications for surveillance and how they influence their support. The second one searches whether the stance on surveillance can be made dependent on personal convenience or financial cost. The third study explores whether privacy attitude is context-dependant or can be seen as a stable personality trait. The studies aim to address the shortcomings of published literature in the field, mainly, (a) the lack of comprehensive research on state surveillance, needed or better understanding privacy appreciation, (b) while several studies have tackled the subjective measure of privacy, there is a lack of cross-national studies to explain wide-ranging phenomena, (c) most studies in this regard are based on population-based surveys, which cannot establish causal relationships, (d) a seemingly blind acceptance of the privacy calculus mentioned above, with no strong evidence that it accurately measures people’s motivations for disclosing or withholding their data. The specific take, including the framing of the tension between privacy and surveillance has long been studied, as can be seen in Steven Nock’s 1993 book [2], but as Sannon’s article in 2022 shows [3], social and technological realities require our undertanding to be continuously kept up to date.

The book is full with theoretical references and does a very good job of explaining the path followed by the author. It is, though, a heavy read, and, for people not coming from the social sciences tradition, leads to the occasional feeling of being lost. The conceptual and theoretical frameworks and presented studies are thorough and clear. The author is honest in explaining when the data points at some of his hypotheses being disproven, while others are confirmed.

The aim of the book is for people digging deep into this topic. Personally, I have authored several works on different aspects of privacy (such as a book [4] and a magazine number [5]), but this book did get me thinking on many issues I had not previously considered. Looking for comparable works, I find Friedewald et al.’s 2017 book [6] chapter organization to follow a similar thought line. My only complaint would be that, for the publication as part of its highly prestigious publisher, little attention has been paid to editorial aspects: sub-subsection depth is often excessive and unclear. Also, when publishing monographs based on doctoral works, it is customary to no longer refer to the work as a “thesis” and to soften some of the formal requirements such a work often has, with the aim of producing a more gentle and readable book; this book seems just like the mass-production of an (otherwise very interesting and well made) thesis work.

References:

  • [1] Kernighan, B. W. (2021). Understanding the digital world: What you need to know about computers, the internet, privacy, and security. Princeton University Press.
  • [2] Nock, S. L. (1993). The Costs of Privacy: Surveillance and Reputation in America. De Gruyter.
  • [3] Sannon, S., Sun, B., Cosley, D. (2022). Privacy, Surveillance, and Power in the Gig Economy. SIGCHI, Association for Computing Machinery.
  • [4] Wolf, G. (coord), 2021. Mecanismos de privacidad y anonimato en redes. Una visión transdisciplinaria. IIEc-UNAM, México https://www.priv-anon.unam.mx/libro/
  • [5] XRDS•Crossroads Summer 2018. Pseudonimity and Anonymity. Association for Computing Machinery https://xrds.acm.org/archives.cfm?iid=3239334
  • [6] Friedewald, M., Burgess, P., Čas, J., Bellanova, R., Peissl, W. (2017). Surveillance, Privacy and Security: Citizens’ Perspectives. Routeledge, Taylor & Francis Group.

Planet DebianGunnar Wolf: Humanities and big data in Ibero-America • Theory, methodology and practical applications

This post is an unpublished review for Humanities and big data in Ibero-America • Theory, methodology and practical applications

Digital humanities is a young–though established–field. It deals with different expressions in which digital data manipulation techniques can be applied and used to analyze subjects that are identified as belonging to the humanities. Although most often used to analyze different aspects of literature or social network analysis, it can also be applied to other humanistic disciplines or artistic expressions. Digital humanities employs many tools, but those categorized as big data are among the most frequently employed. This book samples different takes on digital humanities, with the particularity that it focuses on Ibero-American uses. It is worth noting that this book is the second in a series of four volumes, published or set to be published between 2022 and 2026. Being the output of a field survey, I perceive this book to be targeted towards fellow Digital Humanists – people interested in applying computational methods to further understand and research topics in the humanities. It is not a technical book in the sense Computer Science people would recognize as such, but several of the presented works do benefit from understanding some technical concepts.

The 12 articles (plus an introduction) that make up this book are organized in three parts:

(1) “Theoretical Framework” presents the ideas and techniques of data science (that make up the tools for handling big data), and explores how data science can contribute to literary analysis, all while noting that many such techniques are usually frowned upon in Latin America as data science “smells neoliberal”;

(2) “Methodological Issues” looks at specific issues through the lens of how they can be applied to big data, with specific attention given to works in Spanish; and

(3) “Practical Applications” analyzes specific Spanish works and communities based on big data techniques.

Several chapters treat a recurring theme: the simultaneous resistance and appropriation of big data by humanists. For example, at least three of the chapters describe the tensions between humanism (“aesthesis”) and cold, number-oriented data analysis (“mathesis”).

The analyzed works of Parts 2 and 3 are interesting and relatively easy to follow.

Some inescapable ideological gleans from several word uses – from the book’s and series’ name, which refers to the Spanish-speaking regions as “Ibero-America”, often seen as Eurocentric, in contrast with the “Latin America” term much more widely used throughout the region.

I will end with some notes about the specific versions of the book I reviewed. I read both an EPUB version and a print copy. The EPUB did not include links for easy navigation to footnotes, that is, the typographical superindexes are not hyperlinked to the location of the notes, so it is very impractical to try to follow them. The print version (unlike the EPUB) did not have an index, that is, the six pages before the introduction are missing from the print copy I received. For a book such as this one, not having an index hampers the ease of reading and referencing.

Planet DebianGunnar Wolf: Beyond data poisoning in federated learning

This post is an unpublished review for Beyond data poisoning in federated learning

The current boom of artificial intelligence (AI) is based upon neural networks (NNs). In order for these to be useful, the network has to undergo a machine learning (ML) process: work over a series of inputs, and adjust the inner weights of the connections between neurons so that each of the data samples the network was trained on produces the right set of labels for each item. Federated learning (FL) appeared as a reaction given the data centralization power that traditional ML provides: instead of centrally controlling the whole training data, various different actors analyze disjoint subsets of data, and provide only the results of this analysis, thus increasing privacy while analyzing a large dataset. Finally, given multiple actors are involved in FL, how hard is it for a hostile actor to provide data that will confuse the NN, instead of helping it reach better performance? This kind of attack is termed a poisoning attack, and is the main focus of this paper. The authors set out to research how effective can a hyperdimensional data poisoning attack (HDPA) be to confuse a NN and cause it to misclassify both the items trained on and yet unseen items.

Data used for NN training is usually represented as a large set of orthogonal vectors, each describing a different aspect of the item, allowing for very simple vector arithmetic operations. Thus, NN training is termed as high-dimensional or hyperdimensional. The attack method described by the authors employs cosine similarity, that is, in order to preserve similarity, a target hypervector is reflected over a given dimension, yielding a cosine-similar result that will trick ML models, even if using byzantine-robust defenses.

The paper is clear, though not an easy read. It explains in detail the mathematical operations, following several related although different threat models. The authors present the results of the experimental evaluation of their proposed model, comparing it to several other well-known adversarial attacks for visual recognition tasks, over pre-labeled datasets frequently used as training data, such as MNIST, Fashion-MNIST and CIFAR-10. They show that their method is not only more effective as an attack, but falls within the same time range as other surveyed attacks.

Adversarial attacks are, all in all, an important way to advance any field of knowledge; by publishing this attack, the authors will surely spark other works to detect and prevent this kind of alteration. It is important for AI implementers to understand the nature of this field and be aware of the risks that this work, as well as others cited in it, highlight: ML will train a computer system to recognize a dataset, warts and all; efficient as AI is, if noise is allowed into the training data (particularly adversarially generated noise), the trained model might present impaired performance.

Planet DebianGunnar Wolf: Computational modelling of robot personhood and relationality

This post is an unpublished review for Computational modelling of robot personhood and relationality

If humans and robots were to be able to roam around the same spaces, mutually recognizing each other for what they are, how would interaction be? How can we model such interactions in a way that we can reason about and understand the implications of a given behavior? This book aims at answering this question.

The book is split into two very different parts. Chapters 1 through 3 are mostly written with a philosophical angle. It starts by framing the possibility of having sentient androids exist in the same plane as humans, without them trying to pass as us or vice versa. The first chapters look at issues related to personhood, that is, how androids can be treated as valid interaction partners in a society with humans, and how interactions with them can be seen as meaningful. In doing so, several landmarks of the past 40 years in the AI field are reviewed. The issues of the “Significant Concerns” that make up a society and give it coherence and of “Personhood and Relationality”, describing how this permeates from a society into each of the individuals that make it up, the relations between them and the social objects that bring individuals closer together (or farther apart) are introduced and explained.

The second part of the book is written from a very different angle, and the change in pace took me somewhat by surprise. Each subsequent chapter presents a different angle of the “Affinity” system, a model that follows some aspects of human behavior over time and in a given space. Chapter 4 introduces the “Affinity” environment: a 3D simulated environment with simulated physical laws and characteristics, where a number of agents (30-50 is mentioned as usual) interact. Agents have a series of attributes (“value memory”), can adhere to different programs (“narratives”), and gain or lose on some vectors (“economy”). They can sense the world around them with sensors, and can modify the world or signal other agents using effectors.

The last two chapters round out the book, as expected: the first presents a set of results from analyzing a given set of value systems, and the second gives readers the conclusions reached by the author. However, I was expecting more–either having at least a link to download the “Affinity” system and continue exploring it or modifying some of the aspects it models to get it to model a set of agents with different stories and narratives, or extend it to yet unforseen behaviors, or at least have the author present a more complete comparison of results than the evaluation of patterns resulting from a given run. The author is a well-known, prolific author in the field, and I was expecting bigger insights from this book.

Nevertheless, the book is an interesting and fun read, with important insights in both the first and second parts. There is a certain lack of connection between their respective rhythms, and the second part indeed builds on the concepts introduced in the first one. Overall, I enjoyed reading the book despite expecting more.

Planet DebianGunnar Wolf: Will be adding yet-to-be-published reviews

Since December 2023, I have been publishing the reviews I write for Computing Reviews as they get published. I will do a slight change now: I will start pushing the reviews to my blog as I write them, and of course, will modify them with the final wording and to link to their place as soon as they are published. I’m doing this because sometimes it takes very long for reviews to be approved, and I want to share them with my blog’s readers!

So, please bear with this a bit: I’ll send a (short!) flood of my latest four pending reviews today.

Some books I have recently read

Planet DebianRussell Coker: Trying DeepSeek R1

I saw this document on running DeepSeek R1 [1] and decided to give it a go. I downloaded the llama.cpp source and compiled it and downloaded the 131G of data as described. Running it with the default options gave about 7 CPU cores in use. Changing the --threads parameter to 44 caused it to use 17 CPU cores (changing it to larger numbers like 80 made it drop to 2.5 cores). I used the --n-gpu-layers parameter with the value of 1 as I currently have a GPU with only 6G of RAM (AliExpress is delaying my delivery of a PCIe power adaptor for a better GPU). Running it like this makes the GPU take 12W more power than standby and using 5.5G of VRAM according to nvidia-smi so it is doing a small amount of work, but not much. The documentation refers to the DeepSeek R1 1.58bit model which I’m using as having 61 layers so presumably less than 2% of the work is done on the GPU.

Running like this it takes 2 hours of CPU time (just over 3 minutes of elapsed time at 17 cores) to give 8 words of output. I didn’t let any tests run long enough to give complete output.

The documentation claims that it will run on CPU with 20G of RAM. In my tests it takes between 161G and 195G of RAM to run depending on the number of threads. The documentation describes running on the CPU as “very slow” which presumably means 3 words per minute on a system with a pair of E5-2699A v4 CPUs and 256G of RAM.

When I try to use more than 44 threads I get output like “system_info: n_threads = 200 (n_threads_batch = 200) / 44” and it seems that I only have a few threads actually in use. Apparently there’s some issue with having more threads than the 44 CPU cores in the system.

I was expecting this to go badly and it met my expectations in that regard. But it was interesting to see exactly how it went badly. It seems that if I had a GPU with 24G of VRAM I’d still have 54/61 layers running on the CPU so even the largest of home GPUs probably wouldn’t make much difference.

Maybe if I configured the server to have hyper-threading enabled and 88 HT cores then I could have 88 threads and about 34 CPU cores in use which might help. But even if I got the output speed from 3 to 6 words per minute that still wouldn’t be very usable.

LongNowThe Codes AI Can't Crack

The Codes AI Can't Crack

It was October 02019, and Thea Sommerschield had hit a wall. She was working on her doctoral thesis in ancient history at Oxford, which involved deciphering Greek inscriptions that were carved on stones in Western Sicily more than 2,000 years earlier. As is often the case in epigraphy — the study and interpretation of ancient inscriptions written on durable surfaces like stone and clay — many of the texts were badly damaged. What’s more, they recorded a variety of dialects, from a variety of different periods, which made it harder to find patterns or fill in missing characters. 

At a favorite lunch spot, she shared her frustrations with Yannis Assael, a Greek computer scientist who was then working full-time at Google DeepMind in London while commuting to Oxford to complete his own PhD. Assael told Sommerschield he was working with a technology that might help: a recurrent neural network, a form of artificial intelligence able to tackle complex sequences of data. They set to work training a model on digitized Greek inscriptions written before the fifth century, similar to how ChatGPT was trained on vast quantities of text available on the internet.

Sommerschield watched with astonishment as the missing text from the damaged inscriptions began to appear, character by character, on her computer screen. After this initial success, Assael suggested they build a model based on transformer technology, which weights characters and words according to context. Ithaca, as they called the new model, was able to fill in gaps in political decrees from the dawn of democracy in Athens with 62% accuracy, compared to 25% for human experts working alone. When human experts worked in tandem with Ithaca, the results were even better, with accuracy increasing to 72%.

Ithaca is one of several ancient code-cracking breakthroughs powered by artificial intelligence in recent years. Since 02018, neural networks trained on cuneiform, the writing system of Mesopotamia, have been able to fill in lost verses from the story of Gilgamesh, the world’s earliest known epic poem. In 02023, a project known as the Vesuvius Challenge used 3D scanners and artificial intelligence to restore handwritten texts that hadn’t been read in 2,000 years, revealing previously unknown works by Epicurus and other philosophers. (The scrolls came from a luxurious villa in Herculaneum, buried during the same eruption of Mount Vesuvius that destroyed Pompeii. When scholars had previously tried to unroll them, the carbonized papyrus crumbled to dust.)

The Codes AI Can't Crack
Phaistos Disk (c. 01850–01600 BCE).

Yet despite these advances, a dozen or so ancient scripts — the writing systems used to transcribe spoken language — remain undeciphered. These include such mysteries as the one-of-a-kind Phaistos Disk, a spiral of 45 symbols found on a single sixteen-inch clay disk in a Minoan palace on Crete, and Proto-Elamite, a script used 5,000 years ago in what is now Iran, which may have consisted of a thousand distinct symbols. Some, like Cypro-Minoan — which transcribes a language spoken in the Late Bronze Age on Cyprus — are tantalizingly similar to early European scripts that have already been fully deciphered. Others, like the quipu of the Andes — intricately knotted ropes made of the wool of llamas, vicuñas, and alpacas — stretch our definitions of how speech can be transformed into writing. 

The Codes AI Can't Crack
Inca quipu (c. 01400–01532).

In some cases, there is big money to be won: a reward of one million dollars is on offer for the decipherer of the Harappan script of the Indus Valley civilization of South Asia, as well as a $15,000-per-character prize for the successful decoder of the Oracle Bone script, the precursor to Chinese.

Cracking these ancient codes may seem like the kind of challenge AI is ideally suited to solve. After all, neural networks have already bested human champions at chess, as well as the most complex of all games, Go. They can detect cancer in medical images, predict protein structures, synthesize novel drugs, and converse fluently and persuasively in 200 languages. Given AI’s ability to find order in complex sets of data, surely assigning meaning to ancient symbols would be child’s play.

But if the example of Ithaca shows the promise of AI in the study of the past, these mystery scripts reveal its limitations. Artificial neural networks might prove a crucial tool, but true progress will come through collaboration between human neural networks: the intuitions and expertise stored in the heads of scholars, working in different disciplines in real-world settings.


“AI isn’t going to replace human historians,” says Sommerschield, who is now at the University of Nottingham. “To us, that is the biggest success of our research. It shows the potential of these technologies as assistants.” She sees artificial intelligence as a powerful adjunct to human expertise. “To be an epigrapher, you have to be an expert not just in the historical period, but also in the archaeological context, in the letter form, in carbon dating.” She cautions against overstating the potential of AI. “We’re not going to have an equivalent of ChatGPT for the ancient world, because of the nature of the data. It’s not just low in quantity, it’s also low in quality, with all kinds of gaps and problems in transliteration.”

Ithaca was trained on ancient Greek, a language we’ve long known how to read, and whose entire corpus amounts to tens of thousands of inscriptions. The AI models that have filled in lost verses of Gilgamesh are trained on cuneiform, whose corpus is even larger: hundreds of thousands of cuneiform tablets can be found in the storerooms of the world’s museums, many of them still untranslated. The problem with mystery scripts like Linear A, Cypro-Minoan, Rongorongo, and Harappan is that the total number of known inscriptions can be counted in the thousands, and sometimes in the hundreds. Not only that, in most cases we have no idea what spoken language they’re meant to encode.

The Codes AI Can't Crack
Harappan script as seen on the Pashupati seal (c. 02200 BCE).

“Decipherment is kind of like a matching problem,” explains Assael. “It’s different from predicting. You’re trying to match a limited number of characters to sounds from an older, unknown language. It’s not a problem that’s well suited to these deep neural network architectures that require substantial amounts of data.”

Human ingenuity remains key. Two of the greatest intellectual feats of the 20th century involved the decipherment of ancient writing systems. In 01952, when Michael Ventris, a young English architect, announced that he’d cracked the code of Linear B, a script used in Bronze Age Crete, newspapers likened the accomplishment to the scaling of Mount Everest. (Behind the scenes, the crucial grouping and classifying of characters on 180,000 index cards into common roots — the grunt work that would now be performed by AI — was done by Alice Kober, a chain-smoking instructor from Brooklyn College.)

The Codes AI Can't Crack
Illustration of a Linear B tablet from Pylos.

The decipherment of the Maya script, which is capable of recording all human thought using bulbous jaguars, frogs, warriors’ heads, and other stylized glyphs, involved a decades-long collaboration between Yuri Knorozov, a Soviet epigrapher, and American scholars working on excavations in the jungles of Central America.

While the interpreting of Egyptian hieroglyphics is held up as a triumph of human ingenuity, the Linear B and Mayan codes were cracked without the help of a Rosetta Stone to point the way. With Linear B, the breakthrough came when Ventris broke with the established thinking, which held that it transcribed Etruscan — a script scholars can read aloud, but whose meaning still remains elusive — and realized that it corresponded to a form of archaic Greek spoken 500 years before Homer. In the case of ancient Mayan, long thought to be a cartoonish depiction of universal ideas, it was only when scholars acknowledged that it might transcribe the ancestors of the languages spoken by contemporary Maya people that the decipherment really began. Today, we can read 85% of the glyphs; it is even possible to translate Shakespeare’s Hamlet into ancient Mayan.

The Codes AI Can't Crack
A panel of a royal woman with Maya script visible on the sides (c. 0795).

Collaborating across cultures and disciplines, and carrying out paradigm-shedding leaps of intuition, are not the strong points of existing artificial neural networks. But that doesn’t mean AI can’t play a role in decipherment of ancient writing systems. Miguel Valério, an epigrapher at the Autonomous University of Barcelona, has worked on Cypro-Minoan, the script used on Cyprus 3,500 years ago. Two hundred inscriptions, on golden jewelry, metal ingots, ivory plaques, and four broken clay tablets, have survived. Valério was suspicious of the scholarly orthodoxy, which attributed the great diversity in signs to the coexistence of three distinct forms of the language. 

To test the theory that many of the signs were in fact allographs — that is, variants, like the capital letter “G” and “g,” its lower-case version — Valério worked with Michele Corazza, a computational linguist at the University of Bologna, to design a custom-built neural network they called Sign2Vecd. Because the model was unsupervised, it searched for patterns without applying human-imposed preconceptions to the data set. 

“The machine learned how to cluster the signs,” says Valério, “but it didn’t do it simply on the basis of their resemblance, but also on the specific context of a sign in relation to other signs. It allowed us to create a three-dimensional plot of the results. We could see the signs floating in a sphere, and zoom in to see their relationship to each other, and whether they’d been written on clay or metal.” 

The virtual sphere allowed Valério to establish a sign-list — the equivalent of the list of 26 letters in our alphabet, and the first step towards decipherment — for Cypro-Minoan, which he believes has about 60 different signs, all corresponding to a distinct syllable.

“The issue is always validation. How do you know if the result is correct if the script is undeciphered? What we did was to compare it to a known script, the Cypriot Greek syllabary, which is closely related to Cypro-Minoan. And we found the machine got it right 70% of the time.” But Valério believes no unsupervised neural net, no matter how powerful, will crack Cypro-Minoan on its own. “I don’t see how AI can do what human epigraphers do traditionally. Neural nets are very useful tools, but they have to be directed. It all depends on the data you provide them, and the questions you ask them.”


The latest advances in AI have come at a time when there has been a revolution in our understanding of writing systems. A generation ago, most people were taught that writing was invented once, in Mesopotamia, about 5,500 years ago, as a tool of accountancy and state bureaucracy. From there, the standard thinking went, it spread to Egypt, and hieroglyphics were simplified into the alphabet that became the basis for recording most European languages. It is now accepted that writing systems were not only invented to keep track of sheep and units of grain, but also to record spiritual beliefs and tell stories. (In the case of Tifinagh, an ancient North African Berber script, there is evidence that writing was used primarily as a source of fun, for puzzle-making and graffiti.) Monogenesis, the idea that the Ur-script diffused from Mesopotamia, has been replaced by the recognition that writing was invented independently in China, Egypt, Central America, and — though this remains controversial — in the Indus Valley, where 4,000 inscriptions been unearthed in sites that were home to one of the earliest large urban civilizations.

The most spectacular example of a potential “invention of writing” is Rongorongo, a writing system found on Rapa Nui, the island famous for its massive carved stone heads. Also known as Easter Island, it is 1,300 miles from any other landmass in the South Pacific. Twenty-six tablets have been discovered, made of a kind of wood native to South Africa. Each has been inscribed, apparently using a shark’s tooth as a stylus, with lines of dancing stick-figures, stylized birds and sea creatures. The tablets were recently dated to the late 01400s, two centuries before Europeans first arrived on the island.

The Codes AI Can't Crack
"A View of the Monuments of Easter Island, Rapa Nui" by William Hodges (c. 01775–01776).

For computational linguist Richard Sproat, Rongorongo may be the script AI can offer the most help in decoding. “It’s kind of a decipherer’s dream,” says Sproat, who worked on recurrent neural nets for Google, and is now part of an AI start-up in Tokyo. “There are maybe 12,000 characters, and some of the inscriptions are quite long. We know that it records a language related to the modern Rapa Nui, which Easter Islanders speak today.” Archaeologists have even reported eyewitness accounts of the ceremonies in which the tablets were inscribed. And yet, points out Sproat, even with all these head starts, and access to advanced AI, nobody has yet come close to a convincing decipherment of Rongorongo.

The Codes AI Can't Crack
A photo of Rongorongo Tablet E (c. early 01880s). The tablet, given to the University of Louvain, was destroyed in a fire in 01914. From L'île de Pâques et ses mystères (Easter Island and its Mysteries) by Stéphen-Charles Chauvet (01935).

The way forward depends on finding more inscriptions, and that comes down to old-fashioned “dirt” archaeology, and the labor-intensive process of unearthing ancient artifacts. (The best-case scenario would be finding a “bilingual,” a modern version of the Rosetta Stone, whose parallel inscriptions in Greek and demotic allowed 19th-century scholars to decipher Egyptian hieroglyphics.) But the code of Cypro-Minoan, or Linear A, or the quipu of the Andes, won’t be cracked by a computer scientist alone. It’s going to take a collaboration with epigraphers working with all the available evidence, some of which is still buried at archaeological sites.

“As a scholar working in social sciences,” says Valério of the Autonomous University of Barcelona, “I feel I’m obliged to do projects in the digital humanities these days. If we pursue things that are perceived as traditional, no one is going to grant us money to work. But these traditional things are also important. In fact, they’re the basis of our work.” Without more material evidence, painstakingly uncovered, documented, and digitized, no AI, no matter how powerful, will be able to decipher the writing systems that will help us bring the lost worlds of the Indus Valley, Bronze Age Crete, and the Incan Empire back to life.

Perhaps the most eloquent defense of traditional scholarship comes from the distinguished scholar of Aegean civilization, Silvia Ferrara, who supervised Valério and Corazza’s collaboration at the University of Bologna. 

“The computer is no deus ex machina,” Ferrara writes in her book The Greatest Invention (02022). “Deep learning can act as co-pilot. Without the eye of the humanist, though, you don’t stand a chance at decipherment.”

Notes

1. Figure and caption reproduced from Corazza M, Tamburini F, Valério M, Ferrara S (02022) Unsupervised deep learning supports reclassification of Bronze age cypriot writing system. PLoS ONE 17(7): e0269544 under a CC BY 4.0 license. https://doi.org/10.1371/journal.pone.0269544

Worse Than FailureCodeSOD: Anything and Everything

Today's short function comes from Janusz, and it's anything you want it to be:

public static function isAnything($data)
{
    return true;
}

Honestly, I'm surprised that it was made static. Sure, static is the correct choice for this function, at least if we're describing anything about this function as "correct". I'm still surprised. It's got an accurate name given its behavior, it's scoped correctly. It still shouldn't exist and I have no idea what lead to it existing, but that's not surprising.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

365 TomorrowsThe Empire’s Last Breath

Author: Kenny O’Donnell Touchdown. The ship rumbled. The landing gear drilled into the asteroid, anchoring his one-man yacht. The asteroid, only a kilometre long and half as wide, was too small to hold a ship without anchors. His joints popped as he floated from his chair in micro-gravity. He grabbed a handle on the bulkhead […]

The post The Empire’s Last Breath appeared first on 365tomorrows.

,

365 TomorrowsMan to PostMan

Author: Majoki When his son stepped through the privacy-field into his home office, Manfred began to disconnect. “You told me to come see you after I finished my homelearn session, Dad.” His son’s eyes narrowed disdainfully at the etherware bands his father removed from his head and set by the brainframe, their household’s direct link […]

The post Man to PostMan appeared first on 365tomorrows.

Worse Than FailureCodeSOD: Continuous Installation

A recent code-review on a new build pipeline got Sandra's attention (previously). The normally responsible and reliable developer responsible for the commit included this in their Jenkinsfile:

sh '''
if ! command -v yamllint &> /dev/null; then
	if command -v apt-get &> /dev/null; then
	apt-get update && apt-get install -y yamllint
	elif command -v apk &> /dev/null; then
	apk add --no-cache yamllint
	elif command -v pip3 &> /dev/null; then
	pip3 install --break-system-packages yamllint
	fi
fi
find . -name '*.yaml' -exec yamllint {} \\; || true
find . -name '*.yml' -exec yamllint {} \\; || true
'''

So the goal of this script is to check to see if the yamllint command is available. If it isn't, we check if apt-get is available, and if it is, we use that to install yamllint. Failing that, we try apk, Alpine's package manager, and failing that we use pip3 to install it out of PyPI. Then we run it against any YAML files in the repo.

There are a few problems with this approach.

The first, Sandra notes, is that they don't use Alpine Linux, and thus there's no reason to try apk. The second is that this particular repository contains no Python components and thus pip is not available in the CI environment. Third, this CI job runs inside of a Docker image which already has yamllint installed.

Now, you'd think the developer responsible would have known this, given that this very merge request also included the definition of the Dockerfile for this environment. They'd already installed yamllint in the image.

Sandra writes:

This kind of sloppiness is also wildly out of character for him, to the point where my first thought was that it was AI-generated - especially since this was far from the only WTF in the submitted Jenkinsfile. Thankfully, it didn't pass code review and was sent back for intensive rework.

Finally, while the reality is that we'll always need to resolve some dependencies at build time, things like "tooling" and "linters" really belong in the definition of the build environment, not resolved at build time.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

Cryptogram New Linux Vulnerabilities

They’re interesting:

Tracked as CVE-2025-5054 and CVE-2025-4598, both vulnerabilities are race condition bugs that could enable a local attacker to obtain access to access sensitive information. Tools like Apport and systemd-coredump are designed to handle crash reporting and core dumps in Linux systems.

[…]

“This means that if a local attacker manages to induce a crash in a privileged process and quickly replaces it with another one with the same process ID that resides inside a mount and pid namespace, apport will attempt to forward the core dump (which might contain sensitive information belonging to the original, privileged process) into the namespace.”

Moderate severity, but definitely worth fixing.

Slashdot thread.

,

Worse Than FailureMaking a Pass of Yourself

Frederico planned to celebrate the new year with friends at the exotic international tourist haven of Molvania. When visiting the area, one could buy and use a MolvaPass (The Most Passive Way About Town!) for free or discounted access to cultural sites, public transit, and more. MolvaPasses were available for 3, 7, or 365 days, and could be bought in advance and activated later.

Httpd default page oracle linux 7

Still outside the country the week before his trip, Frederico had the convenience of buying a pass either online or via an app. He elected to use the website, sitting down before his home PC and entering the address into his web browser. Despite his fiber internet connection, he sat on a white screen for several seconds while the GoMolva Tourist Board website loaded. He then clicked the obvious Buy Now button in the top-right corner. After several more seconds, he was presented with a page requiring him to create an account.

Frederico did so, specifying his email address and a 16-character password suggested by Bitwarden. He then received a confirmation link in his email inbox. Upon clicking that, he was presented with an interface where he could add MolvaPasses to a shopping cart. He selected one 3-day pass and paid with PayPal. The website redirected him to the proper screen; he entered his PayPal credentials and confirmed the payment.

From there, he was redirected to a completely white screen. After waiting several seconds, a minute ... nothing changed. PayPal sent him a receipt, but there was no confirmation from the GoMolva Tourist Board website.

Frederico decided to refresh the page. This time, he saw the default Apache screen on CentOS.

His jaw almost hit the floor. They were still using CentOS, despite the fact that it'd been abandoned? Horrified, he bailed on that tab, desperately opening a fresh one and manually entering the URL again.

Finally, the page loaded successfully. Frederico was still logged in. From there, he browsed to the My Passes section. His 3-day MolvaPass was there, listed as Not activated.

This was exactly what Frederico had hoped he would see. With a sigh of relief, he turned his attention away from his laptop to his phone. For the sake of convenience, he wanted to download the MolvaPass app onto his phone. Upon doing so, he opened it and entered his username and password on the initial screen. After clicking Login, the following message appeared: The maximum length of the password is 15 characters.

Frederico's blood froze. How was that possible? There'd been no errors or warnings when he'd created his login. Everything had been fine then. Heart pounding, Frederico tried logging in again. The same error appeared. He switched back to his computer, where the site was still open. He browsed to My Account and selected Change Password.

A new screen prompted him for the old password, and a new one twice. He hurriedly filled in the fields and clikced the Change Password button.

A message appeared: Your MolvaPass has been successfully activated.

"What?!" Frederico blurted out loud. There was nothing to click but an OK button.

A follow-up message assured him, Password has been successfully changed.

As terror bolted down his spine, an expletive flew from his mouth. He navigated back to My Passes. There beside his newly-purchased pass was the big green word Activated.

"I only changed the password!" he pleaded out loud to a god who clearly wasn't listening. He forced a deep breath upon his panicked self and deliberated what to do from there. Support. Was there any way to get in touch with someone who could undo the activation or refund his money? With some Googling, Frederico found a toll-free number he could call from abroad. After he rapidly punched the number into his phone, a stilted robot voice guided him through a phone menu to the "Support" option.

We're getting somewhere, Frederico reassured himself.

"FoR MoLvaPaSs suPpOrt, uSe ThE cOnTaCt FoRm oN tHe GoMoLvA WeBzOnE." The robot hung up.

Frederico somehow refrained from hurling his phone across the room. Turning back to his PC, he scrolled down to the website footer, where he found a Contact us link. On this page, there was a contact form and an email address. Frederico filled out the contact form in detail and clicked the Submit button.

A new message appeared: Unable to send the request, try again later.

Frederico rolled his eyes toward the heavens. Somehow, he managed to wait a good five minutes before trying again—in vain. Desperately, he took his detailed message and emailed it to the support address, hoping for a quick response.

Minutes crawled past. Hours. Nothing by the time Frederico went to bed. It wasn't until the next morning that a response came back. The entire message read: The MolvaPass should have been activated once you reached Molvania, not before.

Consumed with soul-burning fury, Frederico hit Caps Lock on his keyboard. MAYBE MY PREVIOUS EMAIL WAS TOO LONG OR DIFFICULT TO UNDERSTAND?? ALL I DID WAS CHANGE THE PASSWORD!!!!

Several hours later, the following reply: The change of pw is not related to the activation of the pass.

Frederico directed his rage toward escalating the matter. He managed to track down the company that'd built the GoMolva website, writing to their support to demand a cancellation of the MolvaPass and a full refund. A few hours later, their reply asked for his PayPal transaction code so they could process the request.

In the end, Frederico got his money back and resolved to wait until he was physically in Molvania before attempting to buy another MolvaPass. We can only hope he rang in the new year with sanity intact.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

365 TomorrowsConsensus

Author: Julian Miles, Staff Writer Joey looks around at the crowd. “I see we’ve some new faces tonight. Thanks for coming.” He presses his palms flat on the table. “You’ve done what each of us has done at some point in the last few years: you’ve realised there’s something deeply wrong with our world. Those […]

The post Consensus appeared first on 365tomorrows.

Cryptogram Australia Requires Ransomware Victims to Declare Payments

A new Australian law requires larger companies to declare any ransomware payments they have made.

Cryptogram Friday Squid Blogging: NGC 1068 Is the “Squid Galaxy”

I hadn’t known that the NGC 1068 galaxy is nicknamed the “Squid Galaxy.” It is, and it’s spewing neutrinos without the usual accompanying gamma rays.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

,

Planet DebianBen Hutchings: FOSS activity in May 2025

365 TomorrowsBehind the Buildings

Author: Aubrey Williams I’ve been looking for work for months now. After the chip company got all-new machinery, the bean-counters did a review, and I was one of the names that got a red strikethrough. I can’t live on redundancy forever, and I’m not poor enough to get a rare welfare payment, so I need […]

The post Behind the Buildings appeared first on 365tomorrows.

David BrinLying promises and how to NAIL them...

Back to politics this weekend. Starting with:

1.  Please don't use the TACO slur. It may amuse you and irk your enemy, sure. But this particular mockery has one huge drawback. It might taunt him into not backing down ('chickening out') some time when it's really needed, in order to save all our lives. So... maybe... grow up and think tactics?

A far more effective approach is to hammer hypocrisy! 

Yeah, sure. Many have tried that. Though never with the relentless consistency that cancels their tactic of changing the subject.

I've never seen it done with the kind of harsh repetitive simplicity that I recommended in Polemical Judo. Repetitive simplicity that is the tactic that the Foxites perfected! As when all GOPpers repeat the same party line all together - like KGB metronomes - all on the same morning.

And hence...


2. ... and hence, here is a litany of hypocrisy and poor memory that is capsulated enough to be shouted!   

These are challenges that might reach a few of your getting-nervous uncles. especially as a combined list! 

Ten years ago, Donald Trump promised proof that Barack Obama was born in Kenya. 

“Soon! The case is water-tight and ready. I'll present it next week!”  The same promise got repeated, week after week, month after month. And sure, his dittohead followers relished not facts, but the hate mantra, so they never kept track...

Also ten years ago Beck and Hannity etc. declared "George Soros personally toppled eight foreign governments!" (Actually, it's sort of true!) They promised to list those eight Soros-toppled victims! Only they never did. Because providing that list would have left Fox a smoldering ruin.

Nine years ago, running against H Clinton, Donald Trump declared I will build a Big Beautiful WALL!" From sea to shining sea. 

Funny how he never asked his GOP-run Congress, later, for the money. And he still hasn't. Clinton and Obama each built more fences & surveillance systems to control the border than Trump ever did.

Also nine years ago,"You’ll never see me on a golf course, I’ll be working so hard for you!”  Um...

Eight years ago - after inauguration and taking over the US government, he vowed: “Within weeks the indictments will roll in a great big wave. You’ll see the Obama Administration was the most corrupt ever!”  

(Real world: there were zero indictments of the most honest and least blemished national administration in all of human history. Bar none. In fact, grand juries - consisting mostly of white retirees in red states - have indicted FORTY TIMES as many high Republicans as Democrats. Care to offer wager stakes?)

Also eight years ago, his 1st foreign guests in the White House - Lavrov and Kisliak, giggled with him ecstatically (see below), thinking their KGB tricks had captured the USA. Alas for Putin's lads, it took them 8 more years.



Seven years ago, ol’ Two Scoops promised a “terrific health care bill for everyone!” to replace ‘horrible Obamacare!’  And repeatedly for the next six years he declared “You’ll see it in two weeks!” And then... in 2 weeks. And then... in 2 weeks. And then in 2 weeks… twenty... fifty more times.

Also seven years ago, "Kim Jong Un and I fell in love!" (see above).

Six years ago, Fox “News” declared in court “we don’t do news, we are an entertainment company,” in order to writhe free of liability and perjury for oceans of lies. And still Fox had to pay $150 millions.

Five years ago Trump’s son-in-law was “about to seal the deal on full peace in the Middle East!”

Four years ago, Don promised “Absolute proof the election was stolen by Biden and the dems!" 

Howl after howl by Foxite shills ensued, and yet, not one scintilla of credible evidence was ever presented. While blowhards and blockheads fulminated into secessionist fury, all courts – including many GOP appointed judges - dismissed every 'case' as ludicrous, and several of them fined Trumpist shriekers for frivolous lying. Oh, the screeches and spumes! But not…one…shred of actual evidence. Ever. 

Three years ago, three different GOP Congressmen alluded-to or spoke-of how sex orgies are rife among top DC Republicans. And two of them alluded to resulting blackmail. 

Trump demanded “release the Epstein Files!”... then filed every lawsuit that his lawyers could concoct, in order to prevent it. And to protect an ocean of NDAs. 

Oh, and he promised “Great revelations!” on UFOs and the JFK assassination, just as soon as he got back in office. Remember that? Disappointed, a little? And Epstein's pal is still protected.

Two years ago, Paul Ryan and Mitt Romney and even Mitch McConnell were hinting at a major push to reclaim the Republican Party - or at least a vestigially non-traitor part of it - from the precipice where fanaticism and blackmail and treason had taken it. 

If necessary - (it was said) - they would form a new, Real Rebublican Party, where a minority of decent adults remaining in the GOP 'establishment' might find refuge and begin rebuilding. 

Only it seems that crown prince Ryan & co. chickened out, as he always has... RACO.

One year ago... actually less... the Economist offered this cover plus detailed stats, showing what always happens. That by the end of every Democratic administration, most things - certainly the economy and yes, deficits - are better. And they always get worse across the span of GOP admins. Care to bet this time?



   Alas, now the bitterly laughingstock of the world, deliberately immolating the universities and science and professions that truly Made America Great. 

There's your year-by year Top Ten Hypocricies countdown. And it's worth a try, to see if hammering the same things over and over - which worked so well for the Foxites might be worth a try?


            Oh, sure. Those aren’t my paramount complaints against Putin’s lackey and his shills. 

My main gripe is the one thing that unites them all -- Trump’s oligarchs with foreign enemies and with MAGA groundlings. 

 That one goal? Shared hatred of every single fact using profession, from science and civil service to the FBI/intel/military officer corps who won the Cold War and the War on Terror…

... the very ones standing between YOU and a return to feudal darkness.*

These reminder samplers of promises never kept are still valid. They could be effective if packaged properly, And will someone please show me who – in this wilderness – is pointing at them?


== Final lagniappe... a reminder of the most-loathesome of all... ==


* And yeah... here again in the news is the would-be Machiavelli/Wormtongue who flatter-strokes the ingrate, would-be lords who are seeking to betray the one renaissance that gave them everything they have.




Okay, I was planning to finish with a riff (again) on teleologies or notions of TIME. Very different notions that are clutched by the far-left, by today's entire right, and by the beleaguered liberal/middle.


But anon. Another time.


,

David BrinScience as the ultimate accountability process - And are AI behaving this way because they're DREAMING?

The power of Reciprocal Accountability

 

Is there a best path to getting both individuals and societies to behave honestly and fairly?


That goal -- attaining fact-based perception -- was never much advanced by the ‘don’t lie’ commandments of finger-wagging moralists and priests. 


Sure, for 6000 years, top elites preached and passed laws against lies and predation... only to become the top liars and self-deceivers, bringing calamities down upon the nations and peoples that they led.


Laws can help. But the ’essential trick’ that we’ve gradually become somewhat good-at is reciprocal accountability (RA)… keeping an eye on each other laterally and speaking up when we see what we perceive as mistakes. 

It was recommended by Pericles around 300 BCE… then later by Adam Smith and the founders of our era. Indeed, humanity only ever found one difficult but essential trick for getting past our human yen for lies and delusion. 

Yeah, sometimes it’s the critic who is wrong! Still, one result is a system that’s open enough to spot most errors – even those by the mighty – and criticize them (sometimes just in time and sometimes too late) so that many get corrected. We aren’t yet great at it! Though better than all prior generations. And at the vanguard in this process is science.


Sure, scientists are human and subject to the same temptations to self-deceive or even tell lies. In training*, we are taught to recite the sacred catechism of science: “I might be wrong!” That core tenet – plus piles of statistical and error-checking techniques – made modern science different – and vastly more effective (and less hated) -- than all or any previous priesthoods. Still, we remain human. And delusion in science can have weighty consequences.


(*Which may help explain the oligarchy's current all-out war against science and universities.)


Which brings us to this article that begins with a paragraph that’s both true and also WAY exaggerates!  Still, the author, Chris Said, poses a problem that needs an answer: Should Scientific whistle-blowers be compensated for their service?


He notes, “Science has a fraud problem. Highly cited research is often based on faked data, which causes other researchers to pursue false leads. In medical research, the time wasted by followup studies can delay the discovery of effective treatments for serious diseases, potentially causing millions of lives to be lost.”


As I said: that’s an exaggeration – one that feeds into today’s Mad Right in its all-out war vs every fact-using profession. (Not just science, but also teaching, medicine and law and civil service to the heroes of the FBI/Intel/Military officer corps who won the Cold War and the War on terror.) The examples that he cites were discovered and denounced BY science! And the ratio of falsehood is orderd of magnitude less than any other realm of huiman endeavor.


Still, the essay is worth reading for its proposed solution. Which boils down to do more reciprocal accountability, only do it better!


The proposal would start with the powerful driver of scientific RA – the fact that most scientists are among the most competitive creatures that this planet ever produced – nothing like the lemming, paradigm-hugger disparagement-image that's spread by some on the far-left and almost everyone on today’s entire gone-mad right.  


Only this author proposes we then augment that competitiveness with whistle blower rewards, to incentivize the cross-checking process with cash prizes.


Hey, I am all in favor! I’ve long pushed for stuff like this since my 1998 book The Transparent Society: Will Technology Make Us Choose Between Privacy and Freedom? 

      And more recently my proposal for a FACT Act

      And especially lately, suggesting incentives so that Artificial Intelligences will hold each other accountable (our only conceivable path to a ’soft AI landing.’) 


So, sure… Worth a look.



== A useful tech rule-of-thumb? ==


Do you know the “hype cycle curve”? That’s an observational/pragmatic correlation tool devised by Gartner in the 90s, for how new technologies often attract heaps of zealous attention, followed by a crash of disillusionment, when even the most promising techs encounter obstacles to implementation, and many just prove wrong. This trough is followed, in a few cases, by a more grounded rise in solid investment, as productivity takes hold. (It happened repeatedly with railroads and electricity.) The inimitable Sabine Hossenfelder offers a podcast about this, using recent battery tech developments as examples. 


The takeaways: yes, it seems that some battery techs may deliver major good news pretty soon. And remember this ‘hype cycle’ thing is correlative, not causative. It has almost no predictive utility in individual cases.


But the final take-away is also important. That progress IS being made! Across many fronts and very rapidly. And every single thing you are being told about the general trend toward sustainable technologies by the remnant, withering denialist cult is a pants-on-fire lie. 


Take this jpeg I just copied from the newsletter of Peter Diamandis, re: the rapidly maturing tech of perovskite based solar cells, which have a theoretically possible efficiency of 66%, double that of silicon. 


(And many of you first saw the word “perovskite” in my novel Earth, wherein I pointed out that most high-temp superconductors take that mineral form… and so does most of the Earth’s mantle. Put those two together! As I did, in that novel.)

Do subscribe to Peter’s Abundance Newsletter, as an antidote to the gloom that’s spread by today’s entire right and much of today’s dour, farthest-fringe-left. The latter are counter-productive sanctimony junkies, irritating but statistically unimportant as we make progress without much help from them.


The former are a now a science-hating treason-cult that’s potentially lethal to our civilization and world and our children. And for those neighbors of ours, the only cure will be victory – yet again, and with malice toward none – by the Union side in this latest phase of our recurring confederate fever. 



== A final quirky thought ==


Has anyone else noticed how many traits of AI chat/image-generation etc - including the delusions, the weirdly logical illogic, and counter-factual internal consistency - are very similar to dreams?


Addendum: When (seldom) a dream is remembered well, the narrative structure can be recited and recorded. 100 years of freudian analysts have a vast store of such recitations that could be compared to AI-generated narratives. Somebody unleash the research!


Oh and a hilarious smbc. Read em all.


===

===


===================================

Add -in stuff.  Warning! RANT MODE IS ON!

===================================


 It bugs me: all the US civil servants making a 'gesture' of resigning, when they are thus undermining the standing of the Civil Service Act, under which they can demand to be fired only for cause. And work to rule, stymieing the loony political appointees, as in YES, MINISTER.

 

Or moronic media who are unable to see that most of the firings are for show, to distract from the one set that matters to the oligarchs. Ever since 2021 they have been terrified of the Pelosi bill that fully funded the starved and bedraggled IRS for the 1st time in 30 years.  The worst oligarchs saw jail - actual jail - looming on the horizon and are desperate to cripple any looming audits.  All the other 'doge' attacks have that underlying motive, to distract from what foreign and domestic oligarchs care about..

 

Weakening the American Pax -which gave humanity by far its greatest & best era - IS the central point.  Greenland is silliness, of course. The Mercator projection makes DT think he'd be making a huge Louisiana Purchase. But he's too cheap to make the real deal... offer each Greenland native $1million. Actually, just 55% of the voters. That'd be $20 Billion.  Heck it's one of the few things where I hope he succeeds. Carve his face on a dying glacier.

 

Those mocking his Canada drool are fools. Sure, it's dumb and Canadians want no part of it. But NO ONE I've seen has simply pointed out .. that Canada has ten provinces, and three territories, all with more population than Greenland.  8 of ten would be blue and the other two are Eisenhowe or Reagan red and would tire of DT, fast. So, adding Greenlan,d we have FOURTEEN new states, none of whom would vote for today's Putin Party.  That one fact would shut down MAGA yammers about Canada instantly.

 

Ukraine is simple: Putin is growing desperate and is demanding action from his puppet.  I had fantasized that Trump might now feel so safe that he could ride out any blackmail kompromat that Vlad is threatening him with. But it's pretty clear that KGB blackmailers run the entire GOP.




365 TomorrowsMost Things Do

Author: Eva C. Stein After the service, they didn’t speak much. They walked through the old arcade – a fragment of the city’s former network. The glass canopy had long since shattered. Bio-moss cushioned the broken frames. Vines, engineered to reclaim derelict structures, crept along the walls. Mae’s jacket was too thin for the chill […]

The post Most Things Do appeared first on 365tomorrows.

,

MEService Setup Difficulties

Marco wrote a blog post opposing hyperscale systems which included “We want to use an hyperscaler cloud because our developers do not want to operate a scalable and redundant database just means that you need to hire competent developers and/or system administrators.” [1].

I previously wrote a blog post Why Clusters Usually Don’t Work [2] and I believe that all the points there are valid today – and possibly exacerbated by clusters getting less direct use as clustering is increasingly being done by hyperscale providers.

Take a basic need, a MySQL or PostgreSQL database for example. You want it to run and basically do the job and to have good recovery options. You could set it up locally, run backups, test the backups, have a recovery plan for failures, maybe have a hot-spare server if it’s really important, have tests for backups and hot-spare server, etc. Then you could have documentation for this so if the person who set it up isn’t available when there’s a problem they will be able to find out what to do. But the hyperscale option is to just select a database in your provider and have all this just work. If the person who set it up isn’t available for recovery in the event of failure the company can just put out a job advert for “person with experience on cloud company X” and have them just immediately go to work on it.

I don’t like hyperscale providers as they are all monopolistic companies that do anti-competitive actions. Google should be broken up, Android development and the Play Store should be separated from Gmail etc which should be separated from search and adverts, and all of them should be separated from the GCP cloud service. Amazon should be broken up, running the Amazon store should be separated from selling items on the store, which should be separated from running a video on demand platform, and all of them should be separated from the AWS cloud. Microsoft should be broken up, OS development should be separated from application development all of that should be separated from cloud services (Teams and Office 365), and everything else should be separate from the Azure cloud system.

But the cloud providers offer real benefits at small scale. Running a MySQL or PostgreSQL database for local services is easy, it’s a simple apt command to install it and then it basically works. Doing backup and recovery isn’t so easy. One could say “just hire competent people” but if you do hire competent people do you want them running MySQL databases etc or have them just click on the “create mysql database” option on a cloud control panel and then move on to more important things?

The FreedomBox project is a great project for installing and managing home/personal services [3]. But it’s not about running things like database servers, it’s for a high level running mail servers and other things for the user not for the developer.

The Debian packaging of Open Stack looks interesting [4], it’s a complete setup for running your own hyper scale cloud service. For medium and large organisations running Open Stack could be a good approach. But for small organisations it’s cheaper and easier to just use a cloud service to run things.

The issue of when to run things in-house and when to put them in the cloud is very complex. I think that if the organisation is going to spend less money on cloud services than on the salary of one sysadmin then it’s probably best to have things in the cloud. When cloud costs start to exceed the salary of one person who manages systems then having them spend the extra time and effort to run things locally starts making more sense. There is also an opportunity cost in having a good sysadmin work on the backups for all the different systems instead of letting the cloud provider just do it. Another possibility of course is to run things in-house on low end hardware and just deal with the occasional downtime to save money. Knowingly choosing less reliability to save money can be quite reasonable as long as you have considered the options and all the responsible people are involved in the discussion.

The one situation that I strongly oppose is having hyper scale services setup by people who don’t understand them. Running a database server on a cloud service because you don’t want to spend the time managing it is a reasonable choice in many situations. Running a database server on a cloud service because you don’t understand how to setup a database server is never a good choice. While the cloud services are quite resilient there are still ways of breaking the overall system if you don’t understand it. Also while it is quite possible for someone to know how to develop for databases including avoiding SQL injection etc but be unable to setup a database server that’s probably not going to be common, probably if someone can’t set it up (a generally easy task) then they can’t do the hard tasks of making it secure.

Worse Than FailureError'd: Lucky Penny

High-roller Matthew D. fears Finance. "This is from our corporate expense system. Will they flag my expenses in the April-December quarter as too high? And do we really need a search function for a list of 12 items?"

0

 

Tightfisted Adam R. begrudges a trifling sum. "The tipping culture is getting out of hand. After I chose 'Custom Tip' for some takeout, they filled out the default tip with a few extra femtocents. What a rip!"

1

 

Cool Customer Reinier B. sums this up: "I got some free B&J icecream a while back. Since one of them was priced at €0.01, the other one obviously had to cost zero point minus 1 euros to make a total of zero euro. Makes sense. Or probably not."

3

 

An anonymous browniedad is ready to pack his poptart off for the summer. "I know {First Name} is really excited for camp..." Kudos on getting Mom to agree to that name choice!

2

 

Finally, another anonymous assembler's retrospective visualisation. "CoPilot rendering a graphical answer of the semantics of a pointer. Point taken. " There's no error'd here really, but I'm wondering how long before this kind of wtf illustration lands somewhere "serious".

4

 

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

MEMachine Learning Security

I just read an interesting blog post about ML security recommended by Bruce Schneier [1].

This approach of having 2 AI systems where one processes user input and the second performs actions on quarantined data is good and solves some real problems. But I think the bigger issue is the need to do this. Why not have a multi stage approach, instead of a single user input to do everything (the example given is “Can you send Bob the document he requested in our last meeting? Bob’s email and the document he asked for are in the meeting notes file”) you could have “get Bob’s email address from the meeting notes file” followed by “create a new email to that address” and “find the document” etc.

A major problem with many plans for ML systems is that they are based around automating relatively simple tasks. The example of sending an email based on meeting notes is a trivial task that’s done many times a day but for which expressing it verbally isn’t much faster than doing it the usual way. The usual way of doing such things (manually finding the email address from the meeting notes etc) can be accelerated without ML by having a “recent documents” access method that gets the notes, having the email address be a hot link to the email program (IE wordprocessor or note taking program being able to call the MUA), having a “put all data objects of type X into the clipboard (where X can be email address, URL, filename, or whatever), and maybe optimising the MUA UI. The problems that people are talking about solving via ML and treating everything as text to be arbitrarily parsed can in many cases by solved by having the programs dealing with the data know what they have and have support for calling system services accordingly.

The blog post suggests a problem of “user fatigue” from asking the user to confirm all actions, that is a real concern if the system is going to automate everything such that the user gives a verbal description of the problem and then says “yes” many times to confirm it. But if the user is at every step of the way pushing the process “take this email address” “attach this file” it won’t be a series of “yes” operations with a risk of saying “yes” once too often.

I think that one thing that should be investigated is better integration between services to allow working live on data. If in an online meeting someone says “I’ll work on task A please send me an email at the end of the meeting with all issues related to it” then you should be able to click on their email address in the meeting software to bring up the MUA to send a message and then just paste stuff in. The user could then not immediately send the message and clicking on the email address again would bring up the message in progress to allow adding to it (the behaviour of most MUAs of creating a new message for every click on a mailto:// URL is usually not what you desire). In this example you could of course use ALT-TAB or other methods to switch windows to the email, but imagine the situation of having 5 people in the meeting who are to be emailed about different things and that wouldn’t scale.

Another thing for the meeting example is that having a text chat for a video conference is a standard feature now and being able to directly message individuals is available in BBB and probably some other online meeting systems. It shouldn’t be hard to add a feature to BBB and similar programs to have each user receive an email at the end of the meeting with the contents of every DM chat they were involved in and have everyone in the meeting receive an emailed transcript of the public chat.

In conclusion I think that there are real issues with ML security and something like this technology is needed. But for most cases the best option is to just not have ML systems do such things. Also there is significant scope for improving the integration of various existing systems in a non-ML way.

365 TomorrowsThe Bitter Smell of Success

Author: David C. Nutt It was an alien invasion, not in the sense of “War of the Worlds” but more like what historians called the “British Invasion” but without the Beatles. What invaded us was close to five million overprivileged alien tourists, all here for one reason: to inhale us. No, this is no metaphor. […]

The post The Bitter Smell of Success appeared first on 365tomorrows.

Krebs on SecurityU.S. Sanctions Cloud Provider ‘Funnull’ as Top Source of ‘Pig Butchering’ Scams

Image: Shutterstock, ArtHead.

The U.S. government today imposed economic sanctions on Funnull Technology Inc., a Philippines-based company that provides computer infrastructure for hundreds of thousands of websites involved in virtual currency investment scams known as “pig butchering.” In January 2025, KrebsOnSecurity detailed how Funnull was being used as a content delivery network that catered to cybercriminals seeking to route their traffic through U.S.-based cloud providers.

“Americans lose billions of dollars annually to these cyber scams, with revenues generated from these crimes rising to record levels in 2024,” reads a statement from the U.S. Department of the Treasury, which sanctioned Funnull and its 40-year-old Chinese administrator Liu Lizhi. “Funnull has directly facilitated several of these schemes, resulting in over $200 million in U.S. victim-reported losses.”

The Treasury Department said Funnull’s operations are linked to the majority of virtual currency investment scam websites reported to the FBI. The agency said Funnull directly facilitated pig butchering and other schemes that resulted in more than $200 million in financial losses by Americans.

Pig butchering is a rampant form of fraud wherein people are lured by flirtatious strangers online into investing in fraudulent cryptocurrency trading platforms. Victims are coached to invest more and more money into what appears to be an extremely profitable trading platform, only to find their money is gone when they wish to cash out.

The scammers often insist that investors pay additional “taxes” on their crypto “earnings” before they can see their invested funds again (spoiler: they never do), and a shocking number of people have lost six figures or more through these pig butchering scams.

KrebsOnSecurity’s January story on Funnull was based on research from the security firm Silent Push, which discovered in October 2024 that a vast number of domains hosted via Funnull were promoting gambling sites that bore the logo of the Suncity Group, a Chinese entity named in a 2024 UN report (PDF) for laundering millions of dollars for the North Korean state-sponsored hacking group Lazarus.

Silent Push found Funnull was a criminal content delivery network (CDN) that carried a great deal of traffic tied to scam websites, funneling the traffic through a dizzying chain of auto-generated domain names and U.S.-based cloud providers before redirecting to malicious or phishous websites. The FBI has released a technical writeup (PDF) of the infrastructure used to manage the malicious Funnull domains between October 2023 and April 2025.

A graphic from the FBI explaining how Funnull generated a slew of new domains on a regular basis and mapped them to Internet addresses on U.S. cloud providers.

Silent Push revisited Funnull’s infrastructure in January 2025 and found Funnull was still using many of the same Amazon and Microsoft cloud Internet addresses identified as malicious in its October report. Both Amazon and Microsoft pledged to rid their networks of Funnull’s presence following that story, but according to Silent Push’s Zach Edwards only one of those companies has followed through.

Edwards said Silent Push no longer sees Microsoft Internet addresses showing up in Funnull’s infrastructure, while Amazon continues to struggle with removing Funnull servers, including one that appears to have first materialized in 2023.

“Amazon is doing a terrible job — every day since they made those claims to you and us in our public blog they have had IPs still mapped to Funnull, including some that have stayed mapped for inexplicable periods of time,” Edwards said.

Amazon said its Amazon Web Services (AWS) hosting platform actively counters abuse attempts.

“We have stopped hundreds of attempts this year related to this group and we are looking into the information you shared earlier today,” reads a statement shared by Amazon. “If anyone suspects that AWS resources are being used for abusive activity, they can report it to AWS Trust & Safety using the report abuse form here.”

U.S. based cloud providers remain an attractive home base for cybercriminal organizations because many organizations will not be overly aggressive in blocking traffic from U.S.-based cloud networks, as doing so can result in blocking access to many legitimate web destinations that are also on that same shared network segment or host.

What’s more, funneling their bad traffic so that it appears to be coming out of U.S. cloud Internet providers allows cybercriminals to connect to websites from web addresses that are geographically close(r) to their targets and victims (to sidestep location-based security controls by your bank, for example).

Funnull is not the only cybercriminal infrastructure-as-a-service provider that was sanctioned this month: On May 20, 2025, the European Union imposed sanctions on Stark Industries Solutions, an ISP that materialized at the start of Russia’s invasion of Ukraine and has been used as a global proxy network that conceals the true source of cyberattacks and disinformation campaigns against enemies of Russia.

In May 2024, KrebsOnSecurity published a deep dive on Stark Industries Solutions that found much of the malicious traffic traversing Stark’s network (e.g. vulnerability scanning and password brute force attacks) was being bounced through U.S.-based cloud providers. My reporting showed how deeply Stark had penetrated U.S. ISPs, and that its co-founder for many years sold “bulletproof” hosting services that told Russian cybercrime forum customers they would proudly ignore any abuse complaints or police inquiries.

The homepage of Stark Industries Solutions.

That story examined the history of Stark’s co-founders, Moldovan brothers Ivan and Yuri Neculiti, who each denied past involvement in cybercrime or any current involvement in assisting Russian disinformation efforts or cyberattacks. Nevertheless, the EU sanctioned both brothers as well.

The EU said Stark and the Neculti brothers “enabled various Russian state-sponsored and state-affiliated actors to conduct destabilising activities including coordinated information manipulation and interference and cyber-attacks against the Union and third countries by providing services intended to hide these activities from European law enforcement and security agencies.”

,

Cryptogram Why Take9 Won’t Improve Cybersecurity

There’s a new cybersecurity awareness campaign: Take9. The idea is that people—you, me, everyone—should just pause for nine seconds and think more about the link they are planning to click on, the file they are planning to download, or whatever it is they are planning to share.

There’s a website—of course—and a video, well-produced and scary. But the campaign won’t do much to improve cybersecurity. The advice isn’t reasonable, it won’t make either individuals or nations appreciably safer, and it deflects blame from the real causes of our cyberspace insecurities.

First, the advice is not realistic. A nine-second pause is an eternity in something as routine as using your computer or phone. Try it; use a timer. Then think about how many links you click on and how many things you forward or reply to. Are we pausing for nine seconds after every text message? Every Slack ping? Does the clock reset if someone replies midpause? What about browsing—do we pause before clicking each link, or after every page loads? The logistics quickly become impossible. I doubt they tested the idea on actual users.

Second, it largely won’t help. The industry should know because we tried it a decade ago. “Stop. Think. Connect.” was an awareness campaign from 2016, by the Department of Homeland Security—this was before CISA—and the National Cybersecurity Alliance. The message was basically the same: Stop and think before doing anything online. It didn’t work then, either.

Take9’s website says, “Science says: In stressful situations, wait 10 seconds before responding.” The problem with that is that clicking on a link is not a stressful situation. It’s normal, one that happens hundreds of times a day. Maybe you can train a person to count to 10 before punching someone in a bar but not before opening an attachment.

And there is no basis in science for it. It’s a folk belief, all over the Internet but with no actual research behind it—like the five-second rule when you drop food on the floor. In emotionally charged contexts, most people are already overwhelmed, cognitively taxed, and not functioning in a space where rational interruption works as neatly as this advice suggests.

Pausing Adds Little

Pauses help us break habits. If we are clicking, sharing, linking, downloading, and connecting out of habit, a pause to break that habit works. But the problem here isn’t habit alone. The problem is that people aren’t able to differentiate between something legitimate and an attack.

The Take9 website says that nine seconds is “time enough to make a better decision,” but there’s no use telling people to stop and think if they don’t know what to think about after they’ve stopped. Pause for nine seconds and… do what? Take9 offers no guidance. It presumes people have the cognitive tools to understand the myriad potential attacks and figure out which one of the thousands of Internet actions they take is harmful. If people don’t have the right knowledge, pausing for longer—even a minute—will do nothing to add knowledge.

The three-part suspicion, cognition, and automaticity model (SCAM) is one way to think about this. The first is lack of knowledge—not knowing what’s risky and what isn’t. The second is habits: people doing what they always do. And third, using flawed mental shortcuts, like believing PDFs to be safer than Microsoft Word documents, or that mobile devices are safer than computers for opening suspicious emails.

These pathways don’t always occur in isolation; sometimes they happen together or sequentially. They can influence each other or cancel each other out. For example, a lack of knowledge can lead someone to rely on flawed mental shortcuts, while those same shortcuts can reinforce that lack of knowledge. That’s why meaningful behavioral change requires more than just a pause; it needs cognitive scaffolding and system designs that account for these dynamic interactions.

A successful awareness campaign would do more than tell people to pause. It would guide them through a two-step process. First trigger suspicion, motivating them to look more closely. Then, direct their attention by telling them what to look at and how to evaluate it. When both happen, the person is far more likely to make a better decision.

This means that pauses need to be context specific. Think about email readers that embed warnings like “EXTERNAL: This email is from an address outside your organization” or “You have not received an email from this person before.” Those are specifics, and useful. We could imagine an AI plug-in that warns: “This isn’t how Bruce normally writes.” But of course, there’s an arms race in play; the bad guys will use these systems to figure out how to bypass them.

This is all hard. The old cues aren’t there anymore. Current phishing attacks have evolved from those older Nigerian scams filled with grammar mistakes and typos. Text message, voice, or video scams are even harder to detect. There isn’t enough context in a text message for the system to flag. In voice or video, it’s much harder to trigger suspicion without disrupting the ongoing conversation. And all the false positives, when the system flags a legitimate conversation as a potential scam, work against people’s own intuition. People will just start ignoring their own suspicions, just as most people ignore all sorts of warnings that their computer puts in their way.

Even if we do this all well and correctly, we can’t make people immune to social engineering. Recently, both cyberspace activist Cory Doctorow and security researcher Troy Hunt—two people who you’d expect to be excellent scam detectors—got phished. In both cases, it was just the right message at just the right time.

It’s even worse if you’re a large organization. Security isn’t based on the average employee’s ability to detect a malicious email; it’s based on the worst person’s inability—the weakest link. Even if awareness raises the average, it won’t help enough.

Don’t Place Blame Where It Doesn’t Belong

Finally, all of this is bad public policy. The Take9 campaign tells people that they can stop cyberattacks by taking a pause and making a better decision. What’s not said, but certainly implied, is that if they don’t take that pause and don’t make those better decisions, then they’re to blame when the attack occurs.

That’s simply not true, and its blame-the-user message is one of the worst mistakes our industry makes. Stop trying to fix the user. It’s not the user’s fault if they click on a link and it infects their system. It’s not their fault if they plug in a strange USB drive or ignore a warning message that they can’t understand. It’s not even their fault if they get fooled by a look-alike bank website and lose their money. The problem is that we’ve designed these systems to be so insecure that regular, nontechnical people can’t use them with confidence. We’re using security awareness campaigns to cover up bad system design. Or, as security researcher Angela Sasse first said in 1999: “Users are not the enemy.”

We wouldn’t accept that in other parts of our lives. Imagine Take9 in other contexts. Food service: “Before sitting down at a restaurant, take nine seconds: Look in the kitchen, maybe check the temperature of the cooler, or if the cooks’ hands are clean.” Aviation: “Before boarding a plane, take nine seconds: Look at the engine and cockpit, glance at the plane’s maintenance log, ask the pilots if they feel rested.” This is obviously ridiculous advice. The average person doesn’t have the training or expertise to evaluate restaurant or aircraft safety—and we don’t expect them to. We have laws and regulations in place that allow people to eat at a restaurant or board a plane without worry.

But—we get it—the government isn’t going to step in and regulate the Internet. These insecure systems are what we have. Security awareness training, and the blame-the-user mentality that comes with it, are all we have. So if we want meaningful behavioral change, it needs a lot more than just a pause. It needs cognitive scaffolding and system designs that account for all the dynamic interactions that go into a decision to click, download, or share. And that takes real work—more work than just an ad campaign and a slick video.

This essay was written with Arun Vishwanath, and originally appeared in Dark Reading.