Sean sends us a one-line function that is a delight, if by delight you mean "horror". You'll be shocked to know it's PHP.

function proget(){foreach($_GET as $k=>$v){if($k=="h"){$GLOBALS["h"]=1;}$p=explode(",",$k);}return($p);} //function to process GET headers

Based on the comment, proget is a shorthand for process_get_parameters. Which is sort of what it does. Sort of.

Let's go through this. We iterate across our $_GET parameters using $k for the key, $v for the value, but we never reference the value so forget it exists. We're iterating across every key. The first thing we check is if a key "h" exists. We don't look at its value, we just check if it exists, and if it does, we set a global variable. And this, right here, is enough for this to be a WTF. The logic of "set a global variable based on the existence of a query parameter regardless of the value of the query parameter" is… a lot. But then, somehow, this actually gets more out there.

We explode the key on commas (explode being PHP's much cooler name for split), which implies… our keys may be lists of values? Which I feel like is an example of someone not understanding what a "key" is. But worse than that, we just do this for every key, and return the results of performing that operation on the last key. Which means that if this function is doing anything at all, it's entirely dependent on the order of the keys. Which, PHP does order the keys by the order they're added, which I take to mean that if the URL has query params like ?foo=1&h=0&a,b,c,d=wtf. Or, if we're being picky about encoding, ?foo=1&h=0&a%2Cb%2Cc%2Cd=wtf. The only good news here is that PHP handles the encoding/decoding for you, so the explode will work as expected.

This is the kind of bad code that leaves me with lots of questions, and I'm not sure I want any of the answers. How did this happen, and why are questions best left unanswered, because I think the answers might cause more harm.

Author: A.R. McHugh Diamonds won her as a child. Looking at sedimentary quartz under 200x magnification, she was fascinated by the possibility of so much clarity, such mineral perfection. Somewhere between her mother’s flashing ring and her father’s relentless pressure to produce better grades and faster times, a harder carapace around her teenage soul and […]

The post Sand Diamond appeared first on 365tomorrows.

Another minor update 0.3.12 for our nanotime package is now on CRAN. nanotime relies on the RcppCCTZ package (as well as the RcppDate package for additional C++ operations) and offers efficient high(er) resolution time parsing and formatting up to nanosecond resolution, using the bit64 package for the actual integer64 arithmetic. Initially implemented using the S3 system, it has benefitted greatly from a rigorous refactoring by Leonardo who not only rejigged nanotime internals in S4 but also added new S4 types for periods, intervals and durations.

This release responds to recent CRAN nag of also suggesting packages used in demos, makes a small update to the CI script, and improves one version comparison for conditional code.

The NEWS snippet below has the fuller details.

Changes in version 0.3.12 (2025-04-02)

• Update continuous integration to use r-ci action with bootstrap

• Add ggplot2 to Suggests due to use in demo/ which is now scanned

• Refine a version comparison for R 4.5.0 to be greater or equal

Thanks to my CRANberries, there is a diffstat report for this release. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository – and all documentation is provided at the nanotime documentation site.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

If you’ve ever taken a computer security class, you’ve probably learned about the three legs of computer security—confidentiality, integrity, and availability—known as the CIA triad. When we talk about a system being secure, that’s what we’re referring to. All are important, but to different degrees in different contexts. In a world populated by artificial intelligence (AI) systems and artificial intelligent agents, integrity will be paramount.

What is data integrity? It’s ensuring that no one can modify data—that’s the security angle—but it’s much more than that. It encompasses accuracy, completeness, and quality of data—all over both time and space. It’s preventing accidental data loss; the “undo” button is a primitive integrity measure. It’s also making sure that data is accurate when it’s collected—that it comes from a trustworthy source, that nothing important is missing, and that it doesn’t change as it moves from format to format. The ability to restart your computer is another integrity measure.

The CIA triad has evolved with the Internet. The first iteration of the Web—Web 1.0 of the 1990s and early 2000s—prioritized availability. This era saw organizations and individuals rush to digitize their content, creating what has become an unprecedented repository of human knowledge. Organizations worldwide established their digital presence, leading to massive digitization projects where quantity took precedence over quality. The emphasis on making information available overshadowed other concerns.

As Web technologies matured, the focus shifted to protecting the vast amounts of data flowing through online systems. This is Web 2.0: the Internet of today. Interactive features and user-generated content transformed the Web from a read-only medium to a participatory platform. The increase in personal data, and the emergence of interactive platforms for e-commerce, social media, and online everything demanded both data protection and user privacy. Confidentiality became paramount.

We stand at the threshold of a new Web paradigm: Web 3.0. This is a distributed, decentralized, intelligent Web. Peer-to-peer social-networking systems promise to break the tech monopolies’ control on how we interact with each other. Tim Berners-Lee’s open W3C protocol, Solid, represents a fundamental shift in how we think about data ownership and control. A future filled with AI agents requires verifiable, trustworthy personal data and computation. In this world, data integrity takes center stage.

For example, the 5G communications revolution isn’t just about faster access to videos; it’s about Internet-connected things talking to other Internet-connected things without our intervention. Without data integrity, for example, there’s no real-time car-to-car communications about road movements and conditions. There’s no drone swarm coordination, smart power grid, or reliable mesh networking. And there’s no way to securely empower AI agents.

In particular, AI systems require robust integrity controls because of how they process data. This means technical controls to ensure data is accurate, that its meaning is preserved as it is processed, that it produces reliable results, and that humans can reliably alter it when it’s wrong. Just as a scientific instrument must be calibrated to measure reality accurately, AI systems need integrity controls that preserve the connection between their data and ground truth.

This goes beyond preventing data tampering. It means building systems that maintain verifiable chains of trust between their inputs, processing, and outputs, so humans can understand and validate what the AI is doing. AI systems need clean, consistent, and verifiable control processes to learn and make decisions effectively. Without this foundation of verifiable truth, AI systems risk becoming a series of opaque boxes.

Recent history provides many sobering examples of integrity failures that naturally undermine public trust in AI systems. Machine-learning (ML) models trained without thought on expansive datasets have produced predictably biased results in hiring systems. Autonomous vehicles with incorrect data have made incorrect—and fatal—decisions. Medical diagnosis systems have given flawed recommendations without being able to explain themselves. A lack of integrity controls undermines AI systems and harms people who depend on them.

They also highlight how AI integrity failures can manifest at multiple levels of system operation. At the training level, data may be subtly corrupted or biased even before model development begins. At the model level, mathematical foundations and training processes can introduce new integrity issues even with clean data. During execution, environmental changes and runtime modifications can corrupt previously valid models. And at the output level, the challenge of verifying AI-generated content and tracking it through system chains creates new integrity concerns. Each level compounds the challenges of the ones before it, ultimately manifesting in human costs, such as reinforced biases and diminished agency.

Think of it like protecting a house. You don’t just lock a door; you also use safe concrete foundations, sturdy framing, a durable roof, secure double-pane windows, and maybe motion-sensor cameras. Similarly, we need digital security at every layer to ensure the whole system can be trusted.

This layered approach to understanding security becomes increasingly critical as AI systems grow in complexity and autonomy, particularly with large language models (LLMs) and deep-learning systems making high-stakes decisions. We need to verify the integrity of each layer when building and deploying digital systems that impact human lives and societal outcomes.

At the foundation level, bits are stored in computer hardware. This represents the most basic encoding of our data, model weights, and computational instructions. The next layer up is the file system architecture: the way those binary sequences are organized into structured files and directories that a computer can efficiently access and process. In AI systems, this includes how we store and organize training data, model checkpoints, and hyperparameter configurations.

On top of that are the application layers—the programs and frameworks, such as PyTorch and TensorFlow, that allow us to train models, process data, and generate outputs. This layer handles the complex mathematics of neural networks, gradient descent, and other ML operations.

Finally, at the user-interface level, we have visualization and interaction systems—what humans actually see and engage with. For AI systems, this could be everything from confidence scores and prediction probabilities to generated text and images or autonomous robot movements.

Why does this layered perspective matter? Vulnerabilities and integrity issues can manifest at any level, so understanding these layers helps security experts and AI researchers perform comprehensive threat modeling. This enables the implementation of defense-in-depth strategies—from cryptographic verification of training data to robust model architectures to interpretable outputs. This multi-layered security approach becomes especially crucial as AI systems take on more autonomous decision-making roles in critical domains such as healthcare, finance, and public safety. We must ensure integrity and reliability at every level of the stack.

The risks of deploying AI without proper integrity control measures are severe and often underappreciated. When AI systems operate without sufficient security measures to handle corrupted or manipulated data, they can produce subtly flawed outputs that appear valid on the surface. The failures can cascade through interconnected systems, amplifying errors and biases. Without proper integrity controls, an AI system might train on polluted data, make decisions based on misleading assumptions, or have outputs altered without detection. The results of this can range from degraded performance to catastrophic failures.

We see four areas where integrity is paramount in this Web 3.0 world. The first is granular access, which allows users and organizations to maintain precise control over who can access and modify what information and for what purposes. The second is authentication—much more nuanced than the simple “Who are you?” authentication mechanisms of today—which ensures that data access is properly verified and authorized at every step. The third is transparent data ownership, which allows data owners to know when and how their data is used and creates an auditable trail of data providence. Finally, the fourth is access standardization: common interfaces and protocols that enable consistent data access while maintaining security.

Luckily, we’re not starting from scratch. There are open W3C protocols that address some of this: decentralized identifiers for verifiable digital identity, the verifiable credentials data model for expressing digital credentials, ActivityPub for decentralized social networking (that’s what Mastodon uses), Solid for distributed data storage and retrieval, and WebAuthn for strong authentication standards. By providing standardized ways to verify data provenance and maintain data integrity throughout its lifecycle, Web 3.0 creates the trusted environment that AI systems require to operate reliably. This architectural leap for integrity control in the hands of users helps ensure that data remains trustworthy from generation and collection through processing and storage.

Integrity is essential to trust, on both technical and human levels. Looking forward, integrity controls will fundamentally shape AI development by moving from optional features to core architectural requirements, much as SSL certificates evolved from a banking luxury to a baseline expectation for any Web service.

Web 3.0 protocols can build integrity controls into their foundation, creating a more reliable infrastructure for AI systems. Today, we take availability for granted; anything less than 100% uptime for critical websites is intolerable. In the future, we will need the same assurances for integrity. Success will require following practical guidelines for maintaining data integrity throughout the AI lifecycle—from data collection through model training and finally to deployment, use, and evolution. These guidelines will address not just technical controls but also governance structures and human oversight, similar to how privacy policies evolved from legal boilerplate into comprehensive frameworks for data stewardship. Common standards and protocols, developed through industry collaboration and regulatory frameworks, will ensure consistent integrity controls across different AI systems and applications.

Just as the HTTPS protocol created a foundation for trusted e-commerce, it’s time for new integrity-focused standards to enable the trusted AI services of tomorrow.

This essay was written with Davi Ottenheimer, and originally appeared in Communications of the ACM.

Today's anonymous submitter worked for a "large, US-based, e-commerce company." This particular company was, some time back, looking to save money, and like so many companies do, that meant hiring offshore contractors.

Now, I want to stress, there's certainly nothing magical about national borders which turns software engineers into incompetents. The reality is simply that contractors never have their client's best interests at heart; they only want to be good enough to complete their contract. This gets multiplied by the contracting firm's desire to maximize their profits by keeping their contractors as booked as possible. And it gets further multiplied by the remoteness and siloing of the interaction, especially across timezones. Often, the customer sends out requirements, and three months later gets a finished feature, with no more contact than that- and it never goes well.

All that said, let's look at some SQL Server code. It's long, so we'll take it in chunks.

-- ===============================================================================
-- Author     : Ignacius Ignoramus
-- Create date: 04-12-2020
-- Description:	SP of Getting Discrepancy of Allocation Reconciliation Snapshot
-- ===============================================================================

That the comment reinforces that this is an "SP", aka stored procedure, is already not my favorite thing to see. The description is certainly made up of words, and I think I get the gist.

ALTER PROCEDURE [dbo].[Discrepency]
	(
		@startDate DATETIME,
		@endDate DATETIME
	)
AS

BEGIN

Nothing really to see here; it's easy to see that we're going to run a query for a date range. That's fine and common.

	DECLARE @tblReturn TABLE
	(
		intOrderItemId	   INT
	)

Hmm. T-SQL lets you define table variables, which are exactly what they sound like. It's a local variable in this procedure, that acts like a table. You can insert/update/delete/query it. The vague name is a little sketch, and the fact that it holds only one field also makes me go "hmmm", but this isn't bad.

	DECLARE @tblReturn1 TABLE
	(
		intOrderItemId	   INT
	)

Uh oh.

	DECLARE @tblReturn2 TABLE
	(
		intOrderItemId	   INT
	)

Oh no.

	DECLARE @tblReturn3 TABLE
	(
		intOrderItemId	   INT
	)

Oh no no no.

	DECLARE @tblReturn4 TABLE
	(
		intOrderItemId	   INT
	)

This doesn't bode well.

So they've declared five variables called tblReturn, that all hold the same data structure.

What happens next? This next block is gonna be long.

	INSERT INTO @tblReturn --(intOrderItemId) VALUES (@_ordersToBeAllocated)

	/* OrderItemsPlaced */		

		select 		
		intOrderItemId
		from CompanyDatabase..Orders o
		inner join CompanyDatabase..OrderItems oi on oi.intOrderId = o.intOrderId
		where o.dtmTimeStamp between @startDate and  @endDate


		AND intOrderItemId Not In 
		(

		/* _itemsOnBackorder */

		select intOrderItemId			
		from CompanyDatabase..OrderItems oi
		inner join CompanyDatabase..Orders o on o.intOrderId = oi.intOrderId
		where o.dtmTimeStamp between @startDate and  @endDate
		and oi.strstatus='backordered' 
		)

		AND intOrderItemId Not In 
		(

		/* _itemsOnHold */

		select intOrderItemId			
		from CompanyDatabase..OrderItems oi
		inner join CompanyDatabase..Orders o on o.intOrderId = oi.intOrderId
		where o.dtmTimeStamp between @startDate and  @endDate
		and o.strstatus='ONHOLD'
		and oi.strStatus <> 'BACKORDERED' 
		)

		AND intOrderItemId Not In 
		(

		/* _itemsOnReview */

		select  intOrderItemId			
		from CompanyDatabase..OrderItems oi
		inner join CompanyDatabase..Orders o on o.intOrderId = oi.intOrderId
		where o.dtmTimeStamp between @startDate and  @endDate 
		and o.strstatus='REVIEW' 
		and oi.strStatus <> 'BACKORDERED'
		)

		AND intOrderItemId Not In 
		(

		/*_itemsOnPending*/

		select  intOrderItemId			
		from CompanyDatabase..OrderItems oi
		inner join CompanyDatabase..Orders o on o.intOrderId = oi.intOrderId
		where o.dtmTimeStamp between @startDate and  @endDate
		and o.strstatus='PENDING'
		and oi.strStatus <> 'BACKORDERED'
		)

		AND intOrderItemId Not In 
		(

		/*_itemsCancelled */

		select  intOrderItemId			
		from CompanyDatabase..OrderItems oi
		inner join CompanyDatabase..Orders o on o.intOrderId = oi.intOrderId
		where o.dtmTimeStamp between @startDate and  @endDate
		and oi.strstatus='CANCELLED' 
		)

We insert into @tblReturn the result of a query, and this query relies heavily on using a big pile of subqueries to decide if a record should be included in the output- but these subqueries all query the same tables as the root query. I'm fairly certain this could be a simple join with a pretty readable where clause, but I'm also not going to sit here and rewrite it right now, we've got a lot more query to look at.

INSERT INTO @tblReturn1

		
		/* _backOrderItemsReleased */	

		select  intOrderItemId			
		from CompanyDatabase..OrderItems oi
		inner join CompanyDatabase..orders o on o.intorderid = oi.intorderid
		where oi.intOrderItemid in (
			  select intRecordID 
			  from CompanyDatabase..StatusChangeLog
			  where strRecordType = 'OrderItem'
			  and strOldStatus in ('BACKORDERED')
			  and strNewStatus in ('NEW', 'RECYCLED')
			  and dtmTimeStamp between @startDate and  @endDate  
		)
		and o.dtmTimeStamp < @startDate
		

		UNION
		(
			/*_pendingHoldItemsReleased*/

			select  intOrderItemId					
			from CompanyDatabase..OrderItems oi
			inner join CompanyDatabase..orders o on o.intorderid = oi.intorderid
			where oi.intOrderID in (
				  select intRecordID 
				  from CompanyDatabase..StatusChangeLog
				  where strRecordType = 'Order'
				  and strOldStatus in ('REVIEW', 'ONHOLD', 'PENDING')
				  and strNewStatus in ('NEW', 'PROCESSING')
				  and dtmTimeStamp between @startDate and  @endDate  
			)
			and o.dtmTimeStamp < @startDate
			
		)

		UNION

		/* _reallocationsowingtonostock */	
		(
			select oi.intOrderItemID				   	 
			from CompanyDatabase.dbo.StatusChangeLog 
			inner join CompanyDatabase.dbo.OrderItems oi on oi.intOrderItemID = CompanyDatabase.dbo.StatusChangeLog.intRecordID
			inner join CompanyDatabase.dbo.Orders o on o.intOrderId = oi.intOrderId  

			where strOldStatus = 'RECYCLED' and strNewStatus = 'ALLOCATED' 
			and CompanyDatabase.dbo.StatusChangeLog.dtmTimestamp > @endDate and 
			strRecordType = 'OrderItem'
			and intRecordId in 
			(
			  select intRecordId from CompanyDatabase.dbo.StatusChangeLog 
			  where strOldStatus = 'ALLOCATED' and strNewStatus = 'RECYCLED' 
			  and strRecordType = 'OrderItem'
			  and CompanyDatabase.dbo.StatusChangeLog.dtmTimestamp between @startDate and  @endDate  
			)  
		)

Okay, just some unions with more subquery filtering. More of the same. It's the next one that makes this special.

INSERT INTO @tblReturn2

	SELECT intOrderItemId FROM @tblReturn 
	
	UNION

	SELECT intOrderItemId FROM @tblReturn1

Ah, here's the stuff. This is just bonkers. If the goal is to combine the results of these queries into a single table, you could just insert into one table the whole time.

But we know that there are 5 of these tables, so why are we only going through the first two to combine them at this point?

    INSERT INTO @tblReturn3

		/* _factoryAllocation*/

		select 
		oi.intOrderItemId                              
		from CompanyDatabase..Shipments s 
		inner join CompanyDatabase..ShipmentItems si on si.intShipmentID = s.intShipmentID
		inner join Common.CompanyDatabase.Stores stores on stores.intStoreID = s.intLocationID
		inner join CompanyDatabase..OrderItems oi on oi.intOrderItemId = si.intOrderItemId                                      
		inner join CompanyDatabase..Orders o on o.intOrderId = s.intOrderId  
		where s.dtmTimestamp >= @endDate
		and stores.strLocationType = 'FACTORY'
		
		UNION 
		(
	 	  /*_storeAllocations*/

		select oi.intOrderItemId                               
		from CompanyDatabase..Shipments s 
		inner join CompanyDatabase..ShipmentItems si on si.intShipmentID = s.intShipmentID
		inner join Common.CompanyDatabase.Stores stores on stores.intStoreID = s.intLocationID
		inner join CompanyDatabase..OrderItems oi on oi.intOrderItemId = si.intOrderItemId                                      
		inner join CompanyDatabase..Orders o on o.intOrderId = s.intOrderId
		where s.dtmTimestamp >= @endDate
		and stores.strLocationType <> 'FACTORY'
		)

		UNION
		(
		/* _ordersWithAllocationProblems */
    	
			select oi.intOrderItemId
			from CompanyDatabase.dbo.StatusChangeLog
			inner join CompanyDatabase.dbo.OrderItems oi on oi.intOrderItemID = CompanyDatabase.dbo.StatusChangeLog.intRecordID
			inner join CompanyDatabase.dbo.Orders o on o.intOrderId = oi.intOrderId
			where strRecordType = 'orderitem'
			and strNewStatus = 'PROBLEM'
			and strOldStatus = 'NEW'
			and CompanyDatabase.dbo.StatusChangeLog.dtmTimestamp > @endDate
			and o.dtmTimestamp < @endDate
		)

Okay, @tblReturn3 is more of the same. Nothing more to really add.

	 INSERT INTO @tblReturn4
	
	 SELECT intOrderItemId FROM @tblReturn2 WHERE
	 intOrderItemId NOT IN(SELECT intOrderItemId FROM @tblReturn3 )

Ooh, but here we see something a bit different- we're taking the set difference between @tblReturn2 and @tblReturn3. This would almost make sense if there weren't already set operations in T-SQL which would handle all of this.

Which brings us, finally, to the last query in the whole thing:

SELECT 
	 o.intOrderId
	,oi.intOrderItemId
	,o.dtmDate
	,oi.strDescription
	,o.strFirstName + o.strLastName AS 'Name'
	,o.strEmail
	,o.strBillingCountry
	,o.strShippingCountry
	FROM CompanyDatabase.dbo.OrderItems oi
	INNER JOIN CompanyDatabase.dbo.Orders o on o.intOrderId = oi.intOrderId
	WHERE oi.intOrderItemId IN (SELECT intOrderItemId FROM @tblReturn4)
END

At the end of all this, I've determined a few things.

First, the developer responsible didn't understand table variables. Second,they definitely didn't understand joins. Third, they had no sense of the overall workflow of this query and just sorta fumbled through until they got results that the client said were okay.

And somehow, this pile of trash made it through a code review by internal architects and got deployed to production, where it promptly became the worst performing query in their application. Correction: the worst performing query thus far.

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

Author: Julia Rajagopalan Bertrand Dent knew that the lotto was rigged. Everyone knew. Only paid actors won, or friends of the Lotto Commission. Still, Bertrand stood in line at the Station 14 lotto stand to buy a ticket. It was what everyone did when docking at a refueling station. 1. Dock, 2. Secure your ship, […]

The post Paper Tickets, Broken Dreams appeared first on 365tomorrows.

Changes

• ArchiveBot: dashboard fixes and features
• hamster: cleanup warnings
• zygolophodon: support Mastodon redirect pages
• Debian wiki pages: PortsDocs/New

Issues

• Data ignored in swh-lister gitweb support
• Feature in isenkram
• Conffile removal for apparmor-profiles, tracker-miner-fs
• Warnings in emacsen-common

Sponsors

The SWH work was sponsored. All other work was done on a volunteer basis.

John Kelsey and I wrote a short paper for the Rossfest Festschrift: “Rational Astrologies and Security“:

There is another non-security way that designers can spend their security budget: on making their own lives easier. Many of these fall into the category of what has been called rational astrology. First identified by Randy Steve Waldman [Wal12], the term refers to something people treat as though it works, generally for social or institutional reasons, even when there’s little evidence that it works—­and sometimes despite substantial evidence that it does not.

[…]

Both security theater and rational astrologies may seem irrational, but they are rational from the perspective of the people making the decisions about security. Security theater is often driven by information asymmetry: people who don’t understand security can be reassured with cosmetic or psychological measures, and sometimes that reassurance is important. It can be better understood by considering the many non-security purposes of a security system. A monitoring bracelet system that pairs new mothers and their babies may be security theater, considering the incredibly rare instances of baby snatching from hospitals. But it makes sense as a security system designed to alleviate fears of new mothers [Sch07].

Rational astrologies in security result from two considerations. The first is the principal­-agent problem: The incentives of the individual or organization making the security decision are not always aligned with the incentives of the users of that system. The user’s well-being may not weigh as heavily on the developer’s mind as the difficulty of convincing his boss to take a chance by ignoring an outdated security rule or trying some new technology.

The second consideration that can lead to a rational astrology is where there is a social or institutional need for a solution to a problem for which there is actually not a particularly good solution. The organization needs to reassure regulators, customers, or perhaps even a judge and jury that “they did all that could be done” to avoid some problem—even if “all that could be done” wasn’t very much.

Most of my Debian contributions this month were sponsored by Freexian.

You can also support my work directly via Liberapay.

OpenSSH

Changes in dropbear 2025.87 broke OpenSSH’s regression tests. I cherry-picked the fix.

I reviewed and merged patches from Luca Boccassi to send and accept the COLORTERM and NO_COLOR environment variables.

Python team

Following up on last month, I fixed some more uscan errors:

• python-ewokscore
• python-ewoksdask
• python-ewoksdata
• python-ewoksorange
• python-ewoksutils
• python-processview
• python-rsyncmanager

I upgraded these packages to new upstream versions:

• bitstruct
• django-modeltranslation (maintained by Freexian)
• django-yarnpkg
• flit
• isort
• jinja2 (fixing CVE-2025-27516)
• mkdocstrings-python-legacy
• mysql-connector-python (fixing CVE-2025-21548)
• psycopg3
• pydantic-extra-types
• pydantic-settings
• pytest-httpx (fixing a build failure with httpx 0.28)
• python-argcomplete
• python-cymem
• python-djvulibre
• python-ecdsa
• python-expandvars
• python-holidays
• python-json-log-formatter
• python-keycloak (fixing a build failure with httpx 0.28)
• python-limits
• python-mastodon (in the course of which I found #1101140 in blurhash-python and proposed a small cleanup to slidge)
• python-model-bakery
• python-multidict
• python-pip
• python-rsyncmanager
• python-service-identity
• python-setproctitle
• python-telethon
• python-trio
• python-typing-extensions
• responses
• setuptools-scm
• trove-classifiers
• zope.testrunner

In bookworm-backports, I updated python-django to 3:4.2.19-1.

Although Debian’s upgrade to python-click 8.2.0 was reverted for the time being, I fixed a number of related problems anyway since we’re going to have to deal with it eventually:

• celery (contributed upstream)
• magic-wormhole (closed in Debian without action, but contributed upstream)
• python-flasgger (contributed upstream)
• sqlfluff (closed in Debian without action, but contributed upstream)

dh-python dropped its dependency on python3-setuptools in 6.20250306, which was long overdue, but it had quite a bit of fallout; in most cases this was simply a question of adding build-dependencies on python3-setuptools, but in a few cases there was a missing build-dependency on python3-typing-extensions which had previously been pulled in as a dependency of python3-setuptools. I fixed these bugs resulting from this:

• beangrow
• beangulp
• beanprice
• beanquery
• beautifulsoup4
• django-choices-field
• django-modeltranslation (maintained by Freexian)
• flake8-class-newline
• flake8-quotes
• nodeenv
• pygments-ansi-color
• pytest-mypy-plugins
• python-agilent
• python-aiohttp-security
• python-aiohttp-session
• python-djvulibre
• python-ewoksutils
• python-jsonlines
• python-mastodon
• python-pydash
• python-pytest-venv
• python-redfish
• python-ring-doorbell
• python-sluurp
• python-sqlite-migrate
• python-trubar

We agreed to remove python-pytest-flake8. In support of this, I removed unnecessary build-dependencies from pytest-pylint, python-proton-core, python-pyzipper, python-tatsu, python-tatsu-lts, and python-tinycss, and filed #1101178 on eccodes-python and #1101179 on rpmlint.

There was a dnspython autopkgtest regression on s390x. I independently tracked that down to a pylsqpack bug and came up with a reduced test case before realizing that Pranav P had already been working on it; we then worked together on it and I uploaded their patch to Debian.

I fixed various other build/test failures:

• aiomysql (closed with no action needed)
• lazr.uri
• m2crypto (thanks to Sebastian Andrzej Siewior)
• mkdocstrings
• mystic
• poetry-plugin-export
• poetry
• pydantic-extra-types
• pymilter
• python-a2wsgi (contributed upstream)
• python-anyio
• python-cymem
• python-django-timescaledb (only partially successful)
• python-ewokscore
• python-ewoksorange
• python-httpx-sse
• python-ml-collections
• python-opt-einsum (contributed upstream)
• python-passlib
• python-pdoc
• python-ppmd (with a follow-up)
• python-processview
• python-respx
• python-rsyncmanager (contributed upstream: ccc5f66dc7, 51c15ca8d1)
• python-sphobjinv
• python-urllib3
• pytrainer
• tlv8-python

I enabled more tests in python-moto and contributed a supporting fix upstream.

I sponsored Maximilian Engelhardt to reintroduce zope.sqlalchemy.

I fixed various odds and ends of bugs:

• catfish: does not start because of a missing dependency
• jupyterhub: creates /usr/alembic.ini
• pgzero: please add autopkgtests (to add coverage for python3-numpy)
• pyspread: creates /usr/pyspread/share/applications/ instead of using /usr/share/applications/
• python-aiosmtpd: python3-aiosmtpd-doc/trixie misses Breaks and Replaces for python3-aiosmtpd/bookworm
• python-dateutil: will FTBFS during trixie support period (contributed upstream)
• python-passlib: bcrypt warning
• python-pip: pip3-* manual pages could be aliased
• python3-deprecation: installs deprecation-2.0.7.egg-info instead of 2.1.0-2
• quodlibet: crashes on start if libmodplug1 is not installed
• reparser: autopkgtest must be marked superficial

I contributed a small documentation improvement to pybuild-autopkgtest(1).

Rust team

I upgraded rust-asn1 to 0.20.0.

Science team

I finally gave in and joined the Debian Science Team this month, since it often has a lot of overlap with the Python team, and Freexian maintains several packages under it.

I fixed a uscan error in hdf5-blosc (maintained by Freexian), and upgraded it to a new upstream version.

I fixed python-vispy: missing dependency on numpy abi.

Other bits and pieces

I fixed debconf should automatically be noninteractive if input is /dev/null.

I fixed a build failure with GCC 15 in yubihsm-shell (maintained by Freexian).

Prompted by a CI failure in debusine, I submitted a large batch of spelling fixes and some improved static analysis to incus (#1777, #1778) and distrobuilder.

After regaining access to the repository, I fixed telegnome: missing app icon in ‘About’ dialogue and made a new 0.3.7 release.

Another short status update of what happened on my side last month. Some more ModemManager bits landed, Phosh 0.46 is out, haptic feedback is now better tunable plus some more. See below for details (no April 1st joke in there, I promise):

phosh

• Fix swapped arguments in ABI check (MR)
• Sync packaging with Debian so testing packages becomes easier (MR)
• Fix crash when primary output goes away (MR)
• More consistent button press feedback (MR
• Undraft the lockscreen wallpaper branch (MR) - another ~2y old MR out of the way.
• Indicate ongoing WiFi scans (MR)
• Limit ABI compliance check to public headers (MR)
• Document most gsettings in a manpage (MR)
• (Hopefully) make integration test more robust (MR)
• Drop superfluous build invocation in CI by fixing the missing dep (MR)
• Fix top-panel icon size (MR)
• Release 0.46~rc1, 0.46.0
• Simplify adding new symbols (MR)
• Fix crash when taking screenshot on I/O starved system (MR)
• Split media-player and mpris-manger (MR)
• Handle Cell Broadcast notification categories (MR)

phoc

• xwayland: Allow views to use opacity: (MR)
• Track wlroots 0.19.x (MR)
• Initial support for workspaces (MR)
• Don't crash when gtk-layer-shell wants to reposition popups (MR)
• Some cleanups split out of other MRs (MR)
• Release 0.46~rc1, 0.46.0
• Add meson dist job and work around meson not applying patches in meson dist (MR, MR)
• Small render to allow Vulkan renderer to work (MR)
• Fix possible crash when closing applications (MR)
• Rename XdgSurface to XdgToplevel to prevent errors like the above (MR)

phosh-osk-stub

• Make switching into (and out of) symbol2 level more pleasant (MR)
• Simplify UI files as prep for the GTK4 switch (MR)
• Release 0.46~rc1, 0.46.0)

phosh-mobile-settings

• Format meson files (MR)
• Allow to set lockscren wallpaper (MR)
• Allow to set maximum haptic feedback (MR)
• Release 0.46~rc1, 0.46.0
• Avoid warnings when running CI/autopkgtest (MR)

phosh-tour

• Release 0.46.0

pfs

• Add search when opening files (MR)
• Show loading state when opening folders (MR)
• Move demo to its own folder (MR)
• Release 0.0.2

xdg-desktop-portal-gtk

• Add some support for v2 of the notification portal (MR)
• Make two function static (MR)

xdg-desktop-portal-phosh

• Add preview for lockscreen wallpapers (MR)
• Update to newer pfs to support search (MR)
• Release 0.46~rc1), 0.46.0
• Add initial support for notification portal v2 (MR) thus finally allowing flatpaks to submit proper feedback.
• Style consistency (MR, MR)
• Add Cell Broadcast categories (MR)

meta-phosh

• Small release helper tweaks (MR)

feedbackd

• Allow for vibra patterns with different magnitudes (MR)
• Allow to tweak maximum haptic feedback strength (MR)
• Split out libfeedback.h and check more things in CI (MR)
• Tweak haptic in default profile a bit (MR)
• dev-vibra: Allow to use full magnitude range (MR)
• vibra-periodic: Use [0.0, 1.0] as ranges for magnitude (MR)
• Release 0.8.0, 0.8.1)
• Only cancel feedback if ever inited (MR)

feedbackd-device-themes

• Increase button feedback for sarge (MR)

gmobile

• Release 0.2.2
• Format and validate meson files (MR)

livi

• Don't emit properties changed on position changes (MR)

Debian

• libmbim: Update to 1.31.95 (MR)
• libmbim: Upload to unstable and add autopkgtest (MR)
• libqmi: Update to 1.35.95 (MR)
• libqmi: Upload to unstable and add autopkgtest (MR)
• modemmanager: Update to 1.23.95 to experimental and add autopkgtest (MR)
• modemmanager: Upload to unstable (MR)
• modemmanager: Add missing nodoc build deps (MR)
• Package osmo-cbc: (Repo)
• feedbackd: Depend on adduser (MR)
• feedbackd: Release 0.8.0, 0.8.1
• feedbackd-device-themes: Release 0.8.0, 0.8.1
• phosh: Release 0.46~rc1, 0.46.0
• phoc: Release 0.46~rc1, 0.46.0
• phosh-osk-stub: Release 0.46~rc1, 0.46.0
• xdg-desktop-portal-phosh: Release 0.46~rc1, 0.46.0
• phosh-mobile-settings: Release 0.46~rc1, 0.46.0, fix autopkgtest
• phosh-tour: Release 0.46.0
• gmobile: Release 0.2.2-1
• gmobile: Ensure udev rules are applied on updates (MR)

git-buildpackage

• Ease creating packages from scratch and document that better (MR, Testcase MR)

feedbackd-device-themes

• Tweak some haptic for oneplus,fajita (MR)
• Drop superfluous periodic feedbacks and cleanup CI (MR)

wlroots

• xwm: Allow to set opacity (MR)

ModemManager

• Fix typos (MR)
• Add support for setting channels via libmm-glib and mmcli (MR)

Tuba

• Set input-hint for OSK word completion (MR)

xdg-spec

• Propose _NET_WM_WINDOW_OPACITY (which is around since ages) (MR)

gnome-calls

• Help startup ordering (MR)

Reviews

This is not code by me but reviews on other peoples code. The list is (as usual) slightly incomplete. Thanks for the contributions!

• phosh: Remove usage of phosh_{app_grid, overview}handlesearch (MR)
• phosh: app-grid-button: Prepare for GTK 4 by using gestures and other migrations (MR) - merged
• phosh: valign search results (MR) - merged
• phosh: top-panel: Hide setting's details on fold (MR) - merged
• phosh: Show frame with an animation (MR) - merged
• phosh: Use gtk_widget_set_visible (MR) - merged
• phosh: Thumbnail aspect ration tweak (MR) - merged
• phosh: Add clang/llvm ci step (MR)
• mobile-broadband-provider-info: Bild APN (MR) - merged
• iio-sensor-proxy: Buffer driver probing fix (MR) - merged
• iio-sensor-proxy: Double free (MR) - merged
• debian: Autopkgtests for ModemManager (MR)
• debian: gitignore: phosh-pim debian build directory (MR)
• debian: Better autopkgtests for MM (MR) - merged
• feedbackd: tests: Depend on daemon for integration test (MR) - merged
• libcmatrix: Various improvements (MR)
• gmobile/hwdb: Add Sargo (MR) - merged
• gmobile/hwdb: Add xiaomi-daisy (MR) - merged
• gmobile/hwdb: Add SHIFT6mq (MR) - merged
• meta-posh: Add reproducibility check (MR) - merged
• git-buildpackage: Dependency fixes (MR) - merged
• git-buildpackage: Rename tracking (MR)

Help Development

If you want to support my work see donations.

Comments?

Join the Fediverse thread

Many years ago, JP joined a Ruby project. This was in the heyday of Ruby, when every startup on Earth was using it, and if you weren't building your app on Rails, were you even building an app?

Now, Ruby offers a lot of flexibility. One might argue that it offers too much flexibility, especially insofar as it permits "monkey patching": you can always add new methods to an existing class, if you want. Regardless of the technical details, JP and the team saw that massive flexibility and said, "Yes, we should use that. All of it!"

As these stories usually go, that was fine- for awhile. Then one day, a test started failing because a class name wasn't defined. That was already odd, but what was even odder is that when they searched through the code, that class name wasn't actually used anywhere. So yes, there was definitely no class with that name, but also, there was no line of code that was trying to instantiate that class. So where was the problem?

def controller_class(name)
  "#{settings.app_name.camelize}::Controllers".constantize.const_get("#{name.to_s.camelize}")
end

def model_class(name)
  "#{settings.app_name.camelize}".constantize.const_get("#{name.to_s.camelize}")
end

def resource_class(name)
  "#{settings.app_name.camelize}Client".constantize.const_get("#{name.to_s.camelize}")
end

It happened because they were dynamically constructing the class names from a settings field. And not just in this handful of lines- this pattern occurred all over the codebase. There were other places where it referenced a different settings field, and they just hadn't encountered the bug yet, but knew that it was only a matter of time before changing a settings file was going to break more functionality in the application.

They wisely rewrote these sections to not reference the settings, and dubbed the pattern the "Caramelize Pattern". They added that to their coding standards as a thing to avoid, and learned a valuable lesson about how languages provide footguns.

Since today's April Fool's Day, consider the prank the fact that everyone learned their lesson and corrected their mistakes. I suppose that has to happen at least sometimes.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

Author: Majoki What was I thinking? Tiasmet could not put the thought—the picture—out of her head. The chipmunk with its shark-blank eyes and its panicked keening as the tictocs methodically circled and closed on it. The chipmunk should have been able to easily dash away. It was ten times the size of a tic or […]

The post Known appeared first on 365tomorrows.

I’ve released qmpbackup 0.46 which now utilizes the image fleecing technique for backup.

Usually, during backup, Qemu will use a so called copy-before-write filter so that data for new guest writes is sent to the backup target first, the guest write blocks until this operation is finished.

If the backup target is flaky, or becomes unavailable during backup operation, this could lead to high I/O wait times or even complete VM lockups.

To fix this, a so called “fleecing” image is introduced during backup being used as temporary cache for write operations by the guest. This image can be placed on the same storage as the virtual machine disks, so is independent from the backup target performance.

The documentation on which steps are required to get this going, using the Qemu QMP protocol is, lets say.. lacking..

The following examples show the general functionality, but should be enhanced to use transactions where possible. All commands are in qmp-shell command format.

Lets start with a full backup:

# create a new bitmap
block-dirty-bitmap-add node=disk1 name=bitmap persistent=true
# add the fleece image to the virtual machine (same size as original disk required)
blockdev-add driver=qcow2 node-name=fleecie file={"driver":"file","filename":"/tmp/fleece.qcow2"}
# add the backup target file to the virtual machine
blockdev-add driver=qcow2 node-name=backup-target-file file={"driver":"file","filename":"/tmp/backup.qcow2"}
# enable the copy-before-writer for the first disk attached, utilizing the fleece image
blockdev-add driver=copy-before-write node-name=cbw file=disk1 target=fleecie
# "blockdev-replace": make the copy-before-writer filter the major device (use "query-block" to get path parameter value, qdev node)
qom-set path=/machine/unattached/device[20] property=drive value=cbw
# add the snapshot-access filter backing the copy-before-writer
blockdev-add driver=snapshot-access file=cbw node-name=snapshot-backup-source
# create a full backup
blockdev-backup device=snapshot-backup-source target=backup-target-file sync=full job-id=test

[ wait until block job finishes]

# remove the snapshot access filter from the virtual machine
blockdev-del node-name=snapshot-backup-source
# switch back to the regular disk
qom-set path=/machine/unattached/device[20] property=drive value=node-disk1
# remove the copy-before-writer
blockdev-del node-name=cbw
# remove the backup-target-file
blockdev-del node-name=backup-target-file
# detach the fleecing image
blockdev-del node-name=fleecie

After this process, the temporary fleecing image can be deleted/recreated. Now lets go for a incremental backup:

# add the fleecing and backup target image, like before
blockdev-add driver=qcow2 node-name=fleecie file={"driver":"file","filename":"/tmp/fleece.qcow2"}
blockdev-add driver=qcow2 node-name=backup-target-file file={"driver":"file","filename":"/tmp/backup-incremental.qcow2"}
# add the copy-before-write filter, but utilize the bitmap created during full backup
blockdev-add driver=copy-before-write node-name=cbw file=disk1 target=fleecie bitmap={"node":"disk1","name":"bitmap"}
# switch device to the copy-before-write filter
qom-set path=/machine/unattached/device[20] property=drive value=cbw
# add the snapshot-access filter
blockdev-add driver=snapshot-access file=cbw node-name=snapshot-backup-source
# merge the bitmap created during full backup to the snapshot-access device so
# the backup operation can access it. (you better use an transaction here)
block-dirty-bitmap-add node=snapshot-backup-source name=bitmap
block-dirty-bitmap-merge node=snapshot-backup-source target=bitmap bitmaps=[{"node":"disk1","name":"bitmap"}]
# create incremental backup (you better use an transaction here)
blockdev-backup device=snapshot-backup-source target=backup-target-file job-id=test sync=incremental bitmap=bitmap

 [ wait until backup has finished ]
 [ cleanup like before ]

# clear the dirty bitmap (you better use an transaction here)
block-dirty-bitmap-clear node=disk1 name=bitmap

Or, use a simple reproducer by directly passing qmp commands via stdio:

#!/usr/bin/bash
qemu-img create -f raw disk 1M
qemu-img create -f raw fleece 1M
qemu-img create -f raw backup 1M
qemu-system-x86_64 -drive node-name=disk,file=disk,format=file -qmp stdio -nographic -nodefaults <<EOF
{"execute": "qmp_capabilities"}
{"execute": "block-dirty-bitmap-add", "arguments": {"node": "disk", "name": "bitmap"}}
{"execute": "blockdev-add", "arguments": {"node-name": "fleece", "driver": "file", "filename": "fleece"}}
{"execute": "blockdev-add", "arguments": {"node-name": "backup", "driver": "file", "filename": "backup"}}
{"execute": "blockdev-add", "arguments": {"node-name": "cbw", "driver": "copy-before-write", "file": "disk", "target": "fleece", "bitmap": {"node": "disk", "name": "bitmap"}}}
{"execute": "query-block"}
{"execute": "qom-set", "arguments": {"path": "/machine/unattached/device[4]", "property": "drive", "value": "cbw"}}
{"execute": "blockdev-add", "arguments": {"node-name": "snapshot", "driver": "snapshot-access", "file": "cbw"}}
{"execute": "block-dirty-bitmap-add", "arguments": {"node": "snapshot", "name": "tbitmap"}}
{"execute": "block-dirty-bitmap-merge", "arguments": {"node": "snapshot", "target": "tbitmap", "bitmaps": [{"node": "disk", "name": "bitmap"}]}}
[..]
{"execute": "quit"}
EOF

In July 2024, NASA posted an article titled “NASA Has Pride Across the Universe” featuring a pride flag by Rachel Lense where each color band is made up of images from across NASA. Today is the annual International Transgender Day of Visibility. The original NASA article from last year has since been taken offline. But the heroes from archive.org still carry a copy which I now archived myself together with the other source images. Here is NASA’s pride flag in all its glory:

Southern Fried Science has an article about the flag. The original 4000x2547 TIFF image was stored by archive.org but the PNG version in the same resolution can be downloaded here or by clicking on the image.

Version 0.3.16 of the Rblpapi package arrived on CRAN today. Rblpapi provides a direct interface between R and the Bloomberg Terminal via the C++ API provided by Bloomberg (but note that a valid Bloomberg license and installation is required).

This is the sixteenth release since the package first appeared on CRAN in 2016. It contains several enhancements. Two contributed PRs improve an error message, and extended connection options. We cleaned up a bit of internal code. And this release also makes the build conditional on having a valid build environment. This has been driven by the fact CRAN continues to builder under macOS 13 for x86_64, but Bloomberg no longer supplies a library and headers. And our repeated requests to be able to opt out of the build were, well, roundly ignored. So now the builds will succeed, but on unviable platforms such as that one we will only offer ‘empty’ functions. But no more build ERRORS yelling at us for three configurations.

The detailed list of changes follow below.

Changes in Rblpapi version 0.3.16 (2025-03-31)

• A quota error message is now improved (Rodolphe Duge in #400)

• Convert remaining throw into Rcpp::stop (Dirk in #402 fixing #401)

• Add optional appIdentityKey argument to blpConnect (Kai Lin in #404)

• Rework build as function of Blp library availability (Dirk and John in #406, #409, #410 fixing #407, #408)

Courtesy of my CRANberries, there is also a diffstat report for the this release. As always, more detailed information is at the Rblpapi repo or the Rblpapi page. Questions, comments etc should go to the issue tickets system at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

A new maintenance release 0.4.24 of RProtoBuf arrived on CRAN today. RProtoBuf provides R with bindings for the Google Protocol Buffers (“ProtoBuf”) data encoding and serialization library used and released by Google, and deployed very widely in numerous projects as a language and operating-system agnostic protocol.

This release brings an both an upstream API update affecting one function, and an update to our use of the C API of R, also in one function. Nothing user-facing, and no surprises expected.

The following section from the NEWS.Rd file has full details.

Changes in RProtoBuf version 0.4.24 (2025-03-31)

• Add bindings to EnumValueDescriptor::name (Mike Kruskal in #108)

• Replace EXTPTR_PTR with R_ExternalPtrAddr (Dirk)

Thanks to my CRANberries, there is a diff to the previous release. The RProtoBuf page has copies of the (older) package vignette, the ‘quick’ overview vignette, and the pre-print of our JSS paper. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Today I was looking for a way on how to best publish my OpenPGP key on my webserver. Surely, somebody came up with some sort of standard way for where to place that key, right? Turns out, they did: https://datatracker.ietf.org/doc/draft-koch-openpgp-webkey-service/

The TLDR summary is, that my key can now be found here:

https://mister-muffin.de/.well-known/openpgpkey/hu/8yxgr5jjfok88r9um56kb44x9h4dyj7f

Or be downloadable by just running:

$ gpg --locate-key josch@mister-muffin.de

Where does the hash come from? It’s the local part of my email (josch) hashed with sha1 and encoded in z-base32. That computation can be done by gpg:

$ gpg --with-wkd-hash -k josch@mister-muffin.de | grep mister-muffin.de
[...]
8yxgr5jjfok88r9um56kb44x9h4dyj7f@mister-muffin.de

I exported the key that I put there using the following command:

$ gpg --no-options --export --export-options export-minimal,export-clean \
    --export-filter keep-uid="uid = Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>" \
    F83356BBE112B7462A41552F7D5D8C60CF4D3EB4

There is a handy validator for such setups that can be found here: https://www.webkeydirectory.com

I had an interesting debugging experience when I tried to verify my setup in a fresh Debian chroot because I got this error message when I ran above command:

gpg: directory '/root/.gnupg' created
gpg: keybox '/root/.gnupg/pubring.kbx' created
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: error retrieving 'josch@mister-muffin.de' via WKD: General error
gpg: error reading key: General error

That’s not very descriptive… Turns out, that I was missing the ca-certificates package. After installing it, everything worked as expected:

$ gpg --locate-key josch@mister-muffin.de
gpg: key 7D5D8C60CF4D3EB4: public key "Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>" imported
gpg: Total number processed: 1
gpg:               imported: 1
pub   rsa4096 2013-07-04 [SC]
      F83356BBE112B7462A41552F7D5D8C60CF4D3EB4
uid           [ unknown] Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
sub   rsa4096 2013-07-04 [E]
sub   rsa4096 2013-07-04 [S]
sub   rsa4096 2023-07-08 [S]

Life is undeniably real. It defines the very boundary of our reality because it is what we are. Yet despite this fundamental presence, the nature of life has defied precise scientific explanation. While we recognize “life” colloquially and can characterize its more familiar biological forms, we struggle with frontier questions: how does life emerge from non-life? How can we engineer new forms of life? How might we recognize artificial or alien life? What are the sources of novelty and creativity that underlie biology and technology? 

These challenges mirror the limits of our ancestors’ understanding of gravity. They knew objects fell to Earth without understanding why. They observed just a few stars wandering across their night sky and lacked explanations for their motion relative to all the other stars, which remained fixed. It required technological advances — precise mechanical clocks that allowed Tycho Brahe to record planetary motions, Galileo Galilei’s concept of inertial mass, and Isaac Newton’s conception of universal laws — to develop our modern explanation of gravity. While we may be tempted to point to a particular generation that made the conceptual leaps necessary, this transformation took thousands of years of technological and intellectual development before eventually giving rise to theoretical physics as an explanatory framework. The development of physics was based on the premise that reality is comprehensible through abstract descriptions that unify our observations and allow us deeper explanations than our immediate sense perception might otherwise permit. 

Our ability to explain gravity fundamentally changed how we interact with our world. With laws of gravitation, we launch satellites, visit distant worlds, and better understand our place in the cosmos. So too might an explanatory framework for life transform our future.

We now sit at an interesting point in history: one in which it is perhaps evident that we have sufficient technology to understand “life,” and according to some we may even have examples of artificial life and intelligence, but we have not yet landed on the conceptual framing and theoretical abstractions that will allow us to see what this means as clearly as we now see gravity. That is, we lack a formal language to talk about life. 

Life versus Computation

“Life” has historically been difficult to formalize at this deep level of abstraction because of its complexity. Darwin and his contemporaries were successful in explaining some portion of life because their goal was not to inventory the full complexity of living forms, but merely to explain how it is that one form can change into another, and why this should lead to a diversity of forms, some of them more complex than others.  It was not until the advent of the theory of computation roughly 75 years later that it became possible to systematically formalize some notions of complexity (although earlier individual examples of the difficulty of a computation date much earlier). Some thought then, and still think now, that such formalization might be relevant to understanding life. In the historical progression of ideas, proceeding over many many generations, the theory of computation may prove an important step, but not the final or most important one.   

The theory of computation, and its derivative concepts of computational complexity, were not explicitly developed to solve the problem of life, nor were they even devised as a formal approach to life or to physical systems. It is important to maintain this distinction because many alive now confuse computation not only with physical reality, but also more specifically with life itself. In human histories, our best languages for describing the frontier of what we understand are often embedded in the technologies of our time; however, the truly fundamental breakthroughs are often those that allow us to see beyond the current technological horizon. 

The challenge with “computation” begins with the vast spaces we must consider. In chemical space — defined as the space of all possible molecules — there are an estimated 10⁶⁰ possible molecules composed of up to 30 atoms using only the elements carbon, oxygen, nitrogen, and sulfur. This is only a very small subset of all molecules we might imagine, and cheminformaticians who study chemical space have never been able to even estimate its full size. We cannot explore all possible states computationally. You may at first think this is solely a limitation of our computers, but in fact it is a limitation on reality itself. Given all available compute time and resources right now on planet Earth, it would not be possible to generate a foundation model for all possible molecules or their functional properties. But even more revealing about the physical barriers is how, if given all available time and resources in the entire universe, it would not be possible to construct every possible molecule either. And, because chemistry makes things like biological forms, which evolve into technological forms, the limitations at the base layer of chemistry indicate that our universe may be fundamentally unable to explore all possible options even in infinite time. The technical term for this is to say that our universe is non-ergodic: it cannot visit all possible states. But even this terminology is not right because it assumes that the state-space exists at all. If nothing inside the universe can generate the full space, in what sense can we say it exists? 

A much more physical interpretation, and one that keeps all descriptions internal to the universe they describe, is to assume that things do not exist until the universe generates them. Discussing all possible molecules is just one example, but the idea extends to much more familiar things like technology: even with our most advanced generative models, we could never even imagine all possible technologies, so how could we possibly create them all? This feature of living in a universe that is self-constructing is one clue that reality cannot be computational. The fact that we can imagine possibilities that cannot exist all at once is more telling about us as constructive, creative systems within the universe than it is of a landscape of possibilities “out there” that are all equally real. 

This raises deep questions about computational approaches to life, which itself emerges from a backward view of the space of chemistry that the universe can explore; that is, only physical systems that have evolved to be like us can ask such questions about how they came to be. A challenge in the field of chemistry relevant to the issue of defining life is how one can identify molecules with function, that is, ones that have some useful role in sustaining the persistence of a living entity. This is a frontier research area in artificial intelligence-driven chemical design and drug discovery and in questions about biological and machine agency. But function is a post-selected concept. Post-selection is a concept from probability theory, where one conditions the probability space on the occurrence of a given event after the event occurs. “Function” is a concept that can only be defined relative to what already exists and is, therefore, historically contingent. 

A key challenge then emerges based on the limits of our models: we can only calculate the size of the space evolution selects functional structures within by imposing tight restrictions on the space of interest (post-selecting) so we can bound the size of the space to one we can compute. It may be that the only sense in which this counterfactual space is “real” is within the confines of our models of it. Chemical space cannot be computed, nor can the full space be experimentally explored, making probability assignments across all molecules not only impossible but unphysical; there will always be structure outside our models which could be a source for novelty. To stress the point here, I am not indicating this as a limitation on our models themselves, but on reality itself and, by extension, on what laws of physics could possibly explain how life emerges from such large combinatorial spaces. 

Analogies to the theory of computation do not fit, because computation is fundamentally the wrong paradigm for understanding life. But if we were to use such an analogy, it would be like predicting the output of programs that have not yet been run. We know from the very foundations of the theory of computation that this kind of forward-looking algorithm runs into epistemologically uncertain territory. A prime example is the halting problem, and related proofs that one cannot in general determine whether a given program will terminate and produce an output or run forever. One could make a machine that could describe this situation (what is called an oracle) and solve the halting problem in a specific case, but then the oracle itself would introduce new halting problems. I could assume infinity is real and there will always be a system that can describe another, but even this would run into new issues of uncomputability. New uncomputable things lurk no matter how you patch your system to account for other uncomputable things. Furthermore, infinity is a mathematical concept that itself may not correspond to a physical reality beyond the boundaries of the representational forms of the external world constructed within the physical architecture of human minds and human-derived technologies. 

Complexity, in a computational sense of the word, describes the length of the shortest computer program that produces a given output — and it is also generally uncomputable. More important for physics is that it is also not measurable. We might try to approximate complexity with something computable, but this will depend on our choice of language and, therefore, is an observer-dependent quantity and not a good candidate for physical law. If we assume there is a unique shortest program, we must assume infinity is real to do so, and we have again introduced something non-physical. I am advocating that we take the fact that we live in a finite universe with finite resources and finite time seriously, and construct our theories accordingly — particularly in accounting for historical contingency as a fundamental explanation. We need to take seriously our finite, self-constructing universe because this will allow us to embed ideas about life and open-ended creativity into physics, and in turn explain much more about the universe we actually live in. Among the most important aspects of physics is metrology — the science of measurement — because it allows standardization and empirical testing of theory.  It also allows us to define what we consider to be “laws of physics” — laws like those underlying gravitation, which we assume to be independent of the observer or measuring device. Every branch of physics developed to date rests on a foundation of abstract representations built from empirical measurement; it is this process that allows us to see beyond the confines of our own minds. 

For example, in the foundations of physics, we talk about how laws of physics are invariant to an observer’s frame of reference. Einstein’s work on relativity is exemplary in this regard: when experiments showed the speed of light yielded the same value regardless of the measuring instrument’s motion, Einstein equated the speed of light to a law of physics using the principle of invariance. This principle is important because if something is invariant, it does not depend on what the observer is doing; they will always measure it the same way. Einstein’s peers were not willing to take the measurement at face value. Many assumed the conception that the speed of light could change with the observer was correct, consistent with other sense perceptions of the world, and therefore that the measurements must be wrong. They assumed something must be missing from the physical measurements, like the presence of an ether (a substance hypothesized to fill space to explain the data). Indeed, they were missing something physical, but it was because they assumed their current abstractions were correct, and did not take the measurement seriously enough to change their ideas of what was physically real. The invariance of the speed of light had critically important consequences because following this idea to its logical conclusion (what Einstein did in developing special relativity) indicates that simultaneity (the measuring of events happening at the same “time”) and space are relative, and these insights have subsequently been confirmed by other experiments and observations. This example highlights two important features of physical laws: they are grounded in measurement (confirming they exist beyond how our minds label the world) and they are invariant with respect to measurement.

Assembly Theory and the Physics of Life

As an explanation for the physics underlying what we call “life,” my colleagues and I are developing a new approach called assembly theory. Assembly theory as a theory of physics is built on the idea that time is fundamental (you might call it causation) and as a consequence historical contingency is a real physical feature of our universe. The past is deterministic, but the future is undetermined until it happens simply because the universe has yet to construct itself into the future (and the possibility space is so big it cannot exist until it happens). This may seem a radical step, so how did we get here from thinking about life? 

We started with the question of how one might measure the emergence of complex molecules from unconstrained chemical systems. The question was easy to state: how complex does something need to be such that we might say only a living thing can produce it? We were interested in this because we work on the problem of understanding how life arises from non-life, and this requires some way of quantifying the transition from abiotic to living systems. This led to the development of a complexity measure, the assembly index, which my colleague Lee Cronin at the University of Glasgow originally developed from thought experiments on the measurement and physical structure of molecules.

The idea is startlingly simple. The assembly index is formalized as the minimum number of steps to make an object, starting from elementary building blocks, and re-using already assembled parts. For molecules, these parts and operations are chemical bonds. This point on bonds is important: assembly theory uses as its natural language the physical constraints intrinsic to the objects it describes, which can be probed by another system, such as a measuring device. However, we also regard that any mathematical language we use to describe the physical world is not the physical world. What we are looking for is a language that at least allows us to capture the invariant properties of the objects under study, because we are after a law of physics that describes life. We consider the assembly index to represent the minimum causation required to form the object, and this is, in fact, independent of how we label the specific minimum steps. Instead, what it captures is that there is a minimum number of ordered structures necessary for the given structure to come to exist. What the assembly index captures is that causation is a real physical property, automatically implying there is an ordering to what can exist, and that objects are formed in a historically contingent path. This raises the possibility that we may be able to measure the physical complexity of a system, even if it is not possible to compute it. 

Assembly theory’s two observables — assembly index and copy number — provide a generalized quantification of the selective causation necessary to produce an observed configuration of objects. Copy number is countable; it is how many of a given object you observe. Our conjecture is that there is a threshold for life, because objects with high assembly indices do not form in high (detectable) numbers of copies in the absence of life and selective processes. This has been confirmed by experimental tests of assembly theory for molecular life detection.  If we return to the idea of the vastness of chemical space, we can see why this idea is important. If the physics of our universe operated by exhaustive search, we would not exist because there are simply too many possible configurations of matter. What the physics of life indicates is the existence of historically contingent trajectories, where structures made in the past can be used to further elaborate into the space of more complex objects. Assembly theory suggests a phase transition between non-life (breadth-based search of physical structures) and life (depth-first search of physical structures), where the latter is possible because structures the universe has already generated can be used again. Underlying this is an absolute causal structure where complex objects reside, which we call the assembly space. If one assumes everything is possible, and the universe can really do it all, you will entirely miss the structure underlying life and what really gets to exist, and why. 

Determined Pasts, Non-Determinable Futures

An important distinction emerges from the physics of life: you cannot compute the future, but you can compute the past. Assembly theory works precisely because it starts from observed objects and allows reconstructing an invariant, minimum causal ordering for how hard it is for the universe to generate that object through its measurement. This allows us to talk about complexity related to life in an objective way that we expect — if the theory passes the trial and fire of scientific consensus — will play a role like other invariant quantities in physics. This fundamentally differs from computational approaches that depend on the “machine” (or observer), and it builds on the one unique thing theoretical physics has been able to offer the world: the ability to build abstractions that reach deeper than how our brains label data to describe the world. 

By taking measurement in science seriously and recognizing how our theories of physics are built from measurement, assembly theory offers a lens through which we might finally understand life as fundamental — not as a computation to be simulated but as a physical reality to be measured. In this view, life is not merely a special case of computation but something more fundamental: a physical reality that can be measured, quantified, and understood through invariant physical laws rather than observer-dependent computations. This leads to the startling realization that one of the most important features of life is that it produces a set of future states that are not computable, even in principle. This means a paradigm for accurately understanding intelligence, consciousness, and decision making is intrinsically missing in our current science that takes as its foundation the idea that everything can happen and everything can be modeled. This does not mean that life will never be understandable as a purely physical process; it simply points to the fact we are missing the required fundamental physics to be able explain life in a universe that has a future horizon that is inherently undetermined. 

The application of assembly theory in physics introduces contingency at a fundamental level, explaining how the past structures some of the future but not all of it. Life takes inert matter that is predictable and turns it into matter that is unpredictable because of the vast number of possibilities in the phenomenon of evolution, revealing selection as a kind of force that is responsible for the production of complexity in the universe. Life, not computation, unlocks our non-deterministic future. Only by looking beyond our current technological moment to the next technologies creating new life forms will we be able to understand what our future could hold. 

Acknowledgments

Many of the ideas discussed herein come from collaborative work with Leroy Cronin.

Notes

1. Figure and caption reproduced from Sharma, A., Czégel, D., Lachmann, M. et al. Assembly theory explains and quantifies selection and evolution. Nature 622, 321–328 (02023) under a CC BY 4.0 license. https://doi.org/10.1038/s41586-023-06600-9 .

—

Sara Imari Walker is the author of Life As No One Knows It: The Physics of Life’s Emergence (Riverhead Books, 02024) and will be speaking at Long Now on April 1, 02025.

Anarcat’s review of Fish is interesting and shows some benefits I hadn’t previously realised, I’ll have to try it out [1].

Longnow has an insightful article about religion and magic mushrooms [2].

Brian Krebs wrote an informative artivle about DOGE and the many security problems that it has caused to the US government [3].

Techdirt has an insightful article about why they are forced to become a democracy blog after the attacks by Trump et al [4].

Antoine wrote an insightful blog post about the war for the Internet and how in many ways we are losing to fascists [5].

Interesting story about people working for free at Apple to develop a graphing calculator [6]. We need ways for FOSS people to associate to do such projects.

Interesting YouTube video about a wiki for building a cheap road legal car [7].

Interesting video about powering spacecraft with Plutonion 238 and how they are running out [8].

Interesting information about the search for mh370 [9]. I previously hadn’t been convinced that it was hijacked but I am now.

The EFF has an interesting article about the Rayhunter, a tool to detect cellular spying that can run with cheap hardware [10].

I rebuilt (the top-50 popcon) Debian and Ubuntu packages, on amd and arm64, and compared the results a couple of months ago. Since then the Reproduce.Debian.net effort has been launched. Unlike my small experiment, that effort is a full-scale rebuild with more architectures. Their goal is to reproduce what is published in the Debian archive.

One differences between these two approaches are the build inputs: The Reproduce Debian effort use the same build inputs which were used to build the published packages. I’m using the latest version of published packages for the rebuild.

What does that difference imply? I believe reproduce.debian.net will be able to reproduce more of the packages in the archive. If you build a C program using one version of GCC you will get some binary output; and if you use a later GCC version you are likely to end up with a different binary output. This is a good thing: we want GCC to evolve and produce better output over time. However it means in order to reproduce the binaries we publish and use, we need to rebuild them using whatever build dependencies were used to prepare those binaries. The conclusion is that we need to use the old GCC to rebuild the program, and this appears to be the Reproduce.Debian.Net approach.

It would be a huge success if the Reproduce.Debian.net effort were to reach 100% reproducibility, and this seems to be within reach.

However I argue that we need go further than that. Being able to rebuild the packages reproducible using older binary packages only begs the question: can we rebuild those older packages? I fear attempting to do so ultimately leads to a need to rebuild 20+ year old packages, with a non-negligible amount of them being illegal to distribute or are unable to build anymore due to bit-rot. We won’t solve the Trusting Trust concern if our rebuild effort assumes some initial binary blob that we can no longer build from source code.

I’ve made an illustration of the effort I’m thinking of, to reach something that is stronger than reproducible rebuilds. I am calling this concept a Idempotent Rebuild, which is an old concept that I believe is the same as John Gilmore has described many years ago.

The illustration shows how the Debian main archive is used as input to rebuild another “stage #0” archive. This stage #0 archive can be compared with diffoscope to the main archive, and all differences are things that would be nice to resolve. The packages in the stage #0 archive is used to prepare a new container image with build tools, and the stage #0 archive is used as input to rebuild another version of itself, called the “stage #1” archive. The differences between stage #0 and stage #1 are also useful to analyse and resolve. This process can be repeated many times. I believe it would be a useful property if this process terminated at some point, where the stage #N archive was identical to the stage #N-1 archive. If this would happen, I label the output archive as an Idempotent Rebuild of the distribution.

How big is N today? The simplest assumption is that it is infinity. Any build timestamp embedded into binary packages will change on every iteration. This will cause the process to never terminate. Fixing embedded timestamps is something that the Reproduce.Debian.Net effort will also run into, and will have to resolve.

What other causes for differences could there be? It is easy to see that generally if some output is not deterministic, such as the sort order of assembler object code in binaries, then the output will be different. Trivial instances of this problem will be caught by the reproduce.debian.net effort as well.

Could there be higher order chains that lead to infinite N? It is easy to imagine the existence of these, but I don’t know how they would look like in practice.

An ideal would be if we could get down to N=1. Is that technically possible? Compare building GCC, it performs an initial stage 0 build using the system compiler to produce a stage 1 intermediate, which is used to build itself again to stage 2. Stage 1 and 2 is compared, and on success (identical binaries), the compilation succeeds. Here N=2. But this is performed using some unknown system compiler that is normally different from the GCC version being built. When rebuilding a binary distribution, you start with the same source versions. So it seems N=1 could be possible.

I’m unhappy to not be able to report any further technical progress now. The next step in this effort is to publish the stage #0 build artifacts in a repository, so they can be used to build stage #1. I already showed that stage #0 was around ~30% reproducible compared to the official binaries, but I didn’t save the artifacts in a reusable repository. Since the official binaries were not built using the latest versions, it is to be expected that the reproducibility number is low. But what happens at stage #1? The percentage should go up: we are now compare the rebuilds with an earlier rebuild, using the same build inputs. I’m eager to see this materialize, and hope to eventually make progress on this. However to build stage #1 I believe I need to rebuild a much larger number packages in stage #0, it could be roughly similar to the “build-essentials-depends” package set.

I believe the ultimate end goal of Idempotent Rebuilds is to be able to re-bootstrap a binary distribution like Debian from some other bootstrappable environment like Guix. In parallel to working on a achieving the 100% Idempotent Rebuild of Debian, we can setup a Guix environment that build Debian packages using Guix binaries. These builds ought to eventually converge to the same Debian binary packages, or there is something deeply problematic happening. This approach to re-bootstrap a binary distribution like Debian seems simpler than rebuilding all binaries going back to the beginning of time for that distribution.

What do you think?

PS. I fear that Debian main may have already went into a state where it is not able to rebuild itself at all anymore: the presence and assumption of non-free firmware and non-Debian signed binaries may have already corrupted the ability for Debian main to rebuild itself. To be able to complete the idempotent and bootstrapped rebuild of Debian, this needs to be worked out.

Legacy systems are hard to change, and even harder to eliminate. You can't simply do nothing though; as technology and user expectations change, you need to find ways to modernize and adapt the legacy system.

That's what happened to Alicia's team. They had a gigantic, spaghetti-coded, monolithic application that was well past drinking age and had a front-end to match. Someone decided that they couldn't touch the complex business logic, but what they could do was replace the frontend code by creating an adapter service; the front end would call into this adapter, and the adapter would execute the appropriate methods in the backend.

Some clever coder named this "Backend for Frontend" or "BFF".

It was not anyone's BFF. For starters, this system didn't actually allow you to just connect a UI to the backend. No, that'd be too easy. This system was actually a UI generator.

The way this works is that you feed it a schema file, written in JSON. This file specifies what input elements you want, some hints for layout, what validation you want the UI to perform, and even what CSS classes you want. Then you compile this as part of a gigantic .NET application, and deploy it, and then you can see your new UI.

No one likes using it. No one is happy that it exists. Everyone wishes that they could just write frontends like normal people, and not use this awkward schema language.

All that is to say, when Alicia's co-worker stood up shortly before lunch, said, "I'm taking off the rest of the day, BFF has broken me," it wasn't particularly shocking to hear- or even the first time that'd happened.

Alicia, not heeding the warning inherent in that statement, immediately tracked down that dev's last work, and tried to understand what had been so painful.

    "minValue": 1900,
    "maxValue": 99,

This, of course, had to be a bug. Didn't it? How could the maxValue be lower than the minValue?

Let's look at the surrounding context.

{
    "type": "eventValueBetweenValuesValidator",
    "eventType": "CalendarYear",
    "minValue": 1900,
    "maxValue": 99,
    "isCalendarBasedMaxValue": true,
    "message": "CalendarYear must be between {% raw %}{{minValue}}{% endraw %} and {% raw %}{{maxValue}}{% endraw %}."
}

I think this should make it perfectly clear what's happening. Oh, it doesn't? Look at the isCalendarBasedMaxValue field. It's true. There, that should explain everything. No, it doesn't? You're just more confused?

The isCalendarBasedMaxValue says that the maxValue field should not be treated as a literal value, but instead, is the number of years in the future relative to the current year which are considered valid. This schema definition says "accept all years between 1900 and 2124 (at the time of this writing)." Next year, that top value goes up to 2125. Then 2126. And so on.

As features go, it's not a terrible feature. But the implementation of the feature is incredibly counter-intuitive. At the end of the day, this is just bad naming: (ab)using min/max to do something that isn't really a min/max validation is the big issue here.

Alicia writes:

I couldn't come up with something more counterintuitive if I tried.

Oh, don't sell yourself short, Alicia. I'm sure you could write something far, far worse if you tried. The key thing here is that clearly, nobody tried- they just sorta let things happen and definitely didn't think too hard about it.

Review: Ghostdrift, by Suzanne Palmer

Ghostdrift is a science fiction adventure and the fourth (and possibly final) book of the Finder Chronicles. You should definitely read this series in order and not start here, even though the plot of this book would stand alone.

Following The Scavenger Door, in which he made enemies even more dramatically than he had in the previous books, Fergus Ferguson has retired to the beach on Coralla to become a tea master and take care of his cat. It's a relaxing, idyllic life and a much-needed total reset. Also, he's bored. The arrival of his alien friend Qai, in some kind of trouble and searching for him, is a complex balance between relief and disappointment.

Bas Belos is one of the most notorious pirates of the Barrens. He has someone he wants Fergus to find: his twin sister, who disappeared ten years ago. Fergus has an unmatched reputation for finding things, so Belos kidnapped Qai's partner to coerce her into finding Fergus. It's not an auspicious beginning to a relationship, and Qai was ready to fight once they got her partner back, but Belos makes Fergus an offer of payment that, startlingly, is enough for him to take the job mostly voluntarily.

Ghostdrift feels a bit like a return to Finder. Fergus is once again alone among strangers, on an assignment that he's mostly not discussing with others, piecing together clues and navigating tricky social dynamics. I missed his friends, particularly Ignatio, and while there are a few moments with AI ships, they play less of a role.

But Fergus is so very good at what he does, and Palmer is so very good at writing it. This continues to be competence porn at its best. Belos's crew thinks Fergus is a pirate recruited from a prison colony, and he quietly sets out to win their trust with a careful balance of self-deprecation and unflappable skill, helped considerably by the hidden gift he acquired in Finder. The character development is subtle, but this feels like a Fergus who understands friendship and other people at a deeper and more satisfying level than the Fergus we first met three books ago.

Palmer has a real talent for supporting characters and Ghostdrift is no exception. Belos's crew are criminals and murderers, and Palmer does remind the reader of that occasionally, but they're also humans with complex goals and relationships. Belos has earned their loyalty by being loyal and competent in a rough world where those attributes are rare. The morality of this story reminds me of infiltrating a gang: the existence of the gang is not a good thing, and the things they do are often indefensible, but they are an understandable reaction to a corrupt social system. The cops (in this case, the Alliance) are nearly as bad, as we've learned over the past couple of books, and considerably more insufferable. Fergus balances the ethical complexity in a way that I found satisfyingly nuanced, while quietly insisting on his own moral lines.

There is a deep science fiction plot here, possibly the most complex of the series so far. The disappearance of Belos's sister is the tip of an iceberg that leads to novel astrophysics, dangerous aliens, mysterious ruins, and an extended period on a remote and wreck-strewn planet. I groaned a bit when the characters ended up on the planet, since treks across primitive alien terrain with jury-rigged technology are one of my least favorite science fiction tropes, but I need not have worried. Palmer knows what she's doing; the pace of the plot does slow a bit at first, but it quickly picks up again, adding enough new setting and plot complications that I never had a chance to be bored by alien plants. It helps that we get another batch of excellent supporting characters for Fergus to observe and win over.

This series is such great science fiction. Each book becomes my new favorite, and Ghostdrift is no exception. The skeleton of its plot is a satisfying science fiction mystery with multiple competing factions, hints of fascinating galactic politics, complicated technological puzzles, and a sense of wonder that reminds me of reading Larry Niven's Known Space series. But the characters are so much better and more memorable than classic SF; compared to Fergus, Niven's Louis Wu barely exists and is readily forgotten as soon as the story is over. Fergus starts as a quiet problem-solver, but so much character depth unfolds over the course of this series. The ending of this book was delightfully consistent with everything we've learned about Fergus, but also the sort of ending that it's hard to imagine the Fergus from Finder knowing how to want.

Ghostdrift, like each of the books in this series, reaches a satisfying stand-alone conclusion, but there is no reason within the story for this to be the last of the series. The author's acknowledgments, however, says that this the end. I admit to being disappointed, since I want to read more about Fergus and there are numerous loose ends that could be explored. More importantly, though, I hope Palmer will write more novels in any universe of her choosing so that I can buy and read them.

This is fantastic stuff. This review comes too late for the Hugo nominating deadline, but I hope Palmer gets a Best Series nomination for the Finder Chronicles as a whole. She deserves it.

Rating: 9 out of 10

“Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.” -U.S. Constitution, First Amendment.

Image: Shutterstock, zimmytws.

In an address to Congress this month, President Trump claimed he had “brought free speech back to America.” But barely two months into his second term, the president has waged an unprecedented attack on the First Amendment rights of journalists, students, universities, government workers, lawyers and judges.

This story explores a slew of recent actions by the Trump administration that threaten to undermine all five pillars of the First Amendment to the U.S. Constitution, which guarantees freedoms concerning speech, religion, the media, the right to assembly, and the right to petition the government and seek redress for wrongs.

THE RIGHT TO PETITION

The right to petition allows citizens to communicate with the government, whether to complain, request action, or share viewpoints — without fear of reprisal. But that right is being assaulted by this administration on multiple levels. For starters, many GOP lawmakers are now heeding their leadership’s advice to stay away from local town hall meetings and avoid the wrath of constituents affected by the administration’s many federal budget and workforce cuts.

Another example: President Trump recently fired most of the people involved in processing Freedom of Information Act (FOIA) requests for government agencies. FOIA is an indispensable tool used by journalists and the public to request government records, and to hold leaders accountable.

The biggest story by far this week was the bombshell from The Atlantic editor Jeffrey Goldberg, who recounted how he was inadvertently added to a Signal group chat with National Security Advisor Michael Waltz and 16 other Trump administration officials discussing plans for an upcoming attack on Yemen.

One overlooked aspect of Goldberg’s incredible account is that by planning and coordinating the attack on Signal — which features messages that can auto-delete after a short time — administration officials were evidently seeking a way to avoid creating a lasting (and potentially FOIA-able) record of their deliberations.

“Intentional or not, use of Signal in this context was an act of erasure—because without Jeffrey Goldberg being accidentally added to the list, the general public would never have any record of these communications or any way to know they even occurred,” Tony Bradley wrote this week at Forbes.

Petitioning the government, particularly when it ignores your requests, often requires challenging federal agencies in court. But that becomes far more difficult if the most competent law firms start to shy away from cases that may involve crossing the president and his administration.

On March 22, the president issued a memorandum that directs heads of the Justice and Homeland Security Departments to “seek sanctions against attorneys and law firms who engage in frivolous, unreasonable and vexatious litigation against the United States,” or in matters that come before federal agencies.

The POTUS recently issued several executive orders railing against specific law firms with attorneys who worked legal cases against him. On Friday, the president announced that the law firm of Skadden, Arps, Slate, Meager & Flom had agreed to provide $100 million in pro bono work on issues that he supports.

Trump issued another order naming the firm Paul, Weiss, Rifkind, Wharton & Garrison, which ultimately agreed to pledge $40 million in pro bono legal services to the president’s causes.

Other Trump executive orders targeted law firms Jenner & Block and WilmerHale, both of which have attorneys that worked with special counsel Robert Mueller on the investigation into Russian interference in the 2016 election. But this week, two federal judges in separate rulings froze parts of those orders.

“There is no doubt this retaliatory action chills speech and legal advocacy, and that is qualified as a constitutional harm,” wrote Judge Richard Leon, who ruled against the executive order targeting WilmerHale.

President Trump recently took the extraordinary step of calling for the impeachment of federal judges who rule against the administration. Trump called U.S. District Judge James Boasberg a “Radical Left Lunatic” and urged he be removed from office for blocking deportation of Venezuelan alleged gang members under a rarely invoked wartime legal authority.

In a rare public rebuke to a sitting president, U.S. Supreme Court Justice John Roberts issued a statement on March 18 pointing out that “For more than two centuries, it has been established that impeachment is not an appropriate response to disagreement concerning a judicial decision.”

The U.S. Constitution provides that judges can be removed from office only through impeachment by the House of Representatives and conviction by the Senate. The Constitution also states that judges’ salaries cannot be reduced while they are in office.

Undeterred, House Speaker Mike Johnson this week suggested the administration could still use the power of its purse to keep courts in line, and even floated the idea of wholesale eliminating federal courts.

“We do have authority over the federal courts as you know,” Johnson said. “We can eliminate an entire district court. We have power of funding over the courts, and all these other things. But desperate times call for desperate measures, and Congress is going to act, so stay tuned for that.”

FREEDOM OF ASSEMBLY

President Trump has taken a number of actions to discourage lawful demonstrations at universities and colleges across the country, threatening to cut federal funding for any college that supports protests he deems “illegal.”

A Trump executive order in January outlined a broad federal crackdown on what he called “the explosion of antisemitism” on U.S. college campuses. This administration has asserted that foreign students who are lawfully in the United States on visas do not enjoy the same free speech or due process rights as citizens.

Reuters reports that the acting civil rights director at the Department of Education on March 10 sent letters to 60 educational institutions warning they could lose federal funding if they don’t do more to combat anti-semitism. On March 20, Trump issued an order calling for the closure of the Education Department.

Meanwhile, U.S. Immigration and Customs Enforcement (ICE) agents have been detaining and trying to deport pro-Palestinian students who are legally in the United States. The administration is targeting students and academics who spoke out against Israel’s attacks on Gaza, or who were active in campus protests against U.S. support for the attacks. Secretary of State Marco Rubio told reporters Thursday that at least 300 foreign students have seen their visas revoked under President Trump, a far higher number than was previously known.

In his first term, Trump threatened to use the national guard or the U.S. military to deal with protesters, and in campaigning for re-election he promised to revisit the idea.

“I think the bigger problem is the enemy from within,” Trump told Fox News in October 2024. “We have some very bad people. We have some sick people, radical left lunatics. And I think they’re the big — and it should be very easily handled by, if necessary, by National Guard, or if really necessary, by the military, because they can’t let that happen.”

This term, Trump acted swiftly to remove the top judicial advocates in the armed forces who would almost certainly push back on any request by the president to use U.S. soldiers in an effort to quell public protests, or to arrest and detain immigrants. In late February, the president and Defense Secretary Pete Hegseth fired the top legal officers for the military services — those responsible for ensuring the Uniform Code of Military Justice is followed by commanders.

Military.com warns that the purge “sets an alarming precedent for a crucial job in the military, as President Donald Trump has mused about using the military in unorthodox and potentially illegal ways.” Hegseth told reporters the removals were necessary because he didn’t want them to pose any “roadblocks to orders that are given by a commander in chief.”

FREEDOM OF THE PRESS

President Trump has sued a number of U.S. news outlets, including 60 Minutes, CNN, The Washington Post, The New York Times and other smaller media organizations for unflattering coverage.

In a $10 billion lawsuit against 60 Minutes and its parent Paramount, Trump claims they selectively edited an interview with former Vice President Kamala Harris prior to the 2024 election. The TV news show last month published transcripts of the interview at the heart of the dispute, but Paramount is reportedly considering a settlement to avoid potentially damaging its chances of winning the administration’s approval for a pending multibillion-dollar merger.

The president sued The Des Moines Register and its parent company, Gannett, for publishing a poll showing Trump trailing Harris in the 2024 presidential election in Iowa (a state that went for Trump). The POTUS also is suing the Pulitzer Prize board over 2018 awards given to The New York Times and The Washington Post for their coverage of purported Russian interference in the 2016 election.

Whether or not any of the president’s lawsuits against news organizations have merit or succeed is almost beside the point. The strategy behind suing the media is to make reporters and newsrooms think twice about criticizing or challenging the president and his administration. The president also knows some media outlets will find it more expedient to settle.

Trump also sued ABC News and George Stephanopoulos for stating that the president had been found liable for “rape” in a civil case [Trump was found liable of sexually abusing and defaming E. Jean Carroll]. ABC parent Disney settled that claim by agreeing to donate $15 million to the Trump Presidential Library.

Following the attack on the U.S. Capitol on Jan. 6, 2021, Facebook blocked President Trump’s account. Trump sued Meta, and after the president’s victory in 2024 Meta settled and agreed to pay Trump $25 million: $22 million would go to his presidential library, and the rest to legal fees. Meta CEO Mark Zuckerberg also announced Facebook and Instagram would get rid of fact-checkers and rely instead on reader-submitted “community notes” to debunk disinformation on the social media platform.

Brendan Carr, the president’s pick to run the Federal Communications Commission (FCC), has pledged to “dismantle the censorship cartel and restore free speech rights for everyday Americans.” But on January 22, 2025, the FCC reopened complaints against ABC, CBS and NBC over their coverage of the 2024 election. The previous FCC chair had dismissed the complaints as attacks on the First Amendment and an attempt to weaponize the agency for political purposes.

According to Reuters, the complaints call for an investigation into how ABC News moderated the pre-election TV debate between Trump and Biden, and appearances of then-Vice President Harris on 60 Minutes and on NBC’s “Saturday Night Live.”

Since then, the FCC has opened investigations into NPR and PBS, alleging that they are breaking sponsorship rules. The Center for Democracy & Technology (CDT), a think tank based in Washington, D.C., noted that the FCC is also investigating KCBS in San Francisco for reporting on the location of federal immigration authorities.

“Even if these investigations are ultimately closed without action, the mere fact of opening them – and the implicit threat to the news stations’ license to operate – can have the effect of deterring the press from news coverage that the Administration dislikes,” the CDT’s Kate Ruane observed.

Trump has repeatedly threatened to “open up” libel laws, with the goal of making it easier to sue media organizations for unfavorable coverage. But this week, the U.S. Supreme Court declined to hear a challenge brought by Trump donor and Las Vegas casino magnate Steve Wynn to overturn the landmark 1964 decision in New York Times v. Sullivan, which insulates the press from libel suits over good-faith criticism of public figures.

The president also has insisted on picking which reporters and news outlets should be allowed to cover White House events and participate in the press pool that trails the president. He barred the Associated Press from the White House and Air Force One over their refusal to call the Gulf of Mexico by another name.

And the Defense Department has ordered a number of top media outlets to vacate their spots at the Pentagon, including CNN, The Hill, The Washington Post, The New York Times, NBC News, Politico and National Public Radio.

“Incoming media outlets include the New York Post, Breitbart, the Washington Examiner, the Free Press, the Daily Caller, Newsmax, the Huffington Post and One America News Network, most of whom are seen as conservative or favoring Republican President Donald Trump,” Reuters reported.

FREEDOM OF SPEECH

Shortly after Trump took office again in January 2025, the administration began circulating lists of hundreds of words that government staff and agencies shall not use in their reports and communications.

The Brookings Institution notes that in moving to comply with this anti-speech directive, federal agencies have purged countless taxpayer-funded data sets from a swathe of government websites, including data on crime, sexual orientation, gender, education, climate, and global development.

The New York Times reports that in the past two months, hundreds of terabytes of digital resources analyzing data have been taken off government websites.

“While in many cases the underlying data still exists, the tools that make it possible for the public and researchers to use that data have been removed,” The Times wrote.

On Jan. 27, Trump issued a memo (PDF) that paused all federally funded programs pending a review of those programs for alignment with the administration’s priorities. Among those was ensuring that no funding goes toward advancing “Marxist equity, transgenderism, and green new deal social engineering policies.”

According to the CDT, this order is a blatant attempt to force government grantees to cease engaging in speech that the current administration dislikes, including speech about the benefits of diversity, climate change, and LGBTQ issues.

“The First Amendment does not permit the government to discriminate against grantees because it does not like some of the viewpoints they espouse,” the CDT’s Ruane wrote. “Indeed, those groups that are challenging the constitutionality of the order argued as much in their complaint, and have won an injunction blocking its implementation.”

On January 20, the same day Trump issued an executive order on free speech, the president also issued an executive order titled “Reevaluating and Realigning United States Foreign Aid,” which froze funding for programs run by the U.S. Agency for International Development (USAID). Among those were programs designed to empower civil society and human rights groups, journalists and others responding to digital repression and Internet shutdowns.

According to the Electronic Frontier Foundation (EFF), this includes many freedom technologies that use cryptography, fight censorship, protect freedom of speech, privacy and anonymity for millions of people around the world.

“While the State Department has issued some limited waivers, so far those waivers do not seem to cover the open source internet freedom technologies,” the EFF wrote about the USAID disruptions. “As a result, many of these projects have to stop or severely curtail their work, lay off talented workers, and stop or slow further development.”

On March 14, the president signed another executive order that effectively gutted the U.S. Agency for Global Media (USAGM), which oversees or funds media outlets including Radio Free Europe/Radio Liberty and Voice of America (VOA). The USAGM also oversees Radio Free Asia, which supporters say has been one of the most reliable tools used by the government to combat Chinese propaganda.

But this week, U.S. District Court Judge Royce Lamberth, a Reagan appointee, temporarily blocked USAGM’s closure by the administration.

“RFE/RL has, for decades, operated as one of the organizations that Congress has statutorily designated to carry out this policy,” Lamberth wrote in a 10-page opinion. “The leadership of USAGM cannot, with one sentence of reasoning offering virtually no explanation, force RFE/RL to shut down — even if the President has told them to do so.”

FREEDOM OF RELIGION

The Trump administration rescinded a decades-old policy that instructed officers not to take immigration enforcement actions in or near “sensitive” or “protected” places, such as churches, schools, and hospitals.

That directive was immediately challenged in a case brought by a group of Quakers, Baptists and Sikhs, who argued the policy reversal was keeping people from attending services for fear of being arrested on civil immigration violations. On Feb. 24, a federal judge agreed and blocked ICE agents from entering churches or targeting migrants nearby.

The president’s executive order allegedly addressing antisemitism came with a fact sheet that described college campuses as “infested” with “terrorists” and “jihadists.” Multiple faith groups expressed alarm over the order, saying it attempts to weaponize antisemitism and promote “dehumanizing anti-immigrant policies.”

The president also announced the creation of a “Task Force to Eradicate Anti-Christian Bias,” to be led by Attorney General Pam Bondi. Never mind that Christianity is easily the largest faith in America and that Christians are well-represented in Congress.

The Rev. Paul Brandeis Raushenbush, a Baptist minister and head of the progressive Interfaith Alliance, issued a statement accusing Trump of hypocrisy in claiming to champion religion by creating the task force.

“From allowing immigration raids in churches, to targeting faith-based charities, to suppressing religious diversity, the Trump Administration’s aggressive government overreach is infringing on religious freedom in a way we haven’t seen for generations,” Raushenbush said.

A statement from Americans United for Separation of Church and State said the task force could lead to religious persecution of those with other faiths.

“Rather than protecting religious beliefs, this task force will misuse religious freedom to justify bigotry, discrimination, and the subversion of our civil rights laws,” said Rachel Laser, the group’s president and CEO.

Where is President Trump going with all these blatant attacks on the First Amendment? The president has made no secret of his affection for autocratic leaders and “strongmen” around the world, and he is particularly enamored with Hungary’s far-right Prime Minister Viktor Orbán, who has visited Trump’s Mar-a-Lago resort twice in the past year.

A March 15 essay in The Atlantic by Hungarian investigative journalist András Pethő recounts how Orbán rose to power by consolidating control over the courts, and by building his own media universe while simultaneously placing a stranglehold on the independent press.

“As I watch from afar what’s happening to the free press in the United States during the first weeks of Trump’s second presidency — the verbal bullying, the legal harassment, the buckling by media owners in the face of threats — it all looks very familiar,” Pethő wrote. “The MAGA authorities have learned Orbán’s lessons well.”

This week on my podcast, I read Why I don’t like AI art, a column from last week’s Pluralistic newsletter:

Which brings me to art. As a working artist in his third decade of professional life, I’ve concluded that the point of art is to take a big, numinous, irreducible feeling that fills the artist’s mind, and attempt to infuse that feeling into some artistic vessel – a book, a painting, a song, a dance, a sculpture, etc – in the hopes that this work will cause a loose facsimile of that numinous, irreducible feeling to manifest in someone else’s mind.

Art, in other words, is an act of communication – and there you have the problem with AI art. As a writer, when I write a novel, I make tens – if not hundreds – of thousands of tiny decisions that are in service to this business of causing my big, irreducible, numinous feeling to materialize in your mind. Most of those decisions aren’t even conscious, but they are definitely decisions, and I don’t make them solely on the basis of probabilistic autocomplete. One of my novels may be good and it may be bad, but one thing is definitely is is rich in communicative intent. Every one of those microdecisions is an expression of artistic intent.

MP3

(Image: Cryteria, CC BY 3.0, modified)

Berge Schwebs Bjørlo, aged 40, died on March 4th in an avalanche together with his friend Ulf, while on winter holiday.

When writing about someone who recently died, it is common to make lists. Lists of education, of where they worked, on projects they did.

But Berge wasn't common. Berge was an outlier. A paradox, even.

Berge was one of my closest friends; someone who always listened, someone you could always argue with (“I'm a pacifist, but I'm aware that this is an extreme position”) but could rarely be angry at. But if you ask around, you'll see many who say similar things; how could someone be so close to so many at the same time?

Berge had running jokes going on 20 years or more. Many of them would be related to his background from Bergen; he'd often talk about “the un-central east” (aka Oslo), yet had to admit at some point that actually started liking the city. Or about his innate positivity (“I'm in on everything but suicide and marriage!”). I know a lot of people have described his humor as dry, but I found him anything but. Just a free flow of living.

He lived his life in free software, but rarely in actually writing code; I don't think I've seen a patch from him, and only the occasional bug report. Instead, he would spend his time guiding others; he spent a lot of time in PostgreSQL circles, helping people with installation or writing queries or chiding them for using an ORM (“I don't understand why people love to make life so hard for themselves”) or just discussing life, love and everything. Somehow, some people's legacy is just the number of others they touched, and Berge touched everyone he met. Kindness is not something we do well in the free software community, but somehow, it came natural to him. I didn't understand until after he died why he was so chronically bad at reading backlog and hard to get hold of; he was interacting with so many people, always in the present and never caring much about the past.

I remember that Berge once visited my parents' house, and was greeted by our dog, who after a pat promptly went back to relaxing lazily on the floor. “Awh! If I were a dog, that's the kind of dog I'd be.” In retrospect, for someone who lived a lot of his life in 300 km/h (at times quite literally), it was an odd thing to say, but it was just one of those paradoxes.

Berge loved music. He'd argue for intensely political punk, but would really consume everything with great enthuisasm and interest. One of the last albums I know he listened to was Thomas Dybdahl's “… that great October sound”:

Tear us in different ways but leave a thread throughout the maze
In case I need to find my way back home
All these decisions make for people living without faith
Fumbling in the dark nowhere to roam

Dreamweaver
I'll be needing you tomorrow and for days to come
Cause I'm no daydreamer
But I'll need a place to go if memory fails me & let you slip away

Berge wasn't found by a lazy dog. He was found by Shane, a very good dog.

Somehow, I think he would have approved of that, too.

Version 0.0.21 of RcppSpdlog arrived on CRAN today and has been uploaded to Debian. RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich. You can learn more at the nice package documention site.

This release updates the code to the version 1.15.2 of spdlog which was released this weekend as well.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.21 (2025-03-30)

• Upgraded to upstream release spdlog 1.15.2 (including fmt 11.1.4)

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page, or the package documention site.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

NOTE: I offer a bit of a riff about the rarity of science - not just on Earth but possibly across the cosmos - at the end. 

We are gadually trying to resume 'normal' life after our family suffered a 'disruption' in our living arrangements that has left us frazzled, with little time for blog updates. But things are a bit better now, so here is... a roundup of recent* space news and updates.

*(Well, 'recent' as of when these postings were actually drafted, in January, before we realized how crazy things were gonna get!)

== Heading for the moon ==

The landers contain scientific instruments to analyze the lunar regolith and magnetosphere, and set up a moon-based global navigation system, laying the groundwork for future lunar missions.

*As of March 30... well... any space junkies know how it went.

== Rogue planets all over! ==

One of the imperfectly insufficient (by itself) but substantially plausible theories for the Great Silence or “Fermi Paradox” (terrible name) is that interstellar travel… even at just 10% of light speed… is made very difficult by a minefield of hidden obstacles.  No, I am not talking about my short story “Crystal Spheres.”  But rather, these would be rogue planets that are untethered from stars. Every year we find they are more common in the galaxy.

For example, the infrared-sensitive Webb Telescope has found hundreds… down to Saturn size, just in the Orion Nebula, alone! Forty-two of them are in binary pairs. Wow. Implicit: billions of free-floating planets in the darkness between the stars.

One more incredible accomplishment by this fantastic instrument that this fantastic, scientific civilization created, in our steady and accelerating progress as apprentices in the Laboratory of Creation! 

And yet some ignore the almost (or actual) theological significance of these incredible accomplishments (Robots roaming Mars! New human-made life forms! The new skills to save this beautiful world from … ourselves!) Okay, grad students in Creation’s Lab should respect those who clutch the Kindergarten text given to illiterate shepherds. Fine. 

But those who wage all-out war vs science are clearly the real heretics, here.

See more incredible Webb Wonders!  A way-kewl podcast from Fraser Cain. 

== Monitoring Methane Emissions ==

Among the worst criminals alive today are those who are deliberately venting methane into the atmosphere. After GOP Congresses deliberately canceled or slashed the satellites to track down vents and Trump delayed them, we now, at last, have the policing tools. A satellite that measures methane leaks from oil and gas companies is set to start circulating the Earth 15 times a day next month. Google plans to have the data mapped by the end of the year for the whole world to see. (Thanks Sergey.)

Methane is a potent greenhouse gas estimated to be responsible for nearly a third of human-caused global warming. Scientists say slashing methane emissions is one of the fastest ways to slow the climate crisis because methane has 80 times the warming power of carbon dioxide over a decade. Though farming is the largest source of methane emissions from human activities, the energy sector is a close second. Oil, gas, and coal operations are thought to account for 40% of global methane emissions from human activities. The IEA says focusing on the energy sector should be a priority, in part because reducing methane leaks is cost-effective. Leaking gas can be captured and sold, and the technology to do that is relatively cheap.

Two new methane-detecting satellites - Carbon Mapper and MethaneSAT/EDF are now surveying the planet's climate. Because the Biden admin pushed through the quality methane satellites, the information will be so widely seen that members of the public will be able to act on their own - even despite a suborned EPA and Justice dept.  A case where the right may be bitten by the 'market/consumer alternative to government' that they have long raved about.

== Dark comets, Dwarf galaxies - and Dark Matter ==

If I had followed my original scientific path – not lured away by the likes of you telling me to write more scifi – I’d likely have been in the mix of these studies of “dark comets,” whose orbits get significantly altered by gassy or dusty emissions, the way it happens with regular, icy comets, but without any visible signs of watery volatiles. “dark comets are different from another intermediary category between asteroids and comets, known as active asteroids, although there may be some overlap. Active asteroids are objects without ice that produce a cloud of dust around them, for a variety of reasons…” 

Only the Dark Comets – and some include the odd cigar-shaped interstellar visitor ‘Oumuamua' – still have no firm explanation. Though some theories suggest emission of some volatile substance that doesn’t leave an ionized spectral trace.

I have heard stories of more aggressive interrogation of electronic devices at US border crossings. I know a lot about securing computers, but very little about securing phones.

Are there easy ways to delete data—files, photos, etc.—on phones so it can’t be recovered? Does resetting a phone to factory defaults erase data, or is it still recoverable? That is, does the reset erase the old encryption key, or just sever the password that access that key? When the phone is rebooted, are deleted files still available?

We need answers for both iPhones and Android phones. And it’s not just the US; the world is going to become a more dangerous place to oppose state power.

US National Security Advisor Mike Waltz, who started the now-infamous group chat coordinating a US attack against the Yemen-based Houthis on March 15, is seemingly now suggesting that the secure messaging service Signal has security vulnerabilities.

"I didn’t see this loser in the group," Waltz told Fox News about Atlantic editor in chief Jeffrey Goldberg, whom Waltz invited to the chat. "Whether he did it deliberately or it happened in some other technical mean, is something we’re trying to figure out."

Waltz’s implication that Goldberg may have hacked his way in was followed by a report from CBS News that the US National Security Agency (NSA) had sent out a bulletin to its employees last month warning them about a security "vulnerability" identified in Signal.

The truth, however, is much more interesting. If Signal has vulnerabilities, then China, Russia, and other US adversaries suddenly have a new incentive to discover them. At the same time, the NSA urgently needs to find and fix any vulnerabilities quickly as it can—and similarly, ensure that commercial smartphones are free of backdoors—access points that allow people other than a smartphone’s user to bypass the usual security authentication methods to access the device’s contents.

That is essential for anyone who wants to keep their communications private, which should be all of us.

It’s common knowledge that the NSA’s mission is breaking into and eavesdropping on other countries’ networks. (During President George W. Bush’s administration, the NSA conducted warrantless taps into domestic communications as well—surveillance that several district courts ruled to be illegal before those decisions were later overturned by appeals courts. To this day, many legal experts maintain that the program violated federal privacy protections.) But the organization has a secondary, complementary responsibility: to protect US communications from others who want to spy on them. That is to say: While one part of the NSA is listening into foreign communications, another part is stopping foreigners from doing the same to Americans.

Those missions never contradicted during the Cold War, when allied and enemy communications were wholly separate. Today, though, everyone uses the same computers, the same software, and the same networks. That creates a tension.

When the NSA discovers a technological vulnerability in a service such as Signal (or buys one on the thriving clandestine vulnerability market), does it exploit it in secret, or reveal it so that it can be fixed? Since at least 2014, a US government interagency "equities" process has been used to decide whether it is in the national interest to take advantage of a particular security flaw, or to fix it. The trade-offs are often complicated and hard.

Waltz—along with Vice President J.D. Vance, Defense Secretary Pete Hegseth, and the other officials in the Signal group—have just made the trade-offs much tougher to resolve. Signal is both widely available and widely used. Smaller governments that can’t afford their own military-grade encryption use it. Journalists, human rights workers, persecuted minorities, dissidents, corporate executives, and criminals around the world use it. Many of these populations are of great interest to the NSA.

At the same time, as we have now discovered, the app is being used for operational US military traffic. So, what does the NSA do if it finds a security flaw in Signal?

Previously, it might have preferred to keep the flaw quiet and use it to listen to adversaries. Now, if the agency does that, it risks someone else finding the same vulnerability and using it against the US government. And if it was later disclosed that the NSA could have fixed the problem and didn’t, then the results might be catastrophic for the agency.

Smartphones present a similar trade-off. The biggest risk of eavesdropping on a Signal conversation comes from the individual phones that the app is running on. While it’s largely unclear whether the US officials involved had downloaded the app onto personal or government-issued phones—although Witkoff suggested on X that the program was on his "personal devices"—smartphones are consumer devices, not at all suitable for classified US government conversations. An entire industry of spyware companies sells capabilities to remotely hack smartphones for any country willing to pay. More capable countries have more sophisticated operations. Just last year, attacks that were later attributed to China attempted to access both President Donald Trump and Vance’s smartphones. Previously, the FBI—as well as law enforcement agencies in other countries—have pressured both Apple and Google to add "backdoors" in their phones to more easily facilitate court-authorized eavesdropping.

These backdoors would create, of course, another vulnerability to be exploited. A separate attack from China last year accessed a similar capability built into US telecommunications networks.

The vulnerabilities equities have swung against weakened smartphone security and toward protecting the devices that senior government officials now use to discuss military secrets. That also means that they have swung against the US government hoarding Signal vulnerabilities—and toward full disclosure.

This is plausibly good news for Americans who want to talk among themselves without having anyone, government or otherwise, listen in. We don’t know what pressure the Trump administration is using to make intelligence services fall into line, but it isn’t crazy to worry that the NSA might again start monitoring domestic communications.

Because of the Signal chat leak, it’s less likely that they’ll use vulnerabilities in Signal to do that. Equally, bad actors such as drug cartels may also feel safer using Signal. Their security against the US government lies in the fact that the US government shares their vulnerabilities. No one wants their secrets exposed.

I have long advocated for a "defense dominant" cybersecurity strategy. As long as smartphones are in the pocket of every government official, police officer, judge, CEO, and nuclear power plant operator—and now that they are being used for what the White House now calls calls  "sensitive," if not outright classified conversations among cabinet members—we need them to be as secure as possible. And that means no government-mandated backdoors.

We may find out more about how officials—including the vice president of the United States—came to be using Signal on what seem to be consumer-grade smartphones, in a apparent breach of the laws on government records. It’s unlikely that they really thought through the consequences of their actions.

Nonetheless, those consequences are real. Other governments, possibly including US allies, will now have much more incentive to break Signal’s security than they did in the past, and more incentive to hack US government smartphones than they did before March 24.

For just the same reason, the US government has urgent incentives to protect them.

This essay was originally published in Foreign Policy.

A new release 0.1.8 of RcppZiggurat is now on the CRAN network for R, following up on the 0.1.7 release last week which was the first release in four and a half years.

The RcppZiggurat package updates the code for the Ziggurat generator by Marsaglia and others which provides very fast draws from a Normal (or Exponential) distribution. The package provides a simple C++ wrapper class for the generator improving on the very basic macros, and permits comparison among several existing Ziggurat implementations. This can be seen in the figure where Ziggurat from this package dominates accessing the implementations from the GSL, QuantLib and Gretl—all of which are still way faster than the default Normal generator in R (which is of course of higher code complexity).

This release switches the vignette to the standard trick of premaking it as a pdf and including it in a short Sweave document that imports it via pdfpages; this minimizes build-time dependencies on other TeXLive components. It also incorporates a change contributed by Tomas to rely on the system build of the GSL on Windows as well if Rtools 42 or later is found. No other changes.

The NEWS file entry below lists all changes.

Changes in version 0.1.8 (2025-03-30)

• The vignette is now premade and rendered as Rnw via pdfpage to minimize the need for TeXLive package at build / install time (Dirk)

• Windows builds now use the GNU GSL when Rtools is 42 or later (Tomas Kalibera in #25)

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the Rcppziggurat page or the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Author: Jessica Pickard Once again Sam asked himself why he was standing here, in this field, miles out of town, staring into an increasingly dusky sky. Well he was here for the money of course. God knows he could use that right now. But he was also here, if he was honest, for the girl, […]

The post The Unsuitable Girl appeared first on 365tomorrows.

Review: Cascade Failure, by L.M. Sagas

Cascade Failure is a far-future science fiction adventure with a small helping of cyberpunk vibes. It is the first of a (so far) two-book series, and was the author's first novel.

The Ambit is an old and small Guild ship, not much to look at, but it holds a couple of surprises. One is its captain, Eoan, who is an AI with a deep and insatiable curiosity that has driven them and their ship farther and farther out into the Spiral. The other is its surprisingly competent crew: a battle-scarred veteran named Saint who handles the fighting, and a talented engineer named Nash who does literally everything else. The novel opens with them taking on supplies at Aron Outpost. A supposed Guild deserter named Jalsen wanders into the ship looking for work.

An AI ship with a found-family crew is normally my catnip, so I wanted to love this book. Alas, I did not.

There were parts I liked. Nash is great: snarky, competent, and direct. Eoan is a bit distant and slightly more simplistic of a character than I was expecting, but I appreciated the way Sagas put them firmly in charge of the ship and departed from the conventional AI character presentation. Once the plot starts in earnest (more on that in a moment), we meet Anke, the computer hacker, whose charming anxiety reaction is a complete inability to stop talking and who adds some needed depth to the character interactions. There's plenty of action, a plot that makes at least some sense, and a few moments that almost achieved the emotional payoff the author was attempting.

Unfortunately, most of the story focuses on Saint and Jal, and both of them are irritatingly dense cliches.

The moment Jal wanders onto the Ambit in the first chapter, the reader is informed that Jal, Saint, and Eoan have a history. The crew of the Ambit spent a year looking for Jal and aren't letting go of him now that they've found him. Jal, on the other hand, clearly blames Saint for something and is not inclined to trust him. Okay, fine, a bit generic of a setup but the writing moved right along and I was curious enough.

It then takes a full 180 pages before the reader finds out what the hell is going on with Saint and Jal. Predictably, it's a stupid misunderstanding that could have been cleared up with one conversation in the second chapter.

Cascade Failure does not contain a romance (and to the extent that it hints at one, it's a sapphic romance), but I swear Saint and Jal are both the male protagonist from a certain type of stereotypical heterosexual romance novel. They're both the brooding man with the past, who is too hurt to trust anyone and assumes the worst because he's unable to use his words or ask an open question and then listen to the answer. The first half of this book is them being sullen at each other at great length while both of them feel miserable. Jal keeps doing weird and suspicious things to resolve a problem that would have been far more easily resolved by the rest of the crew if he would offer any explanation at all. It's not even suspenseful; we've read about this character enough times to know that he'll turn out to have a heart of gold and everything will be a misunderstanding. I found it tedious. Maybe people who like slow burn romances with this character type will have a less negative reaction.

The real plot starts at about the time Saint and Jal finally get their shit sorted out. It turns out to have almost nothing to do with either of them. The environmental control systems of worlds are suddenly failing (hence the book title), and Anke, the late-arriving computer programmer and terraforming specialist, has a rather wild theory about what's happening. This leads to a lot of action, some decent twists, and a plot that felt very cyberpunk to me, although unfortunately it culminates in an absurdly-cliched action climax.

This book is an action movie that desperately wants to make you feel all the feels, and it worked about as well as that typically works in action movies for me. Jaded cynicism and an inability to communicate are not the ways to get me to have an emotional reaction to a book, and Jal (once he finally starts talking) is so ridiculously earnest that it's like reading the adventures of a Labrador puppy. There was enough going on that it kept me reading, but not enough for the story to feel satisfying. I needed a twist, some depth, way more Nash and Anke and way less of the men, something.

Everyone is going to compare this book to Firefly, but Firefly had better banter, created more complex character interactions due to the larger and more varied crew, and played the cynical mercenary for laughs instead of straight, all of which suited me better. This is not a bad book, particularly once it gets past the halfway point, but it's not that memorable either, at least for me. If you're looking for a space adventure with heavy action hero and military SF vibes that wants to be about Big Feelings but gets there in mostly obvious ways, you could do worse. If you're looking for a found-family starship crew story more like Becky Chambers, I think you'll find this one a bit too shallow and obvious.

Not really recommended, although there's nothing that wrong with it and I'm sure other people's experience will differ.

Followed by Gravity Lost, which I'm unlikely to read.

Rating: 6 out of 10

A second maintenance release of our still young-ish package tinythemes arrived on CRAN today. tinythemes provides the theme_ipsum_rc() function from hrbrthemes by Bob Rudis in a zero (added) dependency way. A simple example is (also available as a demo inside the package) contrasts the default style (on the left) with the one added by this package (on the right):

This version responds solely to things CRAN now nags about. As these are all package quality improvement we generally oblige happily (and generally fix in the respective package repo when we notice). I am currently on a quest to get most/all of my nags down so new releases are sometimes the way to go even when not under a ‘deadline’ gun (as with other releases this week).

The full set of changes since the last release (a little over a year ago) follows.

Changes in tinythemes version 0.0.3 (2025-03-29)

• Updated a badge URL in README.md

• Updated manual pages with proper anchor links

• Rewrote one example without pipe to not require minimum R version

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the repo where comments and suggestions are welcome.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Following the 1.2.0beta1 release two weeks ago, a final 1.2.0 release of theora was wrapped up today. This new release is tagged in the Xiph gitlab theora instance and you can fetch it from the Theora home page as soon as someone with access find time to update the web pages. In the mean time (automatically removed after 14 days) the release tarball is also available as a git build artifact from CI build of the release tag.

The list of changes since The 1.2.0beta release from the CHANGES file in the tarball look like this:

libtheora 1.2.0 (2025 March 29)

• Bumped minor SONAME versions as oc_comment_unpack() implementation changed.
• Added example wrapper script encoder_example_ffmpeg (#1601 #2336).
• Improve comment handling on platforms where malloc(0) return NULL (#2304).
• Added pragma in example code to quiet clang op precedenca warnings.
• Adjusted encoder_example help text.
• Adjusted README, CHANGES, pkg-config and spec files to better reflect current release (#2331 #2328).
• Corrected english typos in source and build system.
• Switched http links to https in doc and comments where relevant. Did not touch RFC drafts.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Author: John Adinolfi Caleb lived alone, as did Cole. Caleb by circumstance, Cole by choice. Trina had entertained a variety of live-in partners, but all were short associations. She lived alone. Each of their homes was unexceptional, except for sharing an extraordinary view of the Pacific below. Sitting on the edge of a cliff, surrounded […]

The post The Memory Hour appeared first on 365tomorrows.

The diffoscope maintainers are pleased to announce the release of diffoscope version 293. This version includes the following changes:

[ Chris Lamb ]
* Correct import masking issue.

You find out more by visiting the project homepage.

The diffoscope maintainers are pleased to announce the release of diffoscope version 292. This version includes the following changes:

[ Ivan Trubach ]
* Ignore st_size entry for directories to avoid spurious diffs as this value
  is essentially filesystem dependent.

[ Chris Lamb ]
* Update copyright years.

You find out more by visiting the project homepage.

Rust, and resistance to it in some parts of the Linux community, has been in my feed recently. One undercurrent seems to be the notion that Rust is woke (and should therefore be rejected as part of culture wars).

I’m going to argue that Rust, the language, is woke. So the opponents are right, in that sense. Of course, as ever, dissing something for being woke is nasty and fascist-adjacent.

• Community
• Technological values - particularly, compared to C/C++
  • Ostensible values
  • Attitude to the programmer’s mistakes
  • The ideology of the hardcore programmer
  • Memory safety as a power struggle
  • Memory safety via Rust as a power struggle
• Notes
  • This is not a RIIR manifesto
  • Disclosure
  • On the meaning of “woke”
• Pithy conclusion

Community

The obvious way that Rust may seem woke is that it has the trappings, and many of the attitudes and outcomes, of a modern, nice, FLOSS community. Rust certainly does better than toxic environments like the Linux kernel, or Debian. This is reflected in a higher proportion of contributors from various kinds of minoritised groups. But Rust is not outstanding in this respect. It certainly has its problems. Many other projects do as well or better.

And this is well-trodden ground. I have something more interesting to say:

Technological values - particularly, compared to C/C++

Rust is woke technology that embodies a woke understanding of what it means to be a programming language.

Ostensible values

Let’s start with Rust’s strapline:

A language empowering everyone to build reliable and efficient software.

Surprisingly, this motto is not mere marketing puff. For Rustaceans, it is a key goal which strongly influences day-to-day decisions (big and small).

Empowering everyone is a key aspect of this, which aligns with my own personal values. In the Rust community, we care about empowerment. We are trying to help liberate our users. And we want to empower everyone because everyone is entitled to technological autonomy. (For a programming language, empowering individuals means empowering their communities, of course.)

This is all very airy-fairy, but it has concrete consequences:

Attitude to the programmer’s mistakes

In Rust we consider it a key part of our job to help the programmer avoid mistakes; to limit the consequences of mistakes; and to guide programmers in useful directions.

If you write a bug in your Rust program, Rust doesn’t blame you. Rust asks “how could the compiler have spotted that bug”.

This is in sharp contrast to C (and C++). C nowadays is an insanely hostile programming environment. A C compiler relentlessly scours your program for any place where you may have violated C’s almost incomprehensible rules, so that it can compile your apparently-correct program into a buggy executable. And then the bug is considered your fault.

These aren’t just attitudes implicitly embodied in the software. They are concrete opinions expressed by compiler authors, and also by language proponents. In other words:

Rust sees programmers writing bugs as a systemic problem, which must be addressed by improvements to the environment and the system. The toxic parts of the C and C++ community see bugs as moral failings by individual programmers.

Sound familiar?

The ideology of the hardcore programmer

Programming has long suffered from the myth of the “rockstar”. Silicon Valley techbro culture loves this notion.

In reality, though, modern information systems are far too complicated for a single person. Developing systems is a team sport. Nontechnical, and technical-adjacent, skills are vital: clear but friendly communication; obtaining and incorporating the insights of every member of your team; willingness to be challenged. Community building. Collaboration. Governance.

The hardcore C community embraces the rockstar myth: they imagine that a few super-programmers (or super-reviewers) are able to spot bugs, just by being so brilliant. Of course this doesn’t actually work at all, as we can see from the atrocious bugfest that is the Linux kernel.

These “rockstars” want us to believe that there is a steep hierarchy in programmming; that they are at the top of this hierarchy; and that being nice isn’t important.

Sound familiar?

Memory safety as a power struggle

Much of the modern crisis of software reliability arises from memory-unsafe programming languages, mostly C and C++.

Addressing this is a big job, requiring many changes. This threatens powerful interests; notably, corporations who want to keep shipping junk. (See also, conniptions over the EU Product Liability Directive.)

The harms of this serious problem mostly fall on society at large, but the convenience of carrying on as before benefits existing powerful interests.

Sound familiar?

Memory safety via Rust as a power struggle

Addressing this problem via Rust is a direct threat to the power of established C programmers such as gatekeepers in the Linux kernel. Supplanting C means they will have to learn new things, and jostle for status against better Rustaceans, or be replaced. More broadly, Rust shows that it is practical to write fast, reliable, software, and that this does not need (mythical) “rockstars”.

So established C programmer “experts” are existing vested interests, whose power is undermined by (this approach to) tackling this serious problem.

Sound familiar?

Notes

This is not a RIIR manifesto

I’m not saying we should rewrite all the world’s C in Rust. We should not try to do that.

Rust is often a good choice for new code, or when a rewrite or substantial overhaul is needed anyway. But we’re going to need other techniques to deal with all of our existing C. CHERI is a very promising approach. Sandboxing, emulation and automatic translation are other possibilities. The problem is a big one and we need a toolkit, not a magic bullet.

But as for Linux: it is a scandal that substantial new drivers and subsystems are still being written in C. We could have been using Rust for new code throughout Linux years ago, and avoided very many bugs. Those bugs are doing real harm. This is not OK.

Disclosure

I first learned C from K&R I in 1989. I spent the first three decades of my life as a working programmer writing lots and lots of C. I’ve written C++ too. I used to consider myself an expert C programmer, but nowadays my C is a bit rusty and out of date. Why is my C rusty? Because I found Rust, and immediately liked and adopted it (despite its many faults).

I like Rust because I care that the software I write actually works: I care that my code doesn’t do harm in the world.

On the meaning of “woke”

The original meaning of “woke” is something much more specific, to do with racism. For the avoidance of doubt, I don’t think Rust is particularly antiracist.

I’m using “woke” (like Rust’s opponents are) in the much broader, and now much more prevalent, culture wars sense.

Pithy conclusion

If you’re a senior developer who knows only C/C++, doesn’t want their authority challenged, and doesn’t want to have to learn how to write better software, you should hate Rust.

Also you should be fired.

comments

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1236 other packages on CRAN, downloaded 39 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 620 times according to Google Scholar.

This release brings a small upstream bug fix to the two FFTW3-interfacing functions, something not likely to hit many CRAN packages.

The changes since the last and fairly recent CRAN release are summarised below.

Changes in RcppArmadillo version 14.4.1-1 (2025-03-27)

• Upgraded to Armadillo release 14.4.1 (Filtered Espresso)

  • Fix for fft() and ifft() when using FFTW3 in multi-threaded contexts (such as OpenMP)

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

In another rare squid/cybersecurity intersection, APT37 is also known as “Squid Werewolf.”

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

We got an unusual rash of submissions at Error'd this week. Here are five reasonably good ones chosen not exactly at random. For those few (everyone) who didn't catch the off-by-one from last week's batch, there's the clue.

"Gotta CAPTCHA 'Em All," puns Alex G. "So do I select them all?" he wondered. I think the correct answer is null.

"What does a null eat?" wondered B.J.H , "and is one null invited or five?". The first question is easily answered. NaaN, of course. Probably garlic. I would expect B.J. to already know the eating habits of a long-standing companion, so I am guessing that the whole family is not meant to tag along. Stick with just the one.

Planespotter Rick R. caught this one at the airport. "Watching my daughter's flight from New York and got surprised by Boeing's new supersonic 737 having already arrived in DFW," he observed. I'm not quite sure what went wrong. It's not the most obvious time zone mistake I can imagine, but I'm pretty sure the cure is the same: all times displayed in any context that is not purely restricted to a single location (and short time frame) should explicitly include the relevant timezone.

Rob H. figures "From my day job's MECM Software Center. It appears that autocorrect has miscalculated, because the internet cannot be calculated." The internet is -1.

Ending this week on a note of hope, global warrior Stewart may have just saved the planet. "Climate change is solved. We just need to replicate the 19 March performance of my new solar panels." Or perhaps I miscalculated.

[Advertisement] Keep all your packages and Docker containers in one place, scan for vulnerabilities, and control who can access different feeds. ProGet installs in minutes and has a powerful free version with a lot of great features that you can upgrade when ready.Learn more.

Author: Hillary Lyon With a well-worn key in hand, Bonnie unlocked the massive front door of her great-uncle Duran’s house. The place sat unoccupied since his passing; it had taken forever for his will to slog through probate. She’d been his favorite family member, and he, hers. His death made her face her own mortality; […]

The post When Next the Fractals Bloom appeared first on 365tomorrows.

As I write this in March 2025, there is a lot of confusion about Signal messenger due to the recent news of people using Signal in government, and subsequent leaks.

The short version is: there was no problem with Signal here. People were using it because they understood it to be secure, not the other way around.

Both the government and the Electronic Frontier Foundation recommend people use Signal. This is an unusual alliance, and in the case of the government, was prompted because it understood other countries had a persistent attack against American telephone companies and SMS traffic.

So let’s dive in. I’ll cover some basics of what security is, what happened in this situation, and why Signal is a good idea.

This post isn’t for programmers that work with cryptography every day. Rather, I hope it can make some of these concepts accessible to everyone else.

What makes communications secure?

When most people are talking about secure communications, they mean some combination of these properties:

1. Privacy - nobody except the intended recipient can decode a message.
2. Authentication - guarantees that the person you are chatting with really is the intended recipient.
3. Ephemerality - preventing a record of the communication from being stored. That is, making it more like a conversation around the table than a written email.
4. Anonymity - keeping your set of contacts to yourself and even obfuscating the fact that communications are occurring.

If you think about it, most people care the most about the first two. In fact, authentication is a key part of privacy. There is an attack known as man in the middle in which somebody pretends to be the intended recipient. The interceptor reads the messages, and then passes them on to the real intended recipient. So we can’t really have privacy without authentication.

I’ll have more to say about these later. For now, let’s discuss attack scenarios.

What compromises security?

There are a number of ways that security can be compromised. Let’s think through some of them:

Communications infrastructure snooping

Let’s say you used no encryption at all, and connected to public WiFi in a coffee shop to send your message. Who all could potentially see it?

• The owner of the coffee shop’s WiFi
• The coffee shop’s Internet provider
• The recipient’s Internet provider
• Any Internet providers along the network between the sender and the recipient
• Any government or institution that can compel any of the above to hand over copies of the traffic
• Any hackers that compromise any of the above systems

Back in the early days of the Internet, most traffic had no encryption. People were careful about putting their credit cards into webpages and emails because they knew it was easy to intercept them. We have been on a decades-long evolution towards more pervasive encryption, which is a good thing.

Text messages (SMS) follow a similar path to the above scenario, and are unencrypted. We know that all of the above are ways people’s texts can be compromised; for instance, governments can issue search warrants to obtain copies of texts, and China is believed to have a persistent hack into western telcos. SMS fails all four of our attributes of secure communication above (privacy, authentication, ephemerality, and anonymity).

Also, think about what information is collected from SMS and by who. Texts you send could be retained in your phone, the recipient’s phone, your phone company, their phone company, and so forth. They might also live in cloud backups of your devices. You only have control over your own phone’s retention.

So defenses against this involve things like:

• Strong end-to-end encryption, so no intermediate party – even the people that make the app – can snoop on it.
• Using strong authentication of your peers
• Taking steps to prevent even app developers from being able to see your contact list or communication history

You may see some other apps saying they use strong encryption or use the Signal protocol. But while they may do that for some or all of your message content, they may still upload your contact list, history, location, etc. to a central location where it is still vulnerable to these kinds of attacks.

When you think about anonymity, think about it like this: if you send a letter to a friend every week, every postal carrier that transports it – even if they never open it or attempt to peak inside – will be able to read the envelope and know that you communicate on a certain schedule with that friend. The same can be said of SMS, email, or most encrypted chat operators. Signal’s design prevents it from retaining even this information, though nation-states or ISPs might still be able to notice patterns (every time you send something via Signal, your contact receives something from Signal a few milliseconds later). It is very difficult to provide perfect anonymity from well-funded adversaries, even if you can provide very good privacy.

Device compromise

Let’s say you use an app with strong end-to-end encryption. This takes away some of the easiest ways someone could get to your messages. But it doesn’t take away all of them.

What if somebody stole your phone? Perhaps the phone has a password, but if an attacker pulled out the storage unit, could they access your messages without a password? Or maybe they somehow trick or compel you into revealing your password. Now what?

An even simpler attack doesn’t require them to steal your device at all. All they need is a few minutes with it to steal your SIM card. Now they can receive any texts sent to your number - whether from your bank or your friend. Yikes, right?

Signal stores your data in an encrypted form on your device. It can protect it in various ways. One of the most important protections is ephemerality - it can automatically delete your old texts. A text that is securely erased can never fall into the wrong hands if the device is compromised later.

An actively-compromised phone, though, could still give up secrets. For instance, what if a malicious keyboard app sent every keypress to an adversary? Signal is only as secure as the phone it runs on – but still, it protects against a wide variety of attacks.

Untrustworthy communication partner

Perhaps you are sending sensitive information to a contact, but that person doesn’t want to keep it in confidence. There is very little you can do about that technologically; with pretty much any tool out there, nothing stops them from taking a picture of your messages and handing the picture off.

Environmental compromise

Perhaps your device is secure, but a hidden camera still captures what’s on your screen. You can take some steps against things like this, of course.

Human error

Sometimes humans make mistakes. For instance, the reason a reporter got copies of messages recently was because a participant in a group chat accidentally added him (presumably that participant meant to add someone else and just selected the wrong name). Phishing attacks can trick people into revealing passwords or other sensitive data. Humans are, quite often, the weakest link in the chain.

Protecting yourself

So how can you protect yourself against these attacks? Let’s consider:

• Use a secure app like Signal that uses strong end-to-end encryption where even the provider can’t access your messages
• Keep your software and phone up-to-date
• Be careful about phishing attacks and who you add to chat rooms
• Be aware of your surroundings; don’t send sensitive messages where people might be looking over your shoulder with their eyes or cameras

There are other methods besides Signal. For instance, you could install GnuPG (GPG) on a laptop that has no WiFi card or any other way to connect it to the Internet. You could always type your messages on that laptop, encrypt them, copy the encrypted text to a floppy disk (or USB device), take that USB drive to your Internet computer, and send the encrypted message by email or something. It would be exceptionally difficult to break the privacy of messages in that case (though anonymity would be mostly lost). Even if someone got the password to your “secure” laptop, it wouldn’t do them any good unless they physically broke into your house or something. In some ways, it is probably safer than Signal. (For more on this, see my article How gapped is your air?)

But, that approach is hard to use. Many people aren’t familiar with GnuPG. You don’t have the convenience of sending a quick text message from anywhere. Security that is hard to use most often simply isn’t used. That is, you and your friends will probably just revert back to using insecure SMS instead of this GnuPG approach because SMS is so much easier.

Signal strikes a unique balance of providing very good security while also being practical, easy, and useful. For most people, it is the most secure option available.

Signal is also open source; you don’t have to trust that it is as secure as it says, because you can inspect it for yourself. Also, while it’s not federated, I previously addressed that.

Government use

If you are a government, particularly one that is highly consequential to the world, you can imagine that you are a huge target. Other nations are likely spending billions of dollars to compromise your communications. Signal itself might be secure, but if some other government can add spyware to your phones, or conduct a successful phishing attack, you can still have your communications compromised.

I have no direct knowledge, but I think it is generally understood that the US government maintains communications networks that are entirely separate from the Internet and can only be accessed from secure physical locations and secure rooms. These can be even more secure than the average person using Signal because they can protect against things like environmental compromise, human error, and so forth. The scandal in March of 2025 happened because government employees were using Signal rather than official government tools for sensitive information, had taken advantage of Signal’s ephemerality (laws require records to be kept), and through apparent human error had directly shared this information with a reporter. Presumably a reporter would have lacked access to the restricted communications networks in the first place, so that wouldn’t have been possible.

This doesn’t mean that Signal is bad. It just means that somebody that can spend billions of dollars on security can be more secure than you. Signal is still a great tool for people, and in many cases defeats even those that can spend lots of dollars trying to defeat it.

And remember - to use those restricted networks, you have to go to specific rooms in specific buildings. They are still not as convenient as what you carry around in your pocket.

Conclusion

Signal is practical security. Do you want phone companies reading your messages? How about Facebook or X? Have those companies demonstrated that they are completely trustworthy throughout their entire history?

I say no. So, go install Signal. It’s the best, most practical tool we have.

This post is also available on my website, where it may be periodically updated.

A new version 0.1.5 of the tint package arrived at CRAN today. tint provides a style ‘not unlike Tufte’ for use in html and pdf documents created from markdown. The github repo shows several examples in its README, more as usual in the package documentation.

This is the first release in one and a half years and contains only routine maintenance. As CRAN now bags about missing anchors in the common \code{\link{}} use, I added these today. Otherwise the usual mix of updates to continuous integration, to badges and URLs and other small packaging details—but nothing user-facing.

Changes in tint version 0.1.5 (2025-03-27)

• Standard package maintenance for continuous integration, URL updates, and packaging conventions

• Correct two minor nags in the Rd file

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Like each month, have a look at the work funded by Freexian’s Debian LTS offering.

Debian LTS contributors

In February, 18 contributors have been paid to work on Debian LTS, their reports are available:

• Abhijith PA did 10.0h (out of 8.0h assigned and 6.0h from previous period), thus carrying over 4.0h to the next month.
• Adrian Bunk did 12.0h (out of 0.0h assigned and 63.5h from previous period), thus carrying over 51.5h to the next month.
• Andrej Shadura did 10.0h (out of 6.0h assigned and 4.0h from previous period).
• Bastien Roucariès did 20.0h (out of 20.0h assigned).
• Ben Hutchings did 12.0h (out of 8.0h assigned and 16.0h from previous period), thus carrying over 12.0h to the next month.
• Chris Lamb did 18.0h (out of 18.0h assigned).
• Daniel Leidert did 23.0h (out of 20.0h assigned and 6.0h from previous period), thus carrying over 3.0h to the next month.
• Emilio Pozuelo Monfort did 53.0h (out of 53.0h assigned and 0.75h from previous period), thus carrying over 0.75h to the next month.
• Guilhem Moulin did 11.0h (out of 3.25h assigned and 16.75h from previous period), thus carrying over 9.0h to the next month.
• Jochen Sprickerhof did 27.0h (out of 30.0h assigned), thus carrying over 3.0h to the next month.
• Lee Garrett did 11.75h (out of 9.5h assigned and 44.25h from previous period), thus carrying over 42.0h to the next month.
• Markus Koschany did 40.0h (out of 40.0h assigned).
• Roberto C. Sánchez did 7.0h (out of 14.75h assigned and 9.25h from previous period), thus carrying over 17.0h to the next month.
• Santiago Ruano Rincón did 19.75h (out of 21.75h assigned and 3.25h from previous period), thus carrying over 5.25h to the next month.
• Sean Whitton did 6.0h (out of 6.0h assigned).
• Sylvain Beucler did 52.5h (out of 14.75h assigned and 39.0h from previous period), thus carrying over 1.25h to the next month.
• Thorsten Alteholz did 11.0h (out of 11.0h assigned).
• Tobias Frost did 17.0h (out of 17.0h assigned).

Evolution of the situation

In February, we have released 38 DLAs.

• Notable security updates:
  • pam-u2f, prepared by Patrick Winnertz, fixed an authentication bypass vulnerability
  • openjdk-17, prepared by Emilio Pozuelo Monfort, fixed an authorization bypass/information disclosure vulnerability
  • firefox-esr, prepared by Emilio Pozuelo Monfort, fixed several vulnerabilities
  • thunderbird, prepared by Emilio Pozuelo Monfort, fixed several vulnerabilities
  • postgresql-13, prepared by Christoph Berg, fixed an SQL injection vulnerability
  • freerdp2, prepared by Tobias Frost, fixed several vulnerabilities
  • openssh, prepared by Colin Watson, fixed a machine-in-the-middle vulnerability

LTS contributors Emilio Pozuelo Monfort and Santiago Ruano Rincón coordinated the administrative aspects of LTS updates of postgresql-13 and pam-u2f, which were prepared by the respective maintainers, to whom we are most grateful.

As has become the custom of the LTS team, work is under way on a number of package updates targeting Debian 12 (codename “bookworm”) with fixes for a variety of vulnerabilities. In February, Guilhem Moulin prepared an upload of sssd, while several other updates are still in progress. Bastien Roucariès prepared an upload of krb5 for unstable as well.

Given the importance of the Debian Security Tracker to the work of the LTS Team, we regularly contribute improvements to it. LTS contributor Emilio Pozuelo Monfort reviewed and merged a change to improve performance, and then dealt with unexpected issues that arose as a result. He also made improvements in the processing of CVEs which are not applicable to Debian.

Looking to the future (the release of Debian 13, codename “trixie”, and beyond), LTS contributor Santiago Ruano Rincón has initiated a conversation among the broader community involved in the development of Debian. The purpose of the discussion is to explore ways to improve the long term supportability of packages in Debian, specifically by focusing effort on ensuring that each Debian release contains the “best” supported upstream version of packages with a history of security issues.

Thanks to our sponsors

Sponsors that joined recently are in bold.

• Platinum sponsors:
  • Toshiba Corporation (for 113 months)
  • Civil Infrastructure Platform (CIP) (for 81 months)
  • VyOS Inc (for 46 months)
• Gold sponsors:
  • Roche Diagnostics International AG (for 124 months)
  • Akamai - Linode (for 118 months)
  • Babiel GmbH (for 107 months)
  • Plat’Home (for 107 months)
  • University of Oxford (for 64 months)
  • Deveryware (for 51 months)
  • EDF SA (for 35 months)
  • Dataport AöR (for 11 months)
  • CERN (for 8 months)
• Silver sponsors:
  • Domeneshop AS (for 128 months)
  • Nantes Métropole (for 122 months)
  • Univention GmbH (for 114 months)
  • Université Jean Monnet de St Etienne (for 114 months)
  • Ribbon Communications, Inc. (for 108 months)
  • Exonet B.V. (for 98 months)
  • Leibniz Rechenzentrum (for 92 months)
  • Ministère de l’Europe et des Affaires Étrangères (for 76 months)
  • Cloudways by DigitalOcean (for 65 months)
  • Dinahosting SL (for 63 months)
  • Bauer Xcel Media Deutschland KG (for 58 months)
  • Platform.sh SAS (for 58 months)
  • Moxa Inc. (for 52 months)
  • sipgate GmbH (for 49 months)
  • OVH US LLC (for 47 months)
  • Tilburg University (for 47 months)
  • GSI Helmholtzzentrum für Schwerionenforschung GmbH (for 39 months)
  • Soliton Systems K.K. (for 36 months)
  • THINline s.r.o. (for 12 months)
  • Copenhagen Airports A/S (for 5 months)
• Bronze sponsors:
  • Evolix (for 129 months)
  • Seznam.cz, a.s. (for 129 months)
  • Linuxhotel GmbH (for 126 months)
  • Intevation GmbH (for 125 months)
  • Daevel SARL (for 124 months)
  • Bitfolk LTD (for 123 months)
  • Megaspace Internet Services GmbH (for 123 months)
  • Greenbone AG (for 122 months)
  • NUMLOG (for 122 months)
  • WinGo AG (for 122 months)
  • Entr’ouvert (for 113 months)
  • Adfinis AG (for 111 months)
  • GNI MEDIA (for 105 months)
  • Laboratoire LEGI - UMR 5519 / CNRS (for 105 months)
  • Tesorion (for 105 months)
  • Bearstech (for 97 months)
  • LiHAS (for 97 months)
  • Catalyst IT Ltd (for 91 months)
  • Supagro (for 87 months)
  • Demarcq SAS (for 85 months)
  • Université Grenoble Alpes (for 71 months)
  • TouchWeb SAS (for 64 months)
  • SPiN AG (for 60 months)
  • CoreFiling (for 56 months)
  • Institut des sciences cognitives Marc Jeannerod (for 51 months)
  • Observatoire des Sciences de l’Univers de Grenoble (for 48 months)
  • Tem Innovations GmbH (for 43 months)
  • WordFinder.pro (for 42 months)
  • CNRS DT INSU Résif (for 41 months)
  • Alter Way (for 34 months)
  • Institut Camille Jordan (for 24 months)
  • SOBIS Software GmbH (for 8 months)
  • Tuxera Inc.

In What is Intelligence?, Blaise Agüera y Arcas, VP, Fellow and CTO of Technology & Society at Google, explores what intelligence really is, and how AI’s emergence is a natural consequence of evolution. Encompassing decades of theory, existing literature, and recent artificial life experiments, Agüera y Arcas’ research argues that certain modern AI systems do indeed have a claim to intelligence, consciousness, and free will.

This talk is presented as part of a larger project on What is Intelligence?, including a printed book alongside experimental formats which challenge the conventions of academic publishing. It is the inaugural collaborative work of Antikythera, a think tank on the philosophy of technology, and MIT Press, a leading publisher of books and journals at the intersection of science, technology, art, social science, and design.

Help us Beta test Kubuntu Plucky Puffin!

Kubuntu work:

Fixed an issue in apparmor preventing QT6 webengine applications from starting.

Beta testing!

KDE Snaps:

Updated Qt6 to 6.8.2

Updated Kf6 6.11.0

Rolling out 25.04 RC applications! You can find them in the –candidate channel!

Life:

I have decided to strike out on my own. I can’t take any more rejections! Honestly, I don’t blame them, I wouldn’t want a one armed engineer either. However, I have persevered and accomplished quite a bit with my one arm! So I have decided to take a leap of faith and with your support for open source work and a resurrected side gig of web development I will survive. If you can help sponsor my work, anything at all, even a dollar! I would be eternally grateful. I have several methods to do so:

• GoFundMe
• Patreon
• Github
• Donorbox

If you want your cool application packaged in a variety of formats please contact me!

If you want focused help with an annoying bug, please contact me!

Contact me for any and all kinds of help, if I can’t do it, I will say so.

Do you need web work? Someone to maintain your website? I can do that too!

Portfolio

Thank you all for your support in this new adventure!

Many successful phishing attacks result in a financial loss or malware infection. But falling for some phishing scams, like those currently targeting Russians searching online for organizations that are fighting the Kremlin war machine, can cost you your freedom or your life.

The real website of the Ukrainian paramilitary group “Freedom of Russia” legion. The text has been machine-translated from Russian.

Researchers at the security firm Silent Push mapped a network of several dozen phishing domains that spoof the recruitment websites of Ukrainian paramilitary groups, as well as Ukrainian government intelligence sites.

The website legiohliberty[.]army features a carbon copy of the homepage for the Freedom of Russia Legion (a.k.a. “Free Russia Legion”), a three-year-old Ukraine-based paramilitary unit made up of Russian citizens who oppose Vladimir Putin and his invasion of Ukraine.

The phony version of that website copies the legitimate site — legionliberty[.]army — providing an interactive Google Form where interested applicants can share their contact and personal details. The form asks visitors to provide their name, gender, age, email address and/or Telegram handle, country, citizenship, experience in the armed forces; political views; motivations for joining; and any bad habits.

“Participation in such anti-war actions is considered illegal in the Russian Federation, and participating citizens are regularly charged and arrested,” Silent Push wrote in a report released today. “All observed campaigns had similar traits and shared a common objective: collecting personal information from site-visiting victims. Our team believes it is likely that this campaign is the work of either Russian Intelligence Services or a threat actor with similarly aligned motives.”

Silent Push’s Zach Edwards said the fake Legion Liberty site shared multiple connections with rusvolcorps[.]net. That domain mimics the recruitment page for a Ukrainian far-right paramilitary group called the Russian Volunteer Corps (rusvolcorps[.]com), and uses a similar Google Forms page to collect information from would-be members.

Other domains Silent Push connected to the phishing scheme include: ciagov[.]icu, which mirrors the content on the official website of the U.S. Central Intelligence Agency; and hochuzhitlife[.]com, which spoofs the Ministry of Defense of Ukraine & General Directorate of Intelligence (whose actual domain is hochuzhit[.]com).

According to Edwards, there are no signs that these phishing sites are being advertised via email. Rather, it appears those responsible are promoting them by manipulating the search engine results shown when someone searches for one of these anti-Putin organizations.

In August 2024, security researcher Artem Tamoian posted on Twitter/X about how he received startlingly different results when he searched for “Freedom of Russia legion” in Russia’s largest domestic search engine Yandex versus Google.com. The top result returned by Google was the legion’s actual website, while the first result on Yandex was a phishing page targeting the group.

“I think at least some of them are surely promoted via search,” Tamoian said of the phishing domains. “My first thread on that accuses Yandex, but apart from Yandex those websites are consistently ranked above legitimate in DuckDuckGo and Bing. Initially, I didn’t realize the scale of it. They keep appearing to this day.”

Tamoian, a native Russian who left the country in 2019, is the founder of the cyber investigation platform malfors.com. He recently discovered two other sites impersonating the Ukrainian paramilitary groups — legionliberty[.]world and rusvolcorps[.]ru — and reported both to Cloudflare. When Cloudflare responded by blocking the sites with a phishing warning, the real Internet address of these sites was exposed as belonging to a known “bulletproof hosting” network called Stark Industries Solutions Ltd.

Stark Industries Solutions appeared two weeks before Russia invaded Ukraine in February 2022, materializing out of nowhere with hundreds of thousands of Internet addresses in its stable — many of them originally assigned to Russian government organizations. In May 2024, KrebsOnSecurity published a deep dive on Stark, which has repeatedly been used to host infrastructure for distributed denial-of-service (DDoS) attacks, phishing, malware and disinformation campaigns from Russian intelligence agencies and pro-Kremlin hacker groups.

In March 2023, Russia’s Supreme Court designated the Freedom of Russia legion as a terrorist organization, meaning that Russians caught communicating with the group could face between 10 and 20 years in prison.

Tamoian said those searching online for information about these paramilitary groups have become easy prey for Russian security services.

“I started looking into those phishing websites, because I kept stumbling upon news that someone gets arrested for trying to join [the] Ukrainian Army or for trying to help them,” Tamoian told KrebsOnSecurity. “I have also seen reports [of] FSB contacting people impersonating Ukrainian officers, as well as using fake Telegram bots, so I thought fake websites might be an option as well.”

Search results showing news articles about people in Russia being sentenced to lengthy prison terms for attempting to aid Ukrainian paramilitary groups.

Tamoian said reports surface regularly in Russia about people being arrested for trying carry out an action requested by a “Ukrainian recruiter,” with the courts unfailingly imposing harsh sentences regardless of the defendant’s age.

“This keeps happening regularly, but usually there are no details about how exactly the person gets caught,” he said. “All cases related to state treason [and] terrorism are classified, so there are barely any details.”

Tamoian said while he has no direct evidence linking any of the reported arrests and convictions to these phishing sites, he is certain the sites are part of a larger campaign by the Russian government.

“Considering that they keep them alive and keep spawning more, I assume it might be an efficient thing,” he said. “They are on top of DuckDuckGo and Yandex, so it unfortunately works.”

Further reading: Silent Push report, Russian Intelligence Targeting its Citizens and Informants.

This is a truly fascinating paper: “Trusted Machine Learning Models Unlock Private Inference for Problems Currently Infeasible with Cryptography.” The basic idea is that AIs can act as trusted third parties:

Abstract: We often interact with untrusted parties. Prioritization of privacy can limit the effectiveness of these interactions, as achieving certain goals necessitates sharing private data. Traditionally, addressing this challenge has involved either seeking trusted intermediaries or constructing cryptographic protocols that restrict how much data is revealed, such as multi-party computations or zero-knowledge proofs. While significant advances have been made in scaling cryptographic approaches, they remain limited in terms of the size and complexity of applications they can be used for. In this paper, we argue that capable machine learning models can fulfill the role of a trusted third party, thus enabling secure computations for applications that were previously infeasible. In particular, we describe Trusted Capable Model Environments (TCMEs) as an alternative approach for scaling secure computation, where capable machine learning model(s) interact under input/output constraints, with explicit information flow control and explicit statelessness. This approach aims to achieve a balance between privacy and computational efficiency, enabling private inference where classical cryptographic solutions are currently infeasible. We describe a number of use cases that are enabled by TCME, and show that even some simple classic cryptographic problems can already be solved with TCME. Finally, we outline current limitations and discuss the path forward in implementing them.

When I was writing Applied Cryptography way back in 1993, I talked about human trusted third parties (TTPs). This research postulates that someday AIs could fulfill the role of a human TTP, with added benefits like (1) being able to audit their processing, and (2) being able to delete it and erase their knowledge when their work is done. And the possibilities are vast.

Here’s a TTP problem. Alice and Bob want to know whose income is greater, but don’t want to reveal their income to the other. (Assume that both Alice and Bob want the true answer, so neither has an incentive to lie.) A human TTP can solve that easily: Alice and Bob whisper their income to the TTP, who announces the answer. But now the human knows the data. There are cryptographic protocols that can solve this. But we can easily imagine more complicated questions that cryptography can’t solve. “Which of these two novel manuscripts has more sex scenes?” “Which of these two business plans is a riskier investment?” If Alice and Bob can agree on an AI model they both trust, they can feed the model the data, ask the question, get the answer, and then delete the model afterwards. And it’s reasonable for Alice and Bob to trust a model with questions like this. They can take the model into their own lab and test it a gazillion times until they are satisfied that it is fair, accurate, or whatever other properties they want.

The paper contains several examples where an AI TTP provides real value. This is still mostly science fiction today, but it’s a fascinating thought experiment.

We are pleased to announce that Viridien has committed to sponsor DebConf25 as a Platinum Sponsor.

Viridien is an advanced technology, digital and Earth data company that pushes the boundaries of science for a more prosperous and sustainable future.

Viridien has been using Debian-based systems to power most of its HPC infrastructure and its cloud platform since 2009 and currently employs two active Debian Project Members.

As a Platinum Sponsor, Viridien is contributing to the Debian annual Developers' conference, directly supporting the progress of Debian and Free Software. Viridien contributes to strengthen the community that collaborates on the Debian project from all around the world throughout all of the year.

Thank you very much, Viridien, for your support of DebConf25!

Become a sponsor too!

DebConf25 will take place from 14 to 20 July 2025 in Brest, France, and will be preceded by DebCamp, from 7 to 13 July 2025.

DebConf25 is accepting sponsors! Interested companies and organizations may contact the DebConf team through sponsors@debconf.org, and visit the DebConf25 website at https://debconf25.debconf.org/sponsors /become-a-sponsor/.

Barry rolled into work at 8:30AM to see the project manager waiting at the door, wringing her hands and sweating. She paced a bit while Barry badged in, and then immediately explained the issue:

Today was a major release of their new features. This wasn't just a mere software change; the new release was tied to major changes to a new product line- actual widgets rolling off an assembly line right now. And those changes didn't work.

"I thought we tested this," Barry said.

"We did! And Stu called in sick today!"

Stu was the senior developer on the project, who had written most of the new code.

"I talked to him for a few minutes, and he's convinced it's a data issue. Something in the metadata or something?"

"I'll take a look," Barry said.

He skipped grabbing a coffee from the carafe and dove straight in.

Prior to the recent project, the code had looked something like this:

if (IsProduct1(_productId))
	_programId = 1;
elseif (IsProduct2(_productId))
	_programId = 2;
elseif (IsProduct3(_productId))
	_programId = 3;

Part of the project, however, was about changing the workflow for "Product 3". So Stu had written this code:

if (IsProduct1(_productId))
	_programId = 1;
else if (IsProduct2(_productId))
	_programId = 2;
else if (IsProduct3(_productId))
	_programId = 3;
	DoSomethingProductId3Specific1();
	DoSomethingProductId3Specific2();
	DoSomethingProductId3Specific3();

Since this is C# and not Python, it took Barry all of 5 seconds to spot this and figure out what the problem was and fix it:

if (IsProduct1(_productId))
{
	_programId = 1;
}
else if (IsProduct2(_productId))
{
	_programId = 2;
}
else if (IsProduct3(_productId))
{
	_programId = 3;
	DoSomethingProductId3Specific1();
	DoSomethingProductId3Specific2();
	DoSomethingProductId3Specific3();
}

This brings us to about 8:32. Now, given the problems, Barry wasn't about to just push this change- in addition to running pipeline tests (and writing tests that Stu clearly hadn't), he pinged the head of QA to get a tester on this fix ASAP. Everyone worked quickly, and that meant by 9:30 the fix was considered good and ready to be merged in and pushed to production. Sometime in there, while waiting for a pipeline to complete, Barry managed to grab a cup of coffee to wake himself up.

While Barry was busy with that, Stu had decided that he wasn't feeling that sick after all, and had rolled into the office around 9:00. Which meant that just as Barry was about to push the button to run the release pipeline, an "URGENT" email came in from Stu.

"Hey, everybody, I fixed that bug. Can we get this released ASAP?"

Barry went ahead and released the version that he'd already tested, but out of morbid curiosity, went and checked Stu's fix.

if (IsProduct1(_productId))
	_programId = 1;
else if (IsProduct2(_productId))
	_programId = 2;
else if (IsProduct3(_productId))
{
	_programId = 3;
}

if (IsProduct3(_productId))
{
	DoSomethingProductId3Specific1();
	DoSomethingProductId3Specific2();
	DoSomethingProductId3Specific3();
}

At least this version would have worked, though I'm not sure Stu fully understands what "{}"s mean in C#. Or in most programming languages, if we're being honest.

With Barry's work, the launch went off just a few minutes later than the scheduled time. Since the launch was successful, at the next company "all hands", the leadership team made sure to congratulate the people instrumental in making it happen: that is to say, the lead developer of the project, Stu.

Author: Alastair Millar Mandy was pretty, vivacious, and my next door neighbour; she’d pop round evenings or at weekends while my spouse was at work to swap gossip, recipes and just chat. But Marco didn’t mind – “you’re such a cliché,” he’d say, laughing, “her gay best friend!”. She was smart, too. Occasionally she’d tell […]

The post The Button appeared first on 365tomorrows.

A new minor release 0.2.5 of our RcppRedis package arrived on CRAN today. RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). RcppRedis does not pretend to be feature complete, but it may do some things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has been “deployed in production” as a risk / monitoring tool on a trading floor for several years. It also supports pub/sub dissemination of streaming market data as per this earlier example.

Given the changes around Redis, it is worth stressing that the package works just as well with valkey – and uses only hiredis which remains proper open source.

This update is again somewhat mechanical as a few maintenance things bubbled up in since the last release in the summer of 2023. As with other packages, continuous integration was updated a few times as were URLs and badges, and we update the use of our RApiSerialize. And, just as we did today and yesterday with littler, RQuantLib and RDieHarder, this addresses a nag from CRAN about implicit bash dependency in configure.ac (which we fixed in January as for the other packages but are under deadline now). Last but not least the newly-added extension of ‘forbidden’ symbols in static libraries revealed that yes indeed I had forgotten to set -DNDEBUG when building the embedded hiredis in fallback mode—and I now also converted its four uses of sprintf to snprintf so we are clean there too.

The detailed changes list follows.

Changes in version 0.2.5 (2025-03-26)

• The continuous integration setup was updated several times

• Badges and URLs in README.md have been updated

• An updated interface from RApiSerialize is now used, and a versioned dependency on version 0.1.4 or later has been added

• The DESCRIPTION file now uses Authors@R

• Two possible bashisms have been converted in configure.ac

• The (fallback if needed) build of libhiredis.a now sets -DNDEBUG, four uses of sprintf converted to snprintf

Courtesy of my CRANberries, there is also a diffstat report for this this release. More information is on the RcppRedis page and at the repository and its issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

An new version 0.2.7 of the random-number generator tester RDieHarder (based on the DieHarder suite developed / maintained by Robert Brown with contributions by David Bauer and myself along with other contributors) is now on CRAN.

This release contains only internal maintenance changes: continuous integration was updated as was a badge URL, and a ‘bashism’ issue in configure.ac was addressed (months ago) but as CRAN now sends NOTEs it triggered this update (just like littler and RQuantLib yesterday).

Thanks to CRANberries, you can also look at the most recent diff to the previous release.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

NIST just released a comprehensive taxonomy of adversarial machine learning attacks and countermeasures.

If you need a unique ID, UUIDs provide a variety of options. It's worth noting that variants 1, 2, and 7 all incorporate a timestamp into the UUID. In the case of variant 7, this has the benefit of making the UUID sortable, which can be convenient in many cases (v1/v2 incorporate a MAC address which means that they're sortable if generated with the same NIC).

I bring this up because Dave inherited some code written by a "guru". Said guru was working before UUIDv7 was a standard, but also didn't have any problems that required sortable UUIDs, and thus had no real reason to use timestamp based UUIDs. They just needed some random identifier and, despite using C#, didn't use the UUID functions built in to the framework. No, they instead did this:

string uniqueID = String.Format("{0:d9}", (DateTime.UtcNow.Ticks / 10) % 1000000000);

A Tick is 100 nanoseconds. We divide that by ten, mod by a billion, and then call that our unique identifier.

This is, as you might guess, not unique. First there's the possibility of timestamp collisions: generating two of these too close together in time would collide. Second, the math is just complete nonsense. We divide Ticks by ten (converting hundreds of nanoseconds into thousands of nanoseconds), then we mod by a billion. So every thousand seconds we loop and have a risk of collision again?

Maybe, maybe, these are short-lived IDs and a thousand seconds is plenty of time. But even if that's true, none of this is a good way to do that.

I suppose the saving grace is they use UtcNow and not Now, thus avoiding situations where collisions also happen because of time zones?

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Author: David C. Nutt I had been working on lucid dreaming off and on for about a year. I never believed the goofier ends of the equation- alternate realities, astral projection, and all that other New Age hooey. All I wanted to do was control my own dream space. Maybe have my own “Grand Theft […]

The post Illegal Astralgants appeared first on 365tomorrows.

A third release of the crc32c package is now on CRAN. The package bring the Google library crc32c to R and offers cyclical checksums with parity in hardware-accelerated form on (recent enough) intel cpus as well as on arm64.

This release is one hundred percent maintenance. Brian Ripley reached out as he already tests the (still very much unreleased) cmake 4.0.0 release, currently at rc5. And that version is now picky about minimum cmake version statements in CMakeLists.txt. As we copied the upstream one here, with its setting of the jurassic 3.1 version, our build conked out. A simple switch to 3.5..4.0, declaring a ‘from .. to’ scheme with a minimally supported version (here 3.5, released in 2016) up to a tested version works. No other changes, really, besides an earlier helping hand from Jeroen concerning cross-compilation support he needed or encountered (and that happened right after the 0.0.2 release).

The NEWS entry for this (as well the initial release) follows.

Changes in version 0.0.3 (2025-03-25)

• Support cross-compilation by setting CC and CXX in Makevars.in (Jeroen Ooms in #1)

• Support pre-release 4.0.0 of cmake by moving the minimum stated version from 3.1 to 3.5 per CRAN request, also sent PR upstream

My CRANberries service provides a comparison to the previous release. The code is available via the GitHub repo, and of course also from its CRAN page and via install.packages("crc32c"). Comments and suggestions are welcome at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

The twentysecond release of littler as a CRAN package landed on CRAN just now, following in the now nineteen year history (!!) as a (initially non-CRAN) package started by Jeff in 2006, and joined by me a few weeks later.

littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only began to do in recent years.

littler lives on Linux and Unix, has its difficulties on macOS due to some-braindeadedness there (who ever thought case-insensitive filesystems as a default were a good idea?) and simply does not exist on Windows (yet – the build system could be extended – see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH. A few examples are highlighted at the Github repo:, as well as in the examples vignette.

This release, the first is almost exactly one year, brings enhancements to six scripts as well as three new ones. The new ones crup.r offers ‘CRan UPloads’ from the command-line, deadliners.r lists CRAN package by CRAN deadline, and wb.r uploads to win-builder (replacing an older shell script of mine). Among the updated ones, kitten.r now creates more complete DESCRIPTION files in the packages it makes, and several scripts support additional options. A number of changes were made to packaging as well, some of which were contributed by Jon and Michael which is of course always greatly appreciated. The trigger for the release was, just like for RQuantLib earlier today, a CRAN nag on ‘bashisms’ half of which actually false here it was in a comment only. Oh well.

The full change description follows.

Changes in littler version 0.3.21 (2025-03-24)

• Changes in examples scripts

  • Usage text for ciw.r is improved, new options were added (Dirk)

  • The ‘noble’ release is supported by r2u.r (Dirk)

  • The installRub.r script has additional options (Dirk)

  • The ttlt.r script has a new load_package argument (Dirk)

  • A new script deadliners.r showing CRAN packages 'under deadline' has been added, and then refined (Dirk)

  • The kitten.r script can now use whoami and argument githubuser on the different *kitten helpers it calls (Dirk)

  • A new script wb.r can upload to win-builder (Dirk)

  • A new script crup.r can upload a CRAN submission (Dirk)

  • In rcc.r, the return from rcmdcheck is now explicitly printed (Dirk)

  • In r2u.r the dry-run option is passed to the build command (Dirk)

• Changes in package

  • Regular updates to badges, continuous integration, DESCRIPTION and configure.ac (Dirk)

  • Errant osVersion return value are handled more robustly (Michael Chirico in #121)

  • The current run-time path is available via variable LITTLER_SCRIPT_PATH (Jon Clayden in #122)

  • The cleanup script remove macOS debug symbols (Jon Clayden in #123)

My CRANberries service provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page, and also on the package docs website. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as (in a day or two) Ubuntu binaries at CRAN thanks to the tireless Michael Rutter. Comments and suggestions are welcome at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Cloudflare has a new feature—available to free users as well—that uses AI to generate random pages to feed to AI web crawlers:

Instead of simply blocking bots, Cloudflare’s new system lures them into a “maze” of realistic-looking but irrelevant pages, wasting the crawler’s computing resources. The approach is a notable shift from the standard block-and-defend strategy used by most website protection services. Cloudflare says blocking bots sometimes backfires because it alerts the crawler’s operators that they’ve been detected.

“When we detect unauthorized crawling, rather than blocking the request, we will link to a series of AI-generated pages that are convincing enough to entice a crawler to traverse them,” writes Cloudflare. “But while real looking, this content is not actually the content of the site we are protecting, so the crawler wastes time and resources.”

The company says the content served to bots is deliberately irrelevant to the website being crawled, but it is carefully sourced or generated using real scientific facts—­such as neutral information about biology, physics, or mathematics—­to avoid spreading misinformation (whether this approach effectively prevents misinformation, however, remains unproven).

It’s basically an AI-generated honeypot. And AI scraping is a growing problem:

The scale of AI crawling on the web appears substantial, according to Cloudflare’s data that lines up with anecdotal reports we’ve heard from sources. The company says that AI crawlers generate more than 50 billion requests to their network daily, amounting to nearly 1 percent of all web traffic they process. Many of these crawlers collect website data to train large language models without permission from site owners….

Presumably the crawlers will now have to up both their scraping stealth and their ability to filter out AI-generated content like this. Which means the honeypots will have to get better at detecting scrapers and more stealthy in their fake content. This arms race is likely to go back and forth, wasting a lot of energy in the process.

A new minor release 0.4.25 of RQuantLib arrived on CRAN this morning, and has just now been uploaded to Debian too.

QuantLib is a rather comprehensice free/open-source library for quantitative finance. RQuantLib connects (some parts of) it to the R environment and language, and has been part of CRAN for nearly twenty-two years (!!) as it was one of the first packages I uploaded to CRAN.

This release of RQuantLib is tickled by a request to remove ‘bashisms’ in shell scripts, or, as in my case here, configure.ac where I used the non-portable form of string comparison. That has of course been there for umpteen years and not bitten anyone as the default shell for most is in fact bash but the change has the right idea. And is of course now mandatory affecting quite a few packages is I tooted yesterday. It also contains an improvement to the macOS 14 build kindly contributed by Jeroen.

Changes in RQuantLib version 0.4.25 (2025-03-24)

• Support macOS 14 with a new compiler flag (Jeroen in #190)

• Correct two bashisms in configure.ac

One more note, though: This may however be the last release I make with Windows support. CRAN now also checks for ‘forbidden’ symbols (such as assert or (s)printf or …) in static libraries, and this release tickled one such warning from the Windows side (which only uses static libraries). I have no desire to get involved in also maintaing QuantLib (no R here) for Windows and may simply turn the package back to OS_type: unix to avoid the hassle. To avoid that, it would be fabulous if someone relying on RQuantLib on Windows could step up and lend a hand looking after that library build.

Courtesy of my CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the rquantlib-devel mailing list. Issue tickets can be filed at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Author: Majoki Juan Dalderis was the creator of LinkJuice, the uber energy drink of the Internet, the black gold, the Texas Tea of web traffic. He could make or break any web platform or presence. He had the power of a techno god, but his mortal self fell seriously ill. A listeria-tainted cantaloupe left him […]

The post Rear Window appeared first on 365tomorrows.

Today's anonymous submitter has managed to find a way to do date formatting wrong that I don't think I've seen yet. That's always remarkable. Like most such bad code, it checks string lengths and then adds a leading zero, if needed. It's not surprising, but again, it's all in the details:

// convert date string to yyyy/MM/DD
return dtmValue.Year + "-" + ((dtmValue.Month.ToString().Length == 1)?  ("0" + dtmValue.Month.ToString()): dtmValue.Month.ToString()) + "-" + ((dtmValue.Day.ToString().Length == 1)? ("0" + dtmValue.Day.ToString()): dtmValue.Day.ToString());

This is only one line, but it has it all, doesn't it. First, we've got good ol' Hungarian notation, which conveys no useful information here. We've got a comment which tells us the code outputs /es, but the code actually outputs -. We've got ternaries that are definitely not helping readability here, plus repeated calls to ToString() instead of maybe just storing the result in a variable.

And, for the record, dtmValue.ToString("yyyy-MM-dd") would have done the correct thing.

[Advertisement] Keep all your packages and Docker containers in one place, scan for vulnerabilities, and control who can access different feeds. ProGet installs in minutes and has a powerful free version with a lot of great features that you can upgrade when ready.Learn more.

Are you a student aspiring to participate in the Google Summer of Code 2025? Would you like to improve the continuous integration pipeline used at salsa.debian.org, the Debian GitLab instance, to help improve the quality of tens of thousands of software packages in Debian?

This summer 2025, I and Emmanuel Arias will be participating as mentors in the GSoC program. We are available to mentor students who propose and develop improvements to the Salsa CI pipeline, as we are members of the Debian team that maintains it.

A post by Santiago Ruano Rincón in the GitLab blog explains what Salsa CI is and its short history since inception in 2018. At the time of the article in fall 2023 there were 9000+ source packages in Debian using Salsa CI. Now in 2025 there are over 27,000 source packages in Debian using it, and since summer 2024 some Ubuntu developers have started using it for enhanced quality assurance of packaging changes before uploading new package revisions to Ubuntu. Personally, I have been using Salsa CI since its inception, and contributing as a team member since 2019. See my blog post about GitLab CI for MariaDB in Debian for a description of an advanced and extensive use case.

Helping Salsa CI is a great way to make a global impact, as it will help avoid regressions and improve the quality of Debian packages. The benefits reach far beyond just Debian, as it will also help hundreds of Debian derivatives, such as Ubuntu, Linux Mint, Tails, Purism PureOS, Pop!_OS, Zorin OS, Raspberry Pi OS, a large portion of Docker containers, and even the Windows Subsystem for Linux.

Improving Salsa CI: more features, robustness, speed

While Salsa CI with contributions from 71 people is already quite mature and capable, there are many ideas floating around about how it could be further extended. For example, Salsa CI issue #147 describes various static analyzers and linters that may be generally useful. Issue #411 proposes using libfaketime to run autopkgtest on arbitrary future dates to test for failures caused by date assumptions, such as the Y2038 issue.

There are also ideas about making Salsa CI more robust and code easier to reuse by refactoring some of the yaml scripts into independent scripts in #230, which could make it easier to run Salsa CI locally as suggested in #169. There are also ideas about improving the Salsa CI’s own CI to avoid regressions from pipeline changes in #318.

The CI system is also better when it’s faster, and some speed improvement ideas have been noted in #412.

Improvements don’t have to be limited to changes in the pipeline itself. A useful project would also be to update more Debian packages to use Salsa CI, and ensure they adopt it in an optimal way as noted in #416. It would also be nice to have a dashboard with statistics about all public Salsa CI pipeline runs as suggested in #413.

These and more ideas can be found in the issue list by filtering for tags Newcomer, Nice-To-Have or Accepting MRs. A Google Summer of Code proposal does not have to be limited to these existing ideas. Participants are also welcome to propose completely novel ideas!

Good time to also learn Debian packaging

Anyone working with Debian team should also take the opportunity to learn Debian packaging, and contribute to the packaging or maintenance of 1-2 packages in parallel to improving the Salsa CI. All Salsa CI team members are also Debian Developers who can mentor and sponsor uploads to Debian.

Maintaining a few packages is a great way to eat your own cooking and experience Salsa CI from the user perspective, and likely to make you better at Salsa CI development.

Apply now!

The contributor applications opened yesterday on March 24, so to participate act now! If you are an eligible student and want to attend, head over to summerofcode.withgoogle.com to learn more.

There are over a thousand participating organizations, with Debian, GitLab and MariaDB being some examples. Within these organizations there may be multiple subteams and projects to choose from. The full list of participating Debian projects can be found in the Debian wiki.

If you are interested in GSoC for Salsa CI specifically, feel free to

1. Reach out to me and Emmanuel by email at otto@ and eamanu@ (debian.org).
2. Sign up at salsa.debian.org for an account (note it takes a few days due to manual vetting and approval process)
3. Read the project README, STRUCTURE and CONTRIBUTING to get a developer’s overview
4. Participate in issue discussions at https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/

Note that you don’t have to wait for GSoC to officially start to contribute. In fact, it may be useful to start immediately by submitting a Merge Request to do some small contribution, just to learn the process and to get more familiar with how everything works, and the team maintaining Salsa CI. Looking forward to seeing new contributors!

Last month, I wrote about the UK forcing Apple to break its Advanced Data Protection encryption in iCloud. More recently, both Sweden and France are contemplating mandating backdoors. Both initiatives are attempting to scare people into supporting backdoors, which are—of course—are terrible idea.

Also: “A Feminist Argument Against Weakening Encryption.”

Citizen Lab has a new report on Paragon’s spyware:

Key Findings:

• Introducing Paragon Solutions. Paragon Solutions was founded in Israel in 2019 and sells spyware called Graphite. The company differentiates itself by claiming it has safeguards to prevent the kinds of spyware abuses that NSO Group and other vendors are notorious for.
• Infrastructure Analysis of Paragon Spyware. Based on a tip from a collaborator, we mapped out server infrastructure that we attribute to Paragon’s Graphite spyware tool. We identified a subset of suspected Paragon deployments, including in Australia, Canada, Cyprus, Denmark, Israel, and Singapore.
• Identifying a Possible Canadian Paragon Customer. Our investigation surfaced potential links between Paragon Solutions and the Canadian Ontario Provincial Police, and found evidence of a growing ecosystem of spyware capability among Ontario-based police services.
• Helping WhatsApp Catch a Zero-Click. We shared our analysis of Paragon’s infrastructure with Meta, who told us that the details were pivotal to their ongoing investigation into Paragon. WhatsApp discovered and mitigated an active Paragon zero-click exploit, and later notified over 90 individuals who it believed were targeted, including civil society members in Italy.
• Android Forensic Analysis: Italian Cluster. We forensically analyzed multiple Android phones belonging to Paragon targets in Italy (an acknowledged Paragon user) who were notified by WhatsApp. We found clear indications that spyware had been loaded into WhatsApp, as well as other apps on their devices.
• A Related Case of iPhone Spyware in Italy. We analyzed the iPhone of an individual who worked closely with confirmed Android Paragon targets. This person received an Apple threat notification in November 2024, but no WhatsApp notification. Our analysis showed an attempt to infect the device with novel spyware in June 2024. We shared details with Apple, who confirmed they had patched the attack in iOS 18.
• Other Surveillance Tech Deployed Against The Same Italian Cluster. We also note 2024 warnings sent by Meta to several individuals in the same organizational cluster, including a Paragon victim, suggesting the need for further scrutiny into other surveillance technology deployed against these individuals.

I am told, by friends who have spent time at Google, about the reason Google Reader finally disappeared. Apparently it had become a 20% Project for those who still cared about it internally, and there was some major change happening to one of it upstream dependencies that was either going to cause a significant amount of work rearchitecting Reader to cope, or create additional ongoing maintenance burden. It was no longer viable to support it as a side project, so it had to go. This was a consequence of an internal culture at Google where service owners are able to make changes that can break downstream users, and the downstream users are the ones who have to adapt.

My experience at Meta goes the other way. If you own a service or other dependency and you want to make a change that will break things for the users, it’s on you to do the migration, or at the very least provide significant assistance to those who own the code. You don’t just get to drop your new release and expect others to clean up; doing that tends to lead to changes being reverted. The culture flows the other way; if you break it, you fix it (nothing is someone else’s problem).

There are pluses and minuses to both approaches. Users having to drive the changes to things they own stops them from blocking progress. Service/code owners having to drive the changes avoids the situation where a wildly used component drops a new release that causes a lot of high priority work for folk in order to adapt.

I started thinking about this in the context of Debian a while back, and a few incidents since have resulted in my feeling that we’re closer to the Google model than the Meta model. Anyone can upload a new version of their package to unstable, and that might end up breaking all the users of it. It’s not quite as extreme as rolling out a new service, because it’s unstable that gets affected (the clue is in the name, I really wish more people would realise that), but it can still result in release critical bugs for lots other Debian contributors.

A good example of this are toolchain changes. Major updates to GCC and friends regularly result in FTBFS issues in lots of packages. Now in this instance the maintainer is usually diligent about a heads up before the default changes, but it’s still a whole bunch of work for other maintainers to adapt (see the list of FTBFS bugs for GCC 15 for instance - these are important, but not serious yet). Worse is when a dependency changes and hasn’t managed to catch everyone who might be affected, so by the time it’s discovered it’s release critical, because at least one package no longer builds in unstable.

Commercial organisations try to avoid this with a decent CI/CD setup that either vendors all dependencies, or tracks changes to them and tries rebuilds before allowing things to land. This is one of the instances where a monorepo can really shine; if everything you need is in there, it’s easier to track the interconnections between different components. Debian doesn’t have a CI/CD system that runs for every upload, allowing us to track exact causes of regressions. Instead we have Lucas, who does a tremendous job of running archive wide rebuilds to make sure we can still build everything. Unfortunately that means I am often unfairly grumpy at him; my heart sinks when I see a bug come in with his name attached, because it often means one of my packages has a new RC bug where I’m going to have to figure out what changed elsewhere to cause it. However he’s just (very usefully) surfacing an issue someone else created, rather than actually being the cause of the problem.

I don’t know if I have a point to this post. I think it’s probably that I wish folk in Free Software would try and be mindful of the incompatible changes they might introducing, and the toil they create for other volunteer developers, often not directly visible to the person making the change. The approach done by the Debian toolchain maintainers strikes me as a good balance; they do a bunch of work up front to try and flag all the places that might need to make changes, far enough in advance of the breaking change actually landing. However they don’t then allow a tardy developer to block progress.

The following contributors got their Debian Developer accounts in the last two months:

• Bo Yu (vimer)
• Maytham Alsudany (maytham)
• Rebecca Natalie Palmer (mpalmer)

The following contributors were added as Debian Maintainers in the last two months:

• NoisyCoil
• Arif Ali
• Julien Plissonneau Duquène
• Maarten Van Geijn
• Ben Collins

Congratulations!

Around a year ago I discussed two concerns with software release archives (tarball artifacts) that could be improved to increase confidence in the supply-chain security of software releases. Repeating the goals for simplicity:

• Release artifacts should be built in a way that can be reproduced by others
• It should be possible to build a project from source tarball that doesn’t contain any generated or vendor files (e.g., in the style of git-archive).

While implementing these ideas for a small project was accomplished within weeks – see my announcement of Libntlm version 1.8 – adressing this in complex projects uncovered concerns with tools that had to be addressed, and things stalled for many months pending that work.

I had the notion that these two goals were easy and shouldn’t be hard to accomplish. I still believe that, but have had to realize that improving tooling to support these goals takes time. It seems clear that these concepts are not universally agreed on and implemented generally.

I’m now happy to recap some of the work that led to releases of libtasn1 v4.20.0, inetutils v2.6, libidn2 v2.3.8, libidn v1.43. These releases all achieve these goals. I am working on a bunch of more projects to support these ideas too.

What have the obstacles so far been to make this happen? It may help others who are in the same process of addressing these concerns to have a high-level introduction to the issues I encountered. Source code for projects above are available and anyone can look at the solutions to learn how the problems are addressed.

First let’s look at the problems we need to solve to make “git-archive” style tarballs usable:

Version Handling

To build usable binaries from a minimal tarballs, it need to know which version number it is. Traditionally this information was stored inside configure.ac in git. However I use gnulib’s git-version-gen to infer the version number from the git tag or git commit instead. The git tag information is not available in a git-archive tarball. My solution to this was to make use of the export-subst feature of the .gitattributes file. I store the file .tarball-version-git in git containing the magic cookie like this:

$Format:%(describe)$

With this, git-archive will replace with a useful version identifier on export, see the libtasn1 patch to achieve this. To make use of this information, the git-version-gen script was enhanced to read this information, see the gnulib patch. This is invoked by ./configure to figure out which version number the package is for.

Translations

We want translations to be included in the minimal source tarball for it to be buildable. Traditionally these files are retrieved by the maintainer from the Translation project when running ./bootstrap, however there are two problems with this. The first one is that there is no strong authentication or versioning information on this data, the tools just download and place whatever wget downloaded into your source tree (printf-style injection attack anyone?). We could improve this (e.g., publish GnuPG signed translations messages with clear versioning), however I did not work on that further. The reason is that I want to support offline builds of packages. Downloading random things from the Internet during builds does not work when building a Debian package, for example. The translation project could solve this by making a monthly tarball with their translations available, for distributors to pick up and provide as a separate package that could be used as a build dependency. However that is not how these tools and projects are designed. Instead I reverted back to storing translations in git, something that I did for most projects back when I was using CVS 20 years ago. Hooking this into ./bootstrap and gettext workflow can be tricky (ideas for improvement most welcome!), but I used a simple approach to store all directly downloaded po/*.po files directly as po/*.po.in and make the ./bootstrap tool move them in place, see the libidn2 commit followed by the actual ‘make update-po’ commit with all the translations where one essential step is:

# Prime po/*.po from fall-back copy stored in git.
for poin in po/*.po.in; do
    po=$(echo $poin | sed 's/.in//')
    test -f $po || cp -v $poin $po
done
ls po/*.po | sed 's|.*/||; s|\.po$||' > po/LINGUAS

Fetching vendor files like gnulib

Most build dependencies are in the shape of “You need a C compiler”. However some come in the shape of “source-code files intended to be vendored”, and gnulib is a huge repository of such files. The latter is a problem when building from a minimal git archive. It is possible to consider translation files as a class of vendor files, since they need to be copied verbatim into the project build directory for things to work. The same goes for *.m4 macros from the GNU Autoconf Archive. However I’m not confident that the solution for all vendor files must be the same. For translation files and for Autoconf Archive macros, I have decided to put these files into git and merge them manually occasionally. For gnulib files, in some projects like OATH Toolkit I also store all gnulib files in git which effectively resolve this concern. (Incidentally, the reason for doing so was originally that running ./bootstrap took forever since there is five gnulib instances used, which is no longer the case since gnulib-tool was rewritten in Python.) For most projects, however, I rely on ./bootstrap to fetch a gnulib git clone when building. I like this model, however it doesn’t work offline. One way to resolve this is to make the gnulib git repository available for offline use, and I’ve made some effort to make this happen via a Gnulib Git Bundle and have explained how to implement this approach for Debian packaging. I don’t think that is sufficient as a generic solution though, it is mostly applicable to building old releases that uses old gnulib files. It won’t work when building from CI/CD pipelines, for example, where I have settled to use a crude way of fetching and unpacking a particular gnulib snapshot, see this Libntlm patch. This is much faster than working with git submodules and cloning gnulib during ./bootstrap. Essentially this is doing:

GNULIB_REVISION=$(. bootstrap.conf >&2; echo $GNULIB_REVISION)
wget -nv https://gitlab.com/libidn/gnulib-mirror/-/archive/$GNULIB_REVISION/gnulib-mirror-$GNULIB_REVISION.tar.gz
gzip -cd gnulib-mirror-$GNULIB_REVISION.tar.gz | tar xf -
rm -fv gnulib-mirror-$GNULIB_REVISION.tar.gz
export GNULIB_SRCDIR=$PWD/gnulib-mirror-$GNULIB_REVISION
./bootstrap --no-git
./configure
make

Test the git-archive tarball

This goes without saying, but if you don’t test that building from a git-archive style tarball works, you are likely to regress at some point. Use CI/CD techniques to continuously test that a minimal git-archive tarball leads to a usable build.

Mission Accomplished

So that wasn’t hard, was it? You should now be able to publish a minimal git-archive tarball and users should be able to build your project from it.

I recommend naming these archives as PROJECT-vX.Y.Z-src.tar.gz replacing PROJECT with your project name and X.Y.Z with your version number. The archive should have only one sub-directory named PROJECT-vX.Y.Z/ containing all the source-code files. This differentiate it against traditional PROJECT-X.Y.Z.tar.gz tarballs in that it embeds the git tag (which typically starts with v) and contains a wildcard-friendly -src substring. Alas there is no consistency around this naming pattern, and GitLab, GitHub, Codeberg etc all seem to use their own slightly incompatible variant.

Let’s go on to see what is needed to achieve reproducible “make dist” source tarballs. This is the release artifact that most users use, and they often contain lots of generated files and vendor files. These files are included to make it easy to build for the user. What are the challenges to make these reproducible?

Build dependencies causing different generated content

The first part is to realize that if you use tool X with version A to generate a file that goes into the tarball, version B of that tool may produce different outputs. This is a generic concern and it cannot be solved. We want our build tools to evolve and produce better outputs over time. What can be addressed is to avoid needless differences. For example, many tools store timestamps and versioning information in the generated files. This causes needless differences, which makes audits harder. I have worked on some of these, like Autoconf Archive timestamps but solving all of these examples will take a long time, and some upstream are reluctant to incorporate these changes. My approach meanwhile is to build things using similar environments, and compare the outputs for differences. I’ve found that the various closely related forks of GNU/Linux distributions are useful for this. Trisquel 11 is based on Ubuntu 22.04, and building my projects using both and comparing the differences only give me the relevant differences to improve. This can be extended to compare AlmaLinux with RockyLinux (for both versions 8 and 9), Devuan 5 against Debian 12, PureOS 10 with Debian 11, and so on.

Timestamps

Sometimes tools store timestamps in files in a way that is harder to fix. Two notable examples of this are *.po translation files and Texinfo manuals. For translation files, I have resolved this by making sure the files use a predictable POT-Creation-Date timestamp, and I set it to the modification timestamps of the NEWS file in the repository (which I set to the git commit of the latest commit elsewhere) like this:

dist-hook: po-CreationDate-to-mtime-NEWS
.PHONY: po-CreationDate-to-mtime-NEWS
po-CreationDate-to-mtime-NEWS: mtime-NEWS-to-git-HEAD
  $(AM_V_GEN)for p in $(distdir)/po/*.po $(distdir)/po/$(PACKAGE).pot; do \
    if test -f "$$p"; then \
      $(SED) -e 's,POT-Creation-Date: .*\\n",POT-Creation-Date: '"$$(env LC_ALL=C TZ=UTC0 stat --format=%y $(srcdir)/NEWS | cut -c1-16,31-)"'\\n",' < $$p > $$p.tmp && \
      if cmp $$p $$p.tmp > /dev/null; then \
        rm -f $$p.tmp; \
      else \
        mv $$p.tmp $$p; \
      fi \
    fi \
  done

Similarily, I set a predictable modification time of the texinfo source file like this:

dist-hook: mtime-NEWS-to-git-HEAD
.PHONY: mtime-NEWS-to-git-HEAD
mtime-NEWS-to-git-HEAD:
  $(AM_V_GEN)if test -e $(srcdir)/.git \
                && command -v git > /dev/null; then \
    touch -m -t "$$(git log -1 --format=%cd \
      --date=format-local:%Y%m%d%H%M.%S)" $(srcdir)/NEWS; \
  fi

However I’ve realized that this needs to happen earlier and probably has to be run during ./configure time, because the doc/version.texi file is generated on first build before running ‘make dist‘ and for some reason the file is not rebuilt at release time. The Automake texinfo integration is a bit inflexible about providing hooks to extend the dependency tracking.

The method to address these differences isn’t really important, and they change over time depending on preferences. What is important is that the differences are eliminated.

ChangeLog

Traditionally ChangeLog files were manually prepared, and still is for some projects. I maintain git2cl but recently I’ve settled with gnulib’s gitlog-to-changelog because doing so avoids another build dependency (although the output formatting is different and arguable worse for my git commit style). So the ChangeLog files are generated from git history. This means a shallow clone will not produce the same ChangeLog file depending on how deep it was cloned. For Libntlm I simply disabled use of generated ChangeLog because I wanted to support an even more extreme form of reproducibility: I wanted to be able to reproduce the full “make dist” source archives from a minimal “git-archive” source archive. However for other projects I’ve settled with a middle ground. I realized that for ‘git describe‘ to produce reproducible outputs, the shallow clone needs to include the last release tag. So it felt acceptable to assume that the clone is not minimal, but instead has some but not all of the history. I settled with the following recipe to produce ChangeLog's covering all changes since the last release.

dist-hook: gen-ChangeLog
.PHONY: gen-ChangeLog
gen-ChangeLog:
  $(AM_V_GEN)if test -e $(srcdir)/.git; then			\
    LC_ALL=en_US.UTF-8 TZ=UTC0					\
    $(top_srcdir)/build-aux/gitlog-to-changelog			\
       --srcdir=$(srcdir) --					\
       v$(PREV_VERSION)~.. > $(distdir)/cl-t &&			\
       { printf '\n\nSee the source repo for older entries\n'	\
         >> $(distdir)/cl-t &&					\
         rm -f $(distdir)/ChangeLog &&				\
         mv $(distdir)/cl-t $(distdir)/ChangeLog; }		\
  fi

I’m undecided about the usefulness of generated ChangeLog files within ‘make dist‘ archives. Before we have stable and secure archival of git repositories widely implemented, I can see some utility of this in case we lose all copies of the upstream git repositories. I can sympathize with the concept of ChangeLog files died when we started to generate them from git logs: the files no longer serve any purpose, and we can ask people to go look at the git log instead of reading these generated non-source files.

Long-term reproducible trusted build environment

Distributions comes and goes, and old releases of them goes out of support and often stops working. Which build environment should I chose to build the official release archives? To my knowledge only Guix offers a reliable way to re-create an older build environment (guix gime-machine) that have bootstrappable properties for additional confidence. However I had two difficult problems here. The first one was that I needed Guix container images that were usable in GitLab CI/CD Pipelines, and this side-tracked me for a while. The second one delayed my effort for many months, and I was inclined to give up. Libidn distribute a C# implementation. Some of the C# source code files included in the release tarball are generated. By what? You guess it, by a C# program, with the source code included in the distribution. This means nobody could reproduce the source tarball of Libidn without trusting someone elses C# compiler binaries, which were built from binaries of earlier releases, chaining back into something that nobody ever attempts to build any more and likely fail to build due to bit-rot. I had two basic choices, either remove the C# implementation from Libidn (which may be a good idea for other reasons, since the C and C# are unrelated implementations) or build the source tarball on some binary-only distribution like Trisquel. Neither felt appealing to me, but a late christmas gift of a reproducible Mono came to Guix that resolve this.

Embedded images in Texinfo manual

For Libidn one section of the manual has an image illustrating some concepts. The PNG, PDF and EPS outputs were generated via fig2dev from a *.fig file (hello 1985!) that I had stored in git. Over time, I had also started to store the generated outputs because of build issues. At some point, it was possible to post-process the PDF outputs with grep to remove some timestamps, however with compression this is no longer possible and actually the grep command I used resulted in a 0-byte output file. So my embedded binaries in git was no longer reproducible. I first set out to fix this by post-processing things properly, however I then realized that the *.fig file is not really easy to work with in a modern world. I wanted to create an image from some text-file description of the image. Eventually, via the Guix manual on guix graph, I came to re-discover the graphviz language and tool called dot (hello 1993!). All well then? Oh no, the PDF output embeds timestamps. Binary editing of PDF’s no longer work through simple grep, remember? I was back where I started, and after some (soul- and web-) searching I discovered that Ghostscript (hello 1988!) pdfmarks could be used to modify things here. Cooperating with automake’s texinfo rules related to make dist proved once again a worthy challenge, and eventually I ended up with a Makefile.am snippet to build images that could be condensed into:

info_TEXINFOS = libidn.texi
libidn_TEXINFOS += libidn-components.png
imagesdir = $(infodir)
images_DATA = libidn-components.png
EXTRA_DIST += components.dot
DISTCLEANFILES = \
  libidn-components.eps libidn-components.png libidn-components.pdf
libidn-components.eps: $(srcdir)/components.dot
  $(AM_V_GEN)$(DOT) -Nfontsize=9 -Teps < $< > $@.tmp
  $(AM_V_at)! grep %%CreationDate $@.tmp
  $(AM_V_at)mv $@.tmp $@
libidn-components.pdf: $(srcdir)/components.dot
  $(AM_V_GEN)$(DOT) -Nfontsize=9 -Tpdf < $< > $@.tmp
# A simple sed on CreationDate is no longer possible due to compression.
# 'exiftool -CreateDate' is alternative to 'gs', but adds ~4kb to file.
# Ghostscript add <1kb.  Why can't 'dot' avoid setting CreationDate?
  $(AM_V_at)printf '[ /ModDate ()\n  /CreationDate ()\n  /DOCINFO pdfmark\n' > pdfmarks
  $(AM_V_at)$(GS) -q -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=$@.tmp2 $@.tmp pdfmarks
  $(AM_V_at)rm -f $@.tmp pdfmarks
  $(AM_V_at)mv $@.tmp2 $@
libidn-components.png: $(srcdir)/components.dot
  $(AM_V_GEN)$(DOT) -Nfontsize=9 -Tpng < $< > $@.tmp
  $(AM_V_at)mv $@.tmp $@
pdf-recursive: libidn-components.pdf
dvi-recursive: libidn-components.eps
ps-recursive: libidn-components.eps
info-recursive: $(top_srcdir)/.version libidn-components.png

Surely this can be improved, but I’m not yet certain in what way is the best one forward. I like having a text representation as the source of the image. I’m sad that the new image size is ~48kb compared to the old image size of ~1kb. I tried using exiftool -CreateDate as an alternative to GhostScript, but using it to remove the timestamp added ~4kb to the file size and naturally I was appalled by this ignorance of impending doom.

Test reproducibility of tarball

Again, you need to continuously test the properties you desire. This means building your project twice using different environments and comparing the results. I’ve settled with a small GitLab CI/CD pipeline job that perform bit-by-bit comparison of generated ‘make dist’ archives. It also perform bit-by-bit comparison of generated ‘git-archive’ artifacts. See the Libidn2 .gitlab-ci.yml 0-compare job which essentially is:

0-compare:
  image: alpine:latest
  stage: repro
  needs: [ B-AlmaLinux8, B-AlmaLinux9, B-RockyLinux8, B-RockyLinux9, B-Trisquel11, B-Ubuntu2204, B-PureOS10, B-Debian11, B-Devuan5, B-Debian12, B-gcc, B-clang, B-Guix, R-Guix, R-Debian12, R-Ubuntu2404, S-Trisquel10, S-Ubuntu2004 ]
  script:
  - cd out
  - sha256sum */*.tar.* */*/*.tar.* | sort | grep    -- -src.tar.
  - sha256sum */*.tar.* */*/*.tar.* | sort | grep -v -- -src.tar.
  - sha256sum */*.tar.* */*/*.tar.* | sort | uniq -c -w64 | sort -rn
  - sha256sum */*.tar.* */*/*.tar.* | grep    -- -src.tar. | sort | uniq -c -w64 | grep -v '^      1 '
  - sha256sum */*.tar.* */*/*.tar.* | grep -v -- -src.tar. | sort | uniq -c -w64 | grep -v '^      1 '
# Confirm modern git-archive tarball reproducibility
  - cmp b-almalinux8/src/*.tar.gz b-almalinux9/src/*.tar.gz
  - cmp b-almalinux8/src/*.tar.gz b-rockylinux8/src/*.tar.gz
  - cmp b-almalinux8/src/*.tar.gz b-rockylinux9/src/*.tar.gz
  - cmp b-almalinux8/src/*.tar.gz b-debian12/src/*.tar.gz
  - cmp b-almalinux8/src/*.tar.gz b-devuan5/src/*.tar.gz
  - cmp b-almalinux8/src/*.tar.gz r-guix/src/*.tar.gz
  - cmp b-almalinux8/src/*.tar.gz r-debian12/src/*.tar.gz
  - cmp b-almalinux8/src/*.tar.gz r-ubuntu2404/src/*v2.*.tar.gz
# Confirm old git-archive (export-subst but long git describe) tarball reproducibility
  - cmp b-trisquel11/src/*.tar.gz b-ubuntu2204/src/*.tar.gz
# Confirm really old git-archive (no export-subst) tarball reproducibility
  - cmp b-debian11/src/*.tar.gz b-pureos10/src/*.tar.gz
# Confirm 'make dist' generated tarball reproducibility
  - cmp b-almalinux8/*.tar.gz b-rockylinux8/*.tar.gz
  - cmp b-almalinux9/*.tar.gz b-rockylinux9/*.tar.gz
  - cmp b-pureos10/*.tar.gz b-debian11/*.tar.gz
  - cmp b-devuan5/*.tar.gz b-debian12/*.tar.gz
  - cmp b-trisquel11/*.tar.gz b-ubuntu2204/*.tar.gz
  - cmp b-guix/*.tar.gz r-guix/*.tar.gz
# Confirm 'make dist' from git-archive tarball reproducibility
  - cmp s-trisquel10/*.tar.gz s-ubuntu2004/*.tar.gz

Notice that I discovered that ‘git archive’ outputs differ over time too, which is natural but a bit of a nuisance. The output of the job is illuminating in the way that all SHA256 checksums of generated tarballs are included, for example the libidn2 v2.3.8 job log:

$ sha256sum */*.tar.* */*/*.tar.* | sort | grep -v -- -src.tar.
368488b6cc8697a0a937b9eb307a014396dd17d3feba3881e6911d549732a293  b-trisquel11/libidn2-2.3.8.tar.gz
368488b6cc8697a0a937b9eb307a014396dd17d3feba3881e6911d549732a293  b-ubuntu2204/libidn2-2.3.8.tar.gz
59db2d045fdc5639c98592d236403daa24d33d7c8db0986686b2a3056dfe0ded  b-debian11/libidn2-2.3.8.tar.gz
59db2d045fdc5639c98592d236403daa24d33d7c8db0986686b2a3056dfe0ded  b-pureos10/libidn2-2.3.8.tar.gz
5bd521d5ecd75f4b0ab0fc6d95d444944ef44a84cad859c9fb01363d3ce48bb8  s-trisquel10/libidn2-2.3.8.tar.gz
5bd521d5ecd75f4b0ab0fc6d95d444944ef44a84cad859c9fb01363d3ce48bb8  s-ubuntu2004/libidn2-2.3.8.tar.gz
7f1dcdea3772a34b7a9f22d6ae6361cdcbe5513e3b6485d40100b8565c9b961a  b-almalinux8/libidn2-2.3.8.tar.gz
7f1dcdea3772a34b7a9f22d6ae6361cdcbe5513e3b6485d40100b8565c9b961a  b-rockylinux8/libidn2-2.3.8.tar.gz
8031278157ce43b5813f36cf8dd6baf0d9a7f88324ced796765dcd5cd96ccc06  b-clang/libidn2-2.3.8.tar.gz
8031278157ce43b5813f36cf8dd6baf0d9a7f88324ced796765dcd5cd96ccc06  b-debian12/libidn2-2.3.8.tar.gz
8031278157ce43b5813f36cf8dd6baf0d9a7f88324ced796765dcd5cd96ccc06  b-devuan5/libidn2-2.3.8.tar.gz
8031278157ce43b5813f36cf8dd6baf0d9a7f88324ced796765dcd5cd96ccc06  b-gcc/libidn2-2.3.8.tar.gz
8031278157ce43b5813f36cf8dd6baf0d9a7f88324ced796765dcd5cd96ccc06  r-debian12/libidn2-2.3.8.tar.gz
acf5cbb295e0693e4394a56c71600421059f9c9bf45ccf8a7e305c995630b32b  r-ubuntu2404/libidn2-2.3.8.tar.gz
cbdb75c38100e9267670b916f41878b6dbc35f9c6cbe60d50f458b40df64fcf1  b-almalinux9/libidn2-2.3.8.tar.gz
cbdb75c38100e9267670b916f41878b6dbc35f9c6cbe60d50f458b40df64fcf1  b-rockylinux9/libidn2-2.3.8.tar.gz
f557911bf6171621e1f72ff35f5b1825bb35b52ed45325dcdee931e5d3c0787a  b-guix/libidn2-2.3.8.tar.gz
f557911bf6171621e1f72ff35f5b1825bb35b52ed45325dcdee931e5d3c0787a  r-guix/libidn2-2.3.8.tar.gz

I’m sure I have forgotten or suppressed some challenges (sprinkling LANG=C TZ=UTC0 helps) related to these goals, but my hope is that this discussion of solutions will inspire you to implement these concepts for your software project too. Please share your thoughts and additional insights in a comment below. Enjoy Happy Hacking in the course of practicing this!

At one point, someone noticed that some financial transactions weren't summing up correctly in the C# application Nancy supported. It didn't require Superman or a Peter Gibbons to figure out why: someone was using floating points for handling dollar amounts.

That kicked off a big refactoring project to replace the usage of double types with decimal types. Everything seemed to go well, at least until there was a network hiccup and the application couldn't connect to the database. Let's see if you can figure out what happened:

MessageBox.Show("Please decimal check the connection details. Also check firewall settings (port 1433) and network connectivity.");

What a clbuttic mistake.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Author: Julian Miles, Staff Writer “Walk with me.” The tall being turns away from Nohane, sweeping it’s cloak out of the way with a graceful, flowing move. Nohane sighs. These trivial, effortless competences are what betray the elder of elders no matter how they try to disguise themselves. It is as if of all the […]

The post Pragmatic appeared first on 365tomorrows.

Oh no, it broke again!

Today, this .gitlab-ci.yml file no longer works in GitLab CI:

build-container-image:
  stage: build
  image: debian:testing
  before_script:
    - apt-get update
    - apt-get install -y buildah ca-certificates
  script:
    - buildah build -t $CI_REGISTRY_IMAGE .

The command buildah build ... fails with this error message:

STEP 2/3: RUN  apt-get update
internal:0:0-0: Error: Could not process rule: No such file or directory
internal:0:0-0: Error: Could not process rule: No such file or directory
error running container: did not get container start message from parent: EOF
Error: building at STEP "RUN apt-get update": setup network: netavark: nftables error: nft did not return successfully while applying ruleset

After some investigation, it's caused by the recent upload of netavark 1.14.0-2. In this version, netavark switched from iptables to nftables as the default firewall driver. That doesn't really fly on GitLab Saas shared runners.

For the complete background, refer to https://discussion.fedoraproject.org/t/125528. Note that the issue with GitLab was reported back in November, but at this point the conversation had died out.

Fortunately, it's easy to workaround, we can tell netavark to keep using iptables via the environment variables NETAVARK_FW. The .gitlab-ci.yml file above becomes:

build-container-image:
  stage: build
  image: debian:testing
  variables:
    # Cf. https://discussion.fedoraproject.org/t/125528/7
    NETAVARK_FW: iptables
  before_script:
    - apt-get update
    - apt-get install -y buildah ca-certificates
  script:
    - buildah build -t $CI_REGISTRY_IMAGE .

And everything works again!

If you're interested in this issue, feel free to fork https://gitlab.com/arnaudr/gitlab-build-container-image and try it by yourself.

Ringlet software updates (2025-03-23)

Recent initial releases of [Ringlet software][r-site] (a fancy name for my pet projects):

• [docker-scry][r-docker-scry] version [0.1.0][r-docker-scry-0.1.0] - examine Docker containers using host tools. Maybe the start of a set of tools that will allow system administrators to see what goes on in minimal containers that may not even have tools like ps or lsof installed.
• [pshlex][r-pshlex] version [0.1.0][r-pshlex-0.1.0] - join various stringifiable objects and quote them for the shell. A trivial Python function that I've embedded in many of my projects over the years and I finally decided to release: a version of [shlex.join()][python-shlex-join] that also accepts [pathlib.Path][python-pathlib-path] objects.
• [uvoxen][r-uvoxen] version [0.1.1][r-uvoxen-0.1.1] - generate test configuration files and run tests. A testing tool for Python projects that can either generate a [Tox configuration file][tox] or run the com...

This week on my podcast, I read my latest Locus Magazine column, “There Were Always Enshittifiers,” about the historical context for my latest novel, Picks and Shovels:

It used to be a much fairer fight. It used to be that if a com­pany figured out how to block copying its floppies, another company – or even just an individual tinkerer – could figure out how to break that “copy protection.” There were plenty of legitimate reasons to want to do this: Maybe you owned more than one computer, or maybe you were just worried that your floppy disk would degrade to the point of unread­ability. That’s a very reasonable fear: Floppies were notoriously unreliable, and every smart computer user learned to make frequent backups against the day that your computer presented you with the dread DISK ERROR message.

In those early days, it was an arms race between companies that wanted to control how their customers used their own computers, and the technological guerrillas who produced the countermeasures that restored command over your computer to you, its owner. It’s true that the companies making the “copy protection” (in scare quotes because the way you protect your data is by making copies of it) typically had far more resources than the toolsmiths who were defending technology users.

MP3

Author: Daniel Rogers “Victor, make coffee and display the weather.” I sank into my kitchen chair, scratching my messed-up mop of hair, wishing I’d gone to bed earlier. “You failed to obtain the recommended eight hours of sleep. It would be beneficial to have a cup of strong coffee.” “No, please. You know I don’t […]

The post Strong Coffee appeared first on 365tomorrows.

A new release 0.1.7 of RcppZiggurat is now on the CRAN network for R. This marks the first release in four and a half years.

The RcppZiggurat package updates the code for the Ziggurat generator by Marsaglia and others which provides very fast draws from a Normal distribution. The package provides a simple C++ wrapper class for the generator improving on the very basic macros, and permits comparison among several existing Ziggurat implementations. This can be seen in the figure where Ziggurat from this package dominates accessing the implementations from the GSL, QuantLib and Gretl—all of which are still way faster than the default Normal generator in R (which is of course of higher code complexity).

This release brings a number of changes. Notably, based on the work we did with the new package zigg (more on that in a second), we now also expose the Exponential generator, and the underlying Uniform generator. Otherwise many aspects of the package have been refreshed: updated builds, updated links, updated CI processes, more use of DOIs and more. The other big news is zigg which should now be the preference for deployment of Ziggurat due to its much lighter-weight and zero-dependency setup.

The NEWS file entry below lists all changes.

Changes in version 0.1.7 (2025-03-22)

• The CI setup was updated to use run.sh from r-ci (Dirk).

• The windows build was updated to GSL 2.7, and UCRT support was added (Jeroen in #16).

• Manual pages now use JSS DOIs for references per CRAN request

• README.md links and badges have been updated

• Continuous integration actions have been updated several times

• The DESCRIPTION file now uses Authors@R as mandated

• Use of multiple cores is eased via a new helper function reflecting option mc.core or architecture defaults, used in tests

• An inline function has been added to avoid a compiler nag

• Support for exponential RNG draws zrexp has been added, the internal uniform generator is now also exposed via zruni

• The vignette bibliography has been updated, and switched to DOIs

• New package zigg is now mentioned in DESCRIPTION and vignette

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the Rcppziggurat page or the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

The Open Source Initiative has two classes of board seats: Affiliate seats, and Individual Member seats. 

In the upcoming election, each affiliate can nominate a candidate, and each affiliate can cast a vote for the Affiliate candidates, but there's only 1 Affiliate seat available. I initially expressed interest in being nominated as an Affiliate candidate via Debian. But since Bradley Kuhn is also running for an Affiliate seat with a similar platform to me, especially with regards to the OSAID, I decided to run as part of an aligned "ticket" as an Individual Member to avoid contention for the 1 Affiliate seat.

Bradley and I discussed running on a similar ticket around 8/9pm Pacific, and I submitted my candidacy around 9pm PT on 17 February. 

I was dismayed when I received the following mail from Nick Vidal:

Dear Luke,

Thank you for your interest in the OSI Board of Directors election. Unfortunately, we are unable to accept your application as it was submitted after the official deadline of Monday Feb 17 at 11:59 pm UTC. To ensure a fair process, we must adhere to the deadline for all candidates.

We appreciate your enthusiasm and encourage you to stay engaged with OSI’s mission. We hope you’ll consider applying in the future or contributing in other meaningful ways.

Best regards,
OSI Election Teams

Nowhere on the "OSI’s board of directors in 2025: details about the elections" page do they list a timezone for closure of nominations; they simply list Monday 17 February. 

The OSI's contact address is in California, so it seems arbitrary and capricious to retroactively define all of these processes as being governed by UTC.

I was not able to participate in the "potential board director" info sessions accordingly, but people who attended heard that the importance of accommodating differing TZ's was discussed during the info session, and that OSI representatives mentioned they try to accommodate TZ's of everyone. This seems in sharp contrast with the above policy. 

I urge the OSI to reconsider this policy and allow me to stand for an Individual seat in the current cycle. 

Upd, N.B.: to people writing about this, I use they/them pronouns

Warning: this is a long ramble I wrote after an outage of my home internet. You'll get your regular scheduled programming shortly.

I didn't realize this until relatively recently, but we're at war.

Fascists and capitalists are trying to take over the world, and it's bringing utter chaos.

We're more numerous than them, of course: this is only a handful of people screwing everyone else over, but they've accumulated so much wealth and media control that it's getting really, really hard to move around.

Everything is surveilled: people are carrying tracking and recording devices in their pockets at all time, or they drive around in surveillance machines. Payments are all turning digital. There's cameras everywhere, including in cars. Personal data leaks are so common people kind of assume their personal address, email address, and other personal information has already been leaked.

The internet itself is collapsing: most people are using the network only as a channel to reach a "small" set of "hyperscalers": mind-boggingly large datacenters that don't really operate like the old internet. Once you reach the local endpoint, you're not on the internet anymore. Netflix, Google, Facebook (Instagram, Whatsapp, Messenger), Apple, Amazon, Microsoft (Outlook, Hotmail, etc), all those things are not really the internet anymore.

Those companies operate over the "internet" (as in the TCP/IP network), but they are not an "interconnected network" as much as their own, gigantic silos so much bigger than everything else that they essentially dictate how the network operates, regardless of standards. You access it over "the web" (as in "HTTP") but the fabric is not made of interconnected links that cross sites: all those sites are trying really hard to keep you captive on their platforms.

Besides, you think you're writing an email to the state department, for example, but you're really writing to Microsoft Outlook. That app your university or border agency tells you to install, the backend is not hosted by those institutions, it's on Amazon. Heck, even Netflix is on Amazon.

Meanwhile I've been operating my own mail server first under my bed (yes, really) and then in a cupboard or the basement for almost three decades now. And what for?

So I can tell people I can? Maybe!

I guess the reason I'm doing this is the same reason people are suddenly asking me about the (dead) mesh again. People are worried and scared that the world has been taken over, and they're right: we have gotten seriously screwed.

It's the same reason I keep doing radio, minimally know how to grow food, ride a bike, build a shed, paddle a canoe, archive and document things, talk with people, host an assembly. Because, when push comes to shove, there's no one else who's going to do it for you, at least not the way that benefits the people.

The Internet is one of humanity's greatest accomplishments. Obviously, oligarchs and fascists are trying to destroy it. I just didn't expect the tech bros to be flipping to that side so easily. I thought we were friends, but I guess we are, after all, enemies.

That said, that old internet is still around. It's getting harder to host your own stuff at home, but it's not impossible. Mail is tricky because of reputation, but it's also tricky in the cloud (don't get fooled!), so it's not that much easier (or cheaper) there.

So there's things you can do, if you're into tech.

Share your wifi with your neighbours.

Build a LAN. Throw a wire over to your neighbour too, it works better than wireless.

Use Tor. Run a relay, a snowflake, a webtunnel.

Host a web server. Build a site with a static site generator and throw it in the wind.

Download and share torrents, and why not a tracker.

Run an IRC server (or Matrix, if you want to federate and lose high availability).

At least use Signal, not Whatsapp or Messenger.

And yes, why not, run a mail server, join a mesh.

Don't write new software, there's plenty of that around already.

(Just kidding, you can write code, cypherpunk.)

You can do many of those things just by setting up a FreedomBox.

That is, after all, the internet: people doing their own thing for their own people.

Otherwise, it's just like sitting in front of the television and watching the ads. Opium of the people, like the good old time.

Let a billion droplets build the biggest multitude of clouds that will storm over this world and rip apart this fascist conspiracy.

Disobey. Revolt. Build.

We are more than them.

This morning, internet was down at home. The last time I had such an issue was in February 2023, when my provider was Oricom. Now I'm with a business service at Teksavvy Internet (TSI), in which I pay 100$ per month for a 250/50 mbps business package, with a static IP address, on which I run, well, everything: email services, this website, etc.

Mitigation

Email

The main problem when the service goes down like this for prolonged outages is email. Mail is pretty resilient to failures like this but after some delay (which varies according to the other end), mail starts to drop. I am actually not sure what the various settings are among different providers, but I would assume mail is typically kept for about 24h, so that's our mark.

Last time, I setup VMs at Linode and Digital Ocean to deal better with this. I have actually kept those VMs running as DNS servers until now, so that part is already done.

I had fantasized about Puppetizing the mail server configuration so that I could quickly spin up mail exchangers on those machines. But now I am realizing that my Puppet server is one of the service that's down, so this would not work, at least not unless the manifests can be applied without a Puppet server (say with puppet apply).

Thankfully, my colleague groente did amazing work to refactor our Postfix configuration in Puppet at Tor, and that gave me the motivation to reproduce the setup in the lab. So I have finally Puppetized part of my mail setup at home. That used to be hand-crafted experimental stuff documented in a couple of pages in this wiki, but is now being deployed by Puppet.

It's not complete yet: spam filtering (including DKIM checks and graylisting) are not implemented yet, but that's the next step, presumably to do during the next outage. The setup should be deployable with puppet apply, however, and I have refined that mechanism a little bit, with the run script.

Heck, it's not even deployed yet. But the hard part / grunt work is done.

Other

The outage was "short" enough (5 hours) that I didn't take time to deploy the other mitigations I had deployed in the previous incident.

But I'm starting to seriously consider deploying a web (and caching) reverse proxy so that I endure such problems more gracefully.

Side note on proper servics

Typically, I tend to think of a properly functioning service as having four things:

1. backups
2. documentation
3. monitoring
4. automation
5. high availability

Yes, I miscounted. This is why you have high availability.

Backups

Duh. If data is maliciously or accidentally destroyed, you need a copy somewhere. Preferably in a way that malicious joe can't get to.

This is harder than you think.

Documentation

I have an entire template for this. Essentially, it boils down to using https://diataxis.fr/ and this "audit" guide. For me, the most important parts are:

• disaster recovery (includes backups, probably)
• playbook
• install/upgrade procedures (see automation)

You probably know this is hard, and this is why you're not doing it. Do it anyways, you'll think it sucks, but you'll be really grateful for whatever scraps you wrote when you're in trouble.

Monitoring

If you don't have monitoring, you'll know it fails too late, and you won't know it recovers. Consider high availability, work hard to reduce noise, and don't have machine wake people up, that's literally torture and is against the Geneva convention.

Consider predictive algorithm to prevent failures, like "add storage within 2 weeks before this disk fills up".

This is harder than you think.

Automation

Make it easy to redeploy the service elsewhere.

Yes, I know you have backups. That is not enough: that typically restores data and while it can also include configuration, you're going to need to change things when you restore, which is what automation (or call it "configuration management" if you will) will do for you anyways.

This also means you can do unit tests on your configuration, otherwise you're building legacy.

This is probably as hard as you think.

High availability

Make it not fail when one part goes down.

Eliminate single points of failures.

This is easier than you think, except for storage and DNS (which, I guess, means it's harder than you think too).

Assessment

In the above 5 items, I check two:

1. backups
2. documentation

And barely: I'm not happy about the offsite backups, and my documentation is much better at work than at home (and even there, I have a 15 year backlog to catchup on).

I barely have monitoring: Prometheus is scraping parts of the infra, but I don't have any sort of alerting -- by which I don't mean "electrocute myself when something goes wrong", I mean "there's a set of thresholds and conditions that define an outage and I can look at it".

Automation is wildly incomplete. My home server is a random collection of old experiments and technologies, ranging from Apache with Perl and CGI scripts to Docker containers running Golang applications. Most of it is not Puppetized (but the ratio is growing). Puppet itself introduces a huge attack vector with kind of catastrophic lateral movement if the Puppet server gets compromised.

And, fundamentally, I am not sure I can provide high availability in the lab. I'm just this one guy running my home network, and I'm growing older. I'm thinking more about winding things down than building things now, and that's just really sad, because I feel we're losing (well that escalated quickly).

Resolution

In the end, I didn't need any mitigation and the problem fixed itself. I did do quite a bit of cleanup so that feels somewhat good, although I despaired quite a bit at the amount of technical debt I've accumulated in the lab.

Timeline

Times are in UTC-4.

• 6:52: IRC bouncer goes offline
• 9:20: called TSI support, waited on the line 15 minutes then was told I'd get a call back
• 9:54: outage apparently detected by TSI
• 11:00: no response, tried calling back support again
• 11:10: confirmed bonding router outage, no official ETA but "today", source of the 9:54 timestamp above
• 12:08: TPA monitoring notices service restored
• 12:34: call back from TSI; service restored, problem was with the "bonder" configuration on their end, which was "fighting between Montréal and Toronto"

Author: Julie Zack “Starlight, Starbright, First star I see tonight, Wish I may, Wish I might, Have this wish, I wish tonight.” Enid loved when her older sister, Tracy, spoke the words at bedtime. “Do you remember the stars?” Enid asked. “I do,” Tracy said, looking somehow both happy and sad. Enid couldn’t understand the […]

The post Honeycomb Dreams appeared first on 365tomorrows.

Authorities in at least two U.S. states last week independently announced arrests of Chinese nationals accused of perpetrating a novel form of tap-to-pay fraud using mobile devices. Details released by authorities so far indicate the mobile wallets being used by the scammers were created through online phishing scams, and that the accused were relying on a custom Android app to relay tap-to-pay transactions from mobile devices located in China.

Image: WLVT-8.

Authorities in Knoxville, Tennessee last week said they arrested 11 Chinese nationals accused of buying tens of thousands of dollars worth of gift cards at local retailers with mobile wallets created through online phishing scams. The Knox County Sheriff’s office said the arrests are considered the first in the nation for a new type of tap-to-pay fraud.

Responding to questions about what makes this scheme so remarkable, Knox County said that while it appears the fraudsters are simply buying gift cards, in fact they are using multiple transactions to purchase various gift cards and are plying their scam from state to state.

“These offenders have been traveling nationwide, using stolen credit card information to purchase gift cards and launder funds,” Knox County Chief Deputy Bernie Lyon wrote. “During Monday’s operation, we recovered gift cards valued at over $23,000, all bought with unsuspecting victims’ information.”

Asked for specifics about the mobile devices seized from the suspects, Lyon said “tap-to-pay fraud involves a group utilizing Android phones to conduct Apple Pay transactions utilizing stolen or compromised credit/debit card information,” [emphasis added].

Lyon declined to offer additional specifics about the mechanics of the scam, citing an ongoing investigation.

Ford Merrill works in security research at SecAlliance, a CSIS Security Group company. Merrill said there aren’t many valid use cases for Android phones to transmit Apple Pay transactions. That is, he said, unless they are running a custom Android app that KrebsOnSecurity wrote about last month as part of a deep dive into the operations of China-based phishing cartels that are breathing new life into the payment card fraud industry (a.k.a. “carding”).

How are these China-based phishing groups obtaining stolen payment card data and then loading it onto Google and Apple phones? It all starts with phishing.

If you own a mobile phone, the chances are excellent that at some point in the past two years it has received at least one phishing message that spoofs the U.S. Postal Service to supposedly collect some outstanding delivery fee, or an SMS that pretends to be a local toll road operator warning of a delinquent toll fee.

These messages are being sent through sophisticated phishing kits sold by several cybercriminals based in mainland China. And they are not traditional SMS phishing or “smishing” messages, as they bypass the mobile networks entirely. Rather, the missives are sent through the Apple iMessage service and through RCS, the functionally equivalent technology on Google phones.

People who enter their payment card data at one of these sites will be told their financial institution needs to verify the small transaction by sending a one-time passcode to the customer’s mobile device. In reality, that code will be sent by the victim’s financial institution in response to a request by the fraudsters to link the phished card data to a mobile wallet.

If the victim then provides that one-time code, the phishers will link the card data to a new mobile wallet from Apple or Google, loading the wallet onto a mobile phone that the scammers control. These phones are then loaded with multiple stolen wallets (often between 5-10 per device) and sold in bulk to scammers on Telegram.

An image from the Telegram channel for a popular Chinese smishing kit vendor shows 10 mobile phones for sale, each loaded with 5-7 digital wallets from different financial institutions.

Merrill found that at least one of the Chinese phishing groups sells an Android app called “Z-NFC” that can relay a valid NFC transaction to anywhere in the world. The user simply waves their phone at a local payment terminal that accepts Apple or Google pay, and the app relays an NFC transaction over the Internet from a phone in China.

“I would be shocked if this wasn’t the NFC relay app,” Merrill said, concerning the arrested suspects in Tennessee.

Merrill said the Z-NFC software can work from anywhere in the world, and that one phishing gang offers the software for $500 a month.

“It can relay both NFC enabled tap-to-pay as well as any digital wallet,” Merrill said. “They even have 24-hour support.”

On March 16, the ABC affiliate in Sacramento (ABC10), Calif. aired a segment about two Chinese nationals who were arrested after using an app to run stolen credit cards at a local Target store. The news story quoted investigators saying the men were trying to buy gift cards using a mobile app that cycled through more than 80 stolen payment cards.

ABC10 reported that while most of those transactions were declined, the suspects still made off with $1,400 worth of gift cards. After their arrests, both men reportedly admitted that they were being paid $250 a day to conduct the fraudulent transactions.

Merrill said it’s not unusual for fraud groups to advertise this kind of work on social media networks, including TikTok.

A CBS News story on the Sacramento arrests said one of the suspects tried to use 42 separate bank cards, but that 32 were declined. Even so, the man still was reportedly able to spend $855 in the transactions.

Likewise, the suspect’s alleged accomplice tried 48 transactions on separate cards, finding success 11 times and spending $633, CBS reported.

“It’s interesting that so many of the cards were declined,” Merrill said. “One reason this might be is that banks are getting better at detecting this type of fraud. The other could be that the cards were already used and so they were already flagged for fraud even before these guys had a chance to use them. So there could be some element of just sending these guys out to stores to see if it works, and if not they’re on their own.”

Merrill’s investigation into the Telegram sales channels for these China-based phishing gangs shows their phishing sites are actively manned by fraudsters who sit in front of giant racks of Apple and Google phones that are used to send the spam and respond to replies in real time.

In other words, the phishing websites are powered by real human operators as long as new messages are being sent. Merrill said the criminals appear to send only a few dozen messages at a time, likely because completing the scam takes manual work by the human operators in China. After all, most one-time codes used for mobile wallet provisioning are generally only good for a few minutes before they expire.

For more on how these China-based mobile phishing groups operate, check out How Phished Data Turns Into Apple and Google Wallets.

The ashtray says: You’ve been phishing all night.