Planet Russell

,

Charles StrossDead Lies Dreaming: Spoilers

I've been head-down in the guts of a novel this month, hence lack of blogging: purely by coincidence, I'm working on the next-but-one sequel to Dead Lies Dreaming.

Which reminds me that Dead Lies Dreaming came out nearly a month ago, and some of you probably have read it and have questions!

So feel free to ask me anything about the book in the comments below.

(Be warned that (a) there will probably be spoilers, and (b) I will probably not answer questions that would supply spoilers for the next books in the ongoing project.)

Charles StrossCountdown to Crazy

This is your official thread for discussing the upcoming US presidential and congressional election on November 3rd; along with its possible outcomes.

Do not chat about the US supreme court, congress, presidency, constitution, constitutional crises (possible), coup (possible), Donald Trump and his hellspawn offspring and associates, or anything about US politics in general on the Laundry Files book launch threads. If you do, your comments will be ruthlessly moderated into oblivion.

You are allowed and encouraged to discuss those topics in the comments below this topic.

(If you want to discuss "Dead Lies Dreaming" here I won't stop you, but there's plenty of other places for that!)

Worse Than FailureError'd: Mandatory Pants Day

"I wonder if the people behind the ad campaign ever said they could 'sell ice to a snowman'," Loren writes.

 

"Umm...Thanks for the suggestion?" John wrote.

 

Kolja writes, "So, wait, do I type 'ENTER' or just press the 'ENTER' key? And what if I hae a 'Return' key instead?"

 

"Given how annoying and defective software licensing usually is, an operation completing successfully should indeed be considered as an error," wrote Carl C.

 

Tim P. writes, "If you're thinking about buying your candy by the kilo, beware of that sugar tax!"

 

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

Planet DebianRussell Coker: KDE Icons Disappearing in Debian/Unstable

One of my workstations is running Debian/Unstable with KDE and SDDM on an AMD Radeon R7 260X video card. Recently it stopped displaying things correctly after a reboot, all the icons failed to display as well as many of the Qt controls. When I ran a KDE application from the command line I got the error “QSGTextureAtlas: texture atlas allocation failed, code=501“. Googling that error gave a blog post about a very similar issue in 2017 [1]. From that blog post I learned that I could stop the problem by setting MESA_EXTENSION_OVERRIDE=”-GL_EXT_bgra -GL_EXT_texture_format_BGRA8888″ in the environment. In a quick test I found that the environment variable setting worked, making the KDE apps display correctly and not report an error about a texture atlas.

I created a file ~/.config/plasma-workspace/env/bgra.sh with the following contents:

export MESA_EXTENSION_OVERRIDE="-GL_EXT_bgra -GL_EXT_texture_format_BGRA8888"

Then after the next login things worked as desired!

Now the issue is, where is the bug? GL, X, and the internals of KDE are things I don’t track much. I welcome suggestions from readers of my blog as to what the culprit might be and where to file a Debian bug – or a URL to a Debian bug report if someone has already filed one.

Update

When I run the game warzone2100 with this setting it crashes with the below output. So this Mesa extension override isn’t always a good thing, just solves one corner case of a bug.

$ warzone2100 
/usr/bin/gdb: warning: Couldn't determine a path for the index cache directory.
27      ../sysdeps/unix/sysv/linux/wait4.c: No such file or directory.
No frame at level 0x7ffc3392ab50.
Saved dump file to '/home/etbe/.local/share/warzone2100-3.3.0//logs/warzone2100.gdmp-VuGo2s'
If you create a bugreport regarding this crash, please include this file.
Segmentation fault (core dumped)

Planet DebianDirk Eddelbuettel: #31: Test your R package against bleeding-edge gcc

Welcome to the 31th post in the rapturously rampant R recommendations series, or R4 for short. This post will once again feature Docker for use with R.

Earlier this week, I received a note from CRAN about how my RcppTOML package was no longer building with the (as of right now of course unreleased) version 11 of the GNU C++ compiler, i.e. g++-11. And very kindly even included a hint about the likely fix (which was of course correct). CRAN, and one of its maintainers in particular, is extremely forward-looking in terms of toolchain changes. A year ago we were asked to updated possible use of global variables in C code as gcc-10 tightened the rules. This changes is a C++ one, and a fairly simple one of simply being more explicit with include headers. Previous g++ release had done the same.

The question now was about the least painful way to get g++-11 onto my machine, with the least amount of side-effects. Regular readers of this blog will know where this is headed, but even use of Docker requires binaries. A look at g++-11 within packages.debian.org comes up empty. No Debian means no Ubuntu. But … there is a PPA for Ubuntu with toolchain builds we have used before. And voilà there we have it: within the PPA for Ubuntu Toolchain repository is the volatile packages PPA with both g++-10 and g++-11. Here Ubuntu 20.10 works with g++-10, but g++-11 requires Ubuntu 21.04. Docker containers are there for either. So with the preliminaries sorted out, the key steps are fairly straightforward:

  • start from ubuntu:21.04 to be able to install g++-11 later
  • install the software-properties-common package to be able to add a PPA
  • (plus a few more packages to deal with the repository signing key)
  • run the sudo add-apt-repository ppa:ubuntu-toolchain-r/volatile command to add the volatile packages PPA
  • install g++-11 (along with, for good measure) gcc-11 and gfortran-11
  • use update-alternative (a clever Debian/Ubuntu command) to make version ‘11’ the default
  • install R itself (via r-base-core) which we simply take from the distro as 21.04 is by construction very recent
  • install Rcpp via the r-cran-rcpp binary which covers all dependencies for the package in question

And that is it! RcppTOML is fairly minimal and could be a member of the tinyverse so no other dependencies are needed—if your package has any you could just use the standard steps to install from source, or binary (including using RSPM or bspm). You can see the resulting Dockerfile which contains a minimal amount of extra stuff to deal with some environment variables and related settings. Nothing critical, but it smoothes the experience somewhat.

This container is now built (under label rocker/r-edge with tags latest and gcc-11), and you can download it from Docker Hub. With that the ‘proof’ of the (now fixed and uploaded) package building becomes as easy as

edd@rob:~/git/rcpptoml(master)$ docker run --rm -ti -v $PWD:/mnt -w /mnt rocker/r-edge:gcc-11 g++ --version
g++ (Ubuntu 11-20201128-0ubuntu2) 11.0.0 20201128 (experimental) [master revision fb6b29c85c4:a331ca6194a:e87559d202d90e614315203f38f9aa2f5881d36e]
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

edd@rob:~/git/rcpptoml(master)$ 
edd@rob:~/git/rcpptoml(master)$ docker run --rm -ti -v $PWD:/mnt -w /mnt rocker/r-edge:gcc-11 R CMD INSTALL RcppTOML_0.1.7.tar.gz
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘RcppTOML’ ...
** using staged installation
** libs
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I../inst/include/ -DCPPTOML_USE_MAP -I'/usr/lib/R/site-library/Rcpp/include'    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-Fuvi9C/r-base-4.0.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I../inst/include/ -DCPPTOML_USE_MAP -I'/usr/lib/R/site-library/Rcpp/include'    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-Fuvi9C/r-base-4.0.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c parse.cpp -o parse.o
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o RcppTOML.so RcppExports.o parse.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-RcppTOML/00new/RcppTOML/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppTOML)
edd@rob:~/git/rcpptoml(master)$ 

I hope both the availability of such a base container with gcc-11 (and g++-11 and gfortran-11) as well as a “recipe” for building similar containers with newer clang version will help other developers.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

,

Planet DebianDaniel Lange: No dog food today - the Linux Foundation annual report

The Linux Foundation has published its annual report today. LWN calls it glossy and yeah, boy, it is shiny.

So shiny that people that work in the publishing industry immediately see this has been produced with the Adobe toolchain which - unfortunately - is one of the big suits of software not yet available for Linux.

Checking the PDF file metadata reveals the keywords "open source, open standards, open hardware, open data". That is what the Linux Foundation is about. Good stuff.

Linux Foundation annual report 2020 cover

The PDF producer meta data for the annual report PDF has been set to "Linux kernel 0.12.1 for Workgroups" and the PDF creator meta data element to "Sharp Zaurus XR-5000 (Maemo5) Edition". Somebody thought to better hide the real data and had some tongue-in-cheek ideas. Kudos.

But nicer would have been to use Open Source software to produce the report, not?

Running strings 2020-Linux-Foundation-Annual-Report_113020.pdf | grep Adobe | wc -l gives us 1229 lines and confirms the suspicion of the toolchain.

A stale /Title (Annual Report 2020) /Producer (macOS Version 10.15.7 \(Build 19H15\) Quartz PDFContext) has been forgotten in the document to tell us about the platform.

So, ladies and gentlemen, the Linux Foundation 2020 annual report has been produced on a Mac.

Running Adobe Creative Cloud on MacOS Catalina 10.15.7.

Which is proprietary software. Its kernel (and some userland pieces) are based on BSD. Not Linux.


The image on the front page also struck me as a bit odd ... using a ballpoint pen on the laptop screen?

Unbranded laptop. Unbranded cup in the foreground.

Kid in the background not paying attention to his tablet.

All of that cries stock image so loud it hurts.

Google currently finds ~560 uses of the picture and any editorial use nicely tells us that it is © Dragana Gordic / Shutterstock.

The image is "Smiling mom working at home with her child on the sofa while writing an email. Young woman working from home, while in quarantine isolation during the Covid-19 health crisis".

See the Daily Mail for a wonderful example of the working mum in context. I hope, if her laptop had been powered on, it would have run Linux. I mean, what else would still run on an old white MacBook with an Intel "Core 2 Duo" processor from 2008?

Daily Mail screenshot of the same stock image used

Continue reading "No dog food today - the Linux Foundation annual report"

Planet DebianSteinar H. Gunderson: plocate 1.1.0 released

I've released version 1.1.0 of plocate. The major new feature is that it now ships its own updatedb; I imported a fair amount of code from mlocate's updatedb (which makes the updatedb implementation GPLv2, unlike the rest of plocate, which is GPLv2+), reworked it heavily and made it read and write plocate databases. The databases need a little extra data, which increases their size by 1% or so, but that's well worth it, because now no longer need the mlocate database; those are typically more than twice the size of plocate's database. Obviously, this also removes the dependency on mlocate.

Like updatedb.mlocate, updatedb.plocate is merging—if the directory hasn't changed since last time, it won't readdir() it. (It will stat() it, though, again like mlocate.) There's no io_uring yet, but for most people, it won't really matter. And databases converted from mlocate won't have the merging information, so the first time you run updatedb.plocate, it will make a full scan.

Debian unstable has the package uploaded; backports to buster will come when it's through the testing quarantine.

Planet DebianAndrew Cater: Preparing for release of Debian 10.7 over the weekend and CentOS / Scientific Linux 6.x and EPEL for 6 now EOL

For those keeping score:

This weekend - 5th December 2020 - should see us release Debian 10.7 - an update to Debian stable (Buster) so I should be spending a day or so in the company of my friends and colleagues.

Red Hat 6.10 is now out of support unless you pay Extended Update subscriptions for individual Red Hat machines. This means that CentOS 6.* has now been removed from CentOS mirrors since these were dependent on Red Hat 6 sources.. Similarly, Scientific Linux have also removed their fork of 6.*. They are continuing to support a Scientific Linux 7 but suggest a move to CentOS 8 thereafter.

Separately, Fedora Linux have removed EPEL 6.* - definitely time to update to Red Hat/CentOS/Scientific Linux 7.* or greater for those affected.

 All the very best to all:

Planet DebianAlberto García: Subcluster allocation for qcow2 images

In previous blog posts I talked about QEMU’s qcow2 file format and how to make it faster. This post gives an overview of how the data is structured inside the image and how that affects performance, and this presentation at KVM Forum 2017 goes further into the topic.

This time I will talk about a new extension to the qcow2 format that seeks to improve its performance and reduce its memory requirements.

Let’s start by describing the problem.

Limitations of qcow2

One of the most important parameters when creating a new qcow2 image is the cluster size. Much like a filesystem’s block size, the qcow2 cluster size indicates the minimum unit of allocation. One difference however is that while filesystems tend to use small blocks (4 KB is a common size in ext4, ntfs or hfs+) the standard qcow2 cluster size is 64 KB. This adds some overhead because QEMU always needs to write complete clusters so it often ends up doing copy-on-write and writing to the qcow2 image more data than what the virtual machine requested. This gets worse if the image has a backing file because then QEMU needs to copy data from there, so a write request not only becomes larger but it also involves additional read requests from the backing file(s).

Because of that qcow2 images with larger cluster sizes tend to:

  • grow faster, wasting more disk space and duplicating data.
  • increase the amount of necessary I/O during cluster allocation,
    reducing the allocation performance.

Unfortunately, reducing the cluster size is in general not an option because it also has an impact on the amount of metadata used internally by qcow2 (reference counts, guest-to-host cluster mapping). Decreasing the cluster size increases the number of clusters and the amount of necessary metadata. This has direct negative impact on I/O performance, which can be mitigated by caching it in RAM, therefore increasing the memory requirements (the aforementioned post covers this in more detail).

Subcluster allocation

The problems described in the previous section are well-known consequences of the design of the qcow2 format and they have been discussed over the years.

I have been working on a way to improve the situation and the work is now finished and available in QEMU 5.2 as a new extension to the qcow2 format called extended L2 entries.

The so-called L2 tables are used to map guest addresses to data clusters. With extended L2 entries we can store more information about the status of each data cluster, and this allows us to have allocation at the subcluster level.

The basic idea is that data clusters are now divided into 32 subclusters of the same size, and each one of them can be allocated separately. This allows combining the benefits of larger cluster sizes (less metadata and RAM requirements) with the benefits of smaller units of allocation (less copy-on-write, smaller images). If the subcluster size matches the block size of the filesystem used inside the virtual machine then we can eliminate the need for copy-on-write entirely.

So with subcluster allocation we get:

  • Sixteen times less metadata per unit of allocation, greatly reducing the amount of necessary L2 cache.
  • Much faster I/O during allocating when the image has a backing file, up to 10-15 times more I/O operations per second for the same cluster size in my tests (see chart below).
  • Smaller images and less duplication of data.

This figure shows the average number of I/O operations per second that I get with 4KB random write requests to an empty 40GB image with a fully populated backing file.

I/O performance comparison between traditional and extended qcow2 images

Things to take into account:

  • The performance improvements described earlier happen during allocation. Writing to already allocated (sub)clusters won’t be any faster.
  • If the image does not have a backing file chances are that the allocation performance is equally fast, with or without extended L2 entries. This depends on the filesystem, so it should be tested before enabling this feature (but note that the other benefits mentioned above still apply).
  • Images with extended L2 entries are sparse, that is, they have holes and because of that their apparent size will be larger than the actual disk usage.
  • It is not recommended to enable this feature in compressed images, as compressed clusters cannot take advantage of any of the benefits.
  • Images with extended L2 entries cannot be read with older versions of QEMU.

How to use this?

Extended L2 entries are available starting from QEMU 5.2. Due to the nature of the changes it is unlikely that this feature will be backported to an earlier version of QEMU.

In order to test this you simply need to create an image with extended_l2=on, and you also probably want to use a larger cluster size (the default is 64 KB, remember that every cluster has 32 subclusters). Here is an example:

$ qemu-img create -f qcow2 -o extended_l2=on,cluster_size=128k img.qcow2 1T

And that’s all you need to do. Once the image is created all allocations will happen at the subcluster level.

More information

This work was presented at the 2020 edition of the KVM Forum. Here is the video recording of the presentation, where I cover all this in more detail:

You can also find the slides here.

Acknowledgments

This work has been possible thanks to Outscale, who have been sponsoring Igalia and my work in QEMU.

Igalia and Outscale

And thanks of course to the rest of the QEMU development team for their feedback and help with this!

Cryptogram Open Source Does Not Equal Secure

Way back in 1999, I wrote about open-source software:

First, simply publishing the code does not automatically mean that people will examine it for security flaws. Security researchers are fickle and busy people. They do not have the time to examine every piece of source code that is published. So while opening up source code is a good thing, it is not a guarantee of security. I could name a dozen open source security libraries that no one has ever heard of, and no one has ever evaluated. On the other hand, the security code in Linux has been looked at by a lot of very good security engineers.

We have some new research from GitHub that bears this out. On average, vulnerabilities in their libraries go four years before being detected. From a ZDNet article:

GitHub launched a deep-dive into the state of open source security, comparing information gathered from the organization’s dependency security features and the six package ecosystems supported on the platform across October 1, 2019, to September 30, 2020, and October 1, 2018, to September 30, 2019.

Only active repositories have been included, not including forks or ‘spam’ projects. The package ecosystems analyzed are Composer, Maven, npm, NuGet, PyPi, and RubyGems.

In comparison to 2019, GitHub found that 94% of projects now rely on open source components, with close to 700 dependencies on average. Most frequently, open source dependencies are found in JavaScript — 94% — as well as Ruby and .NET, at 90%, respectively.

On average, vulnerabilities can go undetected for over four years in open source projects before disclosure. A fix is then usually available in just over a month, which GitHub says “indicates clear opportunities to improve vulnerability detection.”

Open source means that the code is available for security evaluation, not that it necessarily has been evaluated by anyone. This is an important distinction.

Worse Than FailureCodeSOD: WWJSD?

A few months ago, Lee was reviewing a pull request from Eddie. Eddie was a self-appointed "rockstar" developer. Eddie might be a "junior" developer in job title, but they're up on the latest, greatest, bestest practices, and if everyone would just get out of Eddie's way, the software would get so much better.

Which is why the pull request Lee was looking at touched nearly every file. Every change was some variation of this:

- if (obj == null) + if (obj is null)

"Did… did you do a 'Replace in Files' on the entire solution?" Lee asked.

"Well, I had to use the regex find-and-replace," Eddie said, "have you ever used that? It's a great refactoring tool, you can get it with control-H, and then you have to check a box-"

"I know how it works," Lee said. "Why?"

"Well, a regex, or a regular expression is a-"

"No, why did you make the change?"

"Oh! Well, that's the correct way to check for nulls. The equality operator could be overloaded, and that overload might have bugs."

"I mean, it's a correct way," Lee said, "but the equality operator is also fine. Any bugs in our overloads are likely already caught. There's no reason to make a change like this through the whole codebase."

The debate went on for longer than it should have. Eventually, Eddie defended his choice by saying, "Well, it's the way Jon Skeet does it," at which point Lee threw up his hands.

"Fine." Lee approved the request. After all, what was the worst that could happen?

What happened was that a month or two later, the application started throwing NullReferenceExceptions in production. The change which caused the behavior was on line:

Coordinate p = maybeGetCoordinate(); if (p != null) { p.Project(alternateCoordinateSystem); // NullReferenceException }

Look at that, someone using the more traditional equality operator! Maybe Eddie had a point, maybe an overloaded equality operator was the problem. Certainly, the bug went away if Lee used !(p is null) (or, in C# 9.0, p is not null).

Still, it would be good to fix the bug. Lee took a peek at the overloaded operators:

public static bool operator == (Coordinate pt1, Coordinate pt2) => !(pt1 is null) && !(pt2 is null) && pt1.Longitude == pt2.Longitude && pt1.Latitude == pt2.Latitude; public static bool operator !=(Coordinate pt1, Coordinate pt2) => !(pt1 == pt2);

The key bug is !(pt1 is null) && !(pt2 is null)- if one or more of the operands is null, then return false. This means null == null is always false, which means the p != null check returns false if p is null.

As it turned out, Eddie did have a point about favoring is null checks. Someone might add a buggy operator overload which could cause unexpected behavior. Who did add that particular buggy overload? Why, Eddie, of course.

Lee submitted a patch, and gently suggested Eddie might need to spend a little more time in the woodshed before declaring themselves a rockstar.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Planet DebianBdale Garbee: Shifting Emphasis

I joined the Debian project in late 1994, well before the first stable release was issued, and have been involved in various ways continuously ever since. Over the years, I adopted a number of packages that are, or at least were at one time, fundamental to the distribution.

But, not surprisingly, my interests have shifted over time. In the more than quarter century I've contributed to Debian, I've adopted existing packages that needed attention, packaged new software I wanted to use that wasn't yet in Debian, offered packages up for others to adopt, and even sometimes requested the removal of packages that became obsolete or replaced by something better. That all felt completely healthy.

But over the last couple weeks, I realized I'm still "responsible" for some packages I'd had for a very long time, that generally work well but over time have accumulated bugs in functionality I just don't use, and frankly haven't been able to find the motivation to chase down. As one example, I just noticed that I first uploaded the gzip package 25 years ago today, on 2 December 1995. And while the package works fine for me and most other folks, there are 30 outstanding bugs and 3 forwarded bugs that I just can't muster up any energy to address.

So, I just added gzip to a short list of packages I've offered up for adoption recently. I'm pleased that tar already has a new maintainer, and hope that both sudo and gzip will get more attention soon.

It's not that I'm less interested in Debian. I've just been busy recently packaging up more software I use or want to use in designing high power model rockets and the solid propellant motors I fly in them, and would rather spend the time I have available for Debian maintaining those packages and all their various build dependencies than continuing to be responsible for core packages in the distribution that "work fine for me" but could use attention.

I'm writing about this partly to mark the passing of more than a quarter century as a package maintainer for Debian, partly to encourage other Debian package maintainers with the right skills and motivation to consider adopting some of the packages I'm giving up, and finally to encourage other long-time participants in Debian to spend a little time evaluating their own package lists in a similar way.

Planet DebianDirk Eddelbuettel: RcppTOML 0.1.7: Support for g++-11, Minor Updates

A new RcppTOML release arrived on CRAN earlier today evening. RcppTOML brings TOML to R.

TOML is a file format that is most suitable for configurations, as it is meant to be edited by humans but read by computers. It emphasizes strong readability for humans while at the same time supporting strong typing as well as immediate and clear error reports. On small typos you get parse errors, rather than silently corrupted garbage. Much preferable to any and all of XML, JSON or YAML – though sadly these may be too ubiquitous now. TOML has been making inroads with projects such as the Hugo static blog compiler, or the Cargo system of Crates (aka “packages”) for the Rust language.

CRAN had sent us a note that the package no longer compiled under the [unreleased, of course, never change, BDR ;-) ] g++-11 compiler, but were kind enough to hint that it was only lacking an #include <limits>. These things happen: newer compilers are generally more strict, and that is generally a good things. (Last year this time we prepped code for the more stringent view on global variables under gcc-10. Earlier g++ version had similar demands to clarify include headers.) I set up a simple Docker contain with on Ubuntu 21.04 with g++-11, R, and Rcpp to build the package and make this change (which was of course also PR’ed upstream at cpptoml), plus some other small ones that update the package since the last release roughly 18 months ago. We also switched CI use to the r-ci setup I should blog about a little more, removed a bashism and updated a few URLs. The bulleted list of changes in this version follows.

Changes in version 0.1.7 (2020-12-01)

  • Add #include <limits> to header file, also contributed upstream, to permit compilation under the (unreleased) g++-11.

  • Switch the simple cleanup script to sh.

  • Switch CI use to r-ci for focal and bspm.

  • Update several TOML URLs to https://toml.io/en/.

Courtesy of my CRANberries, there is a diffstat report for this release. More information is on the RcppTOML page page. Please use the GitHub issue tracker for issues and bugreports.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

,

Cryptogram Impressive iPhone Exploit

This is a scarily impressive vulnerability:

Earlier this year, Apple patched one of the most breathtaking iPhone vulnerabilities ever: a memory corruption bug in the iOS kernel that gave attackers remote access to the entire device­ — over Wi-Fi, with no user interaction required at all. Oh, and exploits were wormable­ — meaning radio-proximity exploits could spread from one nearby device to another, once again, with no user interaction needed.

[…]

Beer’s attack worked by exploiting a buffer overflow bug in a driver for AWDL, an Apple-proprietary mesh networking protocol that makes things like Airdrop work. Because drivers reside in the kernel — ­one of the most privileged parts of any operating system­ — the AWDL flaw had the potential for serious hacks. And because AWDL parses Wi-Fi packets, exploits can be transmitted over the air, with no indication that anything is amiss.

[…]

Beer developed several different exploits. The most advanced one installs an implant that has full access to the user’s personal data, including emails, photos, messages, and passwords and crypto keys stored in the keychain. The attack uses a laptop, a Raspberry Pi, and some off-the-shelf Wi-Fi adapters. It takes about two minutes to install the prototype implant, but Beer said that with more work a better written exploit could deliver it in a “handful of seconds.” Exploits work only on devices that are within Wi-Fi range of the attacker.

There is no evidence that this vulnerability was ever used in the wild.

EDITED TO ADD: Slashdot thread.

Planet DebianJonathan Dowland: Musick To Play In The Dark

picture of a vinyl record

Music for these dark nights…

Planet DebianJonathan Dowland: OpenJDK Author

I have recently become an OpenJDK Author. Practically this means I can query the Java Bug Database directly, and I can author Mercurial for committing to the relevant OpenJDK project myself, rather than needing sponsors in both cases.

A small milestone but one I'm proud of! Thanks to my team members who have encouraged me and helped me by sponsoring and reviewing my work so far. Next step, Committer!

Krebs on SecurityAccount Hijacking Site OGUsers Hacked, Again

For at least the third time in its existence, OGUsers — a forum overrun with people looking to buy, sell and trade access to compromised social media accounts — has been hacked.

An offer by the apparent hackers of OGUsers, offering to remove account information from the eventual database leak in exchange for payment.

Roughly a week ago, the OGUsers homepage was defaced with a message stating the forum’s user database had been compromised. The hack was acknowledged by the forum’s current administrator, who assured members that their passwords were protected with a password obfuscation technology that was extremely difficult to crack.

But unlike in previous breaches at OGUsers, the perpetrators of this latest incident have not yet released the forum database. In the meantime, someone has been taunting forum members, saying they can have their profiles and private messages removed from an impending database leak by paying between $50 and $100.

OGUsers was hacked at least twice previously, in May 2019 and again in March 2020. In the wake of both incidents, the compromised OGUsers databases were made available for public download.

The leaked databases have been useful in reconstructing who’s behind several high-profile incidents involving compromised social media accounts and virtual currency heists that leveraged SIM swapping, a crime that centers around convincing mobile phone company employees to transfer ownership of the target’s phone number to a device the attackers control.

For example, when several high-profile Twitter accounts were hacked in July 2020 and used to promote bitcoin scams, the profile and private message data from previous OGUser forum compromises proved invaluable in piecing together the “who” behind that scam.

The hacker handles featured in the defacement message left on OGUsers — “Chinese” and “Disco” — correspond to two nicknames used by banned OGUser members who have been trying to generate interest for their own forum that seeks to emulate OGUsers.

Disco, a.k.a “Discoli” a.k.a. “Disco Dog,” is a young man from the United Kingdom who has marketed an automated bot program and service advertised as a way for customers to “cash out” illicit access to OneVanilla Visa prepaid card accounts using PayPal. The same individual also earlier this year founded a corporation in the U.K. called Disco Payments.

Reached via Twitter, Discoli said he and his friends hacked OGUsers via an outdated plugin used by the site. But he claims they have no plans to sell the stolen user data, and said the company was registered as a joke.

“I had a sort of feud with the administrator in the past but this one was more for fun,” Discoli said. “Not too interested in doing damage by releasing database or anything like that.”

As I noted the first time OGUsers got hacked, it’s difficult not to admit feeling a bit of schadenfreude in the continued exposure of a community that has largely specialized in hacking others. Or perhaps in the case of OGUsers, the sentiment may more aptly be described as “schadenfraud.”

Kevin RuddABC 730 – Kevin Rudd on Australia-China relations

E&OE TRANSCRIPT
TV INTERVIEW
ABC 730
01 DECEMBER 2020

 

The post ABC 730 – Kevin Rudd on Australia-China relations appeared first on Kevin Rudd.

Charles StrossUpcoming Attractions!

As you know by now, my next novel, Dead Lies Dreaming comes out next week—on Tuesday the 27th in the US and Thursday 29th in the UK, because I've got different publishers in different territories).

Signed copies can be ordered from Transreal Fiction in Edinburgh via the Hive online mail order service.

(You can also order it via Big River co and all good bookshops, but they don't stock signed copies: Link to Amazon US: Link to Amazon UK. Ebooks are available too, and I gather the audiobook—again, there's a different version in the US, from Audible, and the UK, from Hachette Digital—should be released at the same time.)

COVID-19 has put a brake on any plans I might have had to promote the book in public, but I'm doing a number of webcast events over the next few weeks. Here are the highlights:

Outpost 2020 is a virtual SF convention taking place from Friday 23rd (tomorrow!) to Sunday 25th. I'm on a discussion panel on Saturday 24th at 4pm (UK time), on the subject of "Reborn from the Apocalypse": Both history and current events teach that a Biblical-proportioned apocalypse is not necessarily confined to the realms of fiction. How can we reinvent ourselves, and more importantly, will we?. (Panelists: Charlie Stross, Gabriel Partida, David D. Perlmutter. Moderator: Mike Fatum.)

Orbit Live! As part of a series of Crowdcast events, at 8pm GMT on Thursday 27th RJ Barker is going to host myself and Luke Arnold in conversation about our new books: sign up for the crowdcast here.

Reddit AmA: No book launch is complete these days without an Ask me Anything on Reddit, which in my case is booked for Tuesday 3rd, starting at 5pm, UK time (9am on the US west coast, give or take an hour—the clocks change this weekend in the UK but I'm not sure when the US catches up).

The Nürnberg Digital Festival is a community driven Festival with about 20.000 attendees in Nuremberg, to discuss the future, change and everything that comes with it. Obviously this year it's an extra-digital (i.e. online-only) festival, which has the silver lining of enabling the organizers to invite guests to connect from a long way away. Which is why I'm doing an interview/keynote on Monday November 9th at 5pm (UK time). You can find out more about the Festival here (as well as buying tickets for any or all days' events). It's titled "Are we in dystopian times?" which seems to be an ongoing theme of most of the events I'm being invited to these days, and probably gives you some idea of what my answer is likely to be ...

Anyway, that's all for now: I'll add to this post if new events show up.

Planet DebianNorbert Preining: Debian KDE/Plasma Status 2020-12-02

Another month worth of updates on KDE/Plasma in Debian has accumulated, so here we go. The highlights are: Plasma 5.19.5 based on Qt 5.15 is in Debian/unstable and testing, Plasma 5.20.4 is waiting to be uploaded soon to experimental, and my own builds at OBS have been updated to Plasma 5.20.4, Frameworks 5.76, Apps 20.08.3.

OBS packages

The OBS packages as usual follow the latest release, and currently ship KDE Frameworks 5.76, KDE Apps 20.08.3, and new, Plasma 5.20.4. The package sources are as usual (only the other-dep has disappeared, these packages are now all in Debian proper), for Debian/unstable:

deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/frameworks/Debian_Unstable/ ./
deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/plasma520/Debian_Unstable/ ./
deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/apps2008/Debian_Unstable/ ./
deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/other/Debian_Unstable/ ./

and the same with Testing instead of Unstable for Debian/testing.

The latest update to 5.20.4 will probably need one or two install cycles due to missing Replaces. The reason is that the packages in OBS have been synced with the work we have done for the official Debian packages, and thus some files have moved between packages.

Debian main packages

After a few hiccups, the whole Plasma 5.19.5 stack has landed in Debian/testing, and we are working on an upload of 5.20.4 to Debian/experimental. This takes a bit, because we want to add infrastructure to automatically force all packages to be upgraded only together, so that there is no mix between old and new packages. This has shown quite some problems during the upgrade from 5.17 to 5.19 in testing. Also, I don’t think Debian needs to do something else than everybody else by allowing to mix Plasma released.

That said, I hope that we have an upload of 5.20 to experimental soon, followed by testing and an upload to unstable, so that it can migrate to testing in time for the freeze of Bullseye. Getting 5.20.5 into Bullseye is possible, but not guaranteed, not too much time between the release of 5.20.5 and proper freeze in Debian. But we will see.

Enjoy.

Worse Than FailureA New Bean

Roasted coffee beans

It was Paramdeep's first corporate IT job. He was assigned a mentor, Rajiv, who would train him up on the application he would be supporting: a WebSphere-based Java application full of Enterprise Java Beans.

Paramdeep reserved time with Rajiv in Outlook, arranging to meet at Rajiv's cubicle for half an hour. Rajiv accepted. At the agreed-upon time, Paramdeep walked over with a notebook and pencil in hand, intent on copying down all the pertinent information he would hear. When he reached Rajiv's desk, however, the elder developer waved Paramdeep away from his spare chair before he had a chance to sit down.

"Sorry, more urgent stuff came up," Rajiv said, turning back to his monitor. "The best way to learn about the application is to dive right in. I'll give you one of the simpler tasks. All you need to do is write a bean that'll call a Sybase stored procedure to get the next sequence ID of a domain table, then create the next object based on that ID."

Paramdeep stood there wide-eyed and frozen, pencil hovering uselessly over paper. "What?"

"I've already built an EJB for the database connection layer and all the handling code," Rajiv continued, still intent on his screen. "There's also a stored procedure in the common schema for getting the ID. You just have to put them all together."

"Uh—?"

"I'll send an email with the relevant class names," Rajiv cut him off.

"OK. I'll, uh, let you know if I have any trouble?"

Rajiv was too caught up in his urgent business to respond.

Paramdeep staggered back to his desk: scared, confused, and a bit dejected. But hey, maybe Rajiv was right. Once he dove into the code, he'd have a better understanding of what to do. He'd done JDBC before. It wasn't hard.

Unfortunately, there was a stark difference between a computer science student's homework assignments and the inner workings of a creaky, enterprisey corporate behemoth. Try as he might, Paramdeep simply couldn't connect to the database schema Rajiv had told him about. He was connecting to a default schema instead, which lacked the stored procedure he needed. All of Paramdeep's attempts to solicit help from Rajiv—whether emailed, messaged, or sought in person—met with failure. His boss promised to get Rajiv talking, but that promise never materialized into Paramdeep and Rajiv spending quality time together.

A strict deadline was looming. Desperate times called for desperate measures. Paramdeep emailed the database developer, Amita, suggesting his solution: creating a new stored procedure to call and return the value from the actual stored procedure, which would then return the value to his new bean.

Minutes later, his phone rang. Caller ID showed it was Amita.

"You can't possibly want this!" she declared without preamble. "Just use the stored procedure in the schema."

"I can't connect to it," Paramdeep explained.

"What do you mean, you can't?"

"I just can't!"

"Who's the tech lead on this project?"

"Rajiv."

"Ohhh." A weary understanding permeated her tone, taking all the fight out of it. "OK, I get it. Fine, I'll make a synonym in the default schema."

And that was why the ID generating procedure also existed in the default schema. Paramdeep couldn't help but wonder how many of the procedures in the default schema had gotten there this way.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

,

Planet DebianJunichi Uekawa: Been implementing something for pair programming.

Been implementing something for pair programming. Sharing the display is probably the most core part and the rest are minor improvements.

Planet DebianShirish Agarwal: The Constitution of Knowledge

Truth, Untruths and Education in India.

I read this somewhat disturbing and yet pretty raw truth from foreign affairs. It took me quite a few days to not only digest but also say yes and see the same situation playing out in India. I have been seeing the discourse on Twitter and while a part of it is the equivalent of road rage, a huge part is a disconnect to not acknowledge and be civil. We may come to different conclusions from the same data but being civil seems to be difficult for a lot of people. One part is of course ego, where nobody wants to lose, but more than that are the plain comprehension issues. Most of the literature, good literature is unfortunately based in English.

And while we can have differing opinions of what constitutes good literature, for me it’s books like Battle of Belonging,The:On Nationalism, Patriotism, and What it Means – Shashi Tharoor. From what little I have understood, the book makes the case of civic nationalism which is far more inclusive than the narrow confines of patriotism. Now this begs the question when you have such books and many books which do tell you about different aspects of social, political and knowledge, why are so many people prone to disinformation in India similar to U.S. and probably other countries as well. One of the biggest reasons per-se is lack of education and quality education. When the number of graduates is less than five percent how do you expect that population to be able to take decisions in their economic self-interest? So sadly the understanding is ingrained from WhatsApp and there is no need to check from alternate sources. And just like Mr. Trump followers, they believe those versions to be the unvarnished truth. I do understand that no truth is immutable except for life and death. All others are imperfect unless it is validated by some sort of scientific validation behind it. At the same time, these truths may themselves be invalidated if a stronger scientific evidence establishes itself. This is the reason why hypothesis and facts themselves are challenged again and again. Sharing couple of examples below.

Nationalization of Banking, RERA and RCEP

Most of the people want freedom of the banks i.e. private banking don’t really know that private banking existed at a time in free India before they were nationalized and these banks failed at surprisingly regular intervals. Now it isn’t as if this fact is hidden but it is not as popular as maybe some other facts or ideas. Now the Government in the center obviously doesn’t want to share these facts as they want corporates in banking. And if that fact is known by many people it will be a huge setback to their plans. RBI failures have been to many to count. Even recent legislations like RERA and others which were supposed to bring relief to millions of potential homeowners has become a pawn in the hands of builders and this has been known.

One of the interesting points of RCEP which is not so much in public domain is that RCEP would have a mere 4.5% duty on most products which will go down to 1.5% over a 20 year period. Now with India staying out of it, we have done two things. We have said that we will not be competitive even after 20 years of this which is the more damning part. And we will not take part in the growth that other countries will have due to this.

Contempt Proceedings against Comic Artist because she has an opinion on SC

The fall in SC and constitutional values grows day by day. The AG today consented to have contempt proceedings against a comic artist saying she insulted the SC. Gone are the days when an artist made fun of the PM, and she gave him a Padam Shri (one of the highest civilian honors) for his contributions. Then, even dissent or being cynical was looked as being a contribution to the national effort rather than today. This is the reason why India has been continuously falling in the Global Freedom of Expression Index. I have seen censoring many a time here. I, myself has been locked out of Wikipedia many times. Can you imagine, being locked out of Wikipedia which is perhaps one of the more neutral sites on the web. And then there was this wikibio thing, such a sad thing to happen. Guessing this is the future of the Indian interweb. 😦


Stick figure by Sanitary Panels on SC

Krebs on SecurityBomb Threat, DDoS Purveyor Gets Eight Years

A 22-year-old North Carolina man has been sentenced to nearly eight years in prison for conducting bomb threats against thousands of schools in the U.S. and United Kingdom, running a service that launched distributed denial-of-service (DDoS) attacks, and for possessing sexually explicit images of minors.

Timothy Dalton Vaughn from Winston-Salem, N.C. was a key member of the Apophis Squad, a gang of young ne’er-do-wells who made bomb threats to more than 2,400 schools and launched DDoS attacks against countless Web sites — including KrebsOnSecurity on multiple occasions.

The Justice Department says Vaughn and his gang ran a DDoS-for-hire service that they used to shake down victims.

“In early 2018, Vaughn demanded 1.5 bitcoin (then worth approximately $20,000) from a Long Beach company, to prevent denial-of-service attacks on its website,” reads a statement from Nicola Hanna, U.S. attorney for the Central District of California. “When the company refused to pay, he launched a DDoS attack that disabled the company’s website.”

One of many tweets from the attention-starved Apophis Squad, which launched multiple DDoS attacks against KrebsOnSecurity over the past few months.

Dalton, whose online aliases included “WantedbyFeds” and “Hacker_R_US,” pleaded guilty last year to one count of conspiracy to convey threats to injure, convey false information concerning use of explosive device, and intentionally damage a computer; one count of computer hacking; and one count of possession of child pornography.

Federal judge Otis D. Wright II sentenced Vaughn to 95 months for possessing 200 sexually explicit images and videos depicting children, including at least one toddler, the Justice Department said. Vaughn was sentenced to 60 months in federal prison for the remaining charge. The sentences will be served concurrently.

As KrebsOnSecurity noted in 2019, Vaughn’s identity was revealed by following the trail of clues from a gaming website he used that later got hacked.

Vaughn used multiple aliases on Twitter and elsewhere to crow about his attacks, including “HDGZero,” “WantedByFeds,” and “Xavier Farbel.” Among the Apophis Squad’s targets was encrypted mail service Protonmail, which reached out to this author in 2018 for clues about the identities of the Apophis Squad members after noticing we were both being targeted by them and receiving demands for money in exchange for calling off the attacks.

Protonmail later publicly thanked KrebsOnSecurity for helping to bring about the arrest of Apophis Squad leader George Duke-Cohan — a.k.a. “opt1cz,” “7R1D3n7,” and “Pl3xl3t,” — a 19-year-old from the United Kingdom who was convicted in December 2018 and sentenced to three years in prison. But the real-life identity of HDGZero remained a mystery to both of us, as there was little publicly available information at the time connecting that moniker to anyone.

The DDoS-for-hire service run by Apophis Squad listed their members.

That is, until early January 2019, when news broke that hackers had broken into the servers of computer game maker BlankMediaGames and made off with account details of some 7.6 million people who had signed up to play “Town of Salem,” a browser-based role playing game. That stolen information has since been posted and resold in underground forums.

A review of the leaked BlankMediaGames user database shows that in late 2018, someone who selected the username “hdgzero” signed up to play Town of Salem, registering with the email address xavierfarbel@gmail.com. The data also showed this person registered at the site using a Sprint mobile device with an Internet address that traced back to the Carolinas.

Planet DebianBen Hutchings: Debian LTS work, November 2020

I was assigned 16 hours of work by Freexian's Debian LTS initiative and carried over 4.5 hours from earlier months. I worked 11.5 hours this month, so I will carry over 9 hours to December.

I continued working on backporting fixes for some less urgent security issues in Linux 4.9. I had to give up on some filesystem fixes as they caused regressions. The others have now been applied to the 4.9 stable branch at kernel.org.

I updated the linux packaging branch for stretch to Linux 4.9.246, but haven't made a new package upload yet.

Planet DebianUtkarsh Gupta: FOSS Activites in November 2020

Here’s my (fourteenth) monthly update about the activities I’ve done in the F/L/OSS world.

Debian

This was my 23rd month of contributing to Debian. I became a DM in late March last year and a DD last Christmas! \o/

Apart from doing a bunch of activities like attending KubeCon + RubyConf (blog to follow!), et al and simultaneously giving my undergrad exams, I did (relatively) more work than I had really anticipated!

Here are the following things I did in Debian this month:

Uploads and bug fixes:

Other $things:

  • Attended the Debian Ruby team meeting.
  • Mentoring for newcomers.
  • FTP Trainee reviewing.
  • Moderation of -project mailing list.
  • Sponsored phpmyadmin for William and libexif for Hugh.

Debian (E)LTS

Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success.

And Debian Extended LTS (ELTS) is its sister project, extending support to the Jessie release (+2 years after LTS support).

This was my fourteenth month as a Debian LTS and fourth month as a Debian ELTS paid contributor.
I was assigned 22.75 hours for LTS and 45.00 hours for ELTS and worked on the following things:
(for ELTS, I worked for 5.25 hours last month, so I had to work for 39.75 (+1 extra) hours this month)
(also, I did over-work by 5.00 hours for LTS this month, but I’ll re-compensate it later to avoid so much fuss!)

LTS CVE Fixes and Announcements:

ELTS CVE Fixes and Announcements:

  • Issued ELA 306-1, fixing CVE-2020-25692, for openldap.
    For Debian 8 Jessie, these problems have been fixed in version 2.4.40+dfsg-1+deb8u7.
  • Issued ELA 310-1, fixing CVE-2020-0452, for libexif.
    For Debian 8 Jessie, these problems have been fixed in version 0.6.21-2+deb8u5.
  • Issued ELA 311-1, fixing CVE-2020-8037, for tcpdump.
    For Debian 8 Jessie, these problems have been fixed in version 4.9.3-1~deb8u2.
  • Issued ELA 312-1, backporting a new upstream release, 2020d, for tzdata.
    For Debian 8 Jessie, these problems have been fixed in version 2020d-0+deb8u1.
  • Issued ELA 313-1, fixing CVE-2020-15166, for zeromq3.
    For Debian 8 Jessie, these problems have been fixed in version 4.0.5+dfsg-2+deb8u3.
  • Prepared a debdiff for lxml (3.4.0-1+deb8u2) upload, which Emilio completed and rolled out later.

Other (E)LTS Work:

  • Front-desk duty from 26-10 until 01-10 and from 23-11 until 29-11 for both LTS and ELTS.
  • Triaged openldap, python-cryptography, motion, nvidia-cuda-toolkit, samba, lxml, highlight.js, imagemagick, mongodb, poppler, wordpress, raptor2, and blueman.
  • Marked CVE-2020-25659/python-cryptography as no-dsa for Stretch and Jessie.
  • Marked CVE-2020-25713/raptor2 as postponed for Stretch and Jessie.
  • Marked CVE-2020-27778/poppler as postponed for Stretch and Jessie.
  • Marked CVE-2020-5991/nvidia-cuda-toolkit as ignored for Stretch.
  • Marked CVE-2020-26566/motion as not-affected for Stretch.
  • Marked CVE-2020-26237/highlight.js as postponed for Jessie.
  • Auto EOL’ed libpam-tacplus, motion, blueman, openrc, webcit, wordpress, linux, nvidia-cuda-toolkit, spip, and wireshark for Jessie.
  • Attended the sevent LTS meeting. Logs here.
  • General discussion on LTS private and public mailing list.

Until next time.
:wq for today.

Planet DebianJonathan Carter: Free Software Activities for 2020-11

This month just went past way too fast, didn’t get to all the stuff I wanted to, but managed to cover many essentials (not even listed here) that I’ll cover in follow-up posts. In particular, highlights that I’m thankful for are that we’ve selected the final artwork for Bullseye. We’ve also successfully hosted another two MiniDebConfs. One that was gaming themed, and a Brazilian event all in Portuguese! Videos are up on Debian’s PeerTube instance (Gaming Edition | Brazil) and on the DebConf video archive for direct download.

Remember to take care of yourself out there! Physical safety is high on everyone’s mind in these times, but remember to pay attention to your mental health too. It’s ok if you won’t hit all your usual targets and goals in these times, don’t be too hard on yourself and burn out!

2020-11-01: Upload package gtetrinet (0.7.11+git20200916.46e7ade-2~bpo10+1) to Debian buster-backports.

2020-11-01: Upload package gnome-shell-extension-disconnect-wifi (26-1) to Debian unstable.

2020-11-02: Merge MR!2, MR!4 and MR!5 for zram-tools, follow 3-way merge closing MR!1 and MR!3.

2020-11-02: Upload package zram-swap (0.3.3-1) to Debian unstable (Closes: #917643, #928439, #928443).

2020-11-02: Close live-installer bugs #646704 (fix released a few years ago already), #700642 (nothing left to fix), #835391 (unproducible on latest images), #847446 (graphical d-i installer no longer provided). #714710 (problem not present on latest installation media)

2020-11-02: File ROM for calcoo (#973638) – no longer maintained upstream, GTK-2 only.

2020-11-03: Upload package bundlewrap (4.2.2-1) to Debian unstable.

2020-11-03: Upload package feed2toot (0.14-1) to Debian unstable.

2020-11-03: Upload package feed2toot (0.14-2) to Debian unstable.

2020-11-03: Upload package flask-autoindex (0.6.6-2) to Debian unstable.

2020-11-03: Upload package flask-caching (1.9.0-1) to Debian unstable.

2020-11-03: Upload package flask-restful (0.3.8-5) to Debian unstable.

2020-11-08: Upload package s-tui (1.0.2-2) to Debian unstable (Closes: #961534).

2020-11-09: Merge MR!1 for bluefish (remove old icon).

2020-11-10: Upload package bluefish (2.2.12-1) to Debian unstable.

2020-11-10: Upload package calamares (3.2.33-1) to Debian unstable.

2020-11-11: Upload package calamares-settings-debian (11.0.4-1) to Debian unstable.

2020-11-17: Upload package gnome-shell-extension-multiple-workspaces (22-1) to Debian-unstable.

2020-11-24: Sponsor package xmodem (0.4.6+dfsg-2) for Debian unstable (Python Team request).

2020-11-24: Sponsor package python-opentracing (2.4.0-1) for Debian unstable (Python Team request).

2020-11-24: Sponsor package python-css-parser (1.0.6-1) for Debian unstable (Python Team request).

2020-11-24: Review package buildbot (2.8.4-1) (Needs some more work) (Python Team request).

2020-11-24: Review package gbsplay (0.0.94-1) (Needs some more work) (Games Team request).

2020-11-24: Sponsor package goverlay (0.4.2-1) for Debian unstable (Games Team request).

2020-11-24: Sponsor package lutris (0.5.8-1) for Debian unstable (Games Team request).

2020-11-24: Review package mangohud (0.5.1-1) for Debian unstable (Needs some more work) (Games Team request).

2020-11-24: Sponsor package vkbasalt (0.3.2.3-1) for Debian unstable (Games Team request).

2020-11-25: Sponsor package starfighter (2.3.3-1) for Debian unstable (Games Team request).

2020-11-25: Sponsor package pentobi (18.3-1) for Debian unstable (Games Team request).

2020-11-30: Sponsor package lutris (0.5.8-1) for Debian unstable (Games Team request) (New upload).

Worse Than FailureCodeSOD: A Tight Fitter

Part of the pitch of Python is that it's a language which values simplicity and ease of use. Whether it lives up to that pitch is its own discussion, but the good news is that if you really want to create really complex code with loads of inheritance and nasty interrelationships, you can.

Today's anonymous submitter started out by verifying some of the math in some extremely math-y analytical code. Previously, they'd had problems where the documentation and the code were subtly different, so it was important that they understood exactly what the code was doing.

In one file, they found the core class they cared about:

class WeibullFitter(KnownModelParametricUnivariateFitter): # snip: some math

Now, the math looked more or less right. But there was just one problem: who knows how the superclass interacts with that? Subtle changes in behavior could appear there.

class KnownModelParametricUnivariateFitter(ParametricUnivariateFitter): _KNOWN_MODEL = True

The "base" class isn't terribly "base" at all, as you can see: all it does is set a property to True. So once again, up the inheritance tree we go, to see the base class:

class ParametricUnivariateFitter(UnivariateFitter): # ...snip...

We're still not at the actual base class, though this one has some implementation, at least? Is that a good thing? We haven't hit TRWTF just yet, but this strikes me as a good chance to talk about why "inheritance is considered harmful": inheritance is an automatic dependency. To understand the behavior of a child class, you have to also understand the behavior of its ancestor classes. Certainly, well implemented inheritance should keep those boundaries neat, but as we can see, this example isn't "well implemented".

More than that, Python is purposefully loosely typed, so one of the key benefits of inheritance, polymorphism isn't even a benefit here. And yes, one could use Python's type annotations to get some lint-time type checking, which would sort-of bring back polymorphism, it still doesn't justify this whole inheritance tree.

That all aside, one of the main reasons we use inheritance is so that we can cut out common conditional logic and let the type system worry about it. I call concreteInstance.someMethod() and the right thing happens, even if I have a dozen possible types, each with some differing behavior. I bring this up, because in the ParametricUnivariateFitter class, we have this:

def _fit_model(self, Ts, E, entry, weights, show_progress=True): if utils.CensoringType.is_left_censoring(self): # Oh no. negative_log_likelihood = self._negative_log_likelihood_left_censoring elif utils.CensoringType.is_interval_censoring(self): # Oh no no no. negative_log_likelihood = self._negative_log_likelihood_interval_censoring elif utils.CensoringType.is_right_censoring(self): # This is exactly what I think it is isn't it. negative_log_likelihood = self._negative_log_likelihood_right_censoring # ...snip...

Comments provided by the submitter. In addition to having a whole tree of child classes, each of these child classes may have a censoring type applied, and our behavior is different based on the censoring type. This is 100% a code smell, and it becomes more clear when we take a look at CensoringType.

class CensoringType(Enum): # enum.Enum from the standard library LEFT = "left" INTERVAL = "interval" RIGHT = "right" @classmethod def right_censoring(cls, function: Callable) -> Callable: @wraps(function) # functools.wraps from the standard library def f(model, *args, **kwargs): cls.set_censoring_type(model, cls.RIGHT) return function(model, *args, **kwargs) return f @classmethod def left_censoring(cls, function: Callable) -> Callable: @wraps(function) def f(model, *args, **kwargs): cls.set_censoring_type(model, cls.LEFT) return function(model, *args, **kwargs) return f @classmethod def interval_censoring(cls, function: Callable) -> Callable: @wraps(function) def f(model, *args, **kwargs): cls.set_censoring_type(model, cls.INTERVAL) return function(model, *args, **kwargs) return f @classmethod def is_right_censoring(cls, model) -> bool: return cls.get_censoring_type(model) == cls.RIGHT @classmethod def is_left_censoring(cls, model) -> bool: return cls.get_censoring_type(model) == cls.LEFT @classmethod def is_interval_censoring(cls, model) -> bool: return cls.get_censoring_type(model) == cls.INTERVAL @classmethod def get_censoring_type(cls, model) -> str: return model._censoring_type @classmethod def str_censoring_type(cls, model) -> str: return model._censoring_type.value @classmethod def set_censoring_type(cls, model, censoring_type) -> None: model._censoring_type = censoring_type

For those not up on Python, Enum is exactly what you think it is, and the @classmethod decorator is Python's way of making static methods. In the same way instance methods take self as their first parameter (the Python-ic "this"), static methods take cls as their first parameter- a reference to the class itself.

It's also important to note the methods like right_censoring, because those themselves are "decorator" definitions. See how they def a local function, itself decorated with @wraps? The right_censoring(cls, function) signature expects a callable (probably a constructor method), and replaced its implementation with the implementation of f- the inner function. Here, it tampers with the input parameters to the constructor before calling the constructor itself.

If you aren't doing a lot of Python, you might be completely confused at this point, so let me just show you how this gets used:

@CensoringType.right_censoring class SomeCurveFitterType(SomeHorribleTreeOfBaseClasses): def __init__(self, model, *args, **kwargs): # snip instance = SomeCurveFitterType(model, *args, **kwargs)

On that last line, it doesn't directly call the __init__ constructor, it first passes through that inner f function, which , most notably, does this: cls.set_censoring_type(model, cls.RIGHT) before invoking the constructor.

If you're confused by all of this, don't feel bad. Decorators are a Pythonic way to tamper with the implementation of classes and functions, allowing you to mix declarative programming with more traditional techniques. In the end, to understand what the WeibullFitter class does, you have to walk up a half dozen ancestor classes to reach the BaseFitter type, and you have to note what decorators are applied to it, and any of its ancestor classes, and know what their implementation is.

If you're the person who wrote this code, this mix of decorators and inheritance probably feels wonderfully extensible. It's probably quick and easy to slap a new curve fitting function into the framework, at least if you're the galaxy-brained individual who dreamed up this over-engineered framework. The rest of us just have to poke at it with sticks until the behavior we want falls out.

Our anonymous submitter adds:

I almost feel bad about this. The library is generally good, the math itself is good, this is a very useful library that has made my job a lot easier…
This sort of behavior actually has real performance implications. I had to re-implement a different piece because the sheer number of nested function calls (some of which were actually recursive, in a chain that spanned three different classes) completely destroyed the performance of a conceptually simple process, such that it could take upwards of ten minutes.
The rewritten function runs near-instantly and fits in 20 lines with comments.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Planet DebianC.J. Adams-Collier: Rack coming together

I now have four six-port Qotom systems and one four-port Qotom system in my rack. I have successfully verified that the newest of these is capable of pulling about 745Mbit/s of 1500-byte frames through the switch from six other hosts. I bet it will push data at the same rate. but I can’t get iperf to distribute it over the LAG evenly. That’s another problem for another day. I know it can handle six large streams at the same time, which is great.

Planet DebianPaul Wise: FLOSS Activities November 2020

Focus

This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

Issues

Review

Administration

  • Debian wiki: disable attachments due to security issue, approve accounts

Communication

  • Respond to queries from Debian users and contributors on the mailing lists and IRC

Sponsors

The visdom, apt-listchanges work and lintian-brush bug report were sponsored by my employer. All other work was done on a volunteer basis.

Planet DebianDirk Eddelbuettel: inline 0.3.17: Refactored and New Tests

A new release of the inline package arrived on CRAN this evening and has already been shipped to Debian as well. inline facilitates writing code in-line in simple string expressions or short files. The package was used quite extensively by Rcpp in the days before Rcpp Attributes arrived on the scene proving an even better alternative for its use cases. inline is still use by rstan and a number of other packages.

One of those other packages is mkin, and its author Johannes Ranke overhauled the saving and re-loading of C functions part with a really well-done set of contributions. In the process we also added unit testing via the lovely tinytest, and changed to continuous integration setup to r-ci.

See below for a detailed list of changes extracted from the NEWS file.

Changes in inline version 0.3.17 (2020-11-30)

  • Unit testing is now supported via tinytest (Johannes in #15 addressing #14).

  • CI was updated to use focal and run.sh from r-ci on Travis and GitHub Actions (Dirk)

  • The writing and reading of compiled code was refactored and extended (Johannes in #16 fixing #13).

  • Some minor problems related to CRAN checks and tests were corrected (Johannes and Dirk in #17, Johannes in #18, #19, #20).

  • Small stylistic updates have been applied to some R and Rd files (Dirk).

Courtesy of my CRANberries, there is a comparison to the previous release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

,

Planet DebianChris Lamb: Free software activities in November 2020

Here is my monthly update covering what I have been doing in the free software world during November 2020 (previous month):

  • Merged a pull request from Jens Nistler for django-slack (my library which provides a convenient wrapper between projects using the Django and the Slack chat platform) to make it compatible with Celery version 5. [...]

§


Reproducible Builds

One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.

The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.


This month, I:


I also made the following changes to diffoscope:

  • Improvements:

    • Move the slightly-confusing behaviour if a single file is passed to diffoscope on the command-line to a new --load-existing-diff command. [...]
    • Ensure the new diffoscope-minimal package that was introduced by Mattia Rizzolo has a different short description from the primary diffoscope one. [...]
    • Refresh the long and short descriptions of all of the Debian packages. [...]
  • Bug fixes:

    • Don't depend on radare2 in the Debian 'autopkgtests' as it will not be in bullseye due to security considerations. (#975313)
    • Avoid some incorrectly-formatted error messages. This was caused by diffoscope raising an artificial CalledProcessError exception in a generic handler. [...]
  • Codebase improvements:

    • Add a comment regarding Java tests to aid diffoscope contributors who are not using Debian [...] and don't use the old-style super(...) call [...].

§


Debian

I performed the following uploads to the Debian Linux distribution this month:

  • python-django (2.2.17-1 & 3.1.3-1) — New upstream releases.

  • memcached (1.6.9+dfsg-1) — New upstream release.

  • lintian (2.101.0, 2.102.0, 2.103.0 & 2.104.0) — New upstream releases.

  • xtrlock (2.14) — Mark an autopkgtest as 'superficial'. (#974491)

  • bfs (2.1-1) — New upstream release.

  • splint (3.1.2+dfsg-3) — Re-upload a previous QA upload of mine (3.1.2+dfsg-2) to ensure the package's transition to the testing distribution. (#974872)

I also filed a release-critical bug against the minidlna package which could not be successfully purged from the system without reporting a cannot remove '/var/log/minidlna' error. (#975372)


§


Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project, including:

You can find out more about the Debian LTS project via the following video:

Cryptogram Manipulating Systems Using Remote Lasers

Many systems are vulnerable:

Researchers at the time said that they were able to launch inaudible commands by shining lasers — from as far as 360 feet — at the microphones on various popular voice assistants, including Amazon Alexa, Apple Siri, Facebook Portal, and Google Assistant.

[…]

They broadened their research to show how light can be used to manipulate a wider range of digital assistants — including Amazon Echo 3 — but also sensing systems found in medical devices, autonomous vehicles, industrial systems and even space systems.

The researchers also delved into how the ecosystem of devices connected to voice-activated assistants — such as smart-locks, home switches and even cars — also fail under common security vulnerabilities that can make these attacks even more dangerous. The paper shows how using a digital assistant as the gateway can allow attackers to take control of other devices in the home: Once an attacker takes control of a digital assistant, he or she can have the run of any device connected to it that also responds to voice commands. Indeed, these attacks can get even more interesting if these devices are connected to other aspects of the smart home, such as smart door locks, garage doors, computers and even people’s cars, they said.

Another article. The researchers will present their findings at Black Hat Europe — which, of course, will be happening virtually — on December 10.

Cryptogram Check Washing

I can’t believe that check washing is still a thing:

“Check washing” is a practice where thieves break into mailboxes (or otherwise steal mail), find envelopes with checks, then use special solvents to remove the information on that check (except for the signature) and then change the payee and the amount to a bank account under their control so that it could be deposited at out-state-banks and oftentimes by a mobile phone.

The article suggests a solution: stop using paper checks.

Worse Than FailureCodeSOD: To Coalesce a Null

As we all know, managing null values is its own challenge, especially when you're working in a functional style. So as languages like .NET add functional approaches like LINQ extension methods, they also add null coalescing operators and nullable types, making it easy to pass values around without getting surprised by an unexpected null.

Unless you're whoever wrote the code that Abbie found, because they've managed to keep some surprises.

List<SomeObjects> someObjects; using (var session = dataStore.OpenSession()) { someObjects = session.Query<SomeObjects>().OrderBy(s => s.PrimaryKey).ThenBy(s => s.EventDateLocal).ThenBy(s => s.SystemTransactionDateTimeUtc).ToList(); } return someOtherCollectionOfObjects .GroupJoin(someObjects, o => o.EventDates.PrimaryKey, s => s.PrimaryKey, (o, s) => { var sList = s as IList<SomeObjects> ?? s.ToList(); return new { //omitted }; })

The submitter anonymized the class names and variable names a bit, so it's hard to see exactly what the intent is, but we can still spot the "odd" choices. The goal is to take our someOtherCollectionOfObjects as the "outer" side of a join, and someObjects is the "inner". Each "outer" element will get matched with all of the "inner" elements based on primary keys.

So the first "odd" choice is that when we fetch our inner objects, we sort them. someObjects is sorted first by primary key, then by event date, then by the transaction datetime. That first sort absolutely doesn't matter: since we're joining by primary key, the GroupJoin function preserves the order of the outer elements- the someOtherCollectionOfObjects.

But the line that shows a real fundamental misunderstanding of what's going on is this one:

var sList = s as IList<SomeObjects> ?? s.ToList();

s here, is our "inner" side of the join- the someObjects. They are passed to the lambda as an IEnumerable, which for some reason we choose to cast as an IList… but note the ?? operator. If s is null, we will call s.ToList(). In no case could s ever be null, but if it were, calling a ToList function on it wouldn't work.

Not only is this line unnecessary, but if it were, it wouldn't work anyway.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

Planet DebianJohn Goerzen: Thanksgiving in 2020

With COVID-19, Thanksgiving is a little different this year.

The kids enjoyed doing a little sightseeing by air – in our own plane (all socially-distanced of course!). We built a Prusa 3D printer from a kit (the boys and I, though Martha checked in periodically too). It arrived earlier than expected so that kept us busy for several days. And, of course, there was the Christmas decorating and Zoom church (where only our family is in the building, hosting the service for everyone).

What, so Thanksgiving doesn’t normally involve assembling printers, sightseeing from the sky, and printing tiny cups and dishes for miniature houses on a 3D printer?

I’ll be glad when COVID is over. Meantime, we have some memories to treasure too.

,

Chaotic IdealismHow to deal with nightmares

I’ve read a lot of stuff from people who are having nightmares for the first time due to COVID. Things like forgetting your mask, getting sick, seeing family get sick, etc.

All right, y’all. I’m a veteran nightmare survivor–diagnosed PTSD and everything. Here’s some tips.

First of all: Don’t get mad at yourself for having nightmares. You can’t help the emotions they bring up. When you’re asleep, you’re not thinking rationally, and your emotions are going to run away with you. When you wake up from a nightmare, the best way to deal is to roll your eyes, go, “Well, brain, that was silly,” and remind yourself that you are safe.

Don’t assume that you’re doing something wrong if you have nightmares–you’re not. The vast majority of people who can remember their dreams remember nightmares at least every now and then. Even in PTSD, nightmares are a sign that your brain’s trying to recover, however stuck you may be.

Nightmares come from your brain trying to process stress, or trying to prepare you for possible bad situations you may find yourself in. The first one is necessary so you can defrag your brain from all the thinking you’ve been doing all day; the second one is an evolutionary advantage that helps us practice how to run away from saber-toothed tigers without actually having to deal with any large prehistoric felines.

If you’re any good at lucid dreaming, you can use it to help combat nightmares. If you’re having repetitive ones, look at what they have in common–the dead giveaways that you’re dreaming, not awake–and remind yourself regularly of those signs. For me, for example, a dead giveaway that I’m dreaming is that I’m in my mom’s house and can’t find a way out. For you, it might be “If I’ve forgotten my mask, I must be dreaming.” Because we simply don’t forget our masks nowadays, any more than we forget our shoes. If you manage to realize you’re dreaming while you’re still dreaming, you can just walk away from the whole thing; or you can fight it. Like, I might tell my dream mother, “I’m thirty-seven years old and I live on my own now. I don’t live with you. This is only a dream.” Sure, it’s silly to talk to a figment of my own imagination, but dreams are silly. Let them be silly.

If you wake up from a dream and you’re really on edge–like, heart beating fast, sweating, etc.–that’s probably because your fight-or-flight system got mistakenly activated during the night. Learn some basic calming techniques–breathing, relaxation, that kind of thing. Do those, and your body will start to calm down again. It does take a little while for the adrenaline to wear off, so you will probably have to wait a little while before you can go back to sleep; but at least you can do it in your warm, comfortable bed. Going to the toilet or getting a couple of sips of water can help too, but if you don’t have to get up, don’t, since it’ll make you less sleepy.

You know that old saying “the only thing we have to fear is fear itself”? With nightmares, that’s true. Some people get insomnia when they have regular nightmares because they hate the nightmares so much that they can’t get to sleep. It helps if you see nightmares as annoying, see the fear as an accidental jolt of adrenaline. They’re not a real threat–they are just your brain glitching out a little bit trying to process the stress and frustration of the day. Let it glitch, then go back to sleep.

And don’t feel guilty if these don’t work for you immediately, either. Sometimes it takes practice. I haven’t managed to banish my nightmares entirely, and I’ve been free and safe for nineteen years now. But I’ve sure as heck gotten better at going back to sleep afterwards and at getting a decent night’s rest anyway. Sometimes I even realize they’re nightmares, ignore my mom, phase through the walls, and go flying.

,

Planet DebianMark Brown: Book club: Rust after the honeymoon

Earlier this month Daniel, Lars and myself got together to discuss Bryan Cantrill’s article Rust after the honeymoon. This is an overview of what keeps him enjoying working with Rust after having used it for an extended period of time for low level systems work at Oxide, we were particularly interested to read a perspective from someone who was both very experienced in general and had been working with the language for a while. While I have no experience with Rust both Lars and Daniel have been using it for a while and greatly enjoy it.

One of the first areas we discussed was data bearing enums – these have been very important to Bryan. In keeping with a pattern we all noted these take a construct that’s relatively commonly implemented by hand in C (or skipped as too much effort, as Lars found) and provides direct support in the language for it. For both Daniel and Lars this has been key to their enjoyment of Rust, it makes things that are good practice or common idioms in C and C++ into first class language features which makes them more robust and allows them to fade into the background in a way they can’t when done by hand.

Daniel was also surprised by some omissions, some small such as the ? operator but others much more substantial – the standout one being editions. These aim to address the problems seen with version transitions in other languages like Python, allowing individual parts of a Rust program to adopt potentially incompatible language features while remaining interoperability with older editions of the language rather than requiring the entire program to be upgraded en masse. This helps Rust move forwards with less need to maintain strict source level compatibility, allowing much more rapid evolution and helping deal with any issues that are found. Lars expressed the results of this very clearly, saying that while lots of languages offer a 20%/80% solution which does very well in specific problem domains but has issues for some applications Rust is much more able to move towards a much more general applicability by addressing problems and omissions as they are understood.

This distracted us a bit from the actual content of the article and we had an interesting discussion of the issues with handling OS differences in filenames portably. Rather than mapping filenames onto a standard type within the language and then have to map back out into whatever representation the system actually uses Rust has an explicit type for filenames which must be explicitly converted on those occasions when it’s required, meaning that a lot of file handling never needs to worry about anything except the OS native format and doesn’t run into surprises. This is in keeping with Rust’s general approach to interfacing with things that can’t be represented in its abstractions, rather than hide things it keeps track of where things that might break the assumptions it makes are and requires the programmer to acknowledge and handle them explicitly. Both Lars and Daniel said that this made them feel a lot more confident in the code that they were writing and that they had a good handle on where complexity might lie, Lars noted that Rust is the first languages he’s felt comfortable writing multi threaded code in.

We all agreed that the effect here was more about having idioms which tend to be robust and both encourage writing things well and gives readers tools to help know where particular attention is required – no tooling can avoid problems entirely. This was definitely an interesting discussion for me with my limited familiarity with Rust, hopefully Daniel and Lars also got a lot out of it!

Planet DebianRuss Allbery: Review: Nine Goblins

Review: Nine Goblins, by T. Kingfisher

Publisher: Red Wombat Tea Company
Copyright: 2013
ASIN: B00G9GSEXO
Format: Kindle
Pages: 140

The goblins are at war, a messy multi-sided war also involving humans, elves, and orcs. The war was not exactly their idea, although the humans would claim otherwise. Goblins kept moving farther and farther into the wilderness to avoid human settlements, and then they ran out of wilderness, and it wasn't clear what else to do. For the Nineteenth Infantry, the war is a confusing business, full of boredom and screaming and being miserable and following inexplicable orders. And then they run into a wizard.

Wizards in this world are not right in the head, and by not right I mean completely psychotic. That's the only way that you get magical powers. Wizards are therefore incredibly dangerous and scarily unpredictable, so when the Whinin' Nineteenth run into a human wizard who shoots blue out of his mouth, making him stop shooting blue out of his mouth becomes a high priority. Goblins have only one effective way of stopping things: charge at them and hit them with something until they stop. Wizards have things like emergency escape portals. And that's how the entire troop of nine goblins ended up far, far behind enemy lines.

Sings-to-Trees's problems, in contrast, are rather more domestic. At the start of the book, they involve, well:

Sings-to-Trees had hair the color of sunlight and ashes, delicately pointed ears, and eyes the translucent green of new leaves. His shirt was off, he had the sort of tanned muscle acquired from years of healthy outdoor living, and you could have sharpened a sword on his cheekbones.

He was saved from being a young maiden's fantasy — unless she was a very peculiar young maiden — by the fact that he was buried up to the shoulder in the unpleasant end of a heavily pregnant unicorn.

Sings-to-Trees is the sort of elf who lives by himself, has a healthy appreciation for what nursing wild animals involves, and does it anyway because he truly loves animals. Despite that, he was not entirely prepared to deal with a skeleton deer with a broken limb, or at least with the implications of injured skeleton deer who are attracted by magical disturbances showing up in his yard.

As one might expect, Sings-to-Trees and the goblins run into each other while having to sort out some problems that are even more dangerous than the war the goblins were unexpectedly removed from. But the point of this novella is not a deep or complex plot. It pushes together a bunch of delightfully weird and occasionally grumpy characters, throws a challenge at them, and gives them space to act like fundamentally decent people working within their constraints and preconceptions. It is, in other words, an excellent vehicle for Ursula Vernon (writing as T. Kingfisher) to describe exasperated good-heartedness and stubbornly determined decency.

Sings-to-Trees gazed off in the middle distance with a vague, pleasant expression, the way that most people do when present at other people's minor domestic disputes, and after a moment, the stag had stopped rattling, and the doe had turned back and rested her chin trustingly on Sings-to-Trees' shoulder.

This would have been a touching gesture, if her chin hadn't been made of painfully pointy blades of bone. It was like being snuggled by an affectionate plow.

It's not a book you read for the twists and revelations (the resolution is a bit of an anti-climax). It's strength is in the side moments of characterization, in the author's light-hearted style, and in descriptions like the above. Sings-to-Trees is among my favorite characters in all of Vernon's books, surpassed only by gnoles and a few characters in Digger.

The Kingfisher books I've read recently have involved humans and magic and romance and more standard fantasy plots. This book is from seven years ago and reminds me more of Digger. There is less expected plot machinery, more random asides, more narrator presence, inhuman characters, no romance, and a lot more focus on characters deciding moment to moment how to tackle the problem directly in front of them. I wouldn't call it a children's book (all of the characters are adults), but it has a bit of that simplicity and descriptive focus.

If you like Kingfisher in descriptive mode, or enjoy Vernon's descriptions of D&D campaigns on Twitter, you are probably going to like this. If you don't, you may not. I thought it was slight but perfect for my mood at the time.

Rating: 7 out of 10

,

Planet DebianShirish Agarwal: Farmer Protests and RCEP

Farmer Protests

While I was hoping to write about RCEP exclusively, just today farmer protests have happened against three farm laws which had been passed by our Govt. about a month ago without consulting anybody. The bills benefit only big business houses at the cost of farmers. This has been amply shared by an open letter to one of the biggest business house which will benefit the most.

Now while that is a national experience and what it tells, let me share, some experience from the State I come from, Maharashtra. About 4-5 years back Maharashtra delisted fruit and vegetables from the APMC market. But till date, the APMC market is working, why, the reasons are many. However, what it did was it forced the change to sugarcane, a water guzzling crop much more than previously. This has resulted in lowering the water table in Maharashtra and put them more into debt trap and later they had to commit suicide.

Now let us see why the Punjab farmers have been so agitated that they are walking all the way to Delhi. They are right now, somewhere between Haryana-Delhi border. The reason is that because even their experiments with contract farming have not been good. This is why they are struggling to go to Delhi to make their collective voices heard and get the farm bills rolled back. Even the farmers from Gujarat were sued, but because of elections were put back, the intentions though are clear. This has also happened in Uttar Pradesh and for sugarcane and that too by Bajaj Company. At the end of the day, the laws made by the Govt. leaves our farmer at the mercy of big corporations. It is preposterous to believe that the farmer, with their small land holdings will be able to stand up to the Corporation. Add to that, they cannot go to Court. It is the SDM (Sub-Divisonal Magistrate) who will decide on the matters and has the last word. If this is allowed, in a couple of years there will be only few farmers or corporations who would have large hand-holdings, and they would be easily co-opted by the Government in power.

Just in – A gentleman who turned off water cannon being shot at farmers has been charged for murder 😦

Currently, the Government procures rice in vast quantities and the farmers are assured at least some basic income, in the states of Punjab and Haryana –

Procurement of Rice by Various States

Recently there was also an article in Indian Express which shares the farmer’s apprehensions and does share that it’s a complex problem with no easy solutions. The solution can only be dialogue between the two parties. This was also shared by Vivek Kaul, who is far more knowledgable than me on the subject and made a long read on the subject.

The Canada Way

Recently, while sparring on the Internet, came to know of the Canada way. Here, the Government makes the farmer a corporation and the Government helps them. But the Canada way seems to largely work as the Canadian Government owns the majority of the lands in question. And yes, Indians have benefited from it but that is also due to a. the currency differential between Canadian dollar and Indian Rupee and the 99-year land lease. There may be other advantages that the Canadian Government bestows and that is the reason possibly that most Punjabi farmers go to Canada and UK to farm.

While looking at it, I also came across the situation in the United States and it seems the situation there seems to be becoming even more grim.

RCEP

RCEP stands for Regional Comprehensive Economic Partnership. We were supposed to be part of this partnership. Now why didn’t we join, for two reasons, our judicial infrastructure is the worst. It took 8 years to decide on a tax retrospective case (Vodafone) and that too finally outside India. And that decision, by no means an end. The other thing is all those who have joined RCEP have lesser duties, tariffs then India. What this means is that they are much more competitive than India. While there is fear that perhaps that China may take over its assets as it has done with few countries around the world, the opportunity for those countries was too good to pass up even with the dangers. But, then even India has taken loans from the Asian Infrastructure Investment (AIIB) Bank where China is the biggest shareholder. So it doesn’t make sense to be insecure on that front. And again, it is up to India or any other sovereign country to decide to take loans from some country, some multilateral organization or any other way and on what terms.

What China has done and doing is similar to what IMF (being used primarily by the United States) had done in its past. The only difference is that time it was the United States, now it is China. America co-opted Governments, and got assets, China doing the same, no difference in tactics, more or less the same.

There has also been a somewhat interesting paper which discusses how the RCEP may unfold in different circumstances. In short, it tells that the partners will benefit, some more than others. It also does compare the RCEP to CPTPP (The Comprehensive and Progressive Agreement for Trans-Pacific Partnership). While the study is a bit academic in nature as the United States has walked out and the new president-elect Joe Biden hasn’t made any moves and is unlikely to make any moves as there is deep divide and resentment about multilateral trade partnerships domestically within the United States. This news and understanding was quite shocking to me as it shows that unlike the United States of the past, which was supposed to be a beacon of capitalism and seemed to enjoy capitalism, it seems to be an opportunist only. There is also this truth that under Biden, there is only so many things on which he would need and can spend his political capital on.

Statistica Chart of differences between Republicans and Democrats

As can be seen, economy at least for the democrats, this time around is pretty far round the corner. He has a host of battles and would have to choose which to fight and which to ignore.

In the end, we are left to our own devices. At the moment, India does not know when it’s economy will recover –

PTI News, Nov 27, 2020

There has been another worrying bit of news, now all newspapers will need to get some sort of permission, certification from Govt. of India about any news of the world. This is harking back on the 1970’s, 1980’s era

Planet DebianArturo Borrero González: Netfilter virtual workshop 2020 summary

Netfilter logo

Once a year folks interested in Netfilter technologies gather together to discuss past, ongoing and future works. The Netfilter Workshop is an opportunity to share and discuss new ideas, the state of the project, bring people together to work & hack and to put faces to people who otherwise are just email names. This is an event that has been happening since at least 2001, so we are talking about a genuine community thing here.

It was decided there would be an online format, split in 3 short meetings, once per week on Fridays. I was unable to attend the first session on 2020-11-06 due to scheduling conflict, but I made it to the sessions on 2020-11-13 and 2020-11-20. I would say the sessions were joined by about 8 to 10 people, depending on the day. This post is a summary with some notes on what happened in this edition, with no special order.

Pablo did the classical review of all the changes and updates that happened in all the Netfilter project software components since last workshop. I was unable to watch this presentation, so I have nothing special to comment. However, I’ve been following the development of the project very closely, and there are several interesting things going on, some of them commented below.

Florian Westphal brought to the table status on some open/pending work for mptcp option matching, systemd integration and finally interfacing from nft with cgroupv2. I was unable to participate in the talk for the first two items, so I cannot comment a lot more. On the cgroupv2 side, several options were evaluated to how to match them, identification methods, the hierarchical tree that cgroups present, etc. We will have to wait a bit more to see how the final implementation looks like.

Also, Florian presented his concerns on conntrack hash collisions. There are no real-world known issues at the moment, but there is an old paper that suggests we should keep and eye on this and introduce improvements to prevent future DoS attack vectors. Florian mentioned these attacks are not practical at the moment, but who knows in a few years. He wants to explore introducing RB trees for conntrack. It will probably be a rbtree structure of hash tables in order to keep supporting parallel insertions. He was encouraged by others to go ahead and play/explore with this.

Phil Sutter shared his past and future iptables development efforts. He highlighted fixed bugs and his short/midterm TODO list. I know Phil has been busy lately fixing iptables-legacy/iptables-nft incompatibilities. Basically addressing annoying bugs discovered by all ruleset managers out there (kubernetes, docker, openstack neutron, etc). Lots of work has been done to improve the situation; moreover I myself reported, or forwarded from the Debian bug tracker, several bugs. Anyway I was unable to attend this talk, only learnt a few bits in the following sessions, so I don’t have a lot to comment here.

But when I was fully present, I was asked by Phil about the status of netfilter components in Debian, and future plans. I shared my information. The idea for the next Debian stable release is to don’t include iptables in the installer, and include nftables instead. Since Debian Buster, nftables is the default firewalling tool anyway. He shared the plans for the RedHat-related ecosystem, and we were able to confirm that we are basically in sync.

Pablo commented on the latest Netfilter flowtable enhancements happening. Using the flowtable infrastructure, one can create kernel network bypasses to speed up packet throughput. The latest changes are aimed for bridge and VLAN enabled setups. The flowtable component will now know how to bypass in these 2 network architectures as well as the previously supported ingress hook. This is basically aimed for virtual machines and containers scenarios. There was some debate on use cases and supported setups. I commented that a bunch of virtual machines connected to a classic linux bridge and then doing NAT is basically what Openstack Neutron does, specifically in DVR setups. Same can be found in some container-based environments. Early/simple benchmarks done by Pablo suggest there could be huge performance improvements for those use cases. There was some inevitable comparison of this approach to what others, like DPDK/XDP can do. A point was raised about this being a more generic and operating system-integrated solution, which should make it more extensible and easier to use.

flowtable for bridges

Stefano Bravio commented on several open topics for nftables that he is interested on working on. One of them, issues related to concatenations + vmap issues. He also addressed concerns with people’s expectations when migrating from ipset to nftables. There are several corner features in ipset that aren’t currently supported in nftables, and we should document them. Stefano is also wondering about some tools to help in the migration. A translation layer like there is in place for iptables. Eric Gaver commented there are a couple of semantics that will not be suitable for translation, such as global sets, or sets of sets. But ipset is way simpler than iptables, so a translation mechanism should probably be created. In any case, there was agreement that anything that helps people migrate is more than welcome, even if it doesn’t support 100% of the use cases.

Stefano is planning to write documentation in the nftables wiki on how the pipapo algorithm works and the supported use cases. Other plans by Stefano include to work on some optimisations for faster matches. He mentioned using architecture specific instruction to speed up sets operations, like lookups.

Finally, he commented that some folks working with eBPF have showed interest in reusing some parts of the nftables sets infrastructure (pipapo) because they have detected performance issues in their own data structures in some cases. It is not clear how to best achieve it, how to better bridge the two things together. Probably the ideal is to generalize the pipapo data structures and integrate it into the generic bitmap library or something which can be used by anyone. Anyway, he hopes to get some more time to focus on Netfilter stuff begining with the next year, in a couple of months.

Moving a bit away from the pure software development topics, Pablo commented on the netfilter.org infrastructure. Right now the servers are running on gandi.net, on virtual machines that are being basically donated to us. He pointed that the plan is to simplify the infrastructure. For that reason, for example, FTP services has been shut down. Rsync services have been shut down as well, so basically we no longer have a mirrors infrastructure. The bugzilla and wikis we have need some attention, given they are old software pieces, and we need to migrate them to be more modern. Finally, the new logo that was created was presented.

Later on, we spent a good chunk of the meeting discussing options on how to address the inevitable iptables deprecation situation. There are some open questions, and we discussed several approaches. From doing nothing at all, which means keeping the current status-quo, to setting a deadline date for the deprecation like the python community did with python2. I personally like this deadline idea, but it is perceived like a negative push by other. We all agree that the current ‘do nothing’ approach is not sustainable either. Probably the way to go is basically to be more informative. We need to clearly communicate that choosing iptables for anything in 2020 is a bad idea. There are additional initiatives to help on this topic, without being too aggressive. A FAQ will probably be introduced. Eric Garver suggested we should bring nftables front and center. Given the website still mentions iptables everywhere, we will probably refresh the web content, introduce additional informative banners and similar things.

There was an interesting talk on the topic of nft table ownership. The idea is to attach a table, and all the child objects, to a process. Then, we prevent any modifications to the table or the child objects by external entities. Basically, allocating and locking a table for a certain netlink socket. This is a nice way for ruleset managers, like firewalld, to ensure they have full control of what’s happening to their ruleset, reducing the chances for ending with an inconsistent configuration. There is a proof-of-concept patch by Pablo to support this, and Eric mentioned he is pretty much interested in any improvements to support this use case.

The final time block in the final session day was dedicated to talk about the next workshop. We are all very happy we could meet. Meeting virtually is way easier (and cheaper) than in person. Perhaps we can make it online every 3 or 6 months instead of, or in addition to, one big annual physical event. We will see what happens next year.

That’s all on my side!

Worse Than FailureError'd: You Can't Argue with the Polish Government

"In Poland, if you test positive for COVID-19, or come in contact with someone who has, you must stay home for a mandatory 10-day quarantine. During that time, you must use the government's mobile app named 'Home Quarantine' which tracks your location and requires you to send a selfie every couple of hours," wrote Jan K., "The app also reports if you are using a GPS spoofing app. For example, in this screenshot, it has detected a location spoofing app by the name of ...'Calendar'. Naturally, there are stiff penalties for violating rules of your quarantine like this. Also, as expected, there is no appealing the 'findings' of a buggy app like this."

 

Harry writes, "Of course I can trust this update! My Manufacturer wrote it after all!"

 

"Testing in Production is only good if you tell everybody that you're testing in Production," Nate L. wrote.

 

Bob T. writes, "This is why you add '+1 Day' and not '+24 Hours' when you attempt date math."

 

"Newegg-speak: War is peace, freedom is slavery, ignorance is strength and Out-of-Stock is In-Stock," wrote Nicolas L.

 

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Planet DebianReproducible Builds (diffoscope): diffoscope 162 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 162. This version includes the following changes:

[ Chris Lamb ]
* Don't depends on radare2 in the Debian autopkgtests as it will not be in
  bullseye due to security considerations (#950372). (Closes: #975313)
* Avoid "Command `s p a c e d o u t` failed" messages when creating an
  artificial CalledProcessError instance in our generic from_operation
  feeder creator.
* Overhaul long and short descriptions.
* Use the operation's full name so that "command failed" messages include
  its arguments.
* Add a missing comma in a comment.

[ Jelmer Vernooij ]
* Add missing space to the error message when only one argument is passed to
  diffoscope.

[ Holger Levsen ]
* Update Standards-Version to 4.5.1.

[ Mattia Rizzolo ]
* Split the diffoscope package into a diffoscope-minimal package that
  excludes the larger packages from Recommends. (Closes: #975261)
* Drop support for Python 3.6.

You find out more by visiting the project homepage.

,

Planet DebianJonathan Dowland: Touched by the Hand of God

picture of a vinyl record

In honour of Diego Maradona (RIP), this morning's cobweb-shifter is New Order's "Touched by the Hand of God"

Sam VargheseThe heart of football has stopped beating. Diego Armando Maradona is dead

Un poco con la cabeza de Maradona y otro poco con la mano de Dios,” (“a little with the head of Maradona and a little with the hand of God”). – How Diego Maradona described his exploits to a select few reporters sniffing around for the day’s killer quote after the quarter-final against England in the 1986 World Cup.

Diego Armando Maradona is dead. By any measure, the man was the greatest footballer who ever lived, a short, stumpy man who seemed to have the ball on a string, one who looked terribly clumsy but who had the feet of an angel.

He died of a heart attack, no doubt brought on by the way he abused his body, with cocaine and alcohol use high on his list. The genius on the field was a man who could not control his self-destructive urges.

Maradona came from a poor background, being raised in a shantytown on the outskirts of Buenos Aires. His talent was spotted at an early age, when he appeared for trials with the Argentinos Juniors, for whom he played 10 days before his 16th birthday.

He played in the 1982 World Cup in Spain but did nothing of note. He then went to Barcelona before joining Napoli, the team on which he had his greatest influence. The team won a series of titles and runners-up positions while he was there.

Senior players like Mario Kempes were still in the team, having been there when Argentina won the Cup in 1978, and the senior players did not make the junior players, like Maradona, welcome. Maradona played in all five games, and knocked in two goals against Hungary.

In the final game against Brazil, he retaliated against Batista and was sent off. In an earlier game against Italy, he had been marked by Claudio Gentile and given a very rough time.

But in 1986, the story was different. Maradona shone right through the tournament and his peak form was seen in the game against England when he scored twice, the first being the infamous “hand of God” goal. The second goal was the work of a genius as he dribbled his way past four England players and then beat goalkeeper Peter Shilton with a deft flick into the net.

In the final, too, it was a canny pass from him that put Jorge Burruchaga through to score the winning goal against Germany, after the doughty Germans had fought back to level the scores at 2-all after Argentina seemed to be coasting to victory.

Maradona took his team to the 1990 final as well, but he was savagely marked by Guido Buchenwald, a man about twice his height, and could do little as Argentina lost to Germany 1-0, the lone goal coming through a penalty that should never have been awarded.

Four years later, Maradona was sent home from the Cup in the US after testing positive for drugs. His life unravelled after that, though he took up a number of coaching jobs and was part and parcel of the game.

His death will not remove the memory of a clumsy little man who could make the best ballerina look like a statue. He had magic in his feet and no player in the world has ever, or will, come close to him.

May his troubled soul rest in peace.

Worse Than FailureCodeSOD: Classic WTF: Functional Encryption

It's Thanksgiving Day in the US. Yesterday, we looked at a classic "encryption" story, and today, we should all be thankful that we don't have to support this encryption code. Original --Remy

Richard's company builds, hosts, and maintains a variety of small- and mid-sized web-based applications for their clients. Recently, one of their clients asked Richard to help audit a fraudulent transaction, which meant that Richard needed to dig through the code to see how to decrypt bank account numbers stored in the database. The search led him to H88493247329(), the method responsible for encrypting customer data. After spending a minute to add linebreaks and rename the variables, Richard asked his coworker why he obfuscated the code. His coworker scoffed, you should always encrypt your encryption functions -- it's completely insecure otherwise

function H88493247329($B89424235)
{ 
  //ED: Linkebreaks added
  global $a,$e,$m,$H;
  $X42342234 = $H . "." . $m . "-" . $a;
  
  $KJD234 = fopen($X42342234,"r");
  $MMNVUD884 = fread($KJD234,filesize($X42342234));
  fclose($KJD234);

  $MQUFI3 = mcrypt_module_open('','',''');
  $MMNVUD884 = substr($MMNVUD884,0,mcrypt_enc_get_key_size($MQUFI3));
  
  $JF8_size = mcrypt_enc_get_iv_size($MQUFI3);
  $JF8 = mcrypt_create_iv($JF8_size, MCRYPT_RAND);

  if (mcrypt_generic_init($MQUFI3,$MMNVUD884,$JF8)!=-1)
  {
    $KIDO83R4234FFS = mcrypt_generic($MQUFI3,$B89424235);
    mcrypt_generic_deinit($MQUFI3);
    mcrypt_module_close($MQUFI3);
  }
  return $KIDO83R4234FFS;
}
[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

,

Planet DebianShirish Agarwal: Women state in India and proposal for corporates in Indian banking

Gradle and Kotlin in Debian

Few months back, I was looking at where Gradle and Kotlin were in Debian. They still seem to be a work in progress. I found the Android-tools salsa repo which tells me the state of things. While there has been movement on both, a bit more on Kotlin, it still seems it would take a while. For kotlin, the wiki page is most helpful as well as the android-tool salsa kotlin board page . Ironically, some of the more important info. is shared in a blog post which ideally should also have been reflected in kotlin board page . I did see some of the bugs so know it’s pretty much dependency hell. I can only congratulate and encourage Samyak Jain and Raman Sarda. I also played a bit with the google-android-emulator-installer which is basically a hook which downloads the binary from google. I do not know what the plans are, but perhaps in the future they also might be build locally, who knows. Just sharing/stating here so it’s part of my notes whenever I wanna see what’s happening 🙂

Women in India

I am sure some of you might remember my blog post from last year. It is almost close to a year 2020 now and the question to be asked is, has much changed ? After a lot or hue and cry the Government of India shared the NCRB data of crimes against women and caste crimes. The report shared that crimes against women had risen by 7.3% in a year, similarly crimes against lower castes also went by similar percentage . With the 2020 pandemic, I am sure the number has gone up more. And there is a possibility that just like last year, next year the Government would cite the pandemic and say no data. This year they have done it for migrant deaths during lockdown , for job losses due to the pandemic and so on and so forth. So, it will be no surprise if the Govt. says about NCRB data next year as well. Although media has been showing some in spite of the regular threats to the journalists as shared in the last blog post. There is also data that shows that women participation in labor force has fallen sharply especially in the last few years and the Government seems to have no neither idea nor do they seem to care for the same. There aren’t any concrete plans to bring back the balance even a little bit.

Few Court judgements

But all hope is not lost. There have been a couple of good judgements, one from the CIC (Chief Information Commissioner) wherein in specific cases a wife can know salary details of her husband, especially if there is some kind of maintenance due from the husband. There was so much of hue and cry against this order that it was taken down from the livelaw RTI corner. Luckily, I had downloaded it, hence could upload and share it.

Another one was a case/suit about a legally matured women who had decided to marry without parental consent. In this case, the Delhi High Court had taken women’s side and stated she can marry whom she wants. Interestingly, about a week back Uttar Pradesh (most notorious about crime against women) had made laws called ‘Love Jihad‘ and 2 -3 states have followed them. The idea being to create an atmosphere of hate against Muslims and women have no autonomy about what they want. This is when in a separate suit/case against Sudharshan TV (a far-right leaning channel promoting hate against Muslims) , the Government of India itself put an affidavit stating that Tablighis (a sect of Muslims who came from Malaysia to India for religious discourse and assembly) were not responsible for dissemination of the virus and some media has correctly portrayed the same. Of course, those who are on the side of the Govt. on this topic think a ‘traitor’ has written. They also thought that the Govt. had taken a wrong approach but couldn’t tell of a better approach to the matter.

There are too many matters in the Supreme Court of women asking for justice to tell all here but two instances share how the SC has been buckling under the stress of late, one is a webinar which was chaired by Justice Subramaniam where he shared how the executive is using judicial appointments to do what it wants. The gulf between the executive and the SC has been since Indira Gandhi days, especially the judicial orders which declared that the Emergency is valid by large, it has fallen much more recently and the executive has been muscling in which have resulted in more regressive decisions than progressive.

This observation is also in tune with another study which came to the same result although using data. The raw data from the study could give so much more than what has been shared. For e.g. as an idea for the study, of the ones cited, how many have been in civil law, personal law, criminal or constitutional law. This would give a better understanding of things. Also what is shocking is none of our court orders have been cited in the west in the recent past, when there used to be a time when the west used to take guidance from Indian jurisprudence sometimes and cite the orders to reach similar conclusion or if not conclusion at least be used as a precedent. I guess those days are over.

Government giving Corporate ownership to Private Sector Banks

There was an Internal Working Group report to review extant ownership guidelines and Corporate Structure for Indian Private Sector Banks. – This is the actual title of the report.

Now there were and are concerns about the move which were put forth by Dr. Raghuram Rajan and Viral Acharya. While Dr. Rajan had been the 23rd Governor of RBI from 4th September 2013 to 4th September 2016.

His most commendable work which largely is unknown to most people was the report A hundred small steps which you buy from sage publications. Viral Acharya was the deputy governor from 23rd January 2017 – 23rd July 2019. Mr. Acharya just recently published his book Quest for Restoring Financial Stability in India which can be bought from the same publication house as well.

They also wrote a three page article stating that does India need corporates in banking? More interestingly, he shares two points from history both world war 1 and world war 2. In both cases, the allies had to cut down the businesses who had owned banks. In Germany, it was the same and in Japan, the zaibatsu’s dissolution, both of which were needed to make the world safe again. Now, if we don’t learn lessons from history it is our fault, not history’s.

What was also shared that this idea was taken up in 2013 but was put into cold-storage. He also commented on the pressure on RBI as all co-operative banks have come under its ambit in the last few months. RBI has had a patchy record, especially in the last couple of years, with big scams like ILFS, Yes Bank, PMC Bank, Laxmi Vilas Bank among others. The LVB Bank being the most recent one.

If new banking licenses have to be given they can be given to good NBFC’s who have been in the market for a long time and have shown maturity while dealing with public money. What is the hurry for giving it to Corporate/business houses ? There are many other good points in the report with which both Mr. Rajan and Mr. Acharya are in agreement and do hope the other points/suggestions/proposals are implemented. There was and is an interesting report by Reserve Bank of India called financial sector legislative reforms commission report volume 1 . If and when it gets deleted from RBI, I have put up a copy at my WordPress account, so we shall always have one.

Interestingly, while looking through the people who were part of the committee was a somewhat familiar name Murmu . This is perhaps the first time you see people from a sort of political background being in what should be a cut and dry review which have people normally from careers in finance or Accounts. It also turns out that only one person was in favor of banks going to corporates, all the rest were against.

It seems that the specific person hadn’t heard the terms ‘self-lending’, ‘connected-lending’ and conflict of interest. One of the more interesting comments in the report is if a corporate has a bank, then why would he go to Switzerland, he would just wash the money in his own bank. And if banks were to become to big to fail like it happened in the United States, it would be again private gains, public losses. There was also a Washington Post article which shares some of the reasons that Indian banks fail. I think we need to remind ourselves once again, how things can become –


https://www.youtube.com/watch?v=2gK3s5j7PgA

Positive News at end

At the end I do not want to end on a sour notes, hence sharing a YouTube channel of Films Division India where you can see of the very classic works and interviews of some of the greats in Indian art cinema.

https://www.youtube.com/user/FilmsDivision/videos

Also sharing a bit of funny story I came to know about youtube-dl, apparently it was taken off from github but thanks to efforts from EFF, Hackernews and others, it is now back in action.

Cryptogram Undermining Democracy

Last Thursday, Rudy Giuliani, a Trump campaign lawyer, alleged a widespread voting conspiracy involving Venezuela, Cuba, and China. Another lawyer, Sidney Powell, argued that Mr. Trump won in a landslide, the entire election in swing states should be overturned and the legislatures should make sure that the electors are selected for the president.

The Republican National Committee swung in to support her false claim that Mr. Trump won in a landslide, while Michigan election officials have tried to stop the certification of the vote.

It is wildly unlikely that their efforts can block Joe Biden from becoming president. But they may still do lasting damage to American democracy for a shocking reason: the moves have come from trusted insiders.

American democracy’s vulnerability to disinformation has been very much in the news since the Russian disinformation campaign in 2016. The fear is that outsiders, whether they be foreign or domestic actors, will undermine our system by swaying popular opinion and election results.

This is half right. American democracy is an information system, in which the information isn’t bits and bytes but citizens’ beliefs. When peoples’ faith in the democratic system is undermined, democracy stops working. But as information security specialists know, outsider attacks are hard. Russian trolls, who don’t really understand how American politics works, have actually had a difficult time subverting it.

When you really need to worry is when insiders go bad. And that is precisely what is happening in the wake of the 2020 presidential election. In traditional information systems, the insiders are the people who have both detailed knowledge and high level access, allowing them to bypass security measures and more effectively subvert systems. In democracy, the insiders aren’t just the officials who manage voting but also the politicians who shape what people believe about politics. For four years, Donald Trump has been trying to dismantle our shared beliefs about democracy. And now, his fellow Republicans are helping him.

Democracy works when we all expect that votes will be fairly counted, and defeated candidates leave office. As the democratic theorist Adam Przeworski puts it, democracy is “a system in which parties lose elections.” These beliefs can break down when political insiders make bogus claims about general fraud, trying to cling to power when the election has gone against them.

It’s obvious how these kinds of claims damage Republican voters’ commitment to democracy. They will think that elections are rigged by the other side and will not accept the judgment of voters when it goes against their preferred candidate. Their belief that the Biden administration is illegitimate will justify all sorts of measures to prevent it from functioning.

It’s less obvious that these strategies affect Democratic voters’ faith in democracy, too. Democrats are paying attention to Republicans’ efforts to stop the votes of Democratic voters ­- and especially Black Democratic voters -­ from being counted. They, too, are likely to have less trust in elections going forward, and with good reason. They will expect that Republicans will try to rig the system against them. Mr. Trump is having a hard time winning unfairly, because he has lost in several states. But what if Mr. Biden’s margin of victory depended only on one state? What if something like that happens in the next election?

The real fear is that this will lead to a spiral of distrust and destruction. Republicans ­ who are increasingly committed to the notion that the Democrats are committing pervasive fraud -­ will do everything that they can to win power and to cling to power when they can get it. Democrats ­- seeing what Republicans are doing ­ will try to entrench themselves in turn. They suspect that if the Republicans really win power, they will not ever give it back. The claims of Republicans like Senator Mike Lee of Utah that America is not really a democracy might become a self-fulfilling prophecy.

More likely, this spiral will not directly lead to the death of American democracy. The U.S. federal system of government is complex and hard for any one actor or coalition to dominate completely. But it may turn American democracy into an unworkable confrontation between two hostile camps, each unwilling to make any concession to its adversary.

We know how to make voting itself more open and more secure; the literature is filled with vital and important suggestions. The more difficult problem is this. How do you shift the collective belief among Republicans that elections are rigged?

Political science suggests that partisans are more likely to be persuaded by fellow partisans, like Brad Raffensperger, the Republican secretary of state in Georgia, who said that election fraud wasn’t a big problem. But this would only be effective if other well-known Republicans supported him.

Public outrage, alternatively, can sometimes force officials to back down, as when people crowded in to denounce the Michigan Republican election officials who were trying to deny certification of their votes.

The fundamental problem, however, is Republican insiders who have convinced themselves that to keep and hold power, they need to trash the shared beliefs that hold American democracy together.

They may have long-term worries about the consequences, but they’re unlikely to do anything about those worries in the near-term unless voters, wealthy donors or others whom they depend on make them pay short-term costs.

This essay was written with Henry Farrell, and previously appeared in the New York Times.

Cory DoctorowThe Attack Surface Lectures: Tech in SF


The Attack Surface Lectures were a series of eight panel discussions on the themes in my novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “Tech in SF” hosted by Interabang Books in Dallas, TX, with guest-hosts Annalee Newitz and Ken Liu. It was recorded on October 20, 2020.

Here is the original Youtube link for this program. Please consider subscribing to Interabang’s Youtube channel for access to all their outstanding author events!

MP3

Worse Than FailureCodeSOD: Classic WTF: Top-grade, SHA1 Encryption

Is it that time of year already? Here in the US, we're prepping for the Thanksgiving holiday, so let's take a trip way back into the archives, and learn about the life of a moderately-paid-consultant. Original --Remy

Paul B always thought of himself as a moderately-paid consultant. With no real overhead, a policy against ties when meeting with prospective clients, and a general pickiness about the projects he'll take on, his rates tend to be pretty low. One company that looked right up his alley was a mid-sized manufacturing company that wanted a custom webshop. They went to the highly-paid consultants in town, but weren't too happy with the six-figure price tag. Paul's quote was in the five-figure range, which he felt was pretty moderate given that it was a several month project. Of course, the company wasn't too happy with his quote either, so they searched high and low for a three- or four-figure price. They eventually found one overseas.

Despite losing the bid, Paul never bothered unsubscribing from the company's mailing list - there was always something exciting about learning the latest in gimbal clamps and engine nozzle extensions. About a year and a half later, he received an exciting newsletter announcing that the webshop was finally live. Out of curiosity, he created an account to check things out. A few days later, he received an apology for lost orders - they didn't know who had ordered what, so they sent it to everyone who had signed up. And then came the "data breach" email — everyone's personal data (which, for Paul, was just his throw-away email) was now in the hands of some hackers. You get what you pay for never rang so true.

The day following the breach, the company contacted him to see if he was still available for consulting. Apparently, their overseas programmers couldn't figure out how anyone was getting in the system, since they had used "Top-grade, SHA1 Encryption." Curiosity won the day, so Paul asked for a copy of the source code. He couldn't find anything related to encryption, so he performed a search for "sha1". This was the only line that came up:

$result = mysql_query(
  "SELECT * FROM users " .
  " WHERE SHA1(username) = SHA1('" . $_REQUEST["username"] . "') " . 
  "   AND SHA1(password) = SHA1('" . $_REQUEST["password"] . "')");

Paul told the company he couldn't help them out, and suggested they go to the highly-paid consultants. A few days later, the company's newsletter reported that the webshop was closing down for some "upgrades" - a year later, it's still under construction.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Planet DebianJunichi Uekawa: Grabbing screenshot.

Grabbing screenshot. I wanted to know the size of screenshot generated by canvas.toDataURL so I wrote a web app that just measures the size at 60fps because I could. From output I can see webp: 19087 png: 115818 jpg: 115818, so I figured webp is really good at this, or maybe chrome is really good at using webp. and png and jpg look like they are the same size... hmm.. why? UPDATE: image/jpg generated png and image/jpeg generated jpeg.

,

Cryptogram Cyber Public Health

In a lecture, Adam Shostack makes the case for a discipline of cyber public health. It would relate to cybersecurity in a similar way that public health relates to medicine.

Cory DoctorowTalking interop on EFF’s podcast

How to Fix the Internet is EFF’s amazing new podcast: nuanced discussions of tech law and ethics with incredible experts, interviewed and contextualized by EFF executive director Cindy Cohn and strategy director Danny O’Brien.

https://pluralistic.net/2020/11/13/said-no-one-ever/#fix-it

I devoured the first three episodes. I mean, I started working with EFF nearly 19 years ago (!) but I was learning SO MUCH from them.

Today, the episode I recorded dropped. I’ve never been in such august company.

https://www.eff.org/deeplinks/2020/11/podcast-episode-control-over-users-competitors-and-critics

Our discussion is about the role interoperability plays in helping technology users exercise self-determination, giving them alternatives to bad moderation, abusive lock-in, and poor security choices.

And about how companies love interop when they’re trying to eat another company’s lunch, but then they love to take it away once they win, because without interop, companies can control their customers, critics and competitors.

You can get How to Fix the Internet in your favorite podcatcher. Here’s the RSS:

https://efforg.libsyn.com/rss

and here’s the MP3 for my episode:

https://ia601407.us.archive.org/10/items/eff-podcast-episode-4-interroperability/EFF_Podcast_Episode4_Interroperability.mp3

Cory DoctorowThe Attack Surface Lectures: Sci-Fi Genre

The Attack Surface Lectures were a series of eight panel discussions on the themes in my novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “Sci-Fi Genre” hosted by Fountain Books in Richmond, VA, with guest-hosts Sarah Gailey and Chuck Wendig. It was recorded on October 16, 2020.


Here is the original Youtube link for this program. Please consider subscribing to Fountain Books’s Youtube channel for access to all their outstanding author events!

MP3

Worse Than FailureCodeSOD: Production Comments

A fair bit of "bad code" requires at least a passing understanding of the language in question, or the domain involved. But bad comments transcend programming languages. Vilx sends us this one, which comes from code which is definitely running in production.

// WARNING!!! Special case for [external API] testing. // DO NOT LET THIS PIECE OF CODE FIND IT'S WAY TO PRODUCTION

Adding more commentary almost feels like gilding the lilly. Almost.

One of the main objections to putting loads of comments in your code is that over time the code changes, and if the comments don't change with them, confusion results. Which, not only is this not a comment you want to see in your production code, it's not an accurate comment. Vilx explains:

In all truth the particular code (or even the codebase) isn't all that bad; and the lines that followed are completely harmless in production…

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

,

Planet DebianShirish Agarwal: White Hat Senior and Education

I had been thinking of doing a blog post on RCEP which China signed with 14 countries a week and a day back but this new story has broken and is being viraled a bit on the interwebs, especially twitter and is pretty much in our domain so thought would be better to do a blog post about it. Also, there is quite a lot packed so quite a bit of unpacking to do.

Whitehat, Greyhat and Blackhat

For those of you who may not, there are actually three terms especially in computer science that one comes across. Those are white hats, grey hats and black hats. Now clinically white hats are like the fiery angels or the good guys who basically take permissions to try and find out weakness in an application, program, website, organization and so on and so forth. A somewhat dated reference to hacker could be Sandra Bullock (The Net 1995) , Sneakers (1992), Live Free or Die Hard (2007) . Of the three one could argue that Sandra was actually into viruses which are part of computer security but still she showed some bad-ass skills, but then that is what actors are paid to do 🙂 Sneakers was much more interesting for me because in that you got the best key which can unlock any lock, something like quantum computing is supposed to do. One could equate both the first movies in either as a White hat or a Grey hat . A Grey hat is more flexible in his/her moral values, and they are plenty of such people. For e.g. Julius Assange could be described as a Grey hat, but as you can see and understand those are moral issues.

A black hat on the other hand is one who does things for profit even if it harms the others. The easiest fictitious examples are all Die Hard series, all of them except the 4th one, all had bad guys or black hats. The 4th also had but is the odd one out as it had Matthew Farell (Justin Long) as a Grey hat hacker. In real life Kevin Mitnick, Kevin Poulsen, Robert Tappan Morris, George Hotz, Gary McKinnon are some examples of hackers, most of whom were black hats, most of them reformed into white hats and security specialists. There are many other groups and names but that perhaps is best for another day altogether.

Now why am I sharing this. Because in all of the above, the people who are using and working with the systems have better than average understanding of systems and they arguably would be better than most people at securing their networks, systems etc. but as we shall see in this case there has been lots of issues in the company.

WhiteHat Jr. and 300 Million Dollars

Before I start this, I would like to share that for me this suit in many ways seems to be similar to the suit filed against Krishnaraj Rao . Although the difference is that Krishnaraj Rao’s case/suit is that it was in real estate while this one is in ‘education’ although many things are similar to those cases but also differ in some obvious ways. For e.g. in the suit against Krishnaraj Rao, the plaintiff’s first approached the High Court and then the Supreme Court. Of course Krishnaraj Rao won in the High Court and then in the SC plaintiff’s agreed to Krishnaraj Rao’s demands as they knew they could not win in SC. In that case, a compromise was reached by the plaintiff just before judgement was to be delivered.

In this case, the plaintiff have directly approached the Delhi High Court. The charges against Mr. Poonia (the defendant in this case) are very much similar to those which were made in Krishnaraj Rao’s suit hence won’t be going into those details. They have claimed defamation and filed a 20 crore suit. The idea is basically to silence any whistle-blowers.

Fictional Character Wolf Gupta

The first issue in this case or perhaps one of the most famous or infamous character is an unknown. While he has been reportedly hired by Google India, BJYU, Chandigarh. This has been reported by Yahoo News. I did a cursory search on LinkedIn to see if there indeed is a wolf gupta but wasn’t able to find any person with such a name. I am not even talking the amount of money/salary the fictitious gentleman is supposed to have got and the various variations on the salary figures at different times and the different ads.

If I wanted to, I could have asked few of the kind souls whom I know are working in Google to see if they can find such a person using their own credentials but it probably would have been a waste of time. When you show a LinkedIn profile in your social media, it should come up in the results, in this case it doesn’t. I also tried to find out if somehow BJYU was a partner to Google and came up empty there as well. There is another story done by Kan India but as I’m not a subscriber, I don’t know what they have written but the beginning of the story itself does not bode well.

While I can understand marketing, there is a line between marketing something and being misleading. At least to me, all of the references shared seems misleading at least to me.

Taking down dissent

One of the big no-nos at least from what I perceive, you cannot and should not take down dissent or critique. Indians, like most people elsewhere around the world, critique and criticize day and night. Social media like twitter, mastodon and many others would not exist in the place if criticisms are not there. In fact, one could argue that Twitter and most social media is used to drive engagements to a person, brand etc. It is even an official policy in Twitter. Now you can’t drive engagements without also being open to critique and this is true of all the web, including of WordPress and me 🙂 . What has been happening is that whitehatjr with help of bjyu have been taking out content of people citing copyright violation which seems laughable.

When citizens critique anything, we are obviously going to take the name of the product otherwise people would have to start using new names similar to how Tom Riddle was known as ‘Dark Lord’ , ‘Voldemort’ and ‘He who shall not be named’ . There have been quite a few takedowns, I just provide one for reference, the rest of the takedowns would probably come in the ongoing suit/case.

Whitehat Jr. ad showing investors fighting


Now a brief synopsis of what the ad. is about. The ad is about a kid named ‘Chintu’ who makes an app. The app. Is so good that investors come to his house and right in the lawn and start fighting each other. The parents are enjoying looking at the fight and to add to the whole thing there is also a nosy neighbor who has his own observations. Simply speaking, it is a juvenile ad but it works as most parents in India, as elsewhere are insecure.

Jihan critiquing the whitehatjr ad

Before starting, let me assure that I asked Jihan’s parents if it’s ok to share his ad on my blog and they agreed. What he has done is broken down the ad and showed how juvenile the ad is and using logic and humor as a template for the same. He does make sure to state that he does not know how the product is as he hasn’t used it. His critique was about the ad and not the product as he hasn’t used that.

The Website

If you look at the website, sadly, most of the site only talks about itself rather than giving examples that people can look in detail. For e.g. they say they have few apps. on Google play-store but no link to confirm the same. The same is true of quite a few other things. In another ad a Paralympic star says don’t get into sports and get into coding. Which athlete in their right mind would say that? And it isn’t that we (India) are brimming with athletes at the international level. In the last outing which was had in 2016, India sent a stunning 117 athletes but that was an exception as we had the women’s hockey squad which was of 16 women, and even then they were overshadowed in numbers by the bureaucratic and support staff. There was criticism about the staff bit but that is probably a story for another date.

Most of the site doesn’t really give much value and the point seems to be driving sales to their courses. This is pressurizing small kids as well as teenagers and better who are in the second and third year science-engineering whose parents don’t get that it is advertising and it is fake and think that their kids are incompetent. So this pressurizes both small kids as well as those who are learning, doing in whatever college or educational institution . The teenagers more often than not are unable to tell/share with them that this is advertising and fake. Also most of us have been on a a good diet of ads. Fair and lovely still sells even though we know it doesn’t work.

This does remind me of a similar fake academy which used very much similar symptoms and now nobody remembers them today. There used to be an academy called Wings Academy or some similar name. They used to advertise that you come to us and we will make you into a pilot or an airhostess and it was only much later that it was found out that most kids were doing laundry work in hotels and other such work. Many had taken loans, went bankrupt and even committed suicide because they were unable to pay off the loans due to the dreams given by the company and the harsh realities that awaited them. They were sued in court but dunno what happened but soon they were off the radar so we never came to know what happened to those million of kids whose life dreams were shattered.

Security

Now comes the security part. They have alleged that Mr. Poonia broke into their systems. While this may be true, what I find funny is that with the name Whitehat, how can they justify it? If you are saying you are white hat you are supposed to be much better than this. And while I have not tried to penetrate their systems, I did find it laughable that the site is using an expired https:// certificate. I could have tried further to figure out the systems but I chose not to. How they could not have an automated script to get the certificate fixed is beyond me, this is known as certificate outage and is very well understood in the industry. There are tools like Let’s Encrypt and Certbot (both EFF) and many others. But that is their concern, not mine.

Comparison

A similar offering would be unacademy but as can be seen they neither try to push you in any way and nor do they make any ridiculous claims. In fact how genuine unacademy is can be gauged from the fact that many of its learning resources are available to people to see on YT and if they have tools they can also download it. Now, does this mean that every educational website should have their content for free, of course not. But when a channel has 80% – 90% of it YT content as ads and testimonials then they surely should give a reason to pause both for parents and students alike. But if parents had done that much research, then things would not be where they are now.

Allegations

Just to complete, there are allegations by Mr. Poonia with some screenshots which show the company has been doing a lot of bad things. For e.g. they were harassing an employee at night 2 a.m. who was frustrated and working in the company at the time. Many of the company staff routinely made sexist and offensive, sexual abusive remarks privately between themselves for prospective women who came to interview via webcam (due to the pandemic). There also seems to be a bit of porn on the web/mobile server of the company as well. There also have been allegations that while the company says refund is done next day, many parents who have demanded those refunds have not got it. Now while Mr. Poonia has shared some quotations of the staff while hiding the identities of both the victims and the perpetrators, the language being used in itself tells a lot. I am in two minds whether to share those photos or not hence atm choosing not to. Poonia has also contended that all teachers do not know programming, and they are given scripts to share. There have been some people who did share that experience with him –

Suruchi Sethi

From the company’s side they are alleging he has hacked the company servers and would probably be using the Fruit of the poisonous tree argument which we have seen have been used in many arguments.

Conclusion

Now that lies in the eyes of the Court whether the single bench chooses the literal meaning or use the spirit of the law or the genuine concerns of the people concerned. While in today’s hearing while the company asked for a complete sweeping injunction they were unable to get it. Whatever may happen, we may hope to see some fireworks in the second hearing which is slated to be on 6.01.2021 where all of this plays out. Till later.

Planet DebianVincent Fourmond: QSoas tips and tricks: using meta-data, first level

By essence, QSoas works with \(y = f(x)\) datasets. However, in practice, when working with experimental data (or data generated from simulations), one has often more than one experimental parameter (\(x\)). For instance, one could record series of spectra (\(A = f(\lambda)\)) for different pH values, so that the absorbance is in fact a function of both the pH and \(\lambda\). QSoas has different ways to deal with such situations, and we'll describe one today, using meta-data.

Setting meta-data

Meta-data are simply series of name/values attached to a dataset. It can be numbers, dates or just text. Some of these are automatically detected from certain type of data files (but that is the topic for another day). The simplest way to set meta-data is to use the set-meta command:
QSoas> set-meta pH 7.5
This command sets the meta-data pH to the value 7.5. Keep in mind that QSoas does not know anything about the meaning of the meta-data[1]. It can keep track of the meta-data you give, and manipulate them, but it will not interpret them for you. You can set several meta-data by repeating calls to set-meta, and you can display the meta-data attached to a dataset using the command show. Here is an example:
QSoas> generate-buffer 0 10
QSoas> set-meta pH 7.5
QSoas> set-meta sample "My sample"
QSoas> show 0
Dataset generated.dat: 2 cols, 1000 rows, 1 segments, #0
Flags: 
Meta-data:	pH =	 7.5	sample =	 My sample
Note here the use of quotes around My sample since there is a space inside the value.

Using meta-data

There are many ways to use meta-data in QSoas. In this post, we will discuss just one: using meta-data in the output file. The output file can collect data from several commands, like peak data, statistics and so on. For instance, each time the command 1 is run, a line with the information about the largest peak of the current dataset is written to the output file. It is possible to automatically add meta-data to those lines by using the /meta= option of the output command. Just listing the names of the meta-data will add them to each line of the output file.

As a full example, we'll see how one can take advantage of meta-data to determine the position of the peak of the function \(x^2 \exp (-a\,x)\) depends on \(a\). For that, we first create a script that generates the function for a certain value of \(a\), sets the meta-data a to the corresponding value, and find the peak. Let's call this file do-one.cmds (all the script files can be found in the GitHub repository):

generate-buffer 0 20 x**2*exp(-x*${1})
set-meta a ${1}
1 
This script takes a single argument, the value of \(a\), generates the appropriate dataset, sets the meta-data a and writes the data about the largest (and only in this case) peak to the output file. Let's now run this script with 1 as an argument:
QSoas> @ do-one.cmds 1
This command generates a file out.dat containing the following data:
## buffer       what    x       y       index   width   left_width      right_width     area
generated.dat   max     2.002002002     0.541340590883  100     3.4034034034    1.24124124124   2.162162162161.99999908761
This gives various information about the peak found: the name of the dataset it was found in, whether it's a maximum or minimum, the x and y positions of the peak, the index in the file, the widths of the peak and its area. We are interested here mainly in the x position.

Then, we just run this script for several values of \(a\) using run-for-each, and in particular the option /range-type=lin that makes it interpret values like 0.5..5:80 as 80 values evenly spread between 0.5 and 5. The script is called run-all.cmds:

output peaks.dat /overwrite=true /meta=a
run-for-each do-one.cmds /range-type=lin 0.5..5:80
V all /style=red-to-blue
The first line sets up the output to the output file peaks.dat. The option /meta=a makes sure the meta a is added to each line of the output file, and /overwrite=true make sure the file is overwritten just before the first data is written to it, in order to avoid accumulating the results of different runs of the script. The last line just displays all the curves with a color gradient. It looks like this:
Running this script (with @ run-all.cmds) creates a new file peaks.dat, whose first line looks like this:
## buffer       what    x       y       index   width   left_width      right_width     area    a
The column x (the 3rd) contains the position of the peaks, and the column a (the 10th) contains the meta a (this column wasn't present in the output we described above, because we had not used yet the output /meta=a command). Therefore, to load the peak position as a function of a, one has just to run:
QSoas> load peaks.dat /columns=10,3
This looks like this:
Et voilà !

To train further, you can:

  • improve the resolution in x;
  • improve the resolution in y;
  • plot the magnitude of the peak;
  • extend the range;
  • derive the analytical formula for the position of the peak and verify it !

[1] this is not exactly true. For instance, some commands like unwrap interpret the sr meta-data as a voltammetric scan rate if it is present. But this is the exception.

About QSoas

QSoas is a powerful open source data analysis program that focuses on flexibility and powerful fitting capacities. It is released under the GNU General Public License. It is described in Fourmond, Anal. Chem., 2016, 88 (10), pp 5050–5052. Current version is 2.2. You can download its source code there (or clone from the GitHub repository) and compile it yourself, or buy precompiled versions for MacOS and Windows there.

Cory DoctorowSomeone Comes to Town, Someone Leaves Town (part 24)

Here’s part twenty-four of my new reading of my novel Someone Comes to Town, Someone Leaves Town (you can follow all the installments, as well as the reading I did in 2008/9, here).

This is easily the weirdest novel I ever wrote. Gene Wolfe (RIP) gave me an amazing quote for it: “Someone Comes to Town, Someone Leaves Town is a glorious book, but there are hundreds of those. It is more. It is a glorious book unlike any book you’ve ever read.”

Here’s how my publisher described it when it came out:


Alan is a middle-aged entrepeneur who moves to a bohemian neighborhood of Toronto. Living next door is a young woman who reveals to him that she has wings—which grow back after each attempt to cut them off.

Alan understands. He himself has a secret or two. His father is a mountain, his mother is a washing machine, and among his brothers are sets of Russian nesting dolls.

Now two of the three dolls are on his doorstep, starving, because their innermost member has vanished. It appears that Davey, another brother who Alan and his siblings killed years ago, may have returned, bent on revenge.

Under the circumstances it seems only reasonable for Alan to join a scheme to blanket Toronto with free wireless Internet, spearheaded by a brilliant technopunk who builds miracles from scavenged parts. But Alan’s past won’t leave him alone—and Davey isn’t the only one gunning for him and his friends.

Whipsawing between the preposterous, the amazing, and the deeply felt, Cory Doctorow’s Someone Comes to Town, Someone Leaves Town is unlike any novel you have ever read.

MP3

Cryptogram On That Dusseldorf Hospital Ransomware Attack and the Resultant Death

Wired has a detailed story about the ransomware attack on a Dusseldorf hospital, the one that resulted in an ambulance being redirected to a more distant hospital and the patient dying. The police wanted to prosecute the ransomware attackers for negligent homicide, but the details were more complicated:

After a detailed investigation involving consultations with medical professionals, an autopsy, and a minute-by-minute breakdown of events, Hartmann believes that the severity of the victim’s medical diagnosis at the time she was picked up was such that she would have died regardless of which hospital she had been admitted to. “The delay was of no relevance to the final outcome,” Hartmann says. “The medical condition was the sole cause of the death, and this is entirely independent from the cyberattack.” He likens it to hitting a dead body while driving: while you might be breaking the speed limit, you’re not responsible for the death.

So while this might not be an example of death by cyberattack, the article correctly notes that it’s only a matter of time:

But it’s only a matter of time, Hartmann believes, before ransomware does directly cause a death. “Where the patient is suffering from a slightly less severe condition, the attack could certainly be a decisive factor,” he says. “This is because the inability to receive treatment can have severe implications for those who require emergency services.” Success at bringing a charge might set an important precedent for future cases, thereby deepening the toolkit of prosecutors beyond the typical cybercrime statutes.

“The main hurdle will be one of proof,” Urban says. “Legal causation will be there as soon as the prosecution can prove that the person died earlier, even if it’s only a few hours, because of the hack, but this is never easy to prove.” With the Düsseldorf attack, it was not possible to establish that the victim could have survived much longer, but in general it’s “absolutely possible” that hackers could be found guilty of manslaughter, Urban argues.

And where causation is established, Hartmann points out that exposure for criminal prosecution stretches beyond the hackers. Instead, anyone who can be shown to have contributed to the hack may also be prosecuted, he says. In the Düsseldorf case, for example, his team was preparing to consider the culpability of the hospital’s IT staff. Could they have better defended the hospital by monitoring the network more closely, for instance?

Cory DoctorowThe Attack Surface Lectures: Opsec and Personal Cyber-Security

The Attack Surface Lectures were a series of eight panel discussions on the themes in my novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “OpSec & Personal Cyber-Security: How Can You Be Safe?” hosted by Third Place Books in Seattle, WA, with guest-hosts Runa Sandvik and Window Snyder​. It was recorded on October 22, 2020.

Here is the original Youtube link for this program. Please consider subscribing to Third Place’s Youtube channel for access to all their outstanding author events!

MP3

Worse Than FailureCodeSOD: Pixel Perfect Design

Octavia (previously) didn't just inherit a C# application with dodgy approaches to string handling. It's also an application with questionable understandings of CSS.

CSS is far from perfect, and offers a lot of pitfalls and traps. There's a reason the "impossibility" of vertically centering text is a punchline. It's so flexibly declarative that, in many cases, there are many ways to achieve the same styling result, and it's difficult to pick out the correct one. But one would hope that developers could at least avoid the obviously terrible ones.

<div class="positioning"><span><div class="positioning"><span><div class="positioning"> <!-- repeats several hundred more times -->My Page Title<!-- then the matching closing tags--> </div></span></div></span></div>

This was not generated HTML, at least as it exists in the codebase. Someone checked this in. Whether they scripted it or copy-pasted remains a mystery. What's less mysterious is the purpose.

Octavia doesn't have the "positioning" class to share, but it sets a number of properties. Only one is relevant here: it adjusts the width of the div by one pixel. The gigantic pile of divs and spans above exists to center the text on the page. Horizontally.

There are a lot of wrong ways to do that in CSS, but this is arguably one of the most wrong.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

,

Planet DebianFrançois Marier: Removing a corrupted data pack in a Restic backup

I recently ran into a corrupted data pack in a Restic backup on my GnuBee. It led to consistent failures during the prune operation:

incomplete pack file (will be removed): b45afb51749c0778de6a54942d62d361acf87b513c02c27fd2d32b730e174f2e
incomplete pack file (will be removed): c71452fa91413b49ea67e228c1afdc8d9343164d3c989ab48f3dd868641db113
incomplete pack file (will be removed): 10bf128be565a5dc4a46fc2fc5c18b12ed2e77899e7043b28ce6604e575d1463
incomplete pack file (will be removed): df282c9e64b225c2664dc6d89d1859af94f35936e87e5941cee99b8fbefd7620
incomplete pack file (will be removed): 1de20e74aac7ac239489e6767ec29822ffe52e1f2d7f61c3ec86e64e31984919
hash does not match id: want 8fac6efe99f2a103b0c9c57293a245f25aeac4146d0e07c2ab540d91f23d3bb5, got 2818331716e8a5dd64a610d1a4f85c970fd8ae92f891d64625beaaa6072e1b84
github.com/restic/restic/internal/repository.Repack
        github.com/restic/restic/internal/repository/repack.go:37
main.pruneRepository
        github.com/restic/restic/cmd/restic/cmd_prune.go:242
main.runPrune
        github.com/restic/restic/cmd/restic/cmd_prune.go:62
main.glob..func19
        github.com/restic/restic/cmd/restic/cmd_prune.go:27
github.com/spf13/cobra.(*Command).execute
        github.com/spf13/cobra/command.go:838
github.com/spf13/cobra.(*Command).ExecuteC
        github.com/spf13/cobra/command.go:943
github.com/spf13/cobra.(*Command).Execute
        github.com/spf13/cobra/command.go:883
main.main
        github.com/restic/restic/cmd/restic/main.go:86
runtime.main
        runtime/proc.go:204
runtime.goexit
        runtime/asm_amd64.s:1374

Thanks to the excellent support forum, I was able to resolve this issue by dropping a single snapshot.

First, I identified the snapshot which contained the offending pack:

$ restic -r sftp:hostname.local: find --pack 8fac6efe99f2a103b0c9c57293a245f25aeac4146d0e07c2ab540d91f23d3bb5
repository b0b0516c opened successfully, password is correct
Found blob 2beffa460d4e8ca4ee6bf56df279d1a858824f5cf6edc41a394499510aa5af9e
 ... in file /home/francois/.local/share/akregator/Archive/http___udd.debian.org_dmd_feed_
     (tree 602b373abedca01f0b007fea17aa5ad2c8f4d11f1786dd06574068bf41e32020)
 ... in snapshot 5535dc9d (2020-06-30 08:34:41)

Then, I could simply drop that snapshot:

$ restic -r sftp:hostname.local: forget 5535dc9d
repository b0b0516c opened successfully, password is correct
[0:00] 100.00%  1 / 1 files deleted

and run the prune command to remove the snapshot, as well as the incomplete packs that were also mentioned in the above output but could never be removed due to the other error:

$ restic -r sftp:hostname.local: prune
repository b0b0516c opened successfully, password is correct
counting files in repo
building new index for repo
[20:11] 100.00%  77439 / 77439 packs
incomplete pack file (will be removed): b45afb51749c0778de6a54942d62d361acf87b513c02c27fd2d32b730e174f2e
incomplete pack file (will be removed): c71452fa91413b49ea67e228c1afdc8d9343164d3c989ab48f3dd868641db113
incomplete pack file (will be removed): 10bf128be565a5dc4a46fc2fc5c18b12ed2e77899e7043b28ce6604e575d1463
incomplete pack file (will be removed): df282c9e64b225c2664dc6d89d1859af94f35936e87e5941cee99b8fbefd7620
incomplete pack file (will be removed): 1de20e74aac7ac239489e6767ec29822ffe52e1f2d7f61c3ec86e64e31984919
repository contains 77434 packs (2384522 blobs) with 367.648 GiB
processed 2384522 blobs: 1165510 duplicate blobs, 47.331 GiB duplicate
load all snapshots
find data that is still in use for 15 snapshots
[1:11] 100.00%  15 / 15 snapshots
found 1006062 of 2384522 data blobs still in use, removing 1378460 blobs
will remove 5 invalid files
will delete 13728 packs and rewrite 15140 packs, this frees 142.285 GiB
[4:58:20] 100.00%  15140 / 15140 packs rewritten
counting files in repo
[18:58] 100.00%  50164 / 50164 packs
finding old index files
saved new indexes as [340cb68f 91ff77ef ee21a086 3e5fa853 084b5d4b 3b8d5b7a d5c385b4 5eff0be3 2cebb212 5e0d9244 29a36849 8251dcee 85db6fa2 29ed23f6 fb306aba 6ee289eb 0a74829d]
remove 190 old index files
[0:00] 100.00%  190 / 190 files deleted
remove 28868 old packs
[1:23] 100.00%  28868 / 28868 files deleted
done

Cryptogram More on the Security of the 2020 US Election

Last week I signed on to two joint letters about the security of the 2020 election. The first was as one of 59 election security experts, basically saying that while the election seems to have been both secure and accurate (voter suppression notwithstanding), we still need to work to secure our election systems:

We are aware of alarming assertions being made that the 2020 election was “rigged” by exploiting technical vulnerabilities. However, in every case of which we are aware, these claims either have been unsubstantiated or are technically incoherent. To our collective knowledge, no credible evidence has been put forth that supports a conclusion that the 2020 election outcome in any state has been altered through technical compromise.

That said, it is imperative that the US continue working to bolster the security of elections against sophisticated adversaries. At a minimum, all states should employ election security practices and mechanisms recommended by experts to increase assurance in election outcomes, such as post-election risk-limiting audits.

The New York Times wrote about the letter.

The second was a more general call for election security measures in the US:

Obviously elections themselves are partisan. But the machinery of them should not be. And the transparent assessment of potential problems or the assessment of allegations of security failure — even when they could affect the outcome of an election — must be free of partisan pressures. Bottom line: election security officials and computer security experts must be able to do their jobs without fear of retribution for finding and publicly stating the truth about the security and integrity of the election.

These pile on to the November 12 statement from Cybersecurity and Infrastructure Security Agency (CISA) and the other agencies of the Election Infrastructure Government Coordinating Council (GCC) Executive Committee. While I’m not sure how they have enough comparative data to claim that “the November 3rd election was the most secure in American history,” they are certainly credible in saying that “there is no evidence that any voting system deleted or lost votes, changed votes, or was in any way compromised.”

We have a long way to go to secure our election systems from hacking. Details of what to do are known. Getting rid of touch-screen voting machines is important, but baseless claims of fraud don’t help.

Planet DebianMolly de Blanc: Why should you work on free software (or other technology issues)?

Twice this week I was asked how it can be okay to work on free software when there are issues like climate change and racial injustice. I have a few answers for that.

You can work on injustice while working on free software.

A world in which all technology is just cannot exist under capitalism. It cannot exist under racism or sexism or ableism. It cannot exist in a world that does not exist if we are ravaged by the effects of climate change. At the same time, free software is part of the story of each of these. The modern technology state fuels capitalism, and capitalism fuels it. It cannot exist without transparency at all levels of the creation process. Proprietary software and algorithms reinforce racial and gender injustice. Technology is very guilty of its contributions to the climate crisis. By working on making technology more just, by making it more free, we are working to address these issues. Software makes the world work, and oppressive software creates an oppressive world.

You can work on free software while working on injustice.

Let’s say you do want to devote your time to working on climate justice full time. Activism doesn’t have to only happen in the streets or in legislative buildings. Being a body in a protest is activism, and so is running servers for your community’s federated social network, providing wiki support, developing custom software, and otherwise bringing your free software skills into new environments. As long as your work is being accomplished under an ethos of free software, with free software, and under free software licenses, you’re working on free software issues while saving the world in other ways too!

Not everyone needs to work on everything all the time.

When your house in on fire, you need to put out the fire. However, maybe you can’t help put out the first. Maybe You don’t have the skills or knowledge or physical ability. Maybe your house is on fire, but there’s also an earthquake and a meteor and a airborn toxic event all coming at once. When that happens, we have to split up our efforts and that’s okay.

Planet DebianArturo Borrero González: How to use nftables from python

Netfilter logo

One of the most interesting (and possibly unknown) features of the nftables framework is the native python interface, which allows python programs to access all nft features programmatically, from the source code.

There is a high-level library, libnftables, which is responsible for translating the human-readable syntax from the nft binary into low-level expressions that the nf_tables kernel subsystem can run. The nft command line utility basically wraps this library, where all actual nftables logic lives. You can only imagine how powerful this library is. Originally written in C, ctypes is used to allow native wrapping of the shared lib object using pure python.

To use nftables in your python script or program, first you have to install the libnftables library and the python bindings. In Debian systems, installing the python3-nftables package should be enough to have everything ready to go.

To interact with libnftables you have 2 options, either use the standard nft syntax or the JSON format. The standard format allows you to send commands exactly like you would do using the nft binary. That format is intended for humans and it doesn’t make a lot of sense in a programmatic interaction. Whereas JSON is pretty convenient, specially in a python environment, where there are direct data structure equivalents.

The following code snippet gives you an example of how easy this is to use:

#!/usr/bin/env python3

import nftables
import json

nft = nftables.Nftables()
nft.set_json_output(True)
rc, output, error = nft.cmd("list ruleset")
print(json.loads(output))

This is functionally equivalent to running nft -j list ruleset. Basically, all you have to do in your python code is:

  • import the nftables & json modules
  • init the libnftables instance
  • configure library behavior
  • run commands and parse the output (ideally using JSON)

The key here is to use the JSON format. It allows adding ruleset modification in batches, i.e. to create tables, chains, rules, sets, stateful counters, etc in a single atomic transaction, which is the proper way to update firewalling and NAT policies in the kernel and to avoid inconsistent intermediate states.

The JSON schema is pretty well documented in the libnftables-json(5) manpage. The following example is copy/pasted from there, and illustrates the basic idea behind the JSON format. The structure accepts an arbitrary amount of commands which are interpreted in order of appearance. For instance, the following standard syntax input:

flush ruleset
add table inet mytable
add chain inet mytable mychain
add rule inet mytable mychain tcp dport 22 accept

Translates into JSON as such:

{ "nftables": [
    { "flush": { "ruleset": null }},
    { "add": { "table": {
        "family": "inet",
        "name": "mytable"
    }}},
    { "add": { "chain": {
        "family": "inet",
        "table": "mytable",
        "chain": "mychain"
    }}}
    { "add": { "rule": {
        "family": "inet",
        "table": "mytable",
        "chain": "mychain",
        "expr": [
            { "match": {
                "left": { "payload": {
                    "protocol": "tcp",
                    "field": "dport"
                }},
                "right": 22
            }},
            { "accept": null }
        ]
    }}}
]}

I encourage you to take a look at the manpage if you want to know about how powerful this interface is. I’ve created a git repository to host several source code examples using different features of the library: https://github.com/aborrero/python-nftables-tutorial. I plan to introduce more code examples as I learn and create them.

There are several relevant projects out there using this nftables python integration already. One of the most important pieces of software is firewalld. They started using the JSON format back in 2019.

In the past, people interacting with iptables programmatically would either call the iptables binary directly or, in the case of some C programs, hack libiptc/libxtables libraries into their source code. The native python approach to use libnftables is a huge step forward, which should come handy for developers, network engineers, integrators and other folks using the nftables framework in a pythonic environment.

If you are interested to know how this python binding works, I invite you to take a look at the upstream source code, nftables.py, which contains all the magic behind the scenes.

Kevin RuddNikkei Asia: Why I’m Taking On Murdoch

Published in Nikkei Asia, 22 November 2020

Recent landmark moves by Japan, South Korea and China to embrace a pathway to net-zero emissions beg the question as to why other major economies such as Australia — and until now the United States — have been dragging their feet in the fight against climate change. Part of the answer, quite simply, is Rupert Murdoch.

The Murdoch family’s media empire may be virtually non-existent in Asia, but its political influence across the English-speaking West should not be underestimated. Their crown jewels in the American media include the most-watched cable outlet in Fox News, the biggest-selling metropolitan newspaper in the New York Post, and premier business title The Wall Street Journal — properties that Murdoch surrendered his Australian passport to acquire. These outlets collectively feed American voters a steady diet of climate change denial while running a protection racket for politicians who toe this company line.

In my own country, Australia, Murdoch has ruthlessly deployed his monopoly control of daily newspaper circulation to sow doubt about climate science and destroy politicians who take the problem seriously. Even as our nation was besieged by megafires that burned out almost as much land as the United Kingdom, these newspapers heaped doubt on climate change and spouted trumped up claims of mass-arson.

The lies became so transparent that Rupert Murdoch’s own son, James Murdoch, quit the News Corporation board in disgust at these newspapers’ relentless denial and disinformation on climate change.

At its heart, Murdoch’s position is all about accumulating and exercising political power. In 2006, when conservative governments in both the United States and Australia were inching toward accepting the need for climate action, Murdoch smoothed the way by declaring in Tokyo that “the planet deserved the benefit of the doubt.” But years later, when the world was edging closer toward securing a new landmark deal, he did everything in his power to bring down the progressive governments such as my own that had been working toward it. It was the same modus operandi that Murdoch would use against my conservative successor, Malcolm Turnbull, when he also acted in the national interest to advance climate action.

All along, Murdoch’s strategy has been to use his media mastheads and networks to narrowly frame the debate as one simply between outright climate change denialism at one end, and skepticism as to the humankind’s contribution at the other. This means trotting out his most ardent mouthpieces to proclaim among a long and distinguished rap sheet of disinformation such as that “there is no carbon emissions” and “if there were, we could not see because most carbon is black.” This is, as James Murdoch put it, a campaign to “sow doubt, to obscure fact.”

This tactic of trotting out pseudoscience at every opportunity should not be unfamiliar. It is exactly the same tactic that big tobacco used during the 1950s and 1960s to try and dispel the connection between smoking and cancer. Just as it did then, and is now, this kind of reckless commentary costs lives.

This is part of why I have called Murdoch a cancer on Australia’s democracy, and recently launched a national petition calling for an independent inquiry — known as a royal commission — into media diversity in our country.

In just a few short weeks, the petition garnered 501,876 signatures — the most of any online petition the history of the Australian parliament. It was introduced into the national parliament by Andrew Leigh from the Labor Party. There will now be a parliamentary inquiry to examine the matters raised in the petition. This will only be the beginning.

Our petition not only struck a chord with the Australian people, it struck a chord with the international community that has watched in horror at Australia and the United States’ climate inaction over recent years, and drawn a direct link to Murdoch’s not so invisible hand. This includes some of the most conservative voices in the United States, such as former Republican congressman Bob Inglis.

Unfortunately, the harsh reality is that Murdoch himself is likely to now be just as much of an impediment to President-elect Biden implementing his ambitious climate agenda as a Republican-controlled Senate. Just as he will continue to stand in the way of climate action in Australia, no matter who is in power. This is precisely why it is time the rest of us also declare that enough is enough.

 

The post Nikkei Asia: Why I’m Taking On Murdoch appeared first on Kevin Rudd.

Planet DebianMarkus Koschany: My Free Software Activities in October 2020

Welcome to gambaru.de. Here is my monthly report (+ the first week in November) that covers what I have been doing for Debian. If you’re interested in Java, Games and LTS topics, this might be interesting for you.

Debian Games

  • I released a new version of debian-games, a collection of metapackages for games. As expected the Python 2 removal takes its toll on games in Debian that depend on pygame or other Python 2 libraries. Currently we have lost more games in 2020 than could be newly introduced to the archive. All in all it could be better but also a lot worse.
  • New upstream releases were packaged for freeorion and xaos.
  • Most of the time was spent on upgrading the bullet physics library to version 3.06, testing all reverse-dependencies and requesting a transition for it. (#972395) Similar to bullet I also updated box2d, the 2D counterpart. The only reverse-dependency, caveexpress fails to build from source with box2d 2.4.1, so unless I can fix it, it doesn’t make much sense to upload the package to unstable.
  • Some package polishing: I could fix two bugs in stormbaancoureur, patch by Helmut Grohne, and ardentryst that required a dependency on python3-future to start.
  • I sponsored mgba and pekka-kana-2 for Ryan Tandy and Carlos Donizete Froes
  • and started to work on porting childsplay to Python 3.
  • Finally I did a NMU for bygfoot to work around a GCC 10 FTBFS.

Debian Java

pdfsam
  • I uploaded pdfsam and its related sejda libraries to unstable and applied an upstream patch to fix an error with Debian’s jackson-jr version. Everything should be usable and up-to-date now.
  • I updated mina2 and investigated a related build failure in apache-directory-server, packaged a new upstream release of commons-io and undertow and fixed a security vulnerability in junit4 by upgrading to version 4.13.1.
  • The upgrade of jflex to version 1.8.2 took a while. The package is available in experimental now but regression tests with ratt showed, that several reverse-dependencies FTBFS with 1.8.2. Since all of these projects work fine with 1.7.0, I intend to postpone the upload to unstable. No need to break something.

Misc

  • This month also saw new upstream versions of wabt and binaryen.
  • I intend to update ublock-origin in Buster but I haven’t heard back from the release team yet. (#973695)

Debian LTS

This was my 56. month as a paid contributor and I have been paid to work 20,75 hours on Debian LTS, a project started by Raphaël Hertzog. In that time I did the following:

  • DLA-2440-1. Issued a security update for poppler fixing 9 CVE.
  • DLA-2445-1. Issued a security update for libmaxminddb fixing 1 CVE.
  • DLA-2447-1. Issued a security update for pacemaker fixing 1 CVE. The update had to be reverted because of an unexpected permission problem. I am in contact with one of the users who reported the regression and my intention is to update pacemaker to the latest supported release in the 1.x branch. If further tests show no regressions anymore, a new update will follow shortly.
  • Investigated CVE-2020-24614 in fossil and marked the issue as no-dsa because the impact for Debian users was low.
  • Investigated the open security vulnerabilities in ansible (11) and prepared some preliminary patches. The work is ongoing.
  • Fixed the remaining zsh vulnerabilities in Stretch in line with Debian 8 „Jessie“, so that all versions in Debian are equally protected.

ELTS

Extended Long Term Support (ELTS) is a project led by Freexian to further extend the lifetime of Debian releases. It is not an official Debian project but all Debian users benefit from it without cost. The current ELTS release is Debian 8 „Jessie“. This was my 29. month and I have been paid to work 15 hours on ELTS.

  • ELA-302-1. Issued a security update for poppler fixing 2 CVE. Investigated Debian bug #942391, identified the root cause and reverted the patch for CVE-2018-13988.
  • ELA-303-1. Issued a security update for junit4 fixing 1 CVE.
  • ELA-316-1. Issued a security update for zsh fixing 7 CVE.

Thanks for reading and see you next time.

,

Planet DebianGiovanni Mascellani: Having fun with signal handlers

As every C and C++ programmer knows far too well, if you dereference a pointer that points outside of the space mapped on your process' memory, you get a segmentation fault and your programs crashes. As far as the language itself is concerned, you don't have a second chance and you cannot know in advance whether that dereferencing operation is going to set a bomb off or not. In technical terms, you are invoking undefined behaviour, and you should never do that: you are responsible for knowing in advance if your pointers are valid, and if they are not you keep the pieces.

However, turns out that most actual operating system give you a second chance, although with a lot of fine print attached. So I tried to implement a function that tries to dereference a pointer: if it can, it gives you the value; if it can't, it tells you it couldn't. Again, I stress this should never happen in a real program, except possibly for debugging (or for having fun).

The prototype is

word_t peek(word_t *addr, int *success);

The function is basically equivalent to return *addr, except that if addr is not mapped it doesn't crash, and if success is not NULL it is set to 0 or 1 to indicate that addr was not mapped or mapped. If addr was not mapped the return value is meaningless.

I won't explain it in detail to leave you some fun. Basically the idea is to install a handler for SIGSEGV: if the address is invalid, the handler is called, which basically fixes everything by advancing a little bit the instruction pointer, in order to skip the faulting instruction. The dereferencing instruction is written as hardcoded Assembly bytes, so that I know exactly how many bytes I need to skip.

Of course this is very architecture-dependent: I wrote the i386 and amd64 variants (no x32). And I don't guarantee there are no bugs or subtelties!

Another solution would have been to just parse /proc/self/maps before dereferencing and check whether the pointer is in a mapped area, but it would have suffered of a TOCTTOU problem: another thread might have changed the mappings between the time when /proc/self/maps was parsed and when the pointer was dereferenced (also, parsing that file can take a relatively long amount of time). Another less architecture-dependent but still not pure-C approach would have been to establish a setjmp before attempting the dereference and longjmp-ing back from the signal handler (but again you would need to use different setjmp contexts in different threads to exclude race conditions).

Have fun! (and again, don't try this in real programs)

EDIT I realized I should specify the language for source code highlighting to work decently. Now it's better!

EDIT 2 I also realized that my version of peek has problems when there are other threads, because signal actions are per-process, not per-thread (as I initially thought). See the comments for a better version (though not perfect).

#define _GNU_SOURCE
#include <stdint.h>
#include <signal.h>
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include <ucontext.h>

#ifdef __i386__
typedef uint32_t word_t;
#define IP_REG REG_EIP
#define IP_REG_SKIP 3
#define READ_CODE __asm__ __volatile__(".byte 0x8b, 0x03\n"  /* mov (%ebx), %eax */ \
                                       ".byte 0x41\n"        /* inc %ecx */ \
                                       : "=a"(ret), "=c"(tmp) : "b"(addr), "c"(tmp));
#endif

#ifdef __x86_64__
typedef uint64_t word_t;
#define IP_REG REG_RIP
#define IP_REG_SKIP 6
#define READ_CODE __asm__ __volatile__(".byte 0x48, 0x8b, 0x03\n"  /* mov (%rbx), %rax */ \
                                       ".byte 0x48, 0xff, 0xc1\n"  /* inc %rcx */ \
                                       : "=a"(ret), "=c"(tmp) : "b"(addr), "c"(tmp));
#endif

static void segv_action(int sig, siginfo_t *info, void *ucontext) {
    (void) sig;
    (void) info;
    ucontext_t *uctx = (ucontext_t*) ucontext;
    uctx->uc_mcontext.gregs[IP_REG] += IP_REG_SKIP;
}

struct sigaction peek_sigaction = {
    .sa_sigaction = segv_action,
    .sa_flags = SA_SIGINFO,
    .sa_mask = 0,
};

word_t peek(word_t *addr, int *success) {
    word_t ret;
    int tmp, res;
    struct sigaction prev_act;

    res = sigaction(SIGSEGV, &peek_sigaction, &prev_act);
    assert(res == 0);

    tmp = 0;
    READ_CODE

    res = sigaction(SIGSEGV, &prev_act, NULL);
    assert(res == 0);

    if (success) {
        *success = tmp;
    }

    return ret;
}

int main() {
    int success;
    word_t number = 22;
    word_t value;

    number = 22;
    value = peek(&number, &success);
    printf("%d %d\n", success, value);

    value = peek(NULL, &success);
    printf("%d %d\n", success, value);

    value = peek((word_t*)0x1234, &success);
    printf("%d %d\n", success, value);

    return 0;
}

Krebs on SecurityGoDaddy Employees Used in Attacks on Multiple Cryptocurrency Services

Fraudsters redirected email and web traffic destined for several cryptocurrency trading platforms over the past week. The attacks were facilitated by scams targeting employees at GoDaddy, the world’s largest domain name registrar, KrebsOnSecurity has learned.

The incident is the latest incursion at GoDaddy that relied on tricking employees into transferring ownership and/or control over targeted domains to fraudsters. In March, a voice phishing scam targeting GoDaddy support employees allowed attackers to assume control over at least a half-dozen domain names, including transaction brokering site escrow.com.

And in May of this year, GoDaddy disclosed that 28,000 of its customers’ web hosting accounts were compromised following a security incident in Oct. 2019 that wasn’t discovered until April 2020.

This latest campaign appears to have begun on or around Nov. 13, with an attack on cryptocurrency trading platform liquid.com.

“A domain hosting provider ‘GoDaddy’ that manages one of our core domain names incorrectly transferred control of the account and domain to a malicious actor,” Liquid CEO Mike Kayamori said in a blog post. “This gave the actor the ability to change DNS records and in turn, take control of a number of internal email accounts. In due course, the malicious actor was able to partially compromise our infrastructure, and gain access to document storage.”

In the early morning hours of Nov. 18 Central European Time (CET), cyptocurrency mining service NiceHash disccovered that some of the settings for its domain registration records at GoDaddy were changed without authorization, briefly redirecting email and web traffic for the site. NiceHash froze all customer funds for roughly 24 hours until it was able to verify that its domain settings had been changed back to their original settings.

“At this moment in time, it looks like no emails, passwords, or any personal data were accessed, but we do suggest resetting your password and activate 2FA security,” the company wrote in a blog post.

NiceHash founder Matjaz Skorjanc said the unauthorized changes were made from an Internet address at GoDaddy, and that the attackers tried to use their access to its incoming NiceHash emails to perform password resets on various third-party services, including Slack and Github. But he said GoDaddy was impossible to reach at the time because it was undergoing a widespread system outage in which phone and email systems were unresponsive.

“We detected this almost immediately [and] started to mitigate [the] attack,” Skorjanc said in an email to this author. “Luckily, we fought them off well and they did not gain access to any important service. Nothing was stolen.”

Skorjanc said NiceHash’s email service was redirected to privateemail.com, an email platform run by Namecheap Inc., another large domain name registrar. Using Farsight Security, a service which maps changes to domain name records over time, KrebsOnSecurity instructed the service to show all domains registered at GoDaddy that had alterations to their email records in the past week which pointed them to privateemail.com. Those results were then indexed against the top one million most popular websites according to Alexa.com.

The result shows that several other cryptocurrency platforms also may have been targeted by the same group, including Bibox.com, Celsius.network, and Wirex.app. None of these companies responded to requests for comment.

In response to questions from KrebsOnSecurity, GoDaddy acknowledged that “a small number” of customer domain names had been modified after a “limited” number of GoDaddy employees fell for a social engineering scam. GoDaddy said the outage between 7:00 p.m. and 11:00 p.m. PST on Nov. 17 was not related to a security incident, but rather a technical issue that materialized during planned network maintenance.

“Separately, and unrelated to the outage, a routine audit of account activity identified potential unauthorized changes to a small number of customer domains and/or account information,” GoDaddy spokesperson Dan Race said. “Our security team investigated and confirmed threat actor activity, including social engineering of a limited number of GoDaddy employees.

“We immediately locked down the accounts involved in this incident, reverted any changes that took place to accounts, and assisted affected customers with regaining access to their accounts,” GoDaddy’s statement continued. “As threat actors become increasingly sophisticated and aggressive in their attacks, we are constantly educating employees about new tactics that might be used against them and adopting new security measures to prevent future attacks.”

Race declined to specify how its employees were tricked into making the unauthorized changes, saying the matter was still under investigation. But in the attacks earlier this year that affected escrow.com and several other GoDaddy customer domains, the assailants targeted employees over the phone, and were able to read internal notes that GoDaddy employees had left on customer accounts.

What’s more, the attack on escrow.com redirected the site to an Internet address in Malaysia that hosted fewer than a dozen other domains, including the phishing website servicenow-godaddy.com. This suggests the attackers behind the March incident — and possibly this latest one — succeeded by calling GoDaddy employees and convincing them to use their employee credentials at a fraudulent GoDaddy login page.

In August 2020, KrebsOnSecurity warned about a marked increase in large corporations being targeted in sophisticated voice phishing or “vishing” scams. Experts say the success of these scams has been aided greatly by many employees working remotely thanks to the ongoing Coronavirus pandemic.

A typical vishing scam begins with a series of phone calls to employees working remotely at a targeted organization. The phishers often will explain that they’re calling from the employer’s IT department to help troubleshoot issues with the company’s email or virtual private networking (VPN) technology.

The goal is to convince the target either to divulge their credentials over the phone or to input them manually at a website set up by the attackers that mimics the organization’s corporate email or VPN portal.

On July 15, a number of high-profile Twitter accounts were used to tweet out a bitcoin scam that earned more than $100,000 in a few hours. According to Twitter, that attack succeeded because the perpetrators were able to social engineer several Twitter employees over the phone into giving away access to internal Twitter tools.

An alert issued jointly by the FBI and the Cybersecurity and Infrastructure Security Agency (CISA) says the perpetrators of these vishing attacks compile dossiers on employees at their targeted companies using mass scraping of public profiles on social media platforms, recruiter and marketing tools, publicly available background check services, and open-source research.

The FBI/CISA advisory includes a number of suggestions that companies can implement to help mitigate the threat from vishing attacks, including:

• Restrict VPN connections to managed devices only, using mechanisms like hardware checks or installed certificates, so user input alone is not enough to access the corporate VPN.

• Restrict VPN access hours, where applicable, to mitigate access outside of allowed times.

• Employ domain monitoring to track the creation of, or changes to, corporate, brand-name domains.

• Actively scan and monitor web applications for unauthorized access, modification, and anomalous activities.

• Employ the principle of least privilege and implement software restriction policies or other controls; monitor authorized user accesses and usage.

• Consider using a formalized authentication process for employee-to-employee communications made over the public telephone network where a second factor is used to
authenticate the phone call before sensitive information can be discussed.

• Improve 2FA and OTP messaging to reduce confusion about employee authentication attempts.

• Verify web links do not have misspellings or contain the wrong domain.

• Bookmark the correct corporate VPN URL and do not visit alternative URLs on the sole basis of an inbound phone call.

• Be suspicious of unsolicited phone calls, visits, or email messages from unknown individuals claiming to be from a legitimate organization. Do not provide personal information or information about your organization, including its structure or networks, unless you are certain of a person’s authority to have the information. If possible, try to verify the caller’s identity directly with the company.

• If you receive a vishing call, document the phone number of the caller as well as the domain that the actor tried to send you to and relay this information to law enforcement.

• Limit the amount of personal information you post on social networking sites. The internet is a public resource; only post information you are comfortable with anyone seeing.

• Evaluate your settings: sites may change their options periodically, so review your security and privacy settings regularly to make sure that your choices are still appropriate.

Kevin RuddSCMP: How the G20 Can Overcome Covid-19

By Kevin Rudd and Susilo Bambang Yudhoyono

Published in the South China Morning Post, 21 November 2020

As two of the region’s leaders around the G20 decision-making table during the height of the last global recession in 2008 and 2009, we understand the power of the institution in a crisis.

But after four years of the not-so-subtle degradation of the multilateral order under US President Donald Trump, and especially given his proclivity to see G7 and G20 summits as photo shoots for an “America first” foreign policy in action, it is easy to understand why many are ready to give up on the institution – including for having seemingly been missing in action on the current crisis.

This ignores a simple reality. Just as growth in the Asia-Pacific made the G20 indispensable a decade ago for the global economic recovery, this is likely to be the case again today. And just as it was then, this will be more important for America’s economic recovery than it will be for any other country.

This is something president George W. Bush could see when he first convened us in Washington in November 2008. It was something president Barack Obama also recognised when he brought us to Pittsburgh less than a year later. And it is something that President-elect Joe Biden will hopefully embrace, including as a key global pillar for his twin goals of dealing with the health and economic challenges posed by Covid-19.

To this day, no other institution brings together 90 per cent of the global economy, 80 per cent of global trade, two-thirds of the world’s population and has a balance of developed and developing countries, with representation from every major region of the world. To put that in context, the G7 represents only around 40 per cent of the global economy and remains at its heart a transatlantic gathering with only one representative from Asia.

The election of Biden provides new hope for the region in terms of the future of the G20, and especially its capacity to deal with the current crisis. But for the G20 to be able to reach its full potential in helping usher the world towards a post-Covid-19 world, four things must happen.

First, once Biden takes office, the incoming Italian presidency should consider urgently convening G20 leaders early in the new year on the global health crisis. The only thing worse than the lack of a globally coordinated economic strategy in recent months has been the lack of a globally coordinated strategy to respond to the Covid-19 crisis.

Back in March, it took Saudi Arabia – not the United States – to convene the G20 on the crisis. But the absence of genuine engagement by the White House made Riyadh’s efforts nigh impossible.
As a vaccine hopefully nears, this becomes all the more important to avoid a breakout of intense vaccine nationalism. With many of the world’s major economies – including the US – still experiencing a growth in cases, they also have plenty to learn from the G20 members which have fared much better, all of whom are from Asia. This includes Australia, South Korea, Japan and China.

Second, any attempt by the Biden administration to kick-start an American economic recovery in earnest is likely to fall short without an equal appreciation of the need for a globally coordinated stimulus strategy which only the G20 can provide.
A decade ago, it was the growth in Asian markets that did the heaviest lifting in terms of the global recovery, and the International Monetary Fund’s latest projections indicate this is likely to be the case again today. Whether the US is able to harness this regional growth for its own recovery will therefore be a key question.

Third, the G20 needs to be willing to begin discussing wider economic governance reform. Undeniably, the Bretton Woods era has left the Asia-Pacific with the short end of the stick in terms of their voice in economic institutions – something that is only becoming more glaring by the day.

At the very least, the G20 should invite the host of the Asia-Pacific Economic Cooperation forum to also be a permanent observer among a long cast of positions held by Europeans and Americans. More significantly, it might return to the question of how the IMF’s quotas could be better aligned with today’s global economic make-up.

In this regard, the IMF recently stated that the poor are getting poorer during the pandemic and the economic gap is widening not only between countries but also within countries. We therefore hope that in the G20’s efforts to protect and revive the world economy, social justice will not be forgotten. The G20 should commit itself to a strong, balanced and inclusive economic growth.

Finally, the G20 must urgently re-establish its credentials to also combat the emerging crisis of climate change. This is especially important for the Asia-Pacific where the new economic normal is decarbonisation, following recent announcements by China, Japan and South Korea.

Currently, less than a third of global stimulus is directed towards ensuring a green recovery. For the world to emerge in a stronger position to tackle the longer-term challenge posed by climate change, that share must at least double. Another first step would be for the G20 to finally deliver on the commitment to phase out fossil fuel subsidies, particularly given the crucial signal this would send to the private sector.

The G20 has already taken important steps to support the most vulnerable, including suspending and then extending all debt repayments to June 2021. It will also need to step up its support in other areas, such as through addressing the pledge developed countries made in 2009 at the Copenhagen climate conference to deliver US$100 billion a year in climate finance by 2020, which is not only falling short, but is a drop in the ocean of what is ultimately required.

The G20 represents the one forum of global economic governance where the Asia-Pacific’s voice stands strong. For this reason alone, the G20 remains indispensable to the region itself. But for the rest of the world, it is the Asia-Pacific’s experience dealing with Covid-19 and projected economic growth that makes it even more indispensable as the world navigates its way through this new “Covid New (Ab)Normal”.

Kevin Rudd is a former prime minister of Australia and president of the Asia Society Policy Institute and Susilo Bambang Yudhoyono is a former president of Indonesia. This article is published in a content partnership with the Asia Society Policy Institute’s Covid New (Ab)Normal initiative

The post SCMP: How the G20 Can Overcome Covid-19 appeared first on Kevin Rudd.

Planet DebianMichael Stapelberg: Debian Code Search: positional index, TurboPFor-compressed

See the Conclusion for a summary if you’re impatient :-)

Motivation

Over the last few months, I have been developing a new index format for Debian Code Search. This required a lot of careful refactoring, re-implementation, debug tool creation and debugging.

Multiple factors motivated my work on a new index format:

  1. The existing index format has a 2G size limit, into which we have bumped a few times, requiring manual intervention to keep the system running.

  2. Debugging the existing system required creating ad-hoc debugging tools, which made debugging sessions unnecessarily lengthy and painful.

  3. I wanted to check whether switching to a different integer compression format would improve performance (it does not).

  4. I wanted to check whether storing positions with the posting lists would improve performance of identifier queries (= queries which are not using any regular expression features), which make up 78.2% of all Debian Code Search queries (it does).

I figured building a new index from scratch was the easiest approach, compared to refactoring the existing index to increase the size limit (point ①).

I also figured it would be a good idea to develop the debugging tool in lock step with the index format so that I can be sure the tool works and is useful (point ②).

Integer compression: TurboPFor

As a quick refresher, search engines typically store document IDs (representing source code files, in our case) in an ordered list (“posting list”). It usually makes sense to apply at least a rudimentary level of compression: our existing system used variable integer encoding.

TurboPFor, the self-proclaimed “Fastest Integer Compression” library, combines an advanced on-disk format with a carefully tuned SIMD implementation to reach better speeds (in micro benchmarks) at less disk usage than Russ Cox’s varint implementation in github.com/google/codesearch.

If you are curious about its inner workings, check out my “TurboPFor: an analysis”.

Applied on the Debian Code Search index, TurboPFor indeed compresses integers better:

Disk space

 
8.9G codesearch varint index

 
5.5G TurboPFor index

Switching to TurboPFor (via cgo) for storing and reading the index results in a slight speed-up of a dcs replay benchmark, which is more pronounced the more i/o is required.

Query speed (regexp, cold page cache)

 
18s codesearch varint index

 
14s TurboPFor index (cgo)

Query speed (regexp, warm page cache)

 
15s codesearch varint index

 
14s TurboPFor index (cgo)

Overall, TurboPFor is an all-around improvement in efficiency, albeit with a high cost in implementation complexity.

Positional index: trade more disk for faster queries

This section builds on the previous section: all figures come from the TurboPFor index, which can optionally support positions.

Conceptually, we’re going from:

type docid uint32
type index map[trigram][]docid

…to:

type occurrence struct {
    doc docid
    pos uint32 // byte offset in doc
}
type index map[trigram][]occurrence

The resulting index consumes more disk space, but can be queried faster:

  1. We can do fewer queries: instead of reading all the posting lists for all the trigrams, we can read the posting lists for the query’s first and last trigram only.
    This is one of the tricks described in the paper “AS-Index: A Structure For String Search Using n-grams and Algebraic Signatures” (PDF), and goes a long way without incurring the complexity, computational cost and additional disk usage of calculating algebraic signatures.

  2. Verifying the delta between the last and first position matches the length of the query term significantly reduces the number of files to read (lower false positive rate).

  3. The matching phase is quicker: instead of locating the query term in the file, we only need to compare a few bytes at a known offset for equality.

  4. More data is read sequentially (from the index), which is faster.

Disk space

A positional index consumes significantly more disk space, but not so much as to pose a challenge: a Hetzner EX61-NVME dedicated server (≈ 64 €/month) provides 1 TB worth of fast NVMe flash storage.

 
 6.5G non-positional

 
123G positional

 
  93G positional (posrel)

The idea behind the positional index (posrel) is to not store a (doc,pos) tuple on disk, but to store positions, accompanied by a stream of doc/pos relationship bits: 1 means this position belongs to the next document, 0 means this position belongs to the current document.

This is an easy way of saving some space without modifying the TurboPFor on-disk format: the posrel technique reduces the index size to about ¾.

With the increase in size, the Linux page cache hit ratio will be lower for the positional index, i.e. more data will need to be fetched from disk for querying the index.

As long as the disk can deliver data as fast as you can decompress posting lists, this only translates into one disk seek’s worth of additional latency. This is the case with modern NVMe disks that deliver thousands of MB/s, e.g. the Samsung 960 Pro (used in Hetzner’s aforementioned EX61-NVME server).

The values were measured by running dcs du -h /srv/dcs/shard*/full without and with the -pos argument.

Bytes read

A positional index requires fewer queries: reading only the first and last trigram’s posting lists and positions is sufficient to achieve a lower (!) false positive rate than evaluating all trigram’s posting lists in a non-positional index.

As a consequence, fewer files need to be read, resulting in fewer bytes required to read from disk overall.

As an additional bonus, in a positional index, more data is read sequentially (index), which is faster than random i/o, regardless of the underlying disk.

1.2G
19.8G
21.0G regexp queries

4.2G (index)
10.8G (files)
15.0G identifier queries

The values were measured by running iostat -d 25 just before running bench.zsh on an otherwise idle system.

Query speed

Even though the positional index is larger and requires more data to be read at query time (see above), thanks to the C TurboPFor library, the 2 queries on a positional index are roughly as fast as the n queries on a non-positional index (≈4s instead of ≈3s).

This is more than made up for by the combined i/o matching stage, which shrinks from ≈18.5s (7.1s i/o + 11.4s matching) to ≈1.3s.

3.3s (index)
7.1s (i/o)
11.4s (matching)
21.8s regexp queries

3.92s (index)
≈1.3s
5.22s identifier queries

Note that identifier query i/o was sped up not just by needing to read fewer bytes, but also by only having to verify bytes at a known offset instead of needing to locate the identifier within the file.

Conclusion

The new index format is overall slightly more efficient. This disk space efficiency allows us to introduce a positional index section for the first time.

Most Debian Code Search queries are positional queries (78.2%) and will be answered much quicker by leveraging the positions.

Bottomline, it is beneficial to use a positional index on disk over a non-positional index in RAM.

Planet DebianMichael Stapelberg: Linux package managers are slow

I measured how long the most popular Linux distribution’s package manager take to install small and large packages (the ack(1p) source code search Perl script and qemu, respectively).

Where required, my measurements include metadata updates such as transferring an up-to-date package list. For me, requiring a metadata update is the more common case, particularly on live systems or within Docker containers.

All measurements were taken on an Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz running Docker 1.13.1 on Linux 4.19, backed by a Samsung 970 Pro NVMe drive boasting many hundreds of MB/s write performance. The machine is located in Zürich and connected to the Internet with a 1 Gigabit fiber connection, so the expected top download speed is ≈115 MB/s.

See Appendix C for details on the measurement method and command outputs.

Measurements

Keep in mind that these are one-time measurements. They should be indicative of actual performance, but your experience may vary.

ack (small Perl program)

distribution package manager data wall-clock time rate
Fedora dnf 114 MB 33s 3.4 MB/s
Debian apt 16 MB 10s 1.6 MB/s
NixOS Nix 15 MB 5s 3.0 MB/s
Arch Linux pacman 6.5 MB 3s 2.1 MB/s
Alpine apk 10 MB 1s 10.0 MB/s

qemu (large C program)

distribution package manager data wall-clock time rate
Fedora dnf 226 MB 4m37s 1.2 MB/s
Debian apt 224 MB 1m35s 2.3 MB/s
Arch Linux pacman 142 MB 44s 3.2 MB/s
NixOS Nix 180 MB 34s 5.2 MB/s
Alpine apk 26 MB 2.4s 10.8 MB/s


(Looking for older measurements? See Appendix B (2019).

The difference between the slowest and fastest package managers is 30x!

How can Alpine’s apk and Arch Linux’s pacman be an order of magnitude faster than the rest? They are doing a lot less than the others, and more efficiently, too.

Pain point: too much metadata

For example, Fedora transfers a lot more data than others because its main package list is 60 MB (compressed!) alone. Compare that with Alpine’s 734 KB APKINDEX.tar.gz.

Of course the extra metadata which Fedora provides helps some use case, otherwise they hopefully would have removed it altogether. The amount of metadata seems excessive for the use case of installing a single package, which I consider the main use-case of an interactive package manager.

I expect any modern Linux distribution to only transfer absolutely required data to complete my task.

Pain point: no concurrency

Because they need to sequence executing arbitrary package maintainer-provided code (hooks and triggers), all tested package managers need to install packages sequentially (one after the other) instead of concurrently (all at the same time).

In my blog post “Can we do without hooks and triggers?”, I outline that hooks and triggers are not strictly necessary to build a working Linux distribution.

Thought experiment: further speed-ups

Strictly speaking, the only required feature of a package manager is to make available the package contents so that the package can be used: a program can be started, a kernel module can be loaded, etc.

By only implementing what’s needed for this feature, and nothing more, a package manager could likely beat apk’s performance. It could, for example:

  • skip archive extraction by mounting file system images (like AppImage or snappy)
  • use compression which is light on CPU, as networks are fast (like apk)
  • skip fsync when it is safe to do so, i.e.:
    • package installations don’t modify system state
    • atomic package installation (e.g. an append-only package store)
    • automatically clean up the package store after crashes

Current landscape

Here’s a table outlining how the various package managers listed on Wikipedia’s list of software package management systems fare:

name scope package file format hooks/triggers
AppImage apps image: ISO9660, SquashFS no
snappy apps image: SquashFS yes: hooks
FlatPak apps archive: OSTree no
0install apps archive: tar.bz2 no
nix, guix distro archive: nar.{bz2,xz} activation script
dpkg distro archive: tar.{gz,xz,bz2} in ar(1) yes
rpm distro archive: cpio.{bz2,lz,xz} scriptlets
pacman distro archive: tar.xz install
slackware distro archive: tar.{gz,xz} yes: doinst.sh
apk distro archive: tar.gz yes: .post-install
Entropy distro archive: tar.bz2 yes
ipkg, opkg distro archive: tar{,.gz} yes

Conclusion

As per the current landscape, there is no distribution-scoped package manager which uses images and leaves out hooks and triggers, not even in smaller Linux distributions.

I think that space is really interesting, as it uses a minimal design to achieve significant real-world speed-ups.

I have explored this idea in much more detail, and am happy to talk more about it in my post “Introducing the distri research linux distribution".

There are a couple of recent developments going into the same direction:

Appendix C: measurement details (2020)

ack

You can expand each of these:

Fedora’s dnf takes almost 33 seconds to fetch and unpack 114 MB.

% docker run -t -i fedora /bin/bash
[root@62d3cae2e2f9 /]# time dnf install -y ack
Fedora 32 openh264 (From Cisco) - x86_64     1.9 kB/s | 2.5 kB     00:01
Fedora Modular 32 - x86_64                   6.8 MB/s | 4.9 MB     00:00
Fedora Modular 32 - x86_64 - Updates         5.6 MB/s | 3.7 MB     00:00
Fedora 32 - x86_64 - Updates                 9.9 MB/s |  23 MB     00:02
Fedora 32 - x86_64                            39 MB/s |  70 MB     00:01
[…]
real	0m32.898s
user	0m25.121s
sys	0m1.408s

NixOS’s Nix takes a little over 5s to fetch and unpack 15 MB.

% docker run -t -i nixos/nix
39e9186422ba:/# time sh -c 'nix-channel --update && nix-env -iA nixpkgs.ack'
unpacking channels...
created 1 symlinks in user environment
installing 'perl5.32.0-ack-3.3.1'
these paths will be fetched (15.55 MiB download, 85.51 MiB unpacked):
  /nix/store/34l8jdg76kmwl1nbbq84r2gka0kw6rc8-perl5.32.0-ack-3.3.1-man
  /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31
  /nix/store/9fd4pjaxpjyyxvvmxy43y392l7yvcwy1-perl5.32.0-File-Next-1.18
  /nix/store/czc3c1apx55s37qx4vadqhn3fhikchxi-libunistring-0.9.10
  /nix/store/dj6n505iqrk7srn96a27jfp3i0zgwa1l-acl-2.2.53
  /nix/store/ifayp0kvijq0n4x0bv51iqrb0yzyz77g-perl-5.32.0
  /nix/store/w9wc0d31p4z93cbgxijws03j5s2c4gyf-coreutils-8.31
  /nix/store/xim9l8hym4iga6d4azam4m0k0p1nw2rm-libidn2-2.3.0
  /nix/store/y7i47qjmf10i1ngpnsavv88zjagypycd-attr-2.4.48
  /nix/store/z45mp61h51ksxz28gds5110rf3wmqpdc-perl5.32.0-ack-3.3.1
copying path '/nix/store/34l8jdg76kmwl1nbbq84r2gka0kw6rc8-perl5.32.0-ack-3.3.1-man' from 'https://cache.nixos.org'...
copying path '/nix/store/czc3c1apx55s37qx4vadqhn3fhikchxi-libunistring-0.9.10' from 'https://cache.nixos.org'...
copying path '/nix/store/9fd4pjaxpjyyxvvmxy43y392l7yvcwy1-perl5.32.0-File-Next-1.18' from 'https://cache.nixos.org'...
copying path '/nix/store/xim9l8hym4iga6d4azam4m0k0p1nw2rm-libidn2-2.3.0' from 'https://cache.nixos.org'...
copying path '/nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31' from 'https://cache.nixos.org'...
copying path '/nix/store/y7i47qjmf10i1ngpnsavv88zjagypycd-attr-2.4.48' from 'https://cache.nixos.org'...
copying path '/nix/store/dj6n505iqrk7srn96a27jfp3i0zgwa1l-acl-2.2.53' from 'https://cache.nixos.org'...
copying path '/nix/store/w9wc0d31p4z93cbgxijws03j5s2c4gyf-coreutils-8.31' from 'https://cache.nixos.org'...
copying path '/nix/store/ifayp0kvijq0n4x0bv51iqrb0yzyz77g-perl-5.32.0' from 'https://cache.nixos.org'...
copying path '/nix/store/z45mp61h51ksxz28gds5110rf3wmqpdc-perl5.32.0-ack-3.3.1' from 'https://cache.nixos.org'...
building '/nix/store/m0rl62grplq7w7k3zqhlcz2hs99y332l-user-environment.drv'...
created 49 symlinks in user environment
real	0m 5.60s
user	0m 3.21s
sys	0m 1.66s

Debian’s apt takes almost 10 seconds to fetch and unpack 16 MB.

% docker run -t -i debian:sid
root@1996bb94a2d1:/# time (apt update && apt install -y ack-grep)
Get:1 http://deb.debian.org/debian sid InRelease [146 kB]
Get:2 http://deb.debian.org/debian sid/main amd64 Packages [8400 kB]
Fetched 8546 kB in 1s (8088 kB/s)
[…]
The following NEW packages will be installed:
  ack libfile-next-perl libgdbm-compat4 libgdbm6 libperl5.30 netbase perl perl-modules-5.30
0 upgraded, 8 newly installed, 0 to remove and 23 not upgraded.
Need to get 7341 kB of archives.
After this operation, 46.7 MB of additional disk space will be used.
[…]
real	0m9.544s
user	0m2.839s
sys	0m0.775s

Arch Linux’s pacman takes a little under 3s to fetch and unpack 6.5 MB.

% docker run -t -i archlinux/base
[root@9f6672688a64 /]# time (pacman -Sy && pacman -S --noconfirm ack)
:: Synchronizing package databases...
 core            130.8 KiB  1090 KiB/s 00:00
 extra          1655.8 KiB  3.48 MiB/s 00:00
 community         5.2 MiB  6.11 MiB/s 00:01
resolving dependencies...
looking for conflicting packages...

Packages (2) perl-file-next-1.18-2  ack-3.4.0-1

Total Download Size:   0.07 MiB
Total Installed Size:  0.19 MiB
[…]
real	0m2.936s
user	0m0.375s
sys	0m0.160s

Alpine’s apk takes a little over 1 second to fetch and unpack 10 MB.

% docker run -t -i alpine
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
(1/4) Installing libbz2 (1.0.8-r1)
(2/4) Installing perl (5.30.3-r0)
(3/4) Installing perl-file-next (1.18-r0)
(4/4) Installing ack (3.3.1-r0)
Executing busybox-1.31.1-r16.trigger
OK: 43 MiB in 18 packages
real	0m 1.24s
user	0m 0.40s
sys	0m 0.15s

qemu

You can expand each of these:

Fedora’s dnf takes over 4 minutes to fetch and unpack 226 MB.

% docker run -t -i fedora /bin/bash
[root@6a52ecfc3afa /]# time dnf install -y qemu
Fedora 32 openh264 (From Cisco) - x86_64     3.1 kB/s | 2.5 kB     00:00
Fedora Modular 32 - x86_64                   6.3 MB/s | 4.9 MB     00:00
Fedora Modular 32 - x86_64 - Updates         6.0 MB/s | 3.7 MB     00:00
Fedora 32 - x86_64 - Updates                 334 kB/s |  23 MB     01:10
Fedora 32 - x86_64                            33 MB/s |  70 MB     00:02
[…]

Total download size: 181 M
Downloading Packages:
[…]

real	4m37.652s
user	0m38.239s
sys	0m6.321s

NixOS’s Nix takes almost 34s to fetch and unpack 180 MB.

% docker run -t -i nixos/nix
83971cf79f7e:/# time sh -c 'nix-channel --update && nix-env -iA nixpkgs.qemu'
unpacking channels...
created 1 symlinks in user environment
installing 'qemu-5.1.0'
these paths will be fetched (180.70 MiB download, 1146.92 MiB unpacked):
[…]
real	0m 33.64s
user	0m 16.96s
sys	0m 3.05s

Debian’s apt takes over 95 seconds to fetch and unpack 224 MB.

% docker run -t -i debian:sid
root@b7cc25a927ab:/# time (apt update && apt install -y qemu-system-x86)
Get:1 http://deb.debian.org/debian sid InRelease [146 kB]
Get:2 http://deb.debian.org/debian sid/main amd64 Packages [8400 kB]
Fetched 8546 kB in 1s (5998 kB/s)
[…]
Fetched 216 MB in 43s (5006 kB/s)
[…]
real	1m25.375s
user	0m29.163s
sys	0m12.835s

Arch Linux’s pacman takes almost 44s to fetch and unpack 142 MB.

% docker run -t -i archlinux/base
[root@58c78bda08e8 /]# time (pacman -Sy && pacman -S --noconfirm qemu)
:: Synchronizing package databases...
 core          130.8 KiB  1055 KiB/s 00:00
 extra        1655.8 KiB  3.70 MiB/s 00:00
 community       5.2 MiB  7.89 MiB/s 00:01
[…]
Total Download Size:   135.46 MiB
Total Installed Size:  661.05 MiB
[…]
real	0m43.901s
user	0m4.980s
sys	0m2.615s

Alpine’s apk takes only about 2.4 seconds to fetch and unpack 26 MB.

% docker run -t -i alpine
/ # time apk add qemu-system-x86_64
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
[…]
OK: 78 MiB in 95 packages
real	0m 2.43s
user	0m 0.46s
sys	0m 0.09s

Appendix B: measurement details (2019)

ack

You can expand each of these:

Fedora’s dnf takes almost 30 seconds to fetch and unpack 107 MB.

% docker run -t -i fedora /bin/bash
[root@722e6df10258 /]# time dnf install -y ack
Fedora Modular 30 - x86_64            4.4 MB/s | 2.7 MB     00:00
Fedora Modular 30 - x86_64 - Updates  3.7 MB/s | 2.4 MB     00:00
Fedora 30 - x86_64 - Updates           17 MB/s |  19 MB     00:01
Fedora 30 - x86_64                     31 MB/s |  70 MB     00:02
[…]
Install  44 Packages

Total download size: 13 M
Installed size: 42 M
[…]
real	0m29.498s
user	0m22.954s
sys	0m1.085s

NixOS’s Nix takes 14s to fetch and unpack 15 MB.

% docker run -t -i nixos/nix
39e9186422ba:/# time sh -c 'nix-channel --update && nix-env -i perl5.28.2-ack-2.28'
unpacking channels...
created 2 symlinks in user environment
installing 'perl5.28.2-ack-2.28'
these paths will be fetched (14.91 MiB download, 80.83 MiB unpacked):
  /nix/store/57iv2vch31v8plcjrk97lcw1zbwb2n9r-perl-5.28.2
  /nix/store/89gi8cbp8l5sf0m8pgynp2mh1c6pk1gk-attr-2.4.48
  /nix/store/gkrpl3k6s43fkg71n0269yq3p1f0al88-perl5.28.2-ack-2.28-man
  /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27
  /nix/store/k8lhqzpaaymshchz8ky3z4653h4kln9d-coreutils-8.31
  /nix/store/svgkibi7105pm151prywndsgvmc4qvzs-acl-2.2.53
  /nix/store/x4knf14z1p0ci72gl314i7vza93iy7yc-perl5.28.2-File-Next-1.16
  /nix/store/zfj7ria2kwqzqj9dh91kj9kwsynxdfk0-perl5.28.2-ack-2.28
copying path '/nix/store/gkrpl3k6s43fkg71n0269yq3p1f0al88-perl5.28.2-ack-2.28-man' from 'https://cache.nixos.org'...
copying path '/nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27' from 'https://cache.nixos.org'...
copying path '/nix/store/x4knf14z1p0ci72gl314i7vza93iy7yc-perl5.28.2-File-Next-1.16' from 'https://cache.nixos.org'...
copying path '/nix/store/89gi8cbp8l5sf0m8pgynp2mh1c6pk1gk-attr-2.4.48' from 'https://cache.nixos.org'...
copying path '/nix/store/svgkibi7105pm151prywndsgvmc4qvzs-acl-2.2.53' from 'https://cache.nixos.org'...
copying path '/nix/store/k8lhqzpaaymshchz8ky3z4653h4kln9d-coreutils-8.31' from 'https://cache.nixos.org'...
copying path '/nix/store/57iv2vch31v8plcjrk97lcw1zbwb2n9r-perl-5.28.2' from 'https://cache.nixos.org'...
copying path '/nix/store/zfj7ria2kwqzqj9dh91kj9kwsynxdfk0-perl5.28.2-ack-2.28' from 'https://cache.nixos.org'...
building '/nix/store/q3243sjg91x1m8ipl0sj5gjzpnbgxrqw-user-environment.drv'...
created 56 symlinks in user environment
real	0m 14.02s
user	0m 8.83s
sys	0m 2.69s

Debian’s apt takes almost 10 seconds to fetch and unpack 16 MB.

% docker run -t -i debian:sid
root@b7cc25a927ab:/# time (apt update && apt install -y ack-grep)
Get:1 http://cdn-fastly.deb.debian.org/debian sid InRelease [233 kB]
Get:2 http://cdn-fastly.deb.debian.org/debian sid/main amd64 Packages [8270 kB]
Fetched 8502 kB in 2s (4764 kB/s)
[…]
The following NEW packages will be installed:
  ack ack-grep libfile-next-perl libgdbm-compat4 libgdbm5 libperl5.26 netbase perl perl-modules-5.26
The following packages will be upgraded:
  perl-base
1 upgraded, 9 newly installed, 0 to remove and 60 not upgraded.
Need to get 8238 kB of archives.
After this operation, 42.3 MB of additional disk space will be used.
[…]
real	0m9.096s
user	0m2.616s
sys	0m0.441s

Arch Linux’s pacman takes a little over 3s to fetch and unpack 6.5 MB.

% docker run -t -i archlinux/base
[root@9604e4ae2367 /]# time (pacman -Sy && pacman -S --noconfirm ack)
:: Synchronizing package databases...
 core            132.2 KiB  1033K/s 00:00
 extra          1629.6 KiB  2.95M/s 00:01
 community         4.9 MiB  5.75M/s 00:01
[…]
Total Download Size:   0.07 MiB
Total Installed Size:  0.19 MiB
[…]
real	0m3.354s
user	0m0.224s
sys	0m0.049s

Alpine’s apk takes only about 1 second to fetch and unpack 10 MB.

% docker run -t -i alpine
/ # time apk add ack
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
(1/4) Installing perl-file-next (1.16-r0)
(2/4) Installing libbz2 (1.0.6-r7)
(3/4) Installing perl (5.28.2-r1)
(4/4) Installing ack (3.0.0-r0)
Executing busybox-1.30.1-r2.trigger
OK: 44 MiB in 18 packages
real	0m 0.96s
user	0m 0.25s
sys	0m 0.07s

qemu

You can expand each of these:

Fedora’s dnf takes over a minute to fetch and unpack 266 MB.

% docker run -t -i fedora /bin/bash
[root@722e6df10258 /]# time dnf install -y qemu
Fedora Modular 30 - x86_64            3.1 MB/s | 2.7 MB     00:00
Fedora Modular 30 - x86_64 - Updates  2.7 MB/s | 2.4 MB     00:00
Fedora 30 - x86_64 - Updates           20 MB/s |  19 MB     00:00
Fedora 30 - x86_64                     31 MB/s |  70 MB     00:02
[…]
Install  262 Packages
Upgrade    4 Packages

Total download size: 172 M
[…]
real	1m7.877s
user	0m44.237s
sys	0m3.258s

NixOS’s Nix takes 38s to fetch and unpack 262 MB.

% docker run -t -i nixos/nix
39e9186422ba:/# time sh -c 'nix-channel --update && nix-env -i qemu-4.0.0'
unpacking channels...
created 2 symlinks in user environment
installing 'qemu-4.0.0'
these paths will be fetched (262.18 MiB download, 1364.54 MiB unpacked):
[…]
real	0m 38.49s
user	0m 26.52s
sys	0m 4.43s

Debian’s apt takes 51 seconds to fetch and unpack 159 MB.

% docker run -t -i debian:sid
root@b7cc25a927ab:/# time (apt update && apt install -y qemu-system-x86)
Get:1 http://cdn-fastly.deb.debian.org/debian sid InRelease [149 kB]
Get:2 http://cdn-fastly.deb.debian.org/debian sid/main amd64 Packages [8426 kB]
Fetched 8574 kB in 1s (6716 kB/s)
[…]
Fetched 151 MB in 2s (64.6 MB/s)
[…]
real	0m51.583s
user	0m15.671s
sys	0m3.732s

Arch Linux’s pacman takes 1m2s to fetch and unpack 124 MB.

% docker run -t -i archlinux/base
[root@9604e4ae2367 /]# time (pacman -Sy && pacman -S --noconfirm qemu)
:: Synchronizing package databases...
 core       132.2 KiB   751K/s 00:00
 extra     1629.6 KiB  3.04M/s 00:01
 community    4.9 MiB  6.16M/s 00:01
[…]
Total Download Size:   123.20 MiB
Total Installed Size:  587.84 MiB
[…]
real	1m2.475s
user	0m9.272s
sys	0m2.458s

Alpine’s apk takes only about 2.4 seconds to fetch and unpack 26 MB.

% docker run -t -i alpine
/ # time apk add qemu-system-x86_64
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
[…]
OK: 78 MiB in 95 packages
real	0m 2.43s
user	0m 0.46s
sys	0m 0.09s

Planet DebianMichael Stapelberg: Winding down my Debian involvement

This post is hard to write, both in the emotional sense but also in the “I would have written a shorter letter, but I didn’t have the time” sense. Hence, please assume the best of intentions when reading it—it is not my intention to make anyone feel bad about their contributions, but rather to provide some insight into why my frustration level ultimately exceeded the threshold.

Debian has been in my life for well over 10 years at this point.

A few weeks ago, I have visited some old friends at the Zürich Debian meetup after a multi-year period of absence. On my bike ride home, it occurred to me that the topics of our discussions had remarkable overlap with my last visit. We had a discussion about the merits of systemd, which took a detour to respect in open source communities, returned to processes in Debian and eventually culminated in democracies and their theoretical/practical failings. Admittedly, that last one might be a Swiss thing.

I say this not to knock on the Debian meetup, but because it prompted me to reflect on what feelings Debian is invoking lately and whether it’s still a good fit for me.

So I’m finally making a decision that I should have made a long time ago: I am winding down my involvement in Debian to a minimum.

What does this mean?

Over the coming weeks, I will:

  • transition packages to be team-maintained where it makes sense
  • remove myself from the Uploaders field on packages with other maintainers
  • orphan packages where I am the sole maintainer

I will try to keep up best-effort maintenance of the manpages.debian.org service and the codesearch.debian.net service, but any help would be much appreciated.

For all intents and purposes, please treat me as permanently on vacation. I will try to be around for administrative issues (e.g. permission transfers) and questions addressed directly to me, permitted they are easy enough to answer.

Why?

When I joined Debian, I was still studying, i.e. I had luxurious amounts of spare time. Now, over 5 years of full time work later, my day job taught me a lot, both about what works in large software engineering projects and how I personally like my computer systems. I am very conscious of how I spend the little spare time that I have these days.

The following sections each deal with what I consider a major pain point, in no particular order. Some of them influence each other—for example, if changes worked better, we could have a chance at transitioning packages to be more easily machine readable.

Change process in Debian

The last few years, my current team at work conducted various smaller and larger refactorings across the entire code base (touching thousands of projects), so we have learnt a lot of valuable lessons about how to effectively do these changes. It irks me that Debian works almost the opposite way in every regard. I appreciate that every organization is different, but I think a lot of my points do actually apply to Debian.

In Debian, packages are nudged in the right direction by a document called the Debian Policy, or its programmatic embodiment, lintian.

While it is great to have a lint tool (for quick, local/offline feedback), it is even better to not require a lint tool at all. The team conducting the change (e.g. the C++ team introduces a new hardening flag for all packages) should be able to do their work transparent to me.

Instead, currently, all packages become lint-unclean, all maintainers need to read up on what the new thing is, how it might break, whether/how it affects them, manually run some tests, and finally decide to opt in. This causes a lot of overhead and manually executed mechanical changes across packages.

Notably, the cost of each change is distributed onto the package maintainers in the Debian model. At work, we have found that the opposite works better: if the team behind the change is put in power to do the change for as many users as possible, they can be significantly more efficient at it, which reduces the total cost and time a lot. Of course, exceptions (e.g. a large project abusing a language feature) should still be taken care of by the respective owners, but the important bit is that the default should be the other way around.

Debian is lacking tooling for large changes: it is hard to programmatically deal with packages and repositories (see the section below). The closest to “sending out a change for review” is to open a bug report with an attached patch. I thought the workflow for accepting a change from a bug report was too complicated and started mergebot, but only Guido ever signaled interest in the project.

Culturally, reviews and reactions are slow. There are no deadlines. I literally sometimes get emails notifying me that a patch I sent out a few years ago (!!) is now merged. This turns projects from a small number of weeks into many years, which is a huge demotivator for me.

Interestingly enough, you can see artifacts of the slow online activity manifest itself in the offline culture as well: I don’t want to be discussing systemd’s merits 10 years after I first heard about it.

Lastly, changes can easily be slowed down significantly by holdouts who refuse to collaborate. My canonical example for this is rsync, whose maintainer refused my patches to make the package use debhelper purely out of personal preference.

Granting so much personal freedom to individual maintainers prevents us as a project from raising the abstraction level for building Debian packages, which in turn makes tooling harder.

How would things look like in a better world?

  1. As a project, we should strive towards more unification. Uniformity still does not rule out experimentation, it just changes the trade-off from easier experimentation and harder automation to harder experimentation and easier automation.
  2. Our culture needs to shift from “this package is my domain, how dare you touch it” to a shared sense of ownership, where anyone in the project can easily contribute (reviewed) changes without necessarily even involving individual maintainers.

To learn more about how successful large changes can look like, I recommend my colleague Hyrum Wright’s talk “Large-Scale Changes at Google: Lessons Learned From 5 Yrs of Mass Migrations”.

Fragmented workflow and infrastructure

Debian generally seems to prefer decentralized approaches over centralized ones. For example, individual packages are maintained in separate repositories (as opposed to in one repository), each repository can use any SCM (git and svn are common ones) or no SCM at all, and each repository can be hosted on a different site. Of course, what you do in such a repository also varies subtly from team to team, and even within teams.

In practice, non-standard hosting options are used rarely enough to not justify their cost, but frequently enough to be a huge pain when trying to automate changes to packages. Instead of using GitLab’s API to create a merge request, you have to design an entirely different, more complex system, which deals with intermittently (or permanently!) unreachable repositories and abstracts away differences in patch delivery (bug reports, merge requests, pull requests, email, …).

Wildly diverging workflows is not just a temporary problem either. I participated in long discussions about different git workflows during DebConf 13, and gather that there were similar discussions in the meantime.

Personally, I cannot keep enough details of the different workflows in my head. Every time I touch a package that works differently than mine, it frustrates me immensely to re-learn aspects of my day-to-day.

After noticing workflow fragmentation in the Go packaging team (which I started), I tried fixing this with the workflow changes proposal, but did not succeed in implementing it. The lack of effective automation and slow pace of changes in the surrounding tooling despite my willingness to contribute time and energy killed any motivation I had.

Old infrastructure: package uploads

When you want to make a package available in Debian, you upload GPG-signed files via anonymous FTP. There are several batch jobs (the queue daemon, unchecked, dinstall, possibly others) which run on fixed schedules (e.g. dinstall runs at 01:52 UTC, 07:52 UTC, 13:52 UTC and 19:52 UTC).

Depending on timing, I estimated that you might wait for over 7 hours (!!) before your package is actually installable.

What’s worse for me is that feedback to your upload is asynchronous. I like to do one thing, be done with it, move to the next thing. The current setup requires a many-minute wait and costly task switch for no good technical reason. You might think a few minutes aren’t a big deal, but when all the time I can spend on Debian per day is measured in minutes, this makes a huge difference in perceived productivity and fun.

The last communication I can find about speeding up this process is ganneff’s post from 2008.

How would things look like in a better world?

  1. Anonymous FTP would be replaced by a web service which ingests my package and returns an authoritative accept or reject decision in its response.
  2. For accepted packages, there would be a status page displaying the build status and when the package will be available via the mirror network.
  3. Packages should be available within a few minutes after the build completed.

Old infrastructure: bug tracker

I dread interacting with the Debian bug tracker. debbugs is a piece of software (from 1994) which is only used by Debian and the GNU project these days.

Debbugs processes emails, which is to say it is asynchronous and cumbersome to deal with. Despite running on the fastest machines we have available in Debian (or so I was told when the subject last came up), its web interface loads very slowly.

Notably, the web interface at bugs.debian.org is read-only. Setting up a working email setup for reportbug(1) or manually dealing with attachments is a rather big hurdle.

For reasons I don’t understand, every interaction with debbugs results in many different email threads.

Aside from the technical implementation, I also can never remember the different ways that Debian uses pseudo-packages for bugs and processes. I need them rarely enough to establish a mental model of how they are set up, or working memory of how they are used, but frequently enough to be annoyed by this.

How would things look like in a better world?

  1. Debian would switch from a custom bug tracker to a (any) well-established one.
  2. Debian would offer automation around processes. It is great to have a paper-trail and artifacts of the process in the form of a bug report, but the primary interface should be more convenient (e.g. a web form).

Old infrastructure: mailing list archives

It baffles me that in 2019, we still don’t have a conveniently browsable threaded archive of mailing list discussions. Email and threading is more widely used in Debian than anywhere else, so this is somewhat ironic. Gmane used to paper over this issue, but Gmane’s availability over the last few years has been spotty, to say the least (it is down as I write this).

I tried to contribute a threaded list archive, but our listmasters didn’t seem to care or want to support the project.

Debian is hard to machine-read

While it is obviously possible to deal with Debian packages programmatically, the experience is far from pleasant. Everything seems slow and cumbersome. I have picked just 3 quick examples to illustrate my point.

debiman needs help from piuparts in analyzing the alternatives mechanism of each package to display the manpages of e.g. psql(1). This is because maintainer scripts modify the alternatives database by calling shell scripts. Without actually installing a package, you cannot know which changes it does to the alternatives database.

pk4 needs to maintain its own cache to look up package metadata based on the package name. Other tools parse the apt database from scratch on every invocation. A proper database format, or at least a binary interchange format, would go a long way.

Debian Code Search wants to ingest new packages as quickly as possible. There used to be a fedmsg instance for Debian, but it no longer seems to exist. It is unclear where to get notifications from for new packages, and where best to fetch those packages.

Complicated build stack

See my “Debian package build tools” post. It really bugs me that the sprawl of tools is not seen as a problem by others.

Developer experience pretty painful

Most of the points discussed so far deal with the experience in developing Debian, but as I recently described in my post “Debugging experience in Debian”, the experience when developing using Debian leaves a lot to be desired, too.

I have more ideas

At this point, the article is getting pretty long, and hopefully you got a rough idea of my motivation.

While I described a number of specific shortcomings above, the final nail in the coffin is actually the lack of a positive outlook. I have more ideas that seem really compelling to me, but, based on how my previous projects have been going, I don’t think I can make any of these ideas happen within the Debian project.

I intend to publish a few more posts about specific ideas for improving operating systems here. Stay tuned.

Lastly, I hope this post inspires someone, ideally a group of people, to improve the developer experience within Debian.

Planet DebianKentaro Hayashi: Introduction about recent debexpo (mentors.debian.net)

I've make a presentation about "How to hack debexpo (mentors.debian.net)" at Tokyo Debian (local Debian meeting) 21, November 2020.

Here is the agenda about presentation.

  • What is mentors.debian.net
  • How to setup debexpo development environment
  • One example to hack debexpo (Showing "In Debian" flag)

The presentation slide is published at Rabbit Slide Show (Written in Japanese)

I hope that more people will be involved to hack debexpo!

,

Planet DebianShirish Agarwal: Rights, Press freedom and India

In some ways it is sad and interesting to see how personal liberty is viewed in India. And how it differs from those having the highest fame and power can get a different kind of justice then the rest cannot.

Arnab Goswami

This particular gentleman is a class apart. He is the editor as well as Republic TV, a right-leaning channel which demonizes the minority, women whatever is antithesis to the Central Govt. of India. As a result there have been a spate of cases against him in the past few months. But surprisingly, in each of them he got hearing the day after the suit was filed. This is unique in Indian legal history so much so that a popular legal site which publishes on-going cases put up a post sharing how he was getting prompt hearings. That post itself needs to be updated as there have been 3 more hearings which have been done back to back for him. This is unusual as there have been so many cases pending for the SC attention, some arguably more important than this gentleman . So many precedents have been set which will send a wrong message. The biggest one, that even though a trial is taking place in the sessions court (below High Court) the SC can interject on matters. What this will do to the morale of both lawyers as well as judges of the various Sessions Court is a matter of speculation and yet as shared unprecedented. The saddest part was when Justice Chandrachud said –

Justice Chandrachud – If you don’t like a channel then don’t watch it. – 11th November 2020 .

This is basically giving a free rope to hate speech. How can a SC say like that ? And this is the Same Supreme Court which could not take two tweets from Shri Prashant Bhushan when he made remarks against the judiciary .

J&K pleas in Supreme Court pending since August 2019 (Abrogation 370)

After abrogation of 370, citizens of Jammu and Kashmir, the population of which is 13.6 million people including 4 million Hindus have been stuck with reduced rights and their land being taken away due to new laws. Many of the Hindus which regionally are a minority now rue the fact that they supported the abrogation of 370A . Imagine, a whole state whose answers and prayers have not been heard by the Supreme Court and the people need to move a prayer stating the same.

100 Journalists, activists languishing in Jail without even a hearing

55 Journalists alone have been threatened, booked and in jail for reporting of pandemic . Their fault, they were bring the irregularities, corruption made during the pandemic early months. Activists such as Sudha Bharadwaj, who giving up her American citizenship and settling to fight for tribals is in jail for 2 years without any charges. There are many like her, There are several more petitions lying in the Supreme Court, for e.g. Varavara Rao, not a single hearing from last couple of years, even though he has taken part in so many national movements including the emergency as well as part-responsible for creation of Telengana state out of Andhra Pradesh .

Then there is Devangana kalita who works for gender rights. Similar to Sudha Bharadwaj, she had an opportunity to go to UK and settle here. She did her master’s and came back. And now she is in jail for the things that she studied. While she took part in Anti-CAA sittings, none of her speeches were incendiary but she still is locked up under UAPA (Unlawful Practises Act) . I could go on and on but at the moment these should suffice.

Petitions for Hate Speech which resulted in riots in Delhi are pending, Citizen’s Amendment Act (controversial) no hearings till date. All of the best has been explained in a newspaper article which articulates perhaps all that I wanted to articulate and more. It is and was amazing to see how in certain cases Article 32 is valid and in many it is not. Also a fair reading of Justice Bobde’s article tells you a lot how the SC is functioning. I would like to point out that barandbench along with livelawindia makes it easier for never non-lawyers and public to know how arguments are done in court, what evidences are taken as well as give some clue about judicial orders and judgements. Both of these resources are providing an invaluable service and more often than not, free of charge.

Student Suicide and High Cost of Education

For quite sometime now, the cost of education has been shooting up. While I have visited this topic earlier as well, recently a young girl committed suicide because she was unable to pay the fees as well as additional costs due to pandemic. Further investigations show that this is the case with many of the students who are unable to buy laptops. Now while one could think it is limited to one college then it would be wrong. It is almost across all India and this will continue for months and years. People do know that the pandemic is going to last a significant time and it would be a long time before R value becomes zero . Even the promising vaccine from Pfizer need constant refrigeration which is sort of next to impossible in India. It is going to make things very costly.

Last Nail on Indian Media

Just today the last nail on India has been put. Thankfully Freedom Gazette India did a much better job so just pasting that –

Information and Broadcasting Ministry bringing OTT services as well as news within its ambit.

With this, projects like Scam 1992, The Harshad Mehta Story or Bad Boy Billionaires:India, Test Case, Delhi Crime, Laakhon Mein Ek etc. etc. such kind of series, investigative journalism would be still-births. Many of these web-series also shared tales of woman empowerment while at the same time showed some of the hard choices that women had to contend to live with.

Even western media may be censored where it finds the political discourse not to its liking. There had been so many accounts of Mr. Ravish Kumar, the winner of Ramon Magsaysay, how in his shows the electricity was cut in many places. I too have been the victim when the BJP governed in Maharashtra as almost all Puneities experienced it. Light would go for just half or 45 minutes at the exact time.

There is another aspect to it. The U.S. elections showed how independent media was able to counter Mr. Trump’s various falsehoods and give rise to alternative ideas which lead the team of Bernie Sanders, Joe Biden and Kamala Harris, Biden now being the President-elect while Kamala Harris being the vice-president elect. Although the journey to the white house seems as tough as before. Let’s see what happens.

Hopefully 2021 will bring in some good news.

Update – On 27th November 2020 Martin who runs the planet got an e-mail/notice by a Mr. Nikhil Sethi who runs the wikibio.com property. Mr. Sethi asked to remove the link pointing Devangana Kalita from my blog post to his site as he has used the no follow link. On inquiring further, the gentleman stated that it is an ‘Updated mandate’ (his exact quote) from Google algorithm. To further understand the issue, I went to SERP as they are one of the more known ones on the subject. I also looked it up on Google as well. Found that the gentleman was BSing the whole time. The page basically talks about weightage of a page/site and authoritativeness which is known and yet highly contested ideas. In any case, the point for me was for whatever reason (could be fear, could be something else entirely), Mr. Sethi did not want me to link the content. Hence, I have complied above. I could have dragged it out but I do not wish Mr. Sethi any ill-being or/and further harm unduly and unintentionally caused by me. Hence, have taken down the link.

Cryptogram Indistinguishability Obfuscation

Quanta magazine recently published a breathless article on indistinguishability obfuscation — calling it the “‘crown jewel’ of cryptography” — and saying that it had finally been achieved, based on a recently published paper. I want to add some caveats to the discussion.

Basically, obfuscation makes a computer program “unintelligible” by performing its functionality. Indistinguishability obfuscation is more relaxed. It just means that two different programs that perform the same functionality can’t be distinguished from each other. A good definition is in this paper.

This is a pretty amazing theoretical result, and one to be excited about. We can now do obfuscation, and we can do it using assumptions that make real-world sense. The proofs are kind of ugly, but that’s okay — it’s a start. What it means in theory is that we have a fundamental theoretical result that we can use to derive a whole bunch of other cryptographic primitives.

But — and this is a big one — this result is not even remotely close to being practical. We’re talking multiple days to perform pretty simple calculations, using massively large blocks of computer code. And this is likely to remain true for a very long time. Unless researchers increase performance by many orders of magnitude, nothing in the real world will make use of this work anytime soon.

But but, consider fully homomorphic encryption. It, too, was initially theoretically interesting and completely impractical. And now, after decades of work, it seems to be almost just-barely maybe approaching practically useful. This could very well be on the same trajectory, and perhaps in twenty to thirty years we will be celebrating this early theoretical result as the beginning of a new theory of cryptography.

Krebs on SecurityConvicted SIM Swapper Gets 3 Years in Jail

A 21-year-old Irishman who pleaded guilty to charges of helping to steal millions of dollars in cryptocurrencies from victims has been sentenced to just under three years in prison. The defendant is part of an alleged conspiracy involving at least eight others in the United States who stand accused of theft via SIM swapping, a crime that involves convincing mobile phone company employees to transfer ownership of the target’s phone number to a device the attackers control.

Conor Freeman of Dublin took part in the theft of more than two million dollars worth of cryptocurrency from different victims throughout 2018. Freeman was named as a member of a group of alleged SIM swappers called “The Community” charged last year with wire fraud in connection with SIM swapping attacks that netted in excess of $2.4 million.

Among the eight others accused are three former wireless phone company employees who allegedly helped the gang hijack mobile numbers tied to their targets. Prosecutors say the men would identify people likely to have significant cryptocurrency holdings, then pay their phone company cohorts to transfer the victim’s mobile service to a new SIM card — the smart chip in each phone that ties a customer’s device to their number.

A fraudulent SIM swap allows the bad guys to intercept a target’s incoming phone calls and text messages. This is dangerous because a great many sites and services still allow customers to reset their passwords simply by clicking on a link sent via SMS. From there, attackers can gain access to any accounts that allow password resets via SMS or automated calls, from email and social media profiles to virtual currency trading platforms.

Like other accused members of The Community, Freeman was an active member of OGUsers, a forum that caters to people selling access to hijacked social media and other online accounts. But unlike others in the group, Freeman used his real name (username: Conor), and disclosed his hometown and date of birth to others on the forum. At least twice in the past few years OGUsers was hacked, and its database of profiles and user messages posted online.

According to a report in The Irish Times, Freeman spent approximately €130,000, which he had converted into cash from the stolen cryptocurrency. Conor posted on OGUsers that he spent approximately $14,000 on a Rolex watch. The rest was handed over to the police in the form of an electronic wallet that held the equivalent of more than $2 million.

The Irish Times says the judge in the case insisted the three-year sentence was warranted in order to deter the defendant and to prevent others from following in his footsteps. The judge said stealing money of this order is serious because no one can know the effect it will have on the victim, noting that one victim’s life savings were taken and the proceeds of the sale of his house were stolen.

One way to protect your accounts against SIM swappers is to remove your phone number as a primary or secondary authentication mechanism wherever possible. Many online services require you to provide a phone number upon registering an account, but in many cases that number can be removed from your profile afterwards.

It’s also important for people to use something other than text messages for two-factor authentication on their email accounts when stronger authentication options are available. Consider instead using a mobile app like Authy, Duo, or Google Authenticator to generate the one-time code. Or better yet, a physical security key if that’s an option.

Cory DoctorowThe Attack Surface Lectures: Little Revolutions

The Attack Surface Lectures were a series of eight panel discussions on the themes in my novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “Little Revolutions,” hosted by Skylight Books in Los Angeles, with guest-hosts Tochi Onyebuchi and Bethany C. Morrow. It was recorded on October 21, 2020.

Here is a link to this presentation in Skylight’s archive of author events. Please consider subscribing to Skylight’s feed of these videos to see other outstanding author events!

MP3

Worse Than FailureError'd: Reduced Complexity, Increased Errors

"I tried a more complex password and got the same error message, but after trying with a shorter password, it let me through!" wrote Sameer K.

 

Lucas T. writes, "Translation: 'Dear ladies and gentlemen, because of an internet failure (some identifying info here), the electronic signature and the owl are unavailable. The issue is being worked on. Kind regards, your application support.' Well, isn't this just great. How exactly am I supposed to work without the owl!?"

 

"I was looking into time issues regarding backing up my Mac with TimeMachine and saw that it REALLY MUST BE A TIME MACHINE AFTER ALL!!" Mike S. wrote.

 

Joel B. writes, "3D printing my desserts? Sign. Me. Up."

 

"Huh. Apparently someone hacked my Facebook account before I was born," wrote Rob.

 

[Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!

,

Cryptogram Symantec Reports on Cicada APT Attacks against Japan

Symantec is reporting on an APT group linked to China, named Cicada. They have been attacking organizations in Japan and elsewhere.

Cicada has historically been known to target Japan-linked organizations, and has also targeted MSPs in the past. The group is using living-off-the-land tools as well as custom malware in this attack campaign, including a custom malware — Backdoor.Hartip — that Symantec has not seen being used by the group before. Among the machines compromised during this attack campaign were domain controllers and file servers, and there was evidence of files being exfiltrated from some of the compromised machines.

The attackers extensively use DLL side-loading in this campaign, and were also seen leveraging the ZeroLogon vulnerability that was patched in August 2020.

Interesting details about the group’s tactics.

News article.

Cryptogram The US Military Buys Commercial Location Data

Vice has a long article about how the US military buys commercial location data worldwide.

The U.S. military is buying the granular movement data of people around the world, harvested from innocuous-seeming apps, Motherboard has learned. The most popular app among a group Motherboard analyzed connected to this sort of data sale is a Muslim prayer and Quran app that has more than 98 million downloads worldwide. Others include a Muslim dating app, a popular Craigslist app, an app for following storms, and a “level” app that can be used to help, for example, install shelves in a bedroom.

This isn’t new, this isn’t just data of non-US citizens, and this isn’t the US military. We have lots of instances where the government buys data that it cannot legally collect itself.

Some app developers Motherboard spoke to were not aware who their users’ location data ends up with, and even if a user examines an app’s privacy policy, they may not ultimately realize how many different industries, companies, or government agencies are buying some of their most sensitive data. U.S. law enforcement purchase of such information has raised questions about authorities buying their way to location data that may ordinarily require a warrant to access. But the USSOCOM contract and additional reporting is the first evidence that U.S. location data purchases have extended from law enforcement to military agencies.

Planet DebianMolly de Blanc: Transparency

Technology must be transparent in order to be knowable. Technology must be knowable in order for us to be able to consent to it in good faith. Good faith informed consent is necessary to preserving our (digital) autonomy.

Let’s now look at this in reverse, considering first why informed consent is necessary to our digital autonomy.

Let’s take the concept of our digital autonomy as being one of the highest goods. It is necessary to preserve and respect the value of each individual, and the collectives we choose to form. It is a right to which we are entitled by our very nature, and a prerequisite for building the lives we want, that fulfill us. This is something that we have generally agreed on as important or even sacred. Our autonomy, in whatever form it takes, in whatever part of our life it governs, is necessary and must be protected.

One of the things we must do in order to accomplish this is to build a practice and culture of consent. Giving consent — saying yes — is not enough. This consent must come from a place of understand to that which one is consenting. “Informed consent is consenting to the unknowable.”(1)

Looking at sexual consent as a parallel, even when we have a partner who discloses their sexual history and activities, we cannot know whether they are being truthful and complete. Let’s even say they are and that we can trust this, there is a limit to how much even they know about their body, health, and experience. They might not know the extent of their other partners’ experience. They might be carrying HPV without symptoms; we rarely test for herpes.

Arguably, we have more potential to definitely know what is occurring when it comes to technological consent. Technology can be broken apart. We can share and examine code, schematics, and design documentation. Certainly, lots of information is being hidden from us — a lot of code is proprietary, technical documentation unavailable, and the skills to process these things is treated as special, arcane, and even magical. Tracing the resource pipelines for the minerals and metals essential to building circuit boards is not possible for the average person. Knowing the labor practices of each step of this process, and understanding what those imply for individuals, societies, and the environments they exist in seems improbable at best.

Even though true informed consent might not be possible, it is an ideal towards which we must strive. We must work with what we have, and we must be provided as much as possible.

A periodic conversation that arises in the consideration of technology rights is whether companies should build backdoors into technology for the purpose of government exploitation. A backdoor is a hidden vulnerability in a piece of technology that, when used, would afford someone else access to your device or work or cloud storage or whatever. As long as the source code that powers computing technology is proprietary and opaque, we cannot truly know whether backdoors exist and how secure we are in our digital spaces and even our own computers, phones, and other mobile devices.

We must commit wholly to transparency and openness in order to create the possibility of as-informed-as-possible consent in order to protect our digital autonomy. We cannot exist in a vacuum and practical autonomy relies on networks of truth in order to provide the opportunity for the ideal of informed consent. These networks of truth are created through the open availability and sharing of information, relating to how and why technology works the way it does.

(1) Heintzman, Kit. 2020.

Cory DoctorowThe Attack Surface Lectures: Cyberpunk and Post Cyberpunk

The Attack Surface Lectures were a series of eight panel discussions on the themes in Cory Doctorow’s novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “Cyberpunk and Post-Cyberpunk,” hosted by Anderson’s Books in Napierville, IL, with guest-hosts Bruce Sterling and Christopher Brown. It was recorded on October 19, 2020.

Here is the original Youtube link for this program. Please consider subscribing to Anderson’s Youtube channel for access to all their outstanding author events!

MP3

Planet DebianSteinar H. Gunderson: COVID-19 vaccine confidence intervals

I keep hearing about new vaccines being “at least 90% effective”, “94.5% effective”, “92% effective” etc... and that's obviously really good news. But is that a point estimate, or a confidence interval? Does 92% mean “anything from 70% to 99%”, given that n=20?

I dusted off the memories of how bootstrapping works (I didn't want to try to figure out whether one could really approximate using the Cauchy distribution or not) and wrote some R code. Obviously, don't use this for medical or policy decisions since I don't have a background in neither medicine nor medical statistics. But it's uplifting results nevertheless; here from the Pfizer/BioNTech data that I could find:

> N <- 43538 / 2
> infected_vaccine <- c(rep(1, times = 8), rep(0, times=N-8))
> infected_placebo <- c(rep(1, times = 162), rep(0, times=N-162))
>
> infected <- c(infected_vaccine, infected_placebo)
> vaccine <- c(rep(1, times=N), rep(0, times=N))
> mydata <- data.frame(infected, vaccine)
>
> library(boot)
> rsq <- function(data, indices) {
+   d <- data[indices,]
+   num_infected_vaccine <- sum(d[which(d$vaccine == 1), ]$infected)
+   num_infected_placebo <- sum(d[which(d$vaccine == 0), ]$infected)
+   return(1.0 - num_infected_vaccine / num_infected_placebo)
+ }
>
> results <- boot(data=mydata, statistic=rsq, R=1000)
> results

ORDINARY NONPARAMETRIC BOOTSTRAP


Call:
boot(data = mydata, statistic = rsq, R = 1000)


Bootstrap Statistics :
     original       bias    std. error
t1* 0.9506173 -0.001428342  0.01832874
> boot.ci(results, type="perc")
BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
Based on 1000 bootstrap replicates

CALL :
boot.ci(boot.out = results, type = "perc")

Intervals :
Level     Percentile
95%   ( 0.9063,  0.9815 )
Calculations and Intervals on Original Scale

So that would be a 95% CI of between 90.6% and 98.1% effective, roughly. The confidence intervals might be slightly too wide, since I didn't have enough RAM (!) to run the bootstrap calibrated ones (BCa).

Again, take it with a grain of salt. Corrections welcome. :-)

Planet DebianDaniel Silverstone: Withdrawing Gitano from support

Unfortunately, in Debian in particular, libgit2 is undergoing a transition which is blocked by gall. Despite having had over a month to deal with this, I've not managed to summon the tuits to update Gall to the new libgit2 which means, nominally, I ought to withdraw it from testing and possibly even from unstable given that I'm not really prepared to look after Gitano and friends in Debian any longer.

However, I'd love for Gitano to remain in Debian if it's useful to people. Gall isn't exactly a large piece of C code, and so probably won't be a huge job to do the port, I simply don't have the desire/energy to do it myself.

If someone wanted to do the work and provide a patch / "pull request" to me, then I'd happily take on the change and upload a new package, or if someone wanted to NMU the gall package in Debian I'll take the change they make and import it into upstream. I just don't have the energy to reload all that context and do the change myself.

If you want to do this, email me and let me know, so I can support you and take on the change when it's done. Otherwise I probably will go down the lines of requesting Gitano's removal from Debian in the next week or so.

Worse Than FailureCodeSOD: Prepend Eternal

Octavia inherited a decade old pile of C#, and the code quality was pretty much what one would expect from a decade old pile that hadn't seen any real refactoring: nothing but spaghetti. Worse, it also had an "inner platform" problem, as everything they put in their API could conceivably be called by their customers' own "customizations".

One small block caught her eye, as suspicious:

public void SomeFunctionality { // Other functionality here int x = SomeIntMethod(); String y = PrependZeros(x.ToString()); // Do other things with y here }

That call to PrependZeros looked… suspicious. For starters, how many zeroes? It clearly was meant to padd to a certain length, but what?

public String PrependZeros(string n) { if (n.Length == 1) { return "00" + n; } else if (n.Length == 2) { return "0" + n; } else { return n; } }

We've reimplemented one of the built-in formatting methods, badly, which isn't particularly unusual to see. This method clearly doesn't care if it gets a number that's greater than 3 digits, which maybe that's the correct behavior? Inside the codebase, this would be trivial for Octavia to remove, as its only invoked that one time.

Except she can't do that. Because the original developer placed the code in the namespace accessible to customer customizations. Which means some unknown number of customers might have baked this method into their own code. Octavia can't rename it, can't remove it, and there's no real point in re-implementing it. Maybe someday, they'll ship a new version and release some breaking changes, but for now, PrependZeros must live on, just in case a customer is using it.

Every change breaks somebody's workflow.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

,

Kevin RuddWashington Post: How Biden Could Get the US to lead on climate again

Image: Gage Skidmore/Flickr

By Kevin Rudd and Bill Hare

Published by the Washington Post, 18 November 2020

President-elect Joe Biden has pledged to begin the process of rejoining the Paris agreement with the stroke of a pen. As important as this will be, a more significant demonstration of Biden’s determination to rejoin the international fight against climate change will come in the form of the target the United States then puts on the table as its contribution toward meeting the goals of the agreement.

In 2015, the Obama administration committed to reduce U.S. emissions by 26 to 28 percent below 2005 levels by 2025. The beating heart of the Paris agreement is its requirement that countries ratchet up their ambition every five years. And even though the rollbacks of the Trump administration mean the United States will now miss this target, Biden has committed to quickly put the country on an even steeper trajectory of cuts by 2030, en route to achieving net zero emissions by 2050.

The new 2030 target will be an early signal of where the rubber hits the road internationally for the Biden administration. And, along with the administration’s willingness to use trade policy as a lever for climate action, it will be a key arrow in the United States’ diplomatic quiver for rallying other major emitters to increase their own targets.

Analysis by the Asia Society Policy Institute and Climate Analytics shows that this target is likely to be somewhere between a 38 to 54 percent cut in emissions below 2005 levels by 2030. This is based on an assessment of the Biden campaign’s clean energy and climate package, including his commitment to decarbonize the U.S. energy system by 2035 — the biggest element of the package by far. While this will not be enough to immediately align the United States with the Paris agreement’s goal of limiting global temperature increases to 1.5 degrees Celsius, it would represent a huge first restorative step within a relatively short period.

Importantly, given uncertainties about the makeup of the Senate, the report also finds that around half of Biden’s plans could be achieved without the need for congressional support. However, the possibility of protracted legal challenges seems likely, not least given the conservative balance of courts across the United States.

In developing the new target, the Biden administration will want to take its time to gauge the current state of U.S. efforts across the board, assess the level of congressional support for different measures and seek input from business, industry and civil society. However, it will be essential for the United States to put an ambitious new target on the table well before the U.N. Climate Change Conference in Glasgow, Scotland, in November next year, and ideally around the time of the Group of Seven and Group of 20 summits, which will be key steppingstone moments.

One of the best options could be for the United States to use next September’s U.N. General Assembly as an opportunity to announce its new target and convene the leaders of major emitters at the same time — fulfilling a commitment Biden made during the campaign trail.

Either way, one country will be watching Biden’s decisions more closely than any other: China.

While President Xi Jinping’s announcement in September that China will reach carbon neutrality was a game changer, whether that is seen to cover all greenhouse gases is just as significant. China has also announced that it will update its own 2030 target for curbing emissions before the U.N. climate conference — but when, and with what level of ambition, are important issues.

All this remains the subject of live political debate in Beijing in the wake of the Communist Party leadership’s conference last month and as preparations continue on its all-important Five-Year Plan. A recent landmark effort led by China’s long-serving climate envoy, Xie Zhenhua, confirmed this and recommended that the country accelerate decarbonization in the short term, including by peaking emissions around 2025. This makes sense, but to avoid higher cumulative emissions over time, even deeper cuts will be needed in the medium term, including through halting the construction of all new coal-fired power plants followed by a complete phase out of all coal power by 2040.

We should expect further details of China’s thinking as early as December when the secretary general convenes a summit to celebrate the five-year anniversary of the Paris agreement.

Taken together, Xi’s new goals and Biden’s plans have already wiped off 0.3 to 0.4 degrees Celsius from forecasts for global warming this century. With recent announcements by other big emitters ­— Japan and South Korea — for the first time, the Paris agreement’s 1.5 degrees Celsius temperature limit seems within reach. By any measure, this is a very big deal.

Whether the rest of the world now also chooses to ramp up climate action will in large part rest on the decisions taken in Beijing and Washington, individually and jointly. Last week marked the anniversary of the first U.S.-China joint announcement on climate change in 2014. And six years on, the world will be watching just as closely at what the world’s two largest emitters are able to do.

Kevin Rudd, a former prime minister of Australia, is president of the Asia Society Policy Institute. Bill Hare is chief executive and senior scientist at Climate Analytics.

The post Washington Post: How Biden Could Get the US to lead on climate again appeared first on Kevin Rudd.

Krebs on SecurityTrump Fires Security Chief Christopher Krebs

President Trump on Tuesday fired his top election security official Christopher Krebs (no relation). The dismissal came via Twitter two weeks to the day after Trump lost an election he baselessly claims was stolen by widespread voting fraud.

Chris Krebs. Image: CISA.

Krebs, 43, is a former Microsoft executive appointed by Trump to head the Cybersecurity and Infrastructure Security Agency (CISA), a division of the U.S. Department of Homeland Security. As part of that role, Krebs organized federal and state efforts to improve election security, and to dispel disinformation about the integrity of the voting process.

Krebs’ dismissal was hardly unexpected. Last week, in the face of repeated statements by Trump that the president was robbed of re-election by buggy voting machines and millions of fraudulently cast ballots, Krebs’ agency rejected the claims as “unfounded,” asserting that “the November 3rd election was the most secure in American history.”

In a statement on Nov. 12, CISA declared “there is no evidence that any voting system deleted or lost votes, changed votes, or was in any way compromised.”

But in a tweet Tuesday evening, Trump called that assessment “highly inaccurate,” alleging there were “massive improprieties and fraud — including dead people voting, Poll watchers not allowed into polling locations, ‘glitches’ in the voting machines that changed votes from Trump to Biden, late voting, and many more.”

Twitter, as it has done with a remarkable number of the president’s tweets lately, flagged the statements as disputed.

By most accounts, Krebs was one of the more competent and transparent leaders in the Trump administration. But that same transparency may have cost him his job: Krebs’ agency earlier this year launched “Rumor Control,” a blog that sought to address many of the conspiracy theories the president has perpetuated in recent days.

Sen. Richard Burr, a Republican from North Carolina, said Krebs had done “a remarkable job during a challenging time,” and that the “creative and innovative campaign CISA developed to promote cybersecurity should serve as a model for other government agencies.”

Sen. Angus King, an Independent from Maine and co-chair of a commission to improve the nation’s cyber defense posture, called Krebs “an incredibly bright, high-performing, and dedicated public servant who has helped build up new cyber capabilities in the face of swiftly-evolving dangers.”

“By firing Mr. Krebs for simply doing his job, President Trump is inflicting severe damage on all Americans – who rely on CISA’s defenses, even if they don’t know it,” King said in a written statement. “If there’s any silver lining in this unjust decision, it’s this: I hope that President-elect Biden will recognize Chris’s contributions, and consult with him as the Biden administration charts the future of this critically important agency.”

KrebsOnSecurity has received more than a few messages these past two weeks from readers who wondered why the much-anticipated threat from Russian or other state-sponsored hackers never appeared to materialize in this election cycle.

That seems a bit like asking why the year 2000 came to pass with very few meaningful disruptions from the Y2K computer date rollover problem. After all, in advance of the new millennium, the federal government organized a series of task forces that helped coordinate readiness for the changeover, and to minimize the impact of any disruptions.

But the question also ignores a key goal of previous foreign election interference attempts leading up to the 2016 U.S. presidential and 2018 mid-term elections. Namely, to sow fear, uncertainty, doubt, distrust and animosity among the electorate about the democratic process and its outcomes.

To that end, it’s difficult to see how anyone has done more to advance that agenda than President Trump himself, who has yet to concede the race and continues to challenge the result in state courts and in his public statements.

Kevin RuddBloomberg New Economy Forum: US-China Outlook

INTERVIEW VIDEO
BLOOMBERG NEW ECONOMY FORUM
18 NOVEMBER 2020

The post Bloomberg New Economy Forum: US-China Outlook appeared first on Kevin Rudd.

Cory DoctorowThe Attack Surface Lectures: Intersectionality: Race, Surveillance, and Tech and Its History

The Attack Surface Lectures were a series of eight panel discussions on the themes in my’s novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “​​Intersectionality: Race, Surveillance, and Tech and Its History,” hosted by The Booksmith in San Francisco, with guest-hosts Malkia Devich-Cyril and Meredith Whittaker​​. It was recorded on October 15, 2020.

Here is the original Youtube link for this program. Please consider subscribing to The Booksmith’s Youtube channel for access to all their outstanding author events!

MP3

Charles StrossThe Laundry Files: an updated chronology

I've been writing Laundry Files stories since 1999, and there's now about 1.4 million words in that universe. That's a lot of stuff: a typical novel these days is 100,000 words, but these books trend long, and this count includes 11 novels (of which, #10 comes out later this month) and some shorter work. It occurs to me that while some of you have been following them from the beginning, a lot of people come to them cold in the shape of one story or another.

So below the fold I'm going to explain the Laundry Files time line, the various sub-series that share the setting, and give a running order for the series—including short stories as well as novels.

(The series title, "The Laundry Files", was pinned on me by editorial fiat at a previous publisher whose policy was that any group of 3 or more connected novels had to have a common name. It wasn't my idea: my editor at the time also published Jim Butcher, and Bob—my sole protagonist at that point in the series—worked for an organization disparagingly nicknamed "the Laundry", so the inevitable happened. Using a singular series title gives the impression that it has a singular theme, which would be like calling Terry Pratchett's Discworld books as "the Unseen University series". Anyway ...)

TLDR version: If you just want to know where to start reading, pick one of: The Atrocity Archives, The Rhesus Chart, The Nightmare Stacks, or Dead Lies Dreaming. These are all safe starting points for the series, that don't require prior familiarity. Other books might leave you confused if you dive straight in, so here's an exhaustive run-down of all the books and short stories.




Typographic conventions: story titles are rendered in italics (like this). Book titles are presented in boldface (thus).

Publication dates are presented like this: (pub: 2016). The year in which a story is set is presented like so: (set: 2005).

The list is sorted in story order rather than publication order.




The Atrocity Archive (set: 2002; pub: 2002-3)

  • The short novel which started it all. Originally published in an obscure Scottish SF digest-format magazine called Spectrum SF, it ran from 2002 to 2003, and introduced our protagonist Bob Howard, his (eventual) love interest Mo O'Brien, and a bunch of eccentric minor characters and tentacled horrors. Is a kinda-sorta tribute to spy thriller author Len Deighton.

The Concrete Jungle (set: 2003: pub: see below)

  • Novella, set a year after The Atrocity Archive, in which Bob is awakened in the middle of the night to go and count the concrete cows in Milton Keynes. Winner of the 2005 Hugo award for best SF/F novella.

The Atrocity Archives (set 2002-03, pub: 2003 (hbk), 2006 (trade ppbk))

  • Start reading here! A smaller US publisher, Golden Gryphon, liked The Atrocity Archive and wanted to publish it, but considered it to be too short on its own. So The Concrete Jungle was written, and along with an afterword they were published together as a two-story collection/episodic novel, The Atrocity Archives (note the added 's' at the end). A couple of years later, Ace (part of Penguin group) picked up the US trade and mass market paperback rights and Orbit published it in the UK. (Having won a Hugo award in the meantime really didn't hurt; it's normally quite rare for a small press item such as TAA to get picked up and republished like this.)

The Jennifer Morgue (set: 2005, pub: 2007 (hbk), 2008 (trade ppbk))

  • Golden Gryphon asked for a sequel, hence the James Bond episode in what was now clearly going to be a trilogy of comedy Lovecraftian/spy books. Note that it's riffing off the Broccoli movie franchise version of Bond, not Iain Fleming's original psychopathic British government assassin. Orbit again took UK rights, while Ace picked up the paperbacks. Because I wanted to stick with the previous book's two-story format, I wrote an extra short story:

Pimpf (set: 2006, pub: collected in The Jennifer Morgue)

  • A short story set in what I think of as the Chibi-Laundry continuity; Bob ends up inside a computer running a Neverwinter Nights server (hey, this was before World of Warcraft got big). Chibi-Laundry stories are self-parodies and probably shouldn't be thought of as canonical. (Ahem: there's a big continuity blooper tucked away in this one what comes back to bite me in later books because I forgot about it.)

Down on the Farm (novelette: set 2007, pub. 2008, Tor.com)

  • Novelette: Bob has to investigate strange goings-on at a care home for Laundry agents whose minds have gone. Introduces Krantzberg Syndrome, which plays a major role later in the series.

Equoid (novella: set 2007, pub: 2013, Tor.com)

  • A novella set between The Jennifer Morgue and The Fuller Memorandum; Bob is married to Mo and working for Iris Carpenter. Bob learns why Unicorns are Bad News. Won the 2014 Hugo award for best SF/F novella. Also published as the hardback novella edition Equoid by Subterranean Press.

The Fuller Memorandum (set: 2008, pub: 2010 (US hbk/UK ppbk))

  • Third novel, first to be published in hardback by Ace, published in paperback in the UK by Orbit. The title is an intentional call-out to Adam Hall (aka Elleston Trevor), author of the Quiller series of spy thrillers—but it's actually an Anthony Price homage. This is where we begin to get a sense that there's an overall Laundry Files story arc, and where I realized I wasn't writing a trilogy. Didn't have a short story trailer or afterword because I flamed out while trying to come up with one before the deadline. Bob encounters skullduggery within the organization and has to get to the bottom of it before something really nasty happens: also, what and where is the misplaced "Teapot" that the KGB's London resident keeps asking him about?

Overtime (novelette: set 2009, pub 2009, Tor.com)

  • A heart-warming Christmas tale of Terror. Shortlisted for the Hugo award for best novelette, 2010.

Three Tales from the Laundry Files (ebook-only collection)

  • Collection consisting of Down on the Farm, Overtime, and Equoid published the Tor.com as an ebook.

The Apocalypse Codex (set: 2010, pub: 2012 (US hbk/UK ppbk))

  • Fourth novel, and a tribute to the Modesty Blaise comic strip and books by Peter O'Donnell. A slick televangelist is getting much to cosy with the Prime Minister, and the Laundry—as a civil service agency—is forbidden from investigating. We learn about External Assets, and Bob gets the first inkling that he's being fast-tracked for promotion. Won the Locus Award for best fantasy novel in 2013.

A Conventional Boy (set: ~2011-12, not yet written)

  • Projected interstitial novella, introducing Derek the DM (The Nightmare Stacks) and Camp Sunshine (The Delirium Brief). Not yet written.

The Rhesus Chart (set: spring 2013, pub: 2014 (US hbk/UK hbk))

  • Fifth novel, a new series starting point if you want to bypass the early novels. First of a new cycle remixing contemporary fantasy sub-genres (I got bored with British spy thriller authors). Subject: Banking, Vampires, and what happens when an agile programming team inside a merchant bank develops PHANG syndrome. First to be published in hardcover in the UK by Orbit.

  • Note that the books are now set much closer together. This is a key point: the world of the Laundry Files has now developed its own parallel and gradually diverging history as the supernatural incursions become harder to cover up. Note also that Bob is powering up (the Bob of The Atrocity Archive wouldn't exactly be able to walk into a nest of vampires and escape with only minor damage to his dignity). This is why we don't see much of Bob in the next two novels.

The Annihilation Score (set: summer/autumn 2013, pub: 2015 (US hbk/UK ppbk))

  • Sixth novel, first with a non-Bob viewpoint protagonist—it's told by Mo, his wife, and contains spoilers for The Rhesus Chart. Deals with superheroes, mid-life crises, nervous breakdowns, and the King in Yellow. We're clearly deep into ahistorical territory here as we have a dress circle box for the very last Last Night of the Proms, and Orbit's lawyers made me very carefully describe the female Home Secretary as clearly not being one of her non-fictional predecessors, not even a little bit.

Escape from Puroland (set: March-April 2014, pub: summer 2021, forthcoming)

  • Interstitial novella, explaining why Bob wasn't around in the UK during the events described in The Nightmare Stacks. He was on an overseas liason mission, nailing down the coffin lid on one of Angleton's left-over toxic waste sites—this time, it's near Tokyo.

The Nightmare Stacks (set: March-April 2014, pub: June 2016 (US hbk/UK ppbk))

  • Seventh novel, and another series starting point if you want to dive into the most recent books in the series. Viewpoint character: Alex the PHANG. Deals with, well ... the Laundry has been so obsessed by CASE NIGHTMARE GREEN that they're almost completely taken by surprise when CASE NIGHTMARE RED happens. Implicitly marks the end of the Masquerade. Features a Maniac Pixie Dream Girl and the return of Bob's Kettenkrad from The Atrocity Archive. Oh, and it also utterly destroys the major British city I grew up in, because revenge is a dish best eaten cold.

The Delirium Brief (set: May-June 2014, pub: June 2017 (US hbk/UK ppbk))

  • Eighth novel, primary viewpoint character: Bob again, but with an ensemble of other viewpoints cropping up in their own chapters. And unlike the earlier Bob books it no longer pastiches other works or genres. Deals with the aftermath of The Nightmare Stacks; opens with Bob being grilled live on Newsnight by Jeremy Paxman and goes rapidly downhill from there. (I'm guessing that if the events of the previous novel had just taken place, the BBC's leading current affairs news anchor might have deferred his retirement for a couple of months ...)

The Labyrinth Index (set: winter 2014/early 2015, pub: October 2018, (US hbk/UK ppbk))

  • Ninth novel, viewpoint character: Mhari, working for the New Management in the wake of the drastic governmental changes that took place at the end of "The Delirium Brief". The shit has well and truly hit the fan on a global scale, and the new Prime Minister holds unreasonable expectations ...

Dead Lies Dreaming (set: December 2016: pub: Oct 2020 (US hbk/UK hbk))

  • New spin-off series, new starting point! The marketing blurb describes it as "book 10 in the Laundry Files" but by the time this book is set—after CASE NIGHTMARE GREEN and the end of the main Laundry story arc (some time in 2015-16) the Laundry no longer exists. We meet a cast of entirely new characters, civilians (with powers) living under the aegis of the New Management, ruled by his Dread Majesty, the Black Pharaoh. The start of a new trilogy, Dead Lies Dreaming riffs heavily off "Peter and Wendy", the original grimdark version of Peter Pan (before Walt Disney made him twee).

In His House (set: December 2016, pub: probably 2022)

  • Second book in the Dead Lies Dreaming trilogy: continues the story, riffs off Sweeney Todd and Mary Poppins—again: the latter was much darker than the Disney musical implies. (The book is written, but COVID19 has done weird things to publishers' schedules and it's provisionally in the queue behind Invisible Sun, the final Empire Games book, which is due out in September 2021.)

Bones and Nightmares (set: December 2016 and summer of 1820, pub: possibly 2023)

  • Third book in the Dead Lies Dreaming trilogy: finishes the story, riffs off The Prisoner and Jane Austen: also Kingsley's The Water Babies (with Deep Ones). In development.

Further novels are planned but not definite: there need to be 1-2 more books to finish the main Laundry Files story arc with Bob et al, filling in the time line before Dead Lies Dreaming, but the Laundry is a civil service security agency and the current political madness gripping the UK makes it really hard to satirize HMG, so I'm off on a side-quest following the tribulations of Imp, Eve, Wendy, and the gang (from Dead Lies Dreaming) until I figure out how to get back to the Laundry proper.




That's all for now. I'll attempt to update this entry as I write/publish more material.

Worse Than FailureBig Iron

Skill which you don’t use regularly can get rusty. It might not take too much to get the rust off, and remind yourself of what you’re supposed to be doing, but the process of remembering what you’re supposed to do can get a little… damaging.

Lesli spent a big chunk of her career doing IT for an insurance company. They were a conservative company in a conservative industry, which meant they were still rolling out new mainframes in the early 2000s. “Big iron” was the future for insurance.

Until it wasn’t, of course. Lesli was one of the “x86 kids”, part of the team that started with desktop support and migrated into running important services on commodity hardware.

The “big iron” mainframe folks, led by Erwin, watched the process with bemusement. Erwin had joined the company back when they had installed their first S/370 mainframe, and had a low opinion of the direction the future was taking. Watching the “x86 kids” struggle with managing growing storage needs gave him a sense of vindication, as the mainframe never had that problem.

The early x86 rollouts started in 2003, and just used internal disks. At first, only the mail server had anything as fancy as a SCSI RAID array. But as time wore on, the storage needs got harder to manage, and eventually the “x86 kids” rolled out a SAN.

The company bought a second-hand disk array and an expensive support contract with the vendor. It was stuffed with 160GB disks, RAIDed together into about 3TB of storage- a generous amount for 2004. Gradually every service moved onto the SAN, starting with file servers and moving on to email and even experiments with virtualization.

Erwin just watched, and occasionally commented about how they’d solved that problem “on big iron” a generation ago.

Storage needs grew, and more disks got crammed into the array. More disks meant more chances for failures, and each time a disk died, the vendor needed to send out a support tech to replace it. That wasn’t so bad when it was once a quarter, but when disks needed to be replaced twice a month, the hassle of getting a tech on-site, through the multiple layers of security, and into the server room became a burden.

“Hey,” Lesli’s boss suggested, circa late 2005. “Why don’t we just do it ourselves? They can just courier over the new drives, and we can swap and initialize the disk ourselves.”

Everyone liked that idea. After a quick round of training and confirmation that it was safe, that became the process. The support contract was updated, and this became the process.

Until 2009. The world had changed, and Erwin’s beloved “big iron” was declining in relevance. Many of his peers had retired, but he planned to stick it out for a few more years. As the company retired the last mainframe, they needed to reorganize IT, and that meant all the mainframe operators were now going to be server admins. Erwin was put in charge of the storage array.

The good news was that everyone decided to be cautious. Management didn’t want to set Erwin up for failure. Erwin, who frequently wore both a belt and suspenders, didn’t want to take any risks. The support contract was being renegotiated, so the vendor wanted to make sure they looked good. Everyone was ready to make the transition successful.

The first time a disk failed under Erwin’s stewardship, the vendor sent a technician. While Erwin would do all the steps required, the technician was there to train and supervise.

It started well. “You’ll see a red light on the failed disk,” the technician said.

Erwin pointed at a red light. “Like this?”

“Yes, that exactly. Now you’ll need to replace that with the new one.”

Erwin didn’t move. “And I do that how? Let’s go step-by-step.”

The tech started to explain, but went too fast for Erwin’s tastes. Erwin stopped them, and forced them to slow it down. After each step, Erwin paused to confirm it was correct, and note down what, exactly, he had done.

This turned a normally quick process into a bit of a marathon. The marathon got longer, as the technician hadn’t done this for a few years, and was a bit fuzzy on a few of the steps for this specific array, and had to correct themselves- and Erwin had to update his notes. After what felt like too much time, they closed in on the last few steps.

“Okay,” the tech said, “so you pull up a web browser, go to the admin page. Now, login. Great, hit ‘re-initialize’.”

Erwin followed the steps. “It’s warning me about possible data loss, and wants me to confirm by typing in the word ‘yes’?”

“Yeah, sure, do that,” the tech said.

Erwin did.

The tech thought the work was done, but Erwin had more questions. Since the tech was here, Erwin was going to pick their brain. Which was good, because that meant the tech was still on site when every service failed. From the domain service to SharePoint, from the HR database to the actuarial modeling backend, everything which touched the SAN was dead.

“What happened,” Erwin demanded of the tech.

“I don’t know! Something else must have failed.”

Erwin grabbed the tech, Lesli, and the other admins into a conference room. The tech was certain it couldn’t be related to what they had done, so Erwin escalated to the vendor’s phone support. He bulled through the first tier, pointing out they already had a tech onsite, and got to one of the higher-up support reps.

Erwin pulled out his notes, and in detail, recounted every step he had performed. “Finally, I clicked re-initialize.”

“Oh no!” the support rep said. “You don’t want to do that. You want to initialize the disk, not re-initialize. That re-inits the whole array. That’s why there’s a confirmation step, where you have to type ‘yes’.”

“The on-site tech told me to do exactly that.”

The on-site tech experience what must have been the most uncomfortable silence of their career.

“Oh, well, I’m sorry to hear that,” the support rep said. “That deletes all the header information on the array. The data’s still technically on the disks, but there’s no way to get at it. You’ll need to finish formatting and then recover from backup. And ah… can you take me off speaker and put the on-site tech on the line?”

Erwin handed the phone over to the tech, then rounded up the admins. They were going to have a long day ahead getting the disaster fixed. No one was in the room to hear what the support rep said to the tech. When it was over, the tech scrambled out of the office like the building was on fire, never to be heard from again.

In their defense, however, it had been a few years since they’d done the process themselves. They were a bit rusty.

Speaking of rusty, while Erwin continued to praise his “big iron” as being in every way superior to this newfangled nonsense, he stuck around for a few more years. In that time, he proved that he might never be the fastest admin, but he was the most diligent, cautious, and responsible.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

Cryptogram Michael Ellis as NSA General Counsel

Over at Lawfare, Susan Hennessey has an excellent primer on how Trump loyalist Michael Ellis got to be the NSA General Counsel, over the objections of NSA Director Paul Nakasone, and what Biden can and should do about it.

While important details remain unclear, media accounts include numerous indications of irregularity in the process by which Ellis was selected for the job, including interference by the White House. At a minimum, the evidence of possible violations of civil service rules demand immediate investigation by Congress and the inspectors general of the Department of Defense and the NSA.

The moment also poses a test for President-elect Biden’s transition, which must address the delicate balance between remedying improper politicization of the intelligence community, defending career roles against impermissible burrowing, and restoring civil service rules that prohibit both partisan favoritism and retribution. The Biden team needs to set a marker now, to clarify the situation to the public and to enable a new Pentagon general counsel to proceed with credibility and independence in investigating and potentially taking remedial action upon assuming office.

The NSA general counsel is not a Senate-confirmed role. Unlike the general counsels of the CIA, Pentagon and Office of the Director of National Intelligence (ODNI), all of which require confirmation, the NSA’s general counsel is a senior career position whose occupant is formally selected by and reports to the general counsel of the Department of Defense. It’s an odd setup — ­and one that obscures certain realities, like the fact that the NSA general counsel in practice reports to the NSA director. This structure is the source of a perennial legislative fight. Every few years, Congress proposes laws to impose a confirmation requirement as more appropriately befits an essential administration role, and every few years, the executive branch opposes those efforts as dangerously politicizing what should be a nonpolitical job.

While a lack of Senate confirmation reduces some accountability and legislative screening, this career selection process has the benefit of being designed to eliminate political interference and to ensure the most qualified candidate is hired. The system includes a complex set of rules governing a selection board that interviews candidates, certifies qualifications and makes recommendations guided by a set of independent merit-based principles. The Pentagon general counsel has the final call in making a selection. For example, if the panel has ranked a first-choice candidate, the general counsel is empowered to choose one of the others.

Ryan Goodman has a similar article at Just Security.

,

Krebs on SecurityBe Very Sparing in Allowing Site Notifications

An increasing number of websites are asking visitors to approve “notifications,” browser modifications that periodically display messages on the user’s mobile or desktop device. In many cases these notifications are benign, but several dodgy firms are paying site owners to install their notification scripts and then selling that communications pathway to scammers and online hucksters.

Notification prompts in Firefox (left) and Google Chrome.

When a website you visit asks permission to send notifications and you approve the request, the resulting messages that pop up appear outside of the browser. For example, on Microsoft Windows systems they typically show up in the bottom right corner of the screen — just above the system clock. These so-called “push notifications” rely on an Internet standard designed to work similarly across different operating systems and web browsers.

But many users may not fully grasp what they are consenting to when they approve notifications, or how to tell the difference between a notification sent by a website and one made to appear like an alert from the operating system or another program that’s already installed on the device.

This is evident by the apparent scale of the infrastructure behind a relatively new company based in Montenegro called PushWelcome, which advertises the ability for site owners to monetize traffic from their visitors. The company’s site currently is ranked by Alexa.com as among the top 2,000 sites in terms of Internet traffic globally.

Website publishers who sign up with PushWelcome are asked to include a small script on their page which prompts visitors to approve notifications. In many cases, the notification approval requests themselves are deceptive — disguised as prompts to click “OK” to view video material, or as “CAPTCHA” requests designed to distinguish automated bot traffic from real visitors.

An ad from PushWelcome touting the money that websites can make for embedding their dodgy push notifications scripts.

Approving notifications from a site that uses PushWelcome allows any of the company’s advertising partners to display whatever messages they choose, whenever they wish to, and in real-time. And almost invariably, those messages include misleading notifications about security risks on the user’s system, prompts to install other software, ads for dating sites, erectile disfunction medications, and dubious investment opportunities.

That’s according to a deep analysis of the PushWelcome network compiled by Indelible LLC, a cybersecurity firm based in Portland, Ore. Frank Angiolelli, vice president of security at Indelible, said rogue notifications can be abused for credential phishing, as well as foisting malware and other unwanted applications on users.

“This method is currently being used to deliver something akin to adware or click fraud type activity,” Angiolelli said. “The concerning aspect of this is that it is so very undetected by endpoint security programs, and there is a real risk this activity can be used for much more nefarious purposes.”

Sites affiliated with PushWelcome often use misleading messaging to trick people into approving notifications.

Angiolelli said the external Internet addresses, browser user agents and other telemetry tied to people who’ve accepted notifications is known to PushWelcome, which could give them the ability to target individual organizations and users with any number of fake system prompts.

Indelible also found browser modifications enabled by PushWelcome are poorly detected by antivirus and security products, although he noted Malwarebytes reliably flags as dangerous publisher sites that are associated with the notifications.

Indeed, Malwarebytes’ Pieter Arntz warned about malicious browser push notifications in a January 2019 blog post. That post includes detailed instructions on how to tell which sites you’ve allowed to send notifications, and how to remove them.

KrebsOnSecurity installed PushWelcome’s notifications on a brand new Windows test machine, and found that very soon after the system was peppered with alerts about malware threats supposedly found on the system. One notification was an ad for Norton antivirus; the other was for McAfee. Clicking either ultimately led to “buy now” pages at either Norton.com or McAfee.com.

Clicking on the PushWelcome notification in the bottom right corner of the screen opened a Web site claiming my brand new test system was infected with 5 viruses.

It seems likely that PushWelcome and/or some of its advertisers are trying to generate commissions for referring customers to purchase antivirus products at these companies. McAfee has not yet responded to requests for comment. Norton issued the following statement:

“We do not believe this actor to be an affiliate of NortonLifeLock. We are continuing to investigate this matter. NortonLifeLock takes affiliate fraud and abuse seriously and monitors ongoing compliance. When an affiliate partner abuses its responsibilities and violates our agreements, we take necessary action to remove these affiliate partners from the program and swiftly terminate our relationships. Additionally, any potential commissions earned as a result of abuse are not paid. Furthermore, NortonLifeLock sends notification to all of our affiliate partner networks about the affiliate’s abuse to ensure the affiliate is not eligible to participate in any NortonLifeLock programs in the future.”

Requests for comment sent to PushWelcome via email were returned as undeliverable. Requests submitted through the contact form on the company’s website also failed to send.

While scammy notifications may not be the most urgent threat facing Internet users today, most people are probably unaware of how this communications pathway can be abused.

What’s more, dodgy notification networks could be used for less conspicuous and sneakier purposes, including spreading fake news and malware masquerading as update notices from the user’s operating system. I hope it’s clear that regardless of which browser, device or operating system you use, it’s a good idea to be judicious about which sites you allow to serve notifications.

If you’d like to prevent sites from ever presenting notification requests, check out this guide, which has instructions for disabling notification prompts in Chrome, Firefox and Safari. Doing this for any devices you manage on behalf of friends, colleagues or family members might end up saving everyone a lot of headache down the road.

Cory DoctorowThe Attack Surface Lectures: Cross-Media Sci-Fi

The Attack Surface Lectures were a series of eight panel discussions on the themes in my’s novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “Cross-Media Sci Fi” hosted by the Brookline Booksmith in Brookline, MA, with guest-hosts John Rogers and Amber Benson. It was recorded on October 14, 2020.

Here is the original Youtube link for this program. Please consider subscribing to The Brookline Booksmith’s Youtube channel for access to all their outstanding author events!

MP3

LongNowWhat was the biggest empire in history?

What was the biggest empire in history? The answer, writes Benjamin Plackett in Live Science, depends on whether you think in terms of fraction of living humans or number of living humans, revealing the challenges inherent in attempting to compare time periods:

That’s without getting into the pros and cons of the other ways to measure size: largest land mass; largest contiguous land mass; largest army; largest gross domestic product; and so on.

But one alternative would be counted in years: we should measure empires by their long-term influence and stability, according to Martin Bommas, Director of the Macquarie University History Museum in Sydney:

“I think that to be classed as an empire, you need to have a period of peace to bring prosperity,” Bommas added. “If you look at it through years lasted, the Romans won this competition hands down.”

Planet DebianRaphaël Hertzog: Freexian’s report about Debian Long Term Support, October 2020

A Debian LTS logo Like each month, here comes a report about the work of paid contributors to Debian LTS.

Individual reports

In October, 221.50 work hours have been dispatched among 13 paid contributors. Their reports are available:
  • Abhijith PA did 16.0h (out of 14h assigned and 2h from September).
  • Adrian Bunk did 7h (out of 20.75h assigned and 5.75h from September), thus carrying over 19.5h to November.
  • Ben Hutchings did 11.5h (out of 6.25h assigned and 9.75h from September), thus carrying over 4.5h to November.
  • Brian May did 10h (out of 10h assigned).
  • Chris Lamb did 18h (out of 18h assigned).
  • Emilio Pozuelo Monfort did 20.75h (out of 20.75h assigned).
  • Holger Levsen did 7.0h coordinating/managing the LTS team.
  • Markus Koschany did 20.75h (out of 20.75h assigned).
  • Mike Gabriel gave back the 8h he was assigned. See below 🙂
  • Ola Lundqvist did 10.5h (out of 8h assigned and 2.5h from September).
  • Roberto C. Sánchez did 13.5h (out of 20.75h assigned) and gave back 7.25h to the pool.
  • Sylvain Beucler did 20.75h (out of 20.75h assigned).
  • Thorsten Alteholz did 20.75h (out of 20.75h assigned).
  • Utkarsh Gupta did 20.75h (out of 20.75h assigned).

Evolution of the situation

October was a regular LTS month with a LTS team meeting done via video chat thus there’s no log to be shared. After more than five years of contributing to LTS (and ELTS), Mike Gabriel announced that he founded a new company called Frei(e) Software GmbH and thus would leave us to concentrate on this new endeavor. Best of luck with that, Mike! So, once again, this is a good moment to remind that we are constantly looking for new contributors. Please contact Holger if you are interested!

The security tracker currently lists 42 packages with a known CVE and the dla-needed.txt file has 39 packages needing an update.

Thanks to our sponsors

Sponsors that joined recently are in bold.

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

Planet DebianJaldhar Vyas: Sal Mubarak 2077!

[Celebrating Diwali wearing a mask]

Best wishes to the entire Debian world for a happy, prosperous and safe Gujarati new year, Vikram Samvat 2077 named Paridhawi.

Worse Than FailureCodeSOD: Mod-El Code

Long-lived projects can have… interesting little corners. Choices made 13 years ago can stick around, either because they work well enough, or because, well, every change breaks somebody's workflow.

Today's anonymous submitter was poking around the code base of a large, long-lived JavaScript framework. In a file, not modified since 2007, but still included in the product, they found this function.

_getAdjustedDay: function(/*Date*/dateObj){ //FIXME: use mod instead? //summary: used to adjust date.getDay() values to the new values based on the current first day of the week value var days = [0,1,2,3,4,5,6]; if(this.weekStartsOn>0){ for(var i=0;i<this.weekStartsOn;i++){ days.unshift(days.pop()); } } return days[dateObj.getDay()]; // Number: 0..6 where 0=Sunday }

Look, this is old JavaScript, it's date handling code, and it's handling an unusual date case, so we already know it's going to be bad. That's not a surprise at all.

The core problem is, given a date, we want to find out the day of the week it falls on, but weeks don't have to start on Sunday, so we may need to do some arithmetic to adjust the dates. That arithmetic, as the FIXME comment helpfully points out, could easily be done with the % operator.

Someone knew the right answer here, but didn't get to implementing it. Instead, we have an array of valid values. To calculate the offset, we "roll" the array using a unshift(pop) combo- take the last element off the array and plop it onto the front. We also have a bonus unnecessary "if" statement- the "for" loop would have handled that.

This isn't the first time I've seen "populate an array with values and roll the array instead of using mod", and it probably won't be the last. But there's also a bonus WTF here. This function is invoked in _initFirstDay.

_initFirstDay: function(/*Date*/dateObj, /*Boolean*/adj){ //adj: false for first day of month, true for first day of week adjusted by startOfWeek var d = new Date(dateObj); if(!adj){d.setDate(1);} d.setDate(d.getDate()-this._getAdjustedDay(d,this.weekStartsOn)); d.setHours(0,0,0,0); return d; // Date }

So, first off, this function does two entirely different things, depending on what you pass in for adj. As the comment tells us, if adj is false, we find the first day of the month. If adj is true, we find the first day of the week relative to startOfWeek. Unfortunately, I'm not sure that comment is entirely correct, because whether or not adj is false, we do some arithmetic based on _getAdjustedDay. So, if you try this for a date in November 2020, with weeks starting on Sunday, you get the results you expect- because November 1st was a Sunday. But if you try it for October, the "first day" is September 27th, not October 1st.

Maybe that's by intent and design. Maybe it isn't. It's hard to tell from the comment. But the real bonus WTF is how they call this._getAdjustedDay here- passing in two parameters. To a function which only expects one. But that function does use the value passed in anyway, since it's a property of the class.

Even code that we can safely assume is bad just from knowing its origins can still find new ways to surprise us.

[Advertisement] Otter - Provision your servers automatically without ever needing to log-in to a command prompt. Get started today!

Planet DebianLouis-Philippe Véronneau: A better git diff

A few days ago I wrote a quick patch and missed a dumb mistake that made the program crash. When reviewing the merge request on Salsa, the problem became immediately apparent; Gitlab's diff is much better than what git diff shows by default in a terminal.

Well, it turns out since version 2.9, git bundles a better pager, diff-highlight. À la Gitlab, it will highlight what changed in the line.

The output of git diff using diff-highlight

Sadly, even though diff-highlight comes with the git package in Debian, it is not built by default (925288). You will need to:

$ sudo make --directory /usr/share/doc/git/contrib/diff-highlight

You can then add this line to your .gitconfig file:

[core]
  pager = /usr/share/doc/git/contrib/diff-highlight/diff-highlight | less --tabs=4 -RFX

If you use tig, you'll also need to add this line in your tigrc:

set diff-highlight = /usr/share/doc/git/contrib/diff-highlight/diff-highlight

Planet DebianDirk Eddelbuettel: RcppArmadillo 0.10.1.2.0

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 779 other packages on CRAN.

This release ties up a few loose ends from the recent 0.10.1.0.0.

Changes in RcppArmadillo version 0.10.1.2.0 (2020-11-15)

  • Upgraded to Armadillo release 10.1.2 (Orchid Ambush)

  • Remove three unused int constants (#313)

  • Include main armadillo header using quotes instead of brackets

  • Rewrite version number use in old-school mode because gcc 4.8.5

  • Skipping parts of sparse conversion on Windows as win-builder fails

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Planet DebianDirk Eddelbuettel: RcppAnnoy 0.0.17

annoy image

A new release 0.0.17 of RcppAnnoy is now on CRAN. RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm.

This release brings a new upstream version 1.17, released a few weeks ago, which adds multithreaded index building. This changes the API by adding a new ‘threading policy’ parameter requiring code using the main Annoy header to update. For this reason we waited a little for the dust to settle on the BioConductor 3.12 release before bringing the changes to BiocNeighbors via this commit and to uwot via this simple PR. Aaron and James updated their packages accordingly so by the time I uploaded RcppAnnoy it made for very smooth sailing as we all had done our homework with proper conditional builds, and the package had no other issue preventing automated processing at CRAN. Yay. I also added a (somewhat overdue one may argue) header file RcppAnnoy.h regrouping defines and includes which should help going forward.

Detailed changes follow below.

Changes in version 0.0.17 (2020-11-15)

  • Upgrade to Annoy 1.17, but default to serial use.

  • Add new header file to regroup includes and defines.

  • Upgrade CI script to use R with bspm on focal.

Courtesy of my CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

,

Cory DoctorowAttack Surface Lectures master post

The Attack Surface Lectures were a series of eight panel discussions on the themes in my novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.


1. Politics and Protest with Ron Deibert (Citizen Lab) and Eva Galperin (EFF)

Strand Bookstore, October 13, 2020

Original Youtube link.

MP3


2. Cross-Media SF with John Rogers (Leverage) and Amber Benson (Buffy)

Brookline Booksmith, October 14, 2020

Original Youtube link

MP3


3. Intersectionality: Race, Surveillance, and Tech and Its History with Malkia Devich-Cyril (Media Justice) and Meredith Whittaker​​ (AI Now)

The Booksmith, October 15, 2020

Original Youtube link

MP3


4. Sci-Fi Genre with Sarah Gailey (Upright Women Wanted) and Chuck Wendig (Wanderers)

Fountain books, October 16, 2020

Original Youtube link

MP3


5. Cyberpunk and Post Cyberpunk with Bruce Sterling (Pirate Utopia) and Christopher Brown (Failed State)

Anderson’s, October 19, 2020

Original Youtube link


MP3


6. Tech in SF with Annalee Newitz (Four Lost Cities) and Ken Liu (The Grace of Kings)

Interabang, October 20, 2020


Original Youtube link


MP3


7. Little Revolutions with Tochi Onyebuchi (Riot Baby) and Bethany C Morrow (A Song Below Water)

Skylight Books, October 21, 2020

Original Crowdcast link

MP3


8. Opsec and Personal Cybersecurity with Runa Sandvik (Tor Project) and Window Snyder (Square)

Third Place Books, October 22, 2020

Original Youtube link

MP3

Cory DoctorowThe Attack Surface Lectures: Politics and Protest

The Attack Surface Lectures were a series of eight panel discussions on the themes in my’s novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “Politics and Protest,” hosted by The Strand in NYC, with guest-hosts Eva Galperin and Ron Deibert. It was recorded on October 13, 2020.


Here is the original Youtube link for this program. Please consider subscribing to The Strand’s Youtube channel for access to all their outstanding author events!

MP3

Planet DebianBits from Debian: New Debian Developers and Maintainers (September and October 2020)

The following contributors got their Debian Developer accounts in the last two months:

  • Benda XU (orv)
  • Joseph Nahmias (jello)
  • Marcos Fouces (marcos)
  • Hayashi Kentaro (kenhys)
  • James Valleroy (jvalleroy)
  • Helge Deller (deller)

The following contributors were added as Debian Maintainers in the last two months:

  • Ricardo Ribalda Delgado
  • Pierre Gruet
  • Henry-Nicolas Tourneur
  • Aloïs Micard
  • Jérôme Lebleu
  • Nis Martensen
  • Stephan Lachnit
  • Felix Salfelder
  • Aleksey Kravchenko
  • Étienne Mollier

Congratulations!

Cory DoctorowThe Attack Surface Lectures: Politics and Protest (fixed)

The Attack Surface Lectures were a series of eight panel discussions on the themes in my’s novel Attack Surface, each hosted by a different bookstore and each accompanied by a different pair of guest speakers.

This program is “Politics and Protest,” hosted by The Strand in NYC, with guest-hosts Eva Galperin and Ron Deibert. It was recorded on October 13, 2020.


Here is the original Youtube link for this program. Please consider subscribing to The Strand’s Youtube channel for access to all their outstanding author events!

MP3

Cryptogram On Blockchain Voting

Blockchain voting is a spectacularly dumb idea for a whole bunch of reasons. I have generally quoted Matt Blaze:

Why is blockchain voting a dumb idea? Glad you asked.

For starters:

  • It doesn’t solve any problems civil elections actually have.
  • It’s basically incompatible with “software independence”, considered an essential property.
  • It can make ballot secrecy difficult or impossible.

I’ve also quoted this XKCD cartoon.

But now I have this excellent paper from MIT researchers:

“Going from Bad to Worse: From Internet Voting to Blockchain Voting”
Sunoo Park, Michael Specter, Neha Narula, and Ronald L. Rivest

Abstract: Voters are understandably concerned about election security. News reports of possible election interference by foreign powers, of unauthorized voting, of voter disenfranchisement, and of technological failures call into question the integrity of elections worldwide.This article examines the suggestions that “voting over the Internet” or “voting on the blockchain” would increase election security, and finds such claims to be wanting and misleading. While current election systems are far from perfect, Internet- and blockchain-based voting would greatly increase the risk of undetectable, nation-scale election failures.Online voting may seem appealing: voting from a computer or smart phone may seem convenient and accessible. However, studies have been inconclusive, showing that online voting may have little to no effect on turnout in practice, and it may even increase disenfranchisement. More importantly: given the current state of computer security, any turnout increase derived from with Internet- or blockchain-based voting would come at the cost of losing meaningful assurance that votes have been counted as they were cast, and not undetectably altered or discarded. This state of affairs will continue as long as standard tactics such as malware, zero days, and denial-of-service attacks continue to be effective.This article analyzes and systematizes prior research on the security risks of online and electronic voting, and show that not only do these risks persist in blockchain-based voting systems, but blockchains may introduce additional problems for voting systems. Finally, we suggest questions for critically assessing security risks of new voting system proposals.

You may have heard of Voatz, which uses blockchain for voting. It’s an insecure mess. And this is my general essay on blockchain. Short summary: it’s completely useless.

Cory DoctorowSomeone Comes to Town, Someone Leaves Town (part 23)

Here’s part twenty-three of my new reading of my novel Someone Comes to Town, Someone Leaves Town (you can follow all the installments, as well as the reading I did in 2008/9, here).

This is easily the weirdest novel I ever wrote. Gene Wolfe (RIP) gave me an amazing quote for it: “Someone Comes to Town, Someone Leaves Town is a glorious book, but there are hundreds of those. It is more. It is a glorious book unlike any book you’ve ever read.”

Here’s how my publisher described it when it came out:


Alan is a middle-aged entrepeneur who moves to a bohemian neighborhood of Toronto. Living next door is a young woman who reveals to him that she has wings—which grow back after each attempt to cut them off.

Alan understands. He himself has a secret or two. His father is a mountain, his mother is a washing machine, and among his brothers are sets of Russian nesting dolls.

Now two of the three dolls are on his doorstep, starving, because their innermost member has vanished. It appears that Davey, another brother who Alan and his siblings killed years ago, may have returned, bent on revenge.

Under the circumstances it seems only reasonable for Alan to join a scheme to blanket Toronto with free wireless Internet, spearheaded by a brilliant technopunk who builds miracles from scavenged parts. But Alan’s past won’t leave him alone—and Davey isn’t the only one gunning for him and his friends.

Whipsawing between the preposterous, the amazing, and the deeply felt, Cory Doctorow’s Someone Comes to Town, Someone Leaves Town is unlike any novel you have ever read.

MP3

Worse Than FailureAnnouncements: What the Fun Holiday Activity?

Time just flies right past, and before you know it, the holidays will be here. Which is why you had better hurry up and try your hand at giving us the best WTF Christmas Story ever, to help us found a new holiday tradition. Or at least, give us one bright spot in the yawning abyss of 2020.

Can you teach us the true meaning of WTFMas?

What We Want

We want your best holiday story. Any holiday is valid, though given the time of year, we're expecting one of the many solstice-adjacent holidays. This story can be based on real experiences, or it can be entirely fictional, because what we really want is a new holiday tradition.

The best submissions will:

  • Contain a core WTF, whether it's a bad boss, bad technology decisions, or incompetent team members
  • Prominently feature your chosen holiday
  • End with a valuable moral lesson, that leave us feeling full of holiday cheer

Are you going to write a traditional story? Or maybe a Dr. Seussian rhyme? A long letter to Santa? That's up to you.

How We Want It

Submissions are open from now until December 11th. Use our submission form. Check the "Story" box, and set the subject to WTF Holiday Special. Make sure to fill out the email address field, so we can contact you if you win!

What You Get

The best story will be a feature on our site, and also receive some of our new swag: a brand new TDWTF hoodie, a TDWTF mug, and a variety of stickers and other small swag.

The 2 runners up will also get a mug, stickers and other small swag.

Get writing, and let's create a new holiday tradition where opening the present may create more questions than it answers.

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

Worse Than FailureCodeSOD: Unset-tled

Alleen started by digging into a PHP method which was just annoying. _find_shipment_by_object_id would, when it couldn't find the ID, return false, instead of the more expected null. Not terrible, but annoying. Worse, it didn't return the shipment eihter, just a key which could be used to fetch a shipment from an array.

Again, all that's just annoying.

It was when looking at the delete_shipment method that Alleen had the facepalm moment.

public function delete_shipment($object_id) { $key = $this->_find_shipment_by_object_id($object_id); if ($key !== FALSE) { $obj = $this->_shipments[$key]; unset($obj, $this->_shipments[$key]); } return $this; }

The PHP unset method takes a list of variables, including potentially array elements, and deletes them. For whatever reason, the person who wrote this code decided to fetch the value stored in the array, then delete the variable holding the value and the array index holding the value, when the goal was simply to delete the element from the array.

They just enjoyed deleting so much, that they needed to delete it twice.

Alleen also wonders about the return $this. It seems like the intent was to build a fluent, chainable API, but the code is never used that way. We're left with a simple mystery, but at least they couldn't return twice.

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

,

Cryptogram Friday Squid Blogging: Peru Defends Its Waters against Chinese Squid Fishing Boats

Squid geopolitics.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Read my blog posting guidelines here.

Cryptogram Inrupt’s Solid Announcement

Earlier this year, I announced that I had joined Inrupt, the company commercializing Tim Berners-Lee’s Solid specification:

The idea behind Solid is both simple and extraordinarily powerful. Your data lives in a pod that is controlled by you. Data generated by your things — your computer, your phone, your IoT whatever — is written to your pod. You authorize granular access to that pod to whoever you want for whatever reason you want. Your data is no longer in a bazillion places on the Internet, controlled by you-have-no-idea-who. It’s yours. If you want your insurance company to have access to your fitness data, you grant it through your pod. If you want your friends to have access to your vacation photos, you grant it through your pod. If you want your thermostat to share data with your air conditioner, you give both of them access through your pod.

This week, Inrupt announced the availability of the commercial-grade Enterprise Solid Server, along with a small but impressive list of initial customers of the product and the specification (like the UK National Health Service). This is a significant step forward to realizing Tim’s vision:

The technologies we’re releasing today are a component of a much-needed course correction for the web. It’s exciting to see organizations using Solid to improve the lives of everyday people — through better healthcare, more efficient government services and much more.

These first major deployments of the technology will kick off the network effect necessary to ensure the benefits of Solid will be appreciated on a massive scale. Once users have a Solid Pod, the data there can be extended, linked, and repurposed in valuable new ways. And Solid’s growing community of developers can be rest assured that their apps will benefit from the widespread adoption of reliable Solid Pods, already populated with valuable data that users are empowered to share.

A few news articles. Slashdot thread.

,

Cryptogram Upcoming Speaking Engagements

This is a current list of where and when I am scheduled to speak:

The list is maintained on this page.

Sam VargheseWe need to talk about Tom Switzer’s spin about News Limited

Australia was hit by horrendous bushfires in 2019. Picture: Pixabay

Tom Switzer is a right-wing writer in Melbourne, who is executive director at the Centre for Independent Studies and is a presenter on ABC Radio National.

He often writes in support of Rupert Murdoch and his media empire, for the simple reason that if he were to lose his current gigs, then he could go back on the Murdoch teat.

Thus his defence of Murdoch against criticism by two former Australian prime ministers, Malcolm Turnbull and Kevin Rudd, is not surprising. Sucking up to power is a common game used by writers who have an avenue to vent. Switzer has the Nine newspapers open to his rantings.

One of the claims made by Rudd and Turnbull is that Murdoch publications publish stories that are full of incorrect information and slanted. This is correct. While there is nothing like objective journalism, there is indeed something called fact-based journalism.

Some publications may approach an issue from the left when they comment on it. Others may approach the same subject from a centrist or right perspective. There is nothing wrong with any of these occurrences.

Switzer cites the existence of a vast number of small publications to claim that there is media diversity in Australia. But how much reach do these publications have? And, more importantly, how much influence do they have?

As an example, let me cite the case of Arthur Sinodinos. The former adviser to ex-prime minister John Howard was under a cloud over some financial issues a few years back. Naturally, all newspapers that cover federal politics gave the story plenty of air, with many of them calling for him to step down.

But Sinodinos stayed put – until The Australian’s senior staffer Dennis Shanahan wrote a piece suggesting that he should go. He resigned that very day.

When the Murdoch press takes up an issue, one never knows the extent to which it will go, no matter whether the issue affects a group, company or a single individual. Yasmin Abdel-Meguid, a public figure, felt the effect after she issued a tweet that offended some nationalistic sentiments. There were more than 50 articles writen about her and it stopped when she left the country.

The Murdoch press generally backs the Liberal and other rightist parties in Australia. Occasionally, when it suits Murdoch’s business interests he tilts the other way.

Another of Turnbull’s accusations has been that The Australian spread incorrect information about the cause behind the bushires that Australian experienced in 2019, putting many of them down to arson.

The Murdoch defence was to say that only a small percentage of the total had mentioned arson. But what was forgotten is that if even one article had mentioned arson — when there was no evidence to back this up — then the paper was at fault. You cannot print 200 articles saying that a man was killed by his wife and justify the one article that said he took his own life.

It is true that Turnbull and Rudd have their own skeletons which they do not speak about in public. But that does not mean they cannot speak out about publications that operate in a way that only looks to further their proprietior’s interests.

,

Worse Than FailureError'd: Hate the Error and Hate the Game

"Somehow, a busy day for Blizzard's servers is going to last for around 6 months," writes James G.

 

"So, is interpreting error messages a sport now?" Jay C. wrote.

 

Drew W. writes, "I'm not sure how, but Sparkpost thinks I've had over 130 emials opened for every one I've sent!"

 

"I...may have a problem staying off of my phone," Kevin V. writes.

 

Gordon wrote, "Kind of sums up the 2020 season, doesn't it?"

 

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

Sam VargheseVale, Robert Fisk

The veteran Middle East correspondent Robert Fisk died recently at the age of 74, and his death means one of the Western world’s journalists who best understood the region has left the scene.

Fisk lived in Beirut for most of the 30-plus years he covered the region and reported the troubles in Northern Ireland before venturing out of the country.

He reported on the Soviet invasion of Afghanistan, the Israeli invasion of Lebanon and the continuing woes in that country. Fisk interviewed the al-Qaeda chief Osama bin Laden thrice and also covered the US invasion of Iraq.

Robert Fisk.

Some questioned his approach to journalism; he did not believe in getting opinions from both sides, so-called balanced journalism. Rather, it was his belief that the job of a reporter was to provide an outlet for the underdog.

His famous example was that of the liberation of a concentration camp. And he asked whether one should be expected to get a quote from a SS guard for balance, a query which nobody has attempted to answer.

When the terrorist attacks took place in 2001, Fisk was on a flight which was turned back due to the incident. He was invited on a TV talk show, along with the American lawyer Alan Dershowitz. When the attacks discussed, Fisk asked the natural question: what was the motive for the attacks. For this, he was denounced as an anti-Semite by Dershowitz, and he has often told this tale to illustrate the level of stupidity in the debate over the Middle East.

Fisk got into journalism at the Newcastle Chronicle and then moved to the Sunday Express. From there, he went to work for The Times as a correspondent in Northern Ireland, Portugal and the Middle East, a role for which he based himself in Beirut intermittently from 1976.

After 1989, he worked for The Independent. Fisk received many British and international journalism awards, including the Press Awards Foreign Reporter of the Year seven times.

At one stage of his career, he expressed doubts about whether all the reporting being done to cover trouble spots in the world was really of any use, because it seemed to change nothing.

But then the journalist within him prevailed and he continued filing his dispatches from Beirut until he was taken from this earth.

He was a man with a deep understanding of issues and one who took great pains with his reporting. He will be sorely missed.

,

Cryptogram Friday Squid Blogging: Diplomoceras Maximum

Diplomoceras maximum is an ancient squid-like creature. It lived about 68 million years ago, looked kind of like a giant paperclip, and may have had a lifespan of 200 years.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Read my blog posting guidelines here.

LongNowA Timely Reflection on our Changing Climate

Image for post
Antarctic Sea Ice Melt — 02019 (Source: Maxar)

The Ancient Greeks had two different words fortime. The first, chronos, is time as we think of it now: marching forward, ceaselessly creating our past, present, and future. The second, kairos, is time in the opportune sense: the ideal moment to act, as captured by the phrase, “It’s time.”

My work, like many other photographers, has been a dedicated search forkairos — finding that ideal confluence of place and time that helps to tell a particular story. For me, that story has focused on the manmade world. In 02013, I launched Daily Overview, which features compositions created from satellite imagery focused on the places on the planet where humans have left their mark. Partnerships with some of the world’s best satellite imaging companies gave me access to libraries from which I could compose a visual compendium of the world we are creating. It’s a world that we are harvesting, mining, exploring, and powering. And it’s a world that we are changing faster than ever before.

Image for post
Image for post
Image for post
Left: Mount Whaleback Iron Ore Mine in Australia. Center: Development in Boca Raton, Florida. Right: Singapore Tankers (Source: Maxar)

The atmospheric chemist and Nobel Prize laureate Paul J. Crutzen coined the term Anthropocene to describe this new geological era, one in which a single species — human beings — is the most powerful force affecting the planet’s natural systems. My work to date has captured macro-view moments in this era so that we might get a better understanding of what we, collectively — with all of the good and all of the bad — have done.

Thousands of image installments on Daily Overview over the past six years have covered a lot of ground. But in some ways, our earlier work does not include a crucial element — chronos — needed to convey the severity of what we face in this new Anthropocentric era. A single picture reflects the story of a moment in time. With two or more pictures of that same place, you can tell a richer story about change: its breadth, its pace, its cause. That is the idea behind our newest project, Overview Timelapse. What might we learn when we combine chronoskairos, and this awe-inspiring perspective from above?

Image for post
Las Vegas Expansion — 01989 / 02019 (Source: ESA)

The story of the current moment is that far-reaching human activity on the planet, primarily the continued burning of fossil fuels, is releasing a vast and unprecedented amount of carbon that is, in turn, causing a drastic (and widely-predicted) reaction by the planet’s natural systems. By looking for the locations that convey the magnitude of what is taking place, my co-author Timothy Dougherty and I spent hundreds of hours of seeking out and observing change that has taken place on the macroscale — and the reaction from the climate that we have already begun to see as a result.

Image for post
Amazon Rainforest Deforestation — 01989 / 02019 (Source: ESA)

As I write this, smoke from a nearby wildfire is obscuring the sun outside of my window. Perhaps I’m not as surprised as I should be to see that my home state of California has burned at an extraordinary rate this summer. Or that there were five tropical cyclones in the Atlantic Ocean last month for only the second time on record. Or the horrible destruction and loss of wildlife from the Australian Bushfires earlier this year. Or the once-in-a-thousand-year floods and Derechos in the Midwest. Or the recent reports of faster-than-predicted melting of the Greenland Ice Sheet.

Image for post
Miami Beach Red Tide — May 02017 / June 02018 (Source: Nearmap)

What scares me most now is what this project has taught me about how all of these interconnected events can cascade. These conditions build upon on one another such that something like unprecedented heat leads to drought, which leads to conditions ripe for fires, which leads to fires, which destroys trees, which returns all of the carbon stored in those trees since the Industrial Revolution into the atmosphere, which leads to more unprecedented heat, and so on, and so on, and so on.

Image for post
Westmont Rooftop Solar Project — 02014 / 02017 (Source: Nearmap)

Despite all this, I still maintain a healthy dose of optimism for what lies ahead. I have witnessed a changing climate and all the destruction it brings to bear, but I have also seen solutions which can make for a safer, better civilization and world. Throughout Overview Timelapse we have featured some of these innovations that are slated to bring positive change in the coming years.

Image for post
Great Green Wall of Africa — 02018 / 02019 (Source: Maxar)

So what will come next for a human species trying to thrive on a rapidly warming planet? The only certain constant is change. Looking to the future, it is in our hands, collectively, to determine the nature of the change to come. Let us work together to build awareness of the well-researched and considered solutions that already exist. Ones that get us excited about what lies ahead, not paralyzed by the magnitude of the problem. Ones that can be scaled to meet the severity of the challenge of an increasingly carbon-rich atmosphere.

Perhaps we will soon come to an overdue, yet opportune moment — our kairos — to reverse the course of human-induced planetary warming, and change the Earth for the better.

It’s time.

Learn More

Worse Than FailureCodeSOD: The Default Value

Cicely (previously) returned to the codebase which was providing annoyances last time.

This time, the code is meant for constructing objects based on a URL pattern. Specifically, the URL might have a format like api/resource/{id}. Looking at one of the constructors, though, it didn’t want an ID, it wanted an array of them. Cicely wasn’t passing multiple IDs off the URL, and wasn’t clear, from the documentation, how it worked, how you supplied those IDs, or frankly, what they were used for. Digging into the C# code made it clear, but still raised some additional questions.

int[] ids = Request.FormOrQuerystring("ids").EnsureNotNull().Split(",").
Select(item => item.ToInt32()).Concat(new int[] { id }).ToArray();

Whitespace added for readability, the original was on one line.

This is one of those cases where the code isn’t precisely bad, or wrong. At worst, it’s inefficient with all the LINQs and new arrays. It’s just… why would you do this this way?

At its core, we check the request for an ids property. EnsureNotNull() guarantees that we’ll see a value, whether there is one or not, we Split it on commas, project the text into Int32 using Select… and then concatenate a one element array onto the end, containing our id off the URL.

Perhaps someone wanted to avoid branching logic (because it’s potentially hard to debug) or maybe wanted some “functional purity” in their programming. Maybe they were just trying to see how much they could cram into a single line of code? Regardless, Cicely considers it a “most imaginative way to set a default value”. It’s certainly clever, I’ll give it that.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

,

Cryptogram “Privacy Nutrition Labels” in Apple’s App Store

Apple will start requiring standardized privacy labels for apps in its app store, starting in December:

Apple allows data disclosure to be optional if all of the following conditions apply: if it’s not used for tracking, advertising or marketing; if it’s not shared with a data broker; if collection is infrequent, unrelated to the app’s primary function, and optional; and if the user chooses to provide the data in conjunction with clear disclosure, the user’s name or account name is prominently displayed with the submission.

Otherwise, the privacy labeling is mandatory and requires a fair amount of detail. Developers must disclose the use of contact information, health and financial data, location data, user content, browsing history, search history, identifiers, usage data, diagnostics, and more. If a software maker is collecting the user’s data to display first or third-party adverts, this has to be disclosed.

These disclosures then get translated to a card-style interface displayed with app product pages in the platform-appropriate App Store.

The concept of a privacy nutrition label isn’t new, and has been well-explored at CyLab at Carnegie Mellon University.

Cryptogram New Zealand Election Fraud

It seems that this election season has not gone without fraud. In New Zealand, a vote for “Bird of the Year” has been marred by fraudulent votes:

More than 1,500 fraudulent votes were cast in the early hours of Monday in the country’s annual bird election, briefly pushing the Little-Spotted Kiwi to the top of the leaderboard, organizers and environmental organization Forest & Bird announced Tuesday.

Those votes — which were discovered by the election’s official scrutineers — have since been removed. According to election spokesperson Laura Keown, the votes were cast using fake email addresses that were all traced back to the same IP address in Auckland, New Zealand’s most populous city.

It feels like writing this story was a welcome distraction from writing about the US election:

“No one has to worry about the integrity of our bird election,” she told Radio New Zealand, adding that every vote would be counted.

Asked whether Russia had been involved, she denied any “overseas interference” in the vote.

I’m sure that’s a relief to everyone involved.

Cryptogram The Security Failures of Online Exam Proctoring

Proctoring an online exam is hard. It’s hard to be sure that the student isn’t cheating, maybe by having reference materials at hand, or maybe by substituting someone else to take the exam for them. There are a variety of companies that provide online proctoring services, but they’re uniformly mediocre:

The remote proctoring industry offers a range of services, from basic video links that allow another human to observe students as they take exams to algorithmic tools that use artificial intelligence (AI) to detect cheating.

But asking students to install software to monitor them during a test raises a host of fairness issues, experts say.

“There’s a big gulf between what this technology promises, and what it actually does on the ground,” said Audrey Watters, a researcher on the edtech industry who runs the website Hack Education.

“(They) assume everyone looks the same, takes tests the same way, and responds to stressful situations in the same way.”

The article discusses the usual failure modes: facial recognition systems that are more likely to fail on students with darker faces, suspicious-movement-detection systems that fail on students with disabilities, and overly intrusive systems that collect all sorts of data from student computers.

I teach cybersecurity policy at the Harvard Kennedy School. My solution, which seems like the obvious one, is not to give timed closed-book exams in the first place. This doesn’t work for things like the legal bar exam, which can’t modify itself so quickly. But this feels like an arms race where the cheater has a large advantage, and any remote proctoring system will be plagued with false positives.

Worse Than FailureCodeSOD: Testing Architectures

Marlyn’s employer ships software for a wide variety of CPU architectures. And depending on which branch of the product you were digging into, you might have code that builds for just i386, x86_64, PPC, and PPC64, while another branch might add s390, s390x, and aarch64.

As you might imagine, they have a huge automated test suite, meant to ensure that changes don’t break functionality or compatibility. So it’s a pity that their tests were failing.

The error messages implied that there were either missing or too many files, depending on the branch in question, but Marlyn could see that the correct build outputs were there, so nothing should be missing. It must be the test suite that had the error.

Marlyn dug into the Python script which drove their tests, and found the get_num_archs function, which theoretically would detect how many architectures this branch should output. Unfortunately, its implementation was straight out of XKCD.

def get_num_archs(self):
    return 7  # FIXME

At least they left a comment.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

Krebs on SecurityPatch Tuesday, November 2020 Edition

Adobe and Microsoft each issued a bevy of updates today to plug critical security holes in their software. Microsoft’s release includes fixes for 112 separate flaws, including one zero-day vulnerability that is already being exploited to attack Windows users. Microsoft also is taking flak for changing its security advisories and limiting the amount of information disclosed about each bug.

Some 17 of the 112 issues fixed in today’s patch batch involve “critical” problems in Windows, or those that can be exploited by malware or malcontents to seize complete, remote control over a vulnerable Windows computer without any help from users.

Most of the rest were assigned the rating “important,” which in Redmond parlance refers to a vulnerability whose exploitation could “compromise the confidentiality, integrity, or availability of user data, or of the integrity or availability of processing resources.”

A chief concern among all these updates this month is CVE-2020-17087, which is an “important” bug in the Windows kernel that is already seeing active exploitation. CVE-2020-17087 is not listed as critical because it’s what’s known as a privilege escalation flaw that would allow an attacker who has already compromised a less powerful user account on a system to gain administrative control. In essence, it would have to be chained with another exploit.

Unfortunately, this is exactly what Google researchers described witnessing recently. On Oct. 20, Google released an update for its Chrome browser which fixed a bug (CVE-2020-15999) that was seen being used in conjunction with CVE-2020-17087 to compromise Windows users.

If you take a look at the advisory Microsoft released today for CVE-2020-17087 (or any others from today’s batch), you might notice they look a bit more sparse. That’s because Microsoft has opted to restructure those advisories around the Common Vulnerability Scoring System (CVSS) format to more closely align the format of the advisories with that of other major software vendors.

But in so doing, Microsoft has also removed some useful information, such as the description explaining in broad terms the scope of the vulnerability, how it can be exploited, and what the result of the exploitation might be. Microsoft explained its reasoning behind this shift in a blog post.

Not everyone is happy with the new format. Bob Huber, chief security officer at Tenable, praised Microsoft for adopting an industry standard, but said the company should consider that folks who review Patch Tuesday releases aren’t security practitioners but rather IT counterparts responsible for actually applying the updates who often aren’t able (and shouldn’t have to) decipher raw CVSS data.

“With this new format, end users are completely blind to how a particular CVE impacts them,” Huber said. “What’s more, this makes it nearly impossible to determine the urgency of a given patch. It’s difficult to understand the benefits to end-users. However, it’s not too difficult to see how this new format benefits bad actors. They’ll reverse engineer the patches and, by Microsoft not being explicit about vulnerability details, the advantage goes to attackers, not defenders. Without the proper context for these CVEs, it becomes increasingly difficult for defenders to prioritize their remediation efforts.”

Dustin Childs with Trend Micro‘s Zero Day Initiative also puzzled over the lack of details included in Microsoft advisories tied to two other flaws fixed today — including one in Microsoft Exchange Server (CVE-2020-16875) and CVE-2020-17051, which is a scary-looking weakness in the Windows Network File System (NFS).

The Exchange problem, Childs said, was reported by the winner of the Pwn2Own Miami bug finding contest.

“With no details provided by Microsoft, we can only assume this is the bypass of CVE-2020-16875 he had previously mentioned,” Childs said. “It is very likely he will publish the details of these bugs soon. Microsoft rates this as important, but I would treat it as critical, especially since people seem to find it hard to patch Exchange at all.”

Likewise, with CVE-2020-17051, there was a noticeable lack of detail for bug that earned a CVSS score of 9.8 (10 is the most dangerous).

“With no description to work from, we need to rely on the CVSS to provide clues about the real risk from the bug,” Childs said. “Considering this is listed as no user interaction with low attack complexity, and considering NFS is a network service, you should treat this as wormable until we learn otherwise.”

Separately, Adobe today released updates to plug at least 14 security holes in Adobe Acrobat and Reader. Details about those fixes are available here. There are no security updates for Adobe’s Flash Player, which Adobe has said will be retired at the end of the year. Microsoft, which has bundled versions of Flash with its Web browsers, says it plans to ship an update in December that will remove Flash from Windows PCs, and last month it made the removal tool available for download.

Windows 10 users should be aware that the operating system will download updates and install them on its own schedule, closing out active programs and rebooting the system. If you wish to ensure Windows has been set to pause updating so you can back up your files and/or system, see this guide.

But please do back up your system before applying any of these updates. Windows 10 even has some built-in tools to help you do that, either on a per-file/folder basis or by making a complete and bootable copy of your hard drive all at once.

As always, if you experience glitches or problems installing any of these patches this month, please consider leaving a comment about it below; there’s a better-than-even chance other readers have experienced the same and may chime in here with some helpful tips.

,

Krebs on SecurityRansomware Group Turns to Facebook Ads

It’s bad enough that many ransomware gangs now have blogs where they publish data stolen from companies that refuse to make an extortion payment. Now, one crime group has started using hacked Facebook accounts to run ads publicly pressuring their ransomware victims into paying up.

On the evening of Monday, Nov. 9, an ad campaign apparently taken out by the Ragnar Locker Team began appearing on Facebook. The ad was designed to turn the screws to the Italian beverage vendor Campari Group, which acknowledged on Nov. 3 that its computer systems had been sidelined by a malware attack.

On Nov. 6, Campari issued a follow-up statement saying “at this stage, we cannot completely exclude that some personal and business data has been taken.”

“This is ridiculous and looks like a big fat lie,” reads the Facebook ad campaign from the Ragnar crime group. “We can confirm that confidential data was stolen and we talking about huge volume of data.”

The ad went on to say Ragnar Locker Team had offloaded two terabytes of information and would give the Italian firm until 6 p.m. EST today (Nov. 10) to negotiate an extortion payment in exchange for a promise not to publish the stolen files.

The Facebook ad blitz was paid for by Hodson Event Entertainment, an account tied to Chris Hodson, a deejay based in Chicago. Contacted by KrebsOnSecurity, Hodson said his Facebook account indeed was hacked, and that the attackers had budgeted $500 for the entire campaign.

“I thought I had two-step verification turned on for all my accounts, but now it looks like the only one I didn’t have it set for was Facebook,” Hodson said.

Hodson said a review of his account shows the unauthorized campaign reached approximately 7,150 Facebook users, and generated 770 clicks, with a cost-per-result of 21 cents. Of course, it didn’t cost the ransomware group anything. Hodson said Facebook billed him $35 for the first part of the campaign, but apparently detected the ads as fraudulent sometime this morning before his account could be billed another $159 for the campaign.

The results of the unauthorized Facebook ad campaign. Image: Chris Hodson.

It’s not clear whether this was an isolated incident, or whether the fraudsters also ran ads using other hacked Facebook accounts. A spokesperson for Facebook said the company is still investigating the incident. A request for comment sent via email to Campari’s media relations team was returned as undeliverable.

But it seems likely we will continue to see more of this and other mainstream advertising efforts by ransomware groups going forward, even if victims really have no expectation that paying an extortion demand will result in criminals actually deleting or not otherwise using stolen data.

Fabian Wosar, chief technology officer at computer security firm Emsisoft, said some ransomware groups have become especially aggressive of late in pressuring their victims to pay up.

“They have also started to call victims,” Wosar said. “They’re outsourcing to Indian call centers, who call victims asking when they are going to pay or have their data leaked.”

LongNowScenario Planning for the Long-term

The following transcript has been edited for length and clarity. 

The Role of Mental Maps

This is a map of North America. It was made by a Dutch map maker by the name of Herman Moll, working in London in 01701. I bought it on Portobello Road for about 60 pounds back in 01981. Which is to say, it’s not a particularly valuable map. But there is something unusual about it: California is depicted as an island. 

What’s interesting to me as a scenario planner is how the map came to be, how it was used, and how it was changed.

The Spanish came up from the South, and they found what we now call Mexico, and the tip of the Baja Peninsula, and sailed up into the Sea of Cortez. Those who went further north along the West Coast eventually came to the Strait of Juan de Fuca, a channel separating present-day Washington and British Columbia. Assuming the two bodies of water were connected, they created the Island of California. 

Now, this would only be a historical curiosity were it not for the problem of the missionaries who actually used the map to go inland. And of course, they would have to take their boats with them to cross the Sea of California. And when they went over the Sierra Nevada Mountains down the other side, they found this beach that went on and on and on and on until finally they realized there was no Sea of California.

And they went back to the map makers in Spain and said, “Your bloody map is wrong!” And the map makers fought back and said, “No, no, no, you’re in the wrong place, the map is right.” Anybody who works in a large organization understands this logic very well. Because the map is always right. 

The first maps depicting California as an island were drawn in 01605. In 01685, the King of Spain finally figured out that this was wrong and ordered the maps in Spain to be corrected. But we still have maps dating from 01765 that were drawn this way.

So what’s the message? If you get your facts wrong, you get your map wrong. If you get your map wrong, you do the wrong thing. But worst of all, once you believe a map, it’s very hard to change. 

We make our decisions about the future based on our own mental maps about how the world works. And we are very much prisoners of those mental maps. Part of the function of scenario planning is figuring out how to break out of the constraints. How do we challenge those mental maps that we see about how people behave, how organizations work and how institutions evolve?

The Importance of Diversity in Scenario-Planning 

This is the slide that IBM used to make a decision about the future of personal computers. It is the costliest slide in business history. This is a $200 billion slide.

In 01980, IBM made the above forecast of what they believed the demand of personal computers would be to decide whether or not they should get into the business. They projected that the total computers sold through all channels, over five years, would be 241,683, peaking in 01983 and heading south. After all, why would anyone buy a second computer?

This product [the personal computer] was so funky that the theory was that pursuing it was going to kill Apple. That was the goal: get people back to real computers [large mainframes] because real men use big machines. It was nine men in the room making this call, and they got it totally wrong. The correct answer was 25 million units sold over that five year period. 

So, they were a little bit off and they said, “Okay, 241,000 units, a couple thousand dollars a unit. Well, it’s not worth developing a chip. Intel, give us the chip. And there’s this kid from Seattle, Gates or something, who has got an Operating System called QDOS. He’ll give it to us for free, we’ll put it on the machine, we’ll put it in a box, we’ll call it an IBM computer.”

This was the moment they almost lost the future of the company, because they could not imagine that the world could be so different, that people would like a box with 16K of memory in it. That just seemed inconceivable to these nine men, who were the smartest people in their industry, who knew everything about what they were doing and yet still got it completely wrong. And it literally almost killed IBM as a result. And of course, they no longer make PCs because they could no longer compete, et cetera. This slide is why Bill Gates is one of the richest men in the world. If IBM had said, “You know, maybe there really is a future here, and we’ll develop our own operating system and our own chips,” it might’ve been a very different story.

So, again, part of our challenge today in our thinking is precisely how we challenge each other’s mental maps. And for that, you need diversity. Diversity is the single most important characteristic for thinking about the future. Every time I have been wrong, with no exceptions, it’s because we had inadequate diversity in the room. There are a number of embarrassing moments in the history of the Global Business Network (GBN), the Mexico meeting being maybe the lowest point. Two weeks before the collapse of the Mexican peso, we said there were three scenarios for Mexico: a good scenario, a better scenario, and a best scenario. Two weeks later, it all went in the tank. Why? Because we were all just talking to ourselves, and as a result, we got it completely wrong.

So, one of the most important messages about long-term thinking is the inclusion of diverse points of view. And if you’re trapped in one mindset, you’re going to miss an enormous amount. 

The Spirit of Surprise

“Often do the spirits of great events stride on before the events, and in today already walks tomorrow.”

Friedrich Schiller (01759-01805)

The future is being born. We all remember the Bill Gibson quote: “The future’s already here, it’s just not evenly distributed yet.” The signals are out there.

And why diversity matters so much is because it enables you to pick up on a variety of signals from a variety of different disciplines, contexts, cultures, et cetera. And that’s an important part of what scenario thinking is about.

Scenario planning is rooted in the concept of Multiple Possibilities

The way most organizations have thought about the future is to project out from the present. And then, if they were concerned about uncertainty, they shaded it up and down a little bit, 10% up 10% down in what was called sensitivity analysis. It didn’t require much imagination; it required math. 

But scenario planning involves imagining different possibilities and then figuring out how we can get from here to there. It requires a combination of two things: imagination and analytic realism. If it’s just forecasts, just analysis, it’s pedestrian. You miss the big surprises. You don’t see that new mental map. If it’s just imagination without analysis and rigor behind it, it is just that; it’s good fiction.

So, it’s important that both of these come to bear in the task of thinking about scenarios. 

The Test of a Good Scenario

Scenario planning is not about prediction; it’s about making better decisions. That is, if you really do your homework well in multiple scenarios, you’re probably going to see this future. That’s not the hard part. The hard part is: what do you do? And if your scenarios are brilliant and nobody pays any attention to them, you have failed. Having been a consultant for many years, it was not a way to get more business to say to a CEO, “Well, we gave you the future and you blew it. You didn’t make the right decision.” That was our failure as consultants. Our job was to actually affect the mindset of decision makers. 

In the end, what we want to do with the Organizational Continuity Project is not simply understand long-term institutions, but influence them. How do we actually make better decisions about our societies, our governments, our corporations, our educational institutions, our communities? How do we actually think long-term and make better choices? That’s the real goal here. It’s beginning to think long-term about what is likely to happen. 

The Strategic Conversation and Strategic Options

It’s about a strategic conversation. What we want to empower is thinking about different scenarios, going out subsequently, creating new knowledge, doing research, doing serious homework, beginning to think about how you create what we call emergent strategies—strategies that emerge out of that conversation, as opposed to top-down control. And then testing those emergent strategies against what we’re already doing, and thereby improving the quality of decision-making.

So what this is really about is an orchestrated strategic conversation, with inputs from a variety of different sources, thinking about possible scenarios, thinking about how we might influence the shape of institutions going forward, and what the rules for those might be. And this continues on. This doesn’t stop. This conversation is a perpetual Long Now conversation.

Worse Than FailureCodeSOD: Tranposing the Key

Russell F sends us this C# "fuction", and I have to be honest: I have no idea what it's supposed to do. I can trace through the logic, I can see what it does, but I don't understand why it does it.

private List<LaborService> Tranpose(List<LaborService> laborService) { int half = (int)Math.Ceiling((decimal)(laborService.Count)/2); for (int i = 0; i < laborService.Count; i++) { if (i < half) laborService[i].Order = 2 * i; else laborService[i].Order = (i - half) + 1; } return laborService.OrderBy(x => x.Order).ToList(); }

So this starts by finding the rough midpoint of our list. Then we iterate across each element, and if it's position is less than half, we place double its index into the Order field. If it's half or greater, we store its index minus half, plus one, into its order field. Finally, we sort by Order.

Now, based on the name, we can assume this was inspired by a matrix transposition- oh, I'm sorry, tranposition- based on the method name. It isn't one. It's almost an interleaving operation, but it also isn't one of those.

You can play with the code or just look at this table.

Ceiling of half of 10 is 5. Indexes: 0 1 2 3 4 5 6 7 8 9 Values: A B C D E F G H I J Order: 0 2 4 6 8 1 2 3 4 5 ----------------------------- New Sort: A F B G H C I J D E

What you can notice here is that as we re-number our Orders, the bottom half gets doubled, but the top half increases incrementally. This means that we end up with ties, and that means that we end up with sections where elements from the either half of the list end up next to each other- see G, H, I,J and D, E in my example.

What is this for? Why does this exist? Why does it matter? No idea.

But Russell has another detail to add:

The Order field is never used anywhere but in this one function -- it appears to have been added solely to allow this.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

,

Cryptogram 2020 Was a Secure Election

Over at Lawfare: “2020 Is An Election Security Success Story (So Far).”

What’s more, the voting itself was remarkably smooth. It was only a few months ago that professionals and analysts who monitor election administration were alarmed at how badly unprepared the country was for voting during a pandemic. Some of the primaries were disasters. There were not clear rules in many states for voting by mail or sufficient opportunities for voting early. There was an acute shortage of poll workers. Yet the United States saw unprecedented turnout over the last few weeks. Many states handled voting by mail and early voting impressively and huge numbers of volunteers turned up to work the polls. Large amounts of litigation before the election clarified the rules in every state. And for all the president’s griping about the counting of votes, it has been orderly and apparently without significant incident. The result was that, in the midst of a pandemic that has killed 230,000 Americans, record numbers of Americans voted­ — and voted by mail — ­and those votes are almost all counted at this stage.

On the cybersecurity front, there is even more good news. Most significantly, there was no serious effort to target voting infrastructure. After voting concluded, the director of the Cybersecurity and Infrastructure Security Agency (CISA), Chris Krebs, released a statement, saying that “after millions of Americans voted, we have no evidence any foreign adversary was capable of preventing Americans from voting or changing vote tallies.” Krebs pledged to “remain vigilant for any attempts by foreign actors to target or disrupt the ongoing vote counting and final certification of results,” and no reports have emerged of threats to tabulation and certification processes.

A good summary.

Charles StrossEntanglements!

Entanglements Cover.jpg

Many thanks to Charlie for giving me the chance to write about editing and my latest project. I'm very excited about the publication of Entanglements. The book has received a starred review from Publishers Weekly and terrific reviews in Lightspeed, Science, and the Financial Times. MIT Press has created a very nice "Pubpub" page about Entanglements, with information about the book and its various contributors. The "On the Stories" section has an essay about by Nick Wolven about his amazing story, "Sparkly Bits," and a fun Zoom conversation with James Patrick Kelly, Nancy Kress, and Sam J. Miller. I think the site is well worth checking out, and here's the Pubpub description of the book:

Science fiction authors offer original tales of relationships in a future world of evolving technology.

In a future world dominated by the technological, people will still be entangled in relationships--in romances, friendships, and families. This volume in the Twelve Tomorrows series considers the effects that scientific and technological discoveries will have on the emotional bonds that hold us together.

The strange new worlds in these stories feature AI family therapy, floating fungitecture, and a futuristic love potion. A co-op of mothers attempts to raise a child together, lovers try to resolve their differences by employing a therapeutic sexbot, and a robot helps a woman dealing with Parkinson's disease. Contributions include Xia Jia's novelette set in a Buddhist monastery, translated by the Hugo Award-winning writer Ken Liu; a story by Nancy Kress, winner of six Hugos and two Nebulas; and a profile of Kress by Lisa Yaszek, Professor of Science Fiction Studies at Georgia Tech. Stunning artwork by Tatiana Plakhova--"infographic abstracts" of mixed media software--accompanies the texts.

Worse Than FailureCodeSOD: Utility Functions

As a personal perspective, I don't tend to believe that mastery of a programming tool is nearly as important as mastery of the codebase and problem domain you're working on. But there are some developers who just don't want to learn the codebase or what other developers are doing.

Take Jessica's latest co-worker, which is similar to some previous co-workers. In this case, there was a project in flight that was starting to fall behind schedule. Management did what management does in this situation: they threw warm bodies at the project and ensured that it fell further behind.

Brant was one of those warm bodies, and Brant did not want to learn what was already in the code base. He was going to do part of the JavaScript front end, he was going to rush to get it done, and he was going to copy-paste his way through.

Which lead to this:

function setMailingsReceivedCountLabel(e) { // Implement sting prototye format so that we can use string token replacement if (!String.prototype.format) { String.prototype.format = function() { var args = arguments; return this.replace(/{(\d+)}/g, function(match, number) { return typeof args[number] != 'undefined' ? args[number] : match ; }); }; } // Get values var recordCount = $("#mailingsGrid").data("kendoGrid").dataSource.total(); $("#Mailings_Count").text("(" + recordCount + ")"); }

Now, a format method for strings is a useful function. It's not wrong to implement your own- you can't rely on template literals being supported by every browser. In fact, it's such a useful function that Jessica and the team had already added one in a generic file of utility functions. A more robust one, coupled with some unit tests, and y'know, the one you should use.

Brant had no interest in learning that there was already a function which did what he needed, so instead he implemented this one. In fact, he copy-and-pasted this blob into any method he wrote that might potentially do any sort of string formatting. I stress "might potentially", because as you can see, this method doesn't actually use his format method.

[Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!

Krebs on SecurityBody Found in Canada Identified as Neo-Nazi Spam King

The body of a man found shot inside a burned out vehicle in Canada three years ago has been identified as that of Davis Wolfgang Hawke, a prolific spammer and neo-Nazi who led a failed anti-government march on Washington, D.C. in 1999, according to news reports.

Homicide detectives said they originally thought the man found June 14, 2017 in a torched SUV on a logging road in Squamish, British Columbia was a local rock climber known to others in the area as a politically progressive vegan named Jesse James.

Davis Wolfgang Hawke. Image: Spam Kings, by Brian McWilliams.

But according to a report from CTV News, at a press conference late last month authorities said new DNA evidence linked to a missing persons investigation has confirmed the man’s true identity as Davis Wolfgang Hawke.

A key subject of the book Spam Kings by Brian McWilliams, Hawke was a Jewish-born American who’d legally changed his name from Andrew Britt Greenbaum. For many years, Hawke was a big time purveyor of spam emails hawking pornography and male enhancement supplements, such as herbal Viagra.

Hawke had reportedly bragged about the money he earned from spam, but told friends he didn’t trust banks and decided to convert his earnings into gold and platinum bars. That sparked rumors that he had possibly buried his ill-gotten gains on his parents’ Massachusetts property.

In 2005, AOL won a $12.8 million lawsuit against him for relentlessly spamming its users. A year later, AOL won a court judgment authorizing them to dig on that property, although no precious metals were ever found.

More recently, Hawke’s Jesse James identity penned a book called Psychology of Seduction, which claimed to merge the “shady world of the pickup artist with modern science, unraveling the mystery of attraction using evolutionary biology and examining seduction through the lens of social and evolutionary psychology.”

The book’s “about the author” page said James was a “disruptive technology pioneer” who was into rock climbing and was a resident of Squamish. It also claimed James held a PhD in theoretical physics from Stanford, and that he was an officer in the Israeli Defense Force.

It might be difficult to fathom why, but Hawke may have made a few enemies over the years. Spam Kings author McWilliams notes that Hawke changed his name with regularity and used many pseudonyms.

“I could definitely see this guy making someone so mad at him they’d want to kill him,” McWilliams told CTV. “He was a guy who really pushed people that way and was a crook. I mean, he was a conman. That was what he was and I can see how somebody might get mad. I can also see him staging his own death or committing suicide in a fashion like that, if that’s what he chose to do. He was just a perplexing guy. I still don’t feel like I have a handle on him and I spent the better part of a year trying to figure out what made him tick.”

The father of the deceased, Hy Greenbaum, has offered a $10,000 reward to any tipster who can help solve his son’s homicide. British Columbia’s Integrated Homicide Investigation Team also is seeking clues, and can be reached at ihitinfo@rcmp-grc.gc.ca.

Cory DoctorowSomeone Comes to Town, Someone Leaves Town (part 22)

Here’s part twenty-two of my new reading of my novel Someone Comes to Town, Someone Leaves Town (you can follow all the installments, as well as the reading I did in 2008/9, here).

This is easily the weirdest novel I ever wrote. Gene Wolfe (RIP) gave me an amazing quote for it: “Someone Comes to Town, Someone Leaves Town is a glorious book, but there are hundreds of those. It is more. It is a glorious book unlike any book you’ve ever read.”

Here’s how my publisher described it when it came out:


Alan is a middle-aged entrepeneur who moves to a bohemian neighborhood of Toronto. Living next door is a young woman who reveals to him that she has wings—which grow back after each attempt to cut them off.

Alan understands. He himself has a secret or two. His father is a mountain, his mother is a washing machine, and among his brothers are sets of Russian nesting dolls.

Now two of the three dolls are on his doorstep, starving, because their innermost member has vanished. It appears that Davey, another brother who Alan and his siblings killed years ago, may have returned, bent on revenge.

Under the circumstances it seems only reasonable for Alan to join a scheme to blanket Toronto with free wireless Internet, spearheaded by a brilliant technopunk who builds miracles from scavenged parts. But Alan’s past won’t leave him alone—and Davey isn’t the only one gunning for him and his friends.

Whipsawing between the preposterous, the amazing, and the deeply felt, Cory Doctorow’s Someone Comes to Town, Someone Leaves Town is unlike any novel you have ever read.

MP3

,

MELinks November 2020

KDE has a long term problem of excessive CPU time used by the screen locker [1]. Part of it is due to software GL emulation, and part of it is due to the screen locker doing things like flashing the cursor when nothing else is happening. One of my systems has an NVidia card and enabling GL would cause it to crash. So now I have kscreenlocker using 30% of a CPU core even when the screen is powered down.

Informative NYT article about the latest security features for iPhones [2]. Android needs new features like this!

Russ Allbery wrote an interesting review of the book Hand to Mouth by Linda Tirado [3], it’s about poverty in the US and related things. Linda first became Internet famous for her essay “Why I Make Terrible Decisions or Poverty Thoughts” which is very insightful and well written, this is the latest iteration of that essay [4].

This YouTube video by Ruby Payne gives great insights to class based attitudes towards time and money [5].

News Week has an interesting article about chicken sashimi, apparently you can safely eat raw chicken if it’s prepared well [6].

Vanity Fair has an informative article about how Qanon and Trumpism have infected the Catholic Church [7]. Some of Mel Gibson’s mental illness is affecting a significant portion of the Catholic Church in the US and some parts in the rest of the world.

Noema has an interesting article on toxic Internet culture, Japan’s 2chan, 4chan, 8chan/8kun, and the conspiracy theories they spawned [8].

Benjamin Corey is an ex-Fundie who wrote an amusing analysis of the Biblical statements about the anti-Christ [9].

NYMag has an interesting article The Final Gasp of Donald Trump’s Presidency [10].

Mother Jones has an informative article about the fact that Jim Watkins (the main person behind QAnon) has a history of hosting child porn on sites he runs [11], but we all knew QAnon was never about protecting kids.

Eand has an insightful article America’s Problem is That White People Want It to Be a Failed State [12].

Rondam RamblingsI'm proud to be an American again

,

LongNowThe Role of Geology in US Presidential Elections

In an article in Forbes, David Bressan writes that the giant rift in the USA’s political voting blocs is in part a consequence of collisions between continental plates, the literal giant rift that used to separate the two halves of North America, and recent glacial activity:

The same region that had once been covered in ocean water, leading to the fertile Black Belt, was almost an exact replica of the districts that had voted for Clinton.

The rich coal fields in Ohio, West Virginia, Pennsylvania and Maryland formed as a result of two continents colliding some 300 million years ago. The coal fueled the economic growth of cities like Pittsburg, Detroit, Chicago and Cleveland.

The Driftless Area is a region west to the Great Lakes that escaped glaciation during the last ice-age. Farming is more difficult here. The election map shows that most countries in the Driftless Area voted Democrats in 2012. It seems that more liberal politics, combined with financial hardship experienced by the local farmers and accentuated by the poor soils, convinced them to vote for Barack Obama.

Last year, astrobiologist Lewis Dartnell made a similar point in a Conversation at The Interval:

Worse Than FailureError'd: Not So Smart After All!

"Today I learned that the time between 12 PM and 1 PM is "12:28 noon" according to CNN," Drew W. writes.

 

Robert L. wrote, "The Trump campaign website is a little bit confused as to when election day is."

 

"I changed over to playing on my Nintendo Switch from watching Netflix on my so-called 'smart' TV and apparently, the subtitles didn't get the message," writes Josh.

 

"You know how after watching a really scary horror movie, bumps in the night can leave you feeling on edge?" Matia W. wrote, "Well, seeing error messages popping up while watching a show about professional hackers is a little bit like that."

 

Eric L. writes, "Raising the capital for any of these seems like a pain. Yeah...I'll pass."

 

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

,

Cryptogram Detecting Phishing Emails

Research paper: Rick Wash, “How Experts Detect Phishing Scam Emails“:

Abstract: Phishing scam emails are emails that pretend to be something they are not in order to get the recipient of the email to undertake some action they normally would not. While technical protections against phishing reduce the number of phishing emails received, they are not perfect and phishing remains one of the largest sources of security risk in technology and communication systems. To better understand the cognitive process that end users can use to identify phishing messages, I interviewed 21 IT experts about instances where they successfully identified emails as phishing in their own inboxes. IT experts naturally follow a three-stage process for identifying phishing emails. In the first stage, the email recipient tries to make sense of the email, and understand how it relates to other things in their life. As they do this, they notice discrepancies: little things that are “off” about the email. As the recipient notices more discrepancies, they feel a need for an alternative explanation for the email. At some point, some feature of the email — usually, the presence of a link requesting an action — triggers them to recognize that phishing is a possible alternative explanation. At this point, they become suspicious (stage two) and investigate the email by looking for technical details that can conclusively identify the email as phishing. Once they find such information, then they move to stage three and deal with the email by deleting it or reporting it. I discuss ways this process can fail, and implications for improving training of end users about phishing.

Cryptogram California Proposition 24 Passes

California’s Proposition 24, aimed at improving the California Consumer Privacy Act, passed this week. Analyses are very mixed. I was very mixed on the proposition, but on the whole I supported it. The proposition has some serious flaws, and was watered down by industry, but voting for privacy feels like it’s generally a good thing.

Worse Than FailureCodeSOD: Frist Item

In .NET, if you want to get the first item from an IList object, you could just use the index: list[0]. You also have a handy-dandy function called First, or even better FirstOrDefault. FirstOrDefault helpfully doesn’t throw an exception if the list is empty (though depending on what’s in the list, it may give you a null).

What I’m saying is that there are plenty of easy, and obvious ways to get the first element of a list.

Stevie’s co-worker did this instead:

IList<Order> orderList = db.GetOrdersByDateDescending().ToList();
int i = 1;
foreach (Order order in orderList)
{
    if (i == 1)
    {
        PrintOrder(order);
    }
    i++;
}

So, for starters, GetOrdersByDateDescending() is a LINQ-to-SQL call which invokes a stored procedure. Because LINQ does all sorts of optimizations on how that SQL gets generated, if you were to do GetOrdersByDateDescending().FirstOrDefault(), it would fetch only the first row, cutting down on how much data crosses the network.

But because they did ToList, it will fetch all the rows.

And then… then they loop over the result. Every single row. But they only want the first one, so they have an if that only triggers when i == 1, which I mean, at this point, doing 1-based indexing is just there to taunt us.

Stevie adds: “This is a common ‘pattern’ throughout the project.” Well clearly, the developer responsible isn’t going to do something once when they could do it every single time.

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

,

Krebs on SecurityWhy Paying to Delete Stolen Data is Bonkers

Companies hit by ransomware often face a dual threat: Even if they avoid paying the ransom and can restore things from scratch, about half the time the attackers also threaten to release sensitive stolen data unless the victim pays for a promise to have the data deleted. Leaving aside the notion that victims might have any real expectation the attackers will actually destroy the stolen data, new research suggests a fair number of victims who do pay up may see some or all of the stolen data published anyway.

The findings come in a report today from Coveware, a company that specializes in helping firms recover from ransomware attacks. Coveware says nearly half of all ransomware cases now include the threat to release exfiltrated data.

“Previously, when a victim of ransomware had adequate backups, they would just restore and go on with life; there was zero reason to even engage with the threat actor,” the report observes. “Now, when a threat actor steals data, a company with perfectly restorable backups is often compelled to at least engage with the threat actor to determine what data was taken.”

Coveware said it has seen ample evidence of victims seeing some or all of their stolen data published after paying to have it deleted; in other cases, the data gets published online before the victim is even given a chance to negotiate a data deletion agreement.

“Unlike negotiating for a decryption key, negotiating for the suppression of stolen data has no finite end,” the report continues. “Once a victim receives a decryption key, it can’t be taken away and does not degrade with time. With stolen data, a threat actor can return for a second payment at any point in the future. The track records are too short and evidence that defaults are selectively occurring is already collecting.”

Image: Coveware Q3 2020 report.

The company said it advises clients never to pay a data deletion ransom, but rather to engage competent privacy attorneys, perform an investigation into what data was stolen, and notify any affected customers according to the advice of counsel and application data breach notification laws.

Fabian Wosar, chief technology officer at computer security firm Emsisoft, said ransomware victims often acquiesce to data publication extortion demands when they are trying to prevent the public from learning about the breach.

“The bottom line is, ransomware is a business of hope,” Wosar said. “The company doesn’t want the data to be dumped or sold. So they pay for it hoping the threat actor deletes the data. Technically speaking, whether they delete the data or not doesn’t matter from a legal point of view. The data was lost at the point when it was exfiltrated.”

Ransomware victims who pay for a digital key to unlock servers and desktop systems encrypted by the malware also are relying on hope, Wosar said, because it’s also not uncommon that a decryption key fails to unlock some or all of the infected machines.

“When you look at a lot of ransom notes, you can actually see groups address this very directly and have texts that say stuff along the lines of, Yeah, you are fucked now. But if you pay us, everything can go back to before we fucked you.'”

Cryptogram Determining What Video Conference Participants Are Typing from Watching Shoulder Movements

Accuracy isn’t great, but that it can be done at all is impressive.

Murtuza Jadiwala, a computer science professor heading the research project, said his team was able to identify the contents of texts by examining body movement of the participants. Specifically, they focused on the movement of their shoulders and arms to extrapolate the actions of their fingers as they typed.

Given the widespread use of high-resolution web cams during conference calls, Jadiwala was able to record and analyze slight pixel shifts around users’ shoulders to determine if they were moving left or right, forward or backward. He then created a software program that linked the movements to a list of commonly used words. He says the “text inference framework that uses the keystrokes detected from the video … predict[s] words that were most likely typed by the target user. We then comprehensively evaluate[d] both the keystroke/typing detection and text inference frameworks using data collected from a large number of participants.”

In a controlled setting, with specific chairs, keyboards and webcam, Jadiwala said he achieved an accuracy rate of 75 percent. However, in uncontrolled environments, accuracy dropped to only one out of every five words being correctly identified.

Other factors contribute to lower accuracy levels, he said, including whether long sleeve or short sleeve shirts were worn, and the length of a user’s hair. With long hair obstructing a clear view of the shoulders, accuracy plummeted.

Kevin RuddReimagine Podcast with Eric Schmidt: Democracy After the Pandemic

Podcast originally published 4 November 2020

 

Madeleine Albright (00:05):

We can’t expect miracles immediately. But there has to be an assessment of how the international system works, and also, what America’s role in the world is. I happen to believe that president Clinton was the person that said that we were an indispensable power. He said it first, I just repeated it so often it became identified with me. But there’s nothing about the word indispensable that says, alone. It means that we need to be engaged and a partner, not some country that bosses everybody around and then says that we’ve been victimized. But to have, as a partnership, that deals with what are a whole set of new problems.

Eric Schmidt (00:50):

The Coronavirus pandemic is a global tragedy, but it’s also an opportunity to rethink the world. To make it better, faster for more people than ever before. I’m Eric Schmidt, former CEO of Google and now co-founder of Schmidt Futures, and this is Reimagine, a podcast where trailblazing leaders imagine how we can build back better.

Eric Schmidt (01:17):

In 1945 the world was reeling from successive catastrophes, the previous 30 years included two world wars, The Great Depression, and a global pandemic that left hundreds of millions dead or impoverished. That summer, leaders from democratic nations around the world convened in San Francisco to reimagine global cooperation and leadership. They created a set of institutions and norms that we refer to as the liberal world order, to insure the tragic calamities of the prior decades would never happen again. 75 years later, we find ourselves in a similar place, amid another devastating public health crisis, authoritarianism is surging, the leaders and institutions that have historically guided us through various crises are faltering amid rampant tribalism, conflict and fear.

Eric Schmidt (02:18):

On this episode of Reimagine, former Secretary of State, Madeleine Albright, and former Prime Minister of Australia, Kevin Rudd, will help us understand the trajectory of democracy in global leadership in an increasingly unstable world order. The pandemic has deepened divisions and mistrust and set the world on a different course than it was just a short while ago. We must find a way back to the right path for peace and prosperity to flourish.

Eric Schmidt (02:48):

Joining us now, is former Secretary of State, Madeleine Albright. Secretary Albright was our nation’s 64th Secretary of State, and the first woman to lead the state department. During her illustrious career, she has helped the United States navigate many international crises, and has spent much of the last 50 years advocating for freedom around the world. Born in Prague on the cusp of World War II, Secretary Albright has also seen fascism up close. Her experiences have made her one of the worlds foremost experts on democracy and authoritarianism. She presently teaches at Georgetown and chairs the National Democratic Institute, which works to safeguard elections and promote openness and accountability in governments around the world. Secretary Albright, welcome.

Madeleine Albright (03:31):

Eric, it’s great to be with you. Thank you so much.

Eric Schmidt (03:33):

Now, you’re doing a book tour and promoting a book that you’ve just recently published called Hell and Other Destinations. Tell us about the book. What’s special about this insight?

Madeleine Albright (03:44):

Well, first of all, let me tell you, it kind of starts out by saying, people want to know how I want to be remembered. And I say, “I don’t want to be remembered, because I’m still here.” And I wanted to kind of show the things that I’ve done since I left office. And one of the things I’ve always tried to do is to make whatever I do next more interested than what I did before. Which is a little hard if you’ve been Secretary of State. So the book is based on the fact that as we were leaving the department people were saying, “Well, what are you going to do? You can go back to teaching. You can write books. You can start a company. You can do your democracy work. You can continue with the Truman Foundation. So what do you want to do?” And I said, “I want to do it all.”

Madeleine Albright (04:32):

And so what I’m doing are all those different things, and rationalizing that they all go together and that one informs another, and that I learn an awful lot, and I have. The only problem that I’m having was that I was trying to prove that I was not old, by showing how much I do. And then all of a sudden, I’m categorized as “elderly”, and so making that point, while I’m doing something virtually, is a little bit harder. But I had fun.

Eric Schmidt (05:04):

But you were peripatetic and incredibly productive as Secretary of State, so I think this is just a character of who you are. I don’t think it’s true of before state and after state. You just work this way. This is who you are.

Madeleine Albright (05:17):

Well, I certainly love traveling, and maybe I didn’t like airports, but once I got on the plane it was nice.

Eric Schmidt (05:24):

I’ve always been interested in America’s view of fascism and our lack of understanding of kind of bad government outcomes that we have. In the United States, we assume that democracy is first, always the winner, and second, that it’s always been true. But you have personal knowledge that this is not true. And we hear about fascism, but we don’t really know what it is. Tell us in a way that we can understand, why fascism is to be fought at all costs.

Madeleine Albright (05:53):

Well, first of all, I think people throw around fascism as a term without understanding it, a fascist is somebody that you disagree with or I often talk about a teenage boy who’s father doesn’t allow him to drive and he calls him a fascist. First of all, fascism is not an ideology, it is a method for gaining control, and it is a way of controlling population. And the way I describe it is that, a fascist leader is somebody who take the divisions in society, which happened just to exist for any number of reasons, and exacerbates them. So that it is based on the fact that a fascist leader identifies himself, and by the way, they’re all himselves, and identifies with that group at an expense of another which is then the scapegoat that is to blame for everything and makes the divisions worse. The second characteristic of fascism is that the leader thinks that he’s above the law, and then also calls the press, the enemy of the people. But it is a way to control the population ultimately. But to gain power, and control the population.

Madeleine Albright (07:06):

I decided that in order to understand fascism, I had to go back and see where it originated. And it did obviously originate with Mussolini. And what was interesting about him and how he gained power, was the Italians felt unappreciated because of the role that they had played at the end of World War I by supporting The Allies. So there was an anger and a disappointment. And then also, there were divisions in society and all of a sudden, this leader who was an outsider took advantage of those divisions and exacerbated them. The interesting part was that both he and Hitler, gained power constitutionally. And I think that is also something that is worth thinking about.

Madeleine Albright (07:52):

And so then, I began to look at some of the things that I saw going on in Europe, in Hungary, and in Poland, and then in the Philippians with Duterte, Venezuela. So it’s not something that’s gone, it’s definitely there.

Eric Schmidt (08:07):

So when you think about fascism and you think about democracy, we obviously prefer democracy. We also have authoritarian systems, which don’t appear to me to be too fascist. So for example, China, clearly authoritarian, but it’s at least a system of governance without a lot of freedom. What’s happening with democracy? Is democracy weakening now? For a decade or so, democracy was getting quite a bit stronger.

Madeleine Albright (08:35):

First of all, by the way, I decided that I would say communists were fascist also, because they do control the system. But what I do think is true, is that democracy is a process as much as anything, and it is complicated and it takes time. And it is based on a social contract in which people gave up some of their individual rights, in order to have the government take on duties which were protective or did help the system move forward in exchange for the fact that the citizens would participate and vote and play the role that they need to do in a free society.

Madeleine Albright (09:16):

But, and this is where we have found the issues complicated is, democracy depends on information in many different ways, and democracy also has to provide a system which allows people to speak freely and figure out who they are. But at the same time, also allows them to make a living. And so I always say that, democracy has to deliver both in the political and in the economic field, because people want to vote and eat. But it is complicated.

Eric Schmidt (09:50):

It seems to me that, people are positing what I think is a false choice, between order and freedom. And it should be possible to achieve both. The narrative today about democracy has to do with the impact of the internet and social media, and the fact that specialized groups are getting weaponized if you will, by a combination of finding each other and then exploiting either vulnerabilities loopholes or features of the social media world, where they can get an outlandish level on impact, far greater than they would have before that. Do you believe that this is a threat to the way democracy works or do you think that this is going to get solved relatively easily as people understand it?

Madeleine Albright (10:34):

I think it will get solved. And by the way Eric, something that you don’t know about me is that, I wrote my dissertation on the role of the Czechoslovak press in 1968 because I was always interested in the role of information and political change. And the thing that happened in that was that the people actually knew what the truth was because of Radio for Europe and Voice of America, but their censored pressed wasn’t printing it in any way. So they weren’t able to act on it. They couldn’t figure out how it all went together. And what was interesting was, systematically, the press became uncensored. Also, information played a huge role in what was happening with solidarity in Poland. Which by the way, had a new form of passing on information which was a taped cassette. So when Lech Wałęsa spoke in one factory, they could send it to another one and motivate people to be supportive.

Madeleine Albright (11:33):

And so I’ve always been fascinated by the role of information, and so I am very much, by the technology that is taking place now. I do think that in order for people to participate in a democracy, they need information. That is a key to being able to be a participant that knows what is going on. The question is, and I think this is obviously something that you and others are dealing with is, “How do you allow the freedom to put all kinds of information into the system and yet, not have it be undercut by those who are trying to do something else with it? And how do people distinguish between what is true and what isn’t?”

Madeleine Albright (12:18):

And so I hate to be a relativist in this, but I think it is hard to figure out what the truth is these days. And therefore, just the way any professor, I will say, read or listen to a lot of different sources and try to figure it out. But I do think that at the moment, there is an exploitation by some of the incredible advances in technology that have been made. And the question is, how one has some kind of regulation without undercutting the aspect of the freedom of it. And I think that is very hard, as all of you in silicon valley are really experiencing.

Eric Schmidt (12:58):

It remains an unsolved problem. But lets consider the Chinese argument, and their argument goes something like this, the West has had a disease, and failing for a long time. The Chinese model, which is much more organized, much less free if you will, is more effective at producing the things that people care about. And indeed, if you look at the Coronavirus, even if you take a factor of ten discount on the numbers that they quote, there’s no question that China is largely working. The economic growth is quite strong now, there’s plenty of signals that their demand is growing, while the rest of the world is still struggling with no end in sight to the impact of the virus. One scenario is that this is the beginning of the acceleration of the Chinese model, and that the democracies can not get their act together because of the reasons that we discussed. How do you argue that one way or the other?

Madeleine Albright (13:59):

Well, I can take the opposite view, frankly. First of all, I do think it’s worth going back on something in Chinese history. There is an anger that has created a lot of this, from the fact that China felt disrespected by the West all the years, and imposed upon by some of the Western systems, some good some bad, like the opium war and variety of aspects of things, and felt that there needed to be one party. What is interesting is that we all had a theory, which turns out to have been wrong. Which is, having looked at South Korea, that had a dictatorship, and then that was disposed of and all of a sudden there was the development of a middle class, that the middle class brought with it, a sense of wanting to be able to make decisions about their own lives.

Madeleine Albright (14:53):

They were doing fairly well, but having that capability of not working under a dictatorship, they then began to adopt democratic principles. So there was the thought, that as China was experiencing economic growth and developing a middle class, that they would also go in the direction of having a more open system. It didn’t work, because there was a question about what had happened to the communist party and a new leadership with Xi Jinping, who felt that he had to reinvigorate the base of the party by calling on nationalism in a very strong way, i.e. then going back and say, “We had been limited by the imposition of Western ideas and now we’re going to do things our way.”

Madeleine Albright (15:44):

I do think that I could also argue, that the Chinese system made it difficult for on the virus itself in the beginning, because the people that knew about it were quickly expunged from the system and they weren’t able to speak outside about what was happening in Wuhan and how that was effecting people. And then, because of the way that they hid what was going on, we don’t have to speak about what was going on here, but the Chinese were undermining a lot of the really, way of getting information out. They clearly, have a better system of controlling things. Even if we were functioning better, they can tell people what to do in a way that we never can or want to do.

Madeleine Albright (16:35):

So I think it’s a system that is aggressive in the way that it sees itself and the world. It is, as I said earlier, operating off of the base of nationalism, that they were mistreated, and they still have people that would like to be doing something else. So I don’t see it as a better system. And I can’t, given my own background, see any kind of authoritarian system as one that allows for the evolution of society in a way where people feel that they want and can’t make decisions about their own lives. I can see where it is a competitive system, because at the moment, we are totally disorganized. They have somehow managed also, to get some control over the virus. They have no compunctions. And it isn’t just tracing, as far as the virus is concerned, but it’s literally having images of everybody and knowing where people and what people are doing in society. So once the virus is dealt with, it will be hard to get rid of the control system that has been established by the Chinese Communist Party.

Eric Schmidt (17:48):

I agree with that fear. It turns out the ranking system and the rating system can clearly be used for other forms of social oppression as well as of course, tracking the spread of the disease. Madame Secretary, you mentioned earlier a little bit about fascism and that they were always men. Why is it that most of the successful governments dealing with these problems seem to be headed by women now?

Madeleine Albright (18:15):

Well, I’ve been asked that question and I’ve tried to analyze it, and it is very interesting. First of all, I do think that women have a way of worrying about how other people are doing, and these are generalizations, and our caregivers. I think that one of the aspects is that, I think women are better at multitasking, which allows there to be peripheral vision, to see where the problems are coming from and look at them as ways to solve the problem rather than blaming it one somebody else. I think also, there is an attempt I think, to tell the truth to people and not try to hide how to deal with it and not domineer the aspect of being able to really use the various parts of their governments to spread the word without dominating it.

Madeleine Albright (19:10):

And frankly, I have also made clear that fighting fascism, the women do better with that, frankly. Because again, it is not trying to divide people. Mothers do not like to have one set of their children arguing with another. And I think that thinks are not based so much on ego. The countries that have been successful are Taiwan and New Zealand and Germany, and then Norway and Sweden, Iceland. And a lot has to do with having good communication between the head of state and the people, and trying not to treat them as if they can be totally manipulated, but to level with them and say, “You need to be part of the solution.” And they actually believe in science too, that helps.

Eric Schmidt (20:01):

You were the first female Secretary of State for this country, what advice would you have for Kamala Harris if she were to become the first female Vice President?

Madeleine Albright (20:09):

Well, first of all, it’s an honor to be the first but it’s not the easiest to be first.

Eric Schmidt (20:15):

Yes.

Madeleine Albright (20:16):

Because you are constantly being compared with your predecessors, and there are those, and I’ll say this in my own case, who wonder how I ever got the job. And I have to tell you, when my name came up to be Secretary of State, there were people who said, “Well, Arab countries will not deal with a woman Secretary of State.” And so the Arab ambassadors at the UN got together and said, “We’ve had no problems dealing with Ambassador Albright, we won’t have any trouble deal with Secretary Albright.” I had more problems with the men in our own government.

Eric Schmidt (20:51):

Oh my God. Really?

Madeleine Albright (20:52):

And partially, it had to do with the fact that they had known me too long. I had had them over for dinner, which I helped to pass the plates around. I had been a carpool mother. I was good friends with their family. And then, and I’m sure that this will also happen to Senator Harris, many of them thought, well, why weren’t they in the job when they should be the ones doing it. So I think there will be issues. I think also, that one has to be conscious of the fact that you are also being judged by other women. And I think we have a tendency to be very critical of each other, judgemental, and then also, many times, do project our own sense of inadequacy on other women.

Madeleine Albright (21:40):

And that is partially what I was writing about in this book. Because the most famous statement I ever made was that, “There’s a special place in hell for women who don’t help each other,” which came out of my own experience. So when I was writing that dissertation I was talking about, there were other women who said, “Why aren’t you home with your children or in the carpool line?” And then, and this is very germane to your question just now, I was Geraldine Ferraro’s foreign policy advisor, and traveled with her in 1984, when she was the first women to be on a national ticket. And we were somewhere and a women came up to me and said, “How can she deal with a Russian? I can’t deal with a Russian.” Well, nobody was asking this woman to deal with a Russian.

Madeleine Albright (22:26):

So I think that Kamala will also be judged I think, as to whether X woman could be doing the job that she is doing. So I think we do need to be supportive of each other. That has sometimes been interpreted to mean that I say, “Women have to vote for each other.” I have never said that. I do think however, we need to be supportive of each other.

Eric Schmidt (22:49):

On the COVID response, you’ve actually written extensively about how we need to reorganize ourselves and in particular, around international responses. You’re uniquely, I think, concerned about the structure of the world going forward, after this is hopefully over. I was reading about this, you talked about additional resources for low income countries, especially Latin America and Africa, conflict areas where the disease is going to be terrible, but more importantly, they’re in conflict anyway, and then support democracy and good governance in general. Is that going to happen? How will it happen? How will you make that happen from your position?

Madeleine Albright (23:31):

Well, let me just say, one of the things that I have been very conscious of is, we are operating with international organizations that were created, most of them 1945, at the end of World War II. And they do need refurbishing. They need updating in a number of different ways. So that’s for number one. But I also think that we have to recognize that the threats that are out there now know no borders. So whereas the virus might have started in China, it has definitely spread, climate change in another issue that is multinational, nuclear proliferation. So there are a number of aspects that have to be considered multilaterally. And by the way, Americans don’t like the word multilateralism, it has too many syllables and it ends in an ism, but the bottom line, is that some of the issues can only be solved by more than one country. So that is for starters.

Madeleine Albright (24:31):

I think that what has to happen is to recognize the fact that the virus has hit different countries in different ways, and countries have their own way of dealing with it. And part of the issue, and as you raised it, is that the developing countries have been working very hard in terms of dealing with some of their economic issues as well as their governance issues. And this is hitting them very hard now in terms of how they deal with, what are the combination of the issues of environmental problems that pushes them to have to move into refugee camps or in fact, how to deal with the various struggles that are going on, and then not enough in terms of resources. If they are told to wash their hands every five minutes, they don’t have enough water to drink. So one has to consider what the issues are.

Madeleine Albright (25:30):

I also do believe that the international system has the capabilities of helping them economically as well as with advice. And we have done that in other cases in terms of being able to control smallpox or working also on control of Polio or later, Ebola. But the system has failed on dealing with COVID. And it’s partially because of what the Chinese didn’t do, and then what they did do. Which is I do think that they have contributed a lot more than was expected to the World Health Organization, and there are politics everywhere and it all needs to be fixed in some form or another. But also, the fact that the United States has not seen it as a threat and has not recognized the fact that not only is it that the virus knows no borders, but that it’s effect will also affect our economic policies, trade, what can be done, and how people can exist within their countries and whether it is then contributing to a deficit in democracy. Because we are not the best example at the moment.

Madeleine Albright (26:46):

So there are an awful lot of things that have to happen, we can’t expect miracles immediately. But there has to be an assessment of how the international system works and also what America’s role in the world is. I happen to believe that President Clinton was the person that said that we were an indispensable power. He said it first, I just repeated so often it became identified with me. But there’s nothing about the word indispensable that says, alone. It means that we need to be engaged and a partner. Not some country that bosses everybody around and then says that we’ve been victimized, but as a partnership that deals with what are a whole set of new problems.

Eric Schmidt (27:32):

A few weeks ago, you wrote an op-ed about all of this, talking about the American election. And you wrote and I’ll quote, “Mr. Biden, if elected, will inherit a country diminished by his predecessor’s surge for greatness in all the wrong places. The new president’s task will be daunting to reassure allies, reassert leadership on climate change and world health, forage effect coalitions to check the ambitions of China, Russia, and Iran, and establish the U.S.’s identity as a champion of democracy.” Do you believe incoming Biden presidency will be able to do this?

Madeleine Albright (28:08):

I do believe. I don’t think that it can happen all at once. And I also believe the opposite, that another four years of Trump will make our situation impossible in so many different ways. I really do think that another four years of this will be a disaster. There’s no other way to state that. I have been around enough and even now, virtually, to think that it is un-American in every single way, and we are part of the major issue in the functioning of the world.

Madeleine Albright (28:40):

But I do think that Vice President Biden is Uniquely qualified given his experience, to deal with a variety of these issue. He has seen how the system can work in terms of the international aspect of it. He believes and he’s talked about, having a summit of democracies, which would really look at best practices and what can be done. He also, I think, has talked about the power of our example, that I mentioned, just generally. But I think we have to also recognize that it’s going to take a certain amount of humility. We can’t all of a sudden say, “Okay, we’ve had the selection and now we’re in charge again.”

Madeleine Albright (29:24):

I think it is going to take a deliberate effort to explain where we are, the issues that we’ve had. Then in fact, also spend time as a partner trying to sort out how to generally behave in this 21st century, and think about how technology can be out partner and our friend, how we can acclimate our selves to… I don’t think anythings going to be the same after this whole pandemic. And that we need to sort out what the tools are that we have, in order to have a functional world, where we do not divide people more, and where the United States does have a partnership role, and understand that our domestic situation can only be made better in partnership with others.

Madeleine Albright (30:18):

So it’s a very big assignment, there’s no question about that. And my last foreign trip frankly, was to go to Munich, for the Munich Security Conference. And we were a joke, because Pompeo and Esper were there, and the way they talked about the United States was just totally out of lala land. And the other countries were looking at what some of the solutions could be, were concerned. And I think that we need to get a reality check about the way we are viewed. And by the way, one of the things that we need to go back and look at is, how did this all happen. And the best quote in my book on fascism is from Mussolini, that, “If you pluck a chicken one feather at a time, nobody notices.” So there has been an awful lot of feather plucking and we need to either get a new chicken or stop the feather plucking.

Eric Schmidt (31:15):

Madame Secretary, I want to congratulate you on your new book which is called, Hell and Other Destinations: A 21st-Century Memoir. Thank you again, I look forward to your next book and the product of your great work at Georgetown.

Madeleine Albright (31:28):

Thank you very much. I’ve enjoyed being with you Eric, and what you’re doing in your podcast.

Eric Schmidt (31:32):

The primary goal of a democracy is to keeps it’s people safe and get them to be prosperous. Our democracies have failed on both parts of that so far. We accept that democracies are really groups of people who are lobbying and shaping information, and so forth. But ultimately, great leaders should emerge, leaders that somehow can judge where the risks are and make the right balance of trade offs, so that the society can, at the end of the transaction, be more prosperous, safer, and so forth. Where will those leaders come from? They’re not going to come from leaders who spend all their day testing their popularity, and they’re not going to come from leaders that are beholden to special interests. They’re going to come from the leaders of the old time. The ones who started with a principle of what they were trying to do and stuck to it, a principle around greatness, and success, and safety, and so forth. The leaders who choose to pander to the crowds, to ignore facts, and to focus only on themselves and their own narcissism are destined for a terrible history.

Eric Schmidt (32:48):

Secretary Albright’s experience is invaluable as we lay the foundation for the next chapter of international coexistence. Our next guest, former Australian Prime Minister, Kevin Rudd, will help us continue to look toward the future by helping us understand one of the growing forces shaping world affairs, China. Prime Minister Rudd is an expert on China and currently serves as the president of the Asia Society Policy Institute in New York City. As Prime Minister of Australia from 2007 to 2010, and then in 2013, Kevin was an active leader in global affairs. He ratified the Kyoto Protocol and committed Australia to decreasing carbon emissions. On the domestic front, he helped Australia survive the global financial crisis as the only major developed country to not slip into a recession. Among many other accomplishments, he delivered Australia’s first national apology to indigenous Australians as his first act as prime minister, and made significant investments in schools and education. Prime Minister Rudd, thank you so much for being here with me.

Kevin Rudd (33:50):

It’s good to be with you, Eric.

Eric Schmidt (33:51):

So lets look at what happened with Australia and COVID. As best I can tell, the COVID crisis accelerated a break between Australia and China. Can you explain how this break happened and how the positioning of COVID in China now feel if you’re in Australia?

Kevin Rudd (34:11):

I think the first think is, as you and I both know, is that China has significantly changed. Xi Jinping’s China is radically different from the China before 2012, 2013. It’s certainly more assertive in terms of it’s international policy across the board. And so that’s been building over the last six or seven years. Plus the second point is this, being a Western country located in the East, we’ve kind of been the first Western country down the Chinese mineshaft, that is first Western canary down Chinese mineshaft so we’ve experienced first and upfront a lot of the direct challenges in terms of the ultimate tension between economic policy and security policy. Australia is one of America’s oldest allies. China takes, would you believe, more than one third of Australian turtle exports. And of course, we’re from radically different human rights traditions. So for those reasons, it’s structural.

Kevin Rudd (35:05):

And finally, what’s happened most recently, I think it’s because we had the eruption of the virus coming out of China, we had it’s impact on all countries in the world, including the horror that unfolds in the United States, and a more manageable problem here in Australia. But still, big questions on the mind of the Australian public as to how this thing came about in the first place. Put all those things together and Australian advocacy for international inquiry into the origins of the Coronavirus, and it adds up to a cocktail of a deeply negative state of the Australia, China relationship.

Kevin Rudd (35:46):

And one final point is, what our Chinese friends have been doing with various American allies around the world, and friends around the world, is kind of make and example of them. You’ve seen that with the Canadians, over the Madame Meng affair on Huawei. You’ve seen it recently with the Swedes who have had their own human rights challenges with China considering various of their Chinese Swedish citizens. You now see it of course, with emerging problems for the British over Huawei. And then there’s the Australians. So I think what tends to happen is that, individual countries are singled out to particular treatment if they don’t comply with China’s foreign policy wishes, in order to set examples for the rest.

Eric Schmidt (36:31):

Well, it’s interesting that Australia was the first to call for an independent investigation of what was going on, which ultimately the WHO took on, and Australia and the current prime minister pushed very hard. What penalty has China extracted from Australia today, in your opinion?

Kevin Rudd (36:54):

Well, the complexity of this is a bit like this, I suppose number one it, as a middle power like Australia again, to call for such an independent investigation of the origins of Coronavirus, it’s usually helpful to hunt in packs. By which I mean, bring a Coalition of the Policy Willing with you. What the Australian government did was, went out there and unilaterally call for this, which makes it much easier for the Chinese then to single you out.

Kevin Rudd (37:18):

The second point is, just for the clarity of the record, that the independent inquiry into the origins of the virus is somewhat different to what we ended up with, with this WHO investigation into effectively, the WHO’s performance and not much beyond that. But it is something. And I suppose on the key question of, how is Australia been singled out, I suppose I’d point to three measures. One, is travel warnings to Chinese tourists not to come back to Australia because it’s unsafe. Two, warnings to Chinese students studying in Australia, that it’s also unsafe because of alleged racist reaction to Chinese in Australia. And number three, in specific commodity areas, like Australian barley, Australian Beef, and potentially Australian wines. The Chinese have used various so called quarantine and WTO related measures to effectively switch their sources of supply. And ironically, American supplies is moving into some of those opportunities. So there you go. That’s the background.

Eric Schmidt (38:30):

But building on this, you have been critical of the American response. I’m quoting you, “America would have mobilized the world, but in this time, in America’s absence, no one did.” And indeed France convened the G7, and the G20 was summited by Saudi Arabia and so forth. Do you have a view now of this that’s different? Do you see any change in the American role? Is it getting worse or better from your perspective?

Kevin Rudd (39:02):

If you’re concerned about the stability and effectiveness of the global rules based order, which through painstaking leadership, Americans, together with their friends and allies have put together out of the ashes of the second world war, then you’ve got to stand back and look at the policies and posture and actions of the Trump administration and just kind of scratch your head. So take the COVID-19 crisis, yes, it’s been a domestic challenge for all of us. But when you have a monumental global assault on public health and a global assault on the economy and employment in virtually all countries, than the instantaneous response for those of us who are friends and allies in the United States, and others, is to look for American global leadership.

Kevin Rudd (39:51):

Instead what we found with Trump was, the guy behaving domestically, as I read recently, like some 19th century quack apothecary, recommending kind of unbelievable medical treatments for this condition. But when it came to global action, either the global provision of PPE protective equipment, or global leadership on vaccine development, et cetera then the America we’ve come to know and respect, and most of us to love over the decades, was just not there.

Kevin Rudd (40:28):

So this creates a significant vacuum in the mind of global public opinion. And this fall, we look forward to the next presidential election and Joe Biden’s elected, it’s what I’ve described and stuff I’ve written recently for Foreign Affairs Magazine, is kind of the last chance saloon for American global leadership. We want America back. We want you to work closely with your allies. There’s so much to be done in the world not just on pandemics but climate and the rest, and having America back in the saddle is what we’d really like to see. But it is frankly, a last chance saloon to get this done.

Eric Schmidt (41:05):

In the Foreign Affairs piece that you’d recently published, you actually argue that both china and the U.S. will emerge “Severely damaged,” I think is the phrase. And severe damage is a pretty strong statement. And it seems to me that it’s a race to the bottom, whether it’s the politics or the change in the politics inside of China, she is as you pointed out, much more authoritarian, Trump is a different kind of leader than our previous presidents, as everyone is established. Describe the weakening and then tell us how you would fix it, on both sides.

Kevin Rudd (41:45):

Wow, there’s a big question, or a couple of big questions. Firstly on the diagnostics, lets just look at the United States first. There’s a huge economic hit on the United States, which we will not know the full dimensions of for several years. And that’s going to effect the future budget resilience of the United States as well. Ultimately, America can only print money for so long. Ultimately, there has to be a rebalancing of the system. And I say that as someone who has a deeply cleansing approach for how you fix economies in a time of systemic international crisis. But the truth is, the objective truth is, it’s a massive economic hit and it’s a massive budgetary hit. Which obviously then has implications to what you can do with the government in the future and funding the future of the U.S. [inaudible 00:42:39] and the rest.

Kevin Rudd (42:40):

But do you know something? There’s also been this hit on the American soft power. What we talked about before Eric, was American global leadership. And frankly, you friends and allies are just around the world, holding their breath and waiting for November for a decision by the American people as to what leadership they want America to exercise in the world in the future as well. But in the meantime, there’s been a huge reputational hit on the U.S. standing.

Kevin Rudd (43:07):

But what I find, is people often then therefore go into an automatic equation which says, “America down, therefore China up.” Well, not so. The Chinese economy has taken a huge hit itself. We really have to go back to the cultural revolution to see such disastrous economic numbers as we’ve seen emerge from China in recent quarters. And therefore, that flows through their own budgetary capacity to fund The Belt and Road Initiative, to fund what they’re doing through their military, to fund their expanding international development program, et cetera. And so it becomes a huge economic and financial equation for the Chinese State as well. And remember, China is probably, I wouldn’t double dependent, but significantly dependent on the global economy through trade and investment flows as a key part of their formula for long term, sustainable growth for themselves.

Kevin Rudd (44:06):

So what do I say emerges as a result of that, post COVID, whenever post COVID comes, Eric, is likely to see these two wounded elephants roaming around in the global living room, and as a consequence, we no longer have anyone leading effectively, the global order, and the systems and institutions of international governance, which have kept us basically, outside of barbarism for the last three quarters of a century. And what I see is these institutions dying the death of a thousand cuts, and now increasingly becoming, as it were, vulcanized into pro-American and pro-Chinese camps with neither of the super powers willing or able to exercise effective leadership. So it leads to what I’ve described as an emerging international anarchy.

Kevin Rudd (44:54):

So what can you do about it? Two things, perhaps three. Start with those of us who are not Americans and Chinese, what I’ve written about extensively in the Economist and elsewhere is, it’s time for a Coalition of the Policy Willing what I call the M7 or the M10, the middle power 7, or the middle power 10, countries like France, Germany, the U.K., once it decides what it wants to do in the future, maybe the Swedes, the Japanese, the South Koreans, the Australians, the Indians, and the Canadians, and the Mexicans, these are all democracies, they’re all middle powers, and that is, how do you exercise through them financial, diplomatic, and political measures to triage the international system until we have the reestablishment of the level of geo-political equilibrium, involving the great powers.

Kevin Rudd (45:49):

And as for the United States, as I said, it really hinges on November. If Americans decide they wish to be the worlds leaders in the future or be it perhaps in a different way in the past, and not simply a replication of past forms, then the world is looking to see what America under Biden would do. And that means fixing your house at home, Black Lives Matter, but basically the inequality which drives it, and rediscovering your confidence in the world.

Kevin Rudd (46:19):

And as for China, China’s not a done deal under Xi Jinping. Now you’ve just said before in your intro to this part of our conversation, Eric, that you and I share many friends. And lets say there are world views in China quite different to the ones we see articulated by Xi Jinping’s administration. And these are essentially internationalizing world views. These are more liberal internationalist world views. These are more open economy and increasingly open society world view, though with a question mark on the continued centrality of the Chinese Communist Party in a one party state. And so it really depends where shakes down in Chinese politics and the lead up to the 2022, 20th Party Congress, and whether Xi Jinping easily secures his reappointment.

Eric Schmidt (47:06):

My final question, you’ve spent your whole life studying China, you studied the language, you did it academically, you wrote a PhD on the dissidents. Did you foresee the rise of China in this way, the new strong, powerful china? When did you know this was the path?

Kevin Rudd (47:29):

Did I actually see China turning out this way? I think most of us who lived and worked Beijing as I did in the 1980s, when I was a Junior Woodchuck in the Australian Embassy back then, analyzing the earliest days of political and economic reform in the Chinese system, we had a degree of optimism that China would evolve in the direction of more open economy, more open society, and perhaps in time open politics. I think though, having been myself, in Tiananmen Square about a week or so before the tanks moved in, and having spent the better part of a week prior to that walking around and talking to the students back then in the square, many of whom were subsequently killed, I was always deeply skeptical about whether a Leninist Party, like the Chinese Communist Party, would ever voluntarily surrender political power. As we saw with a combination of Galsnost, and Perestroika in the then, Soviet Union.

Kevin Rudd (48:38):

So I’ve seen China as moving in the direction of certainly a more open economy, because they don’t want to return to poverty. I see that as generating the social pressures that you and I have both experienced in China in people wanting more freedom in their personal lives. But to be honest, I’ve always been skeptical as to whether the communist party, being deeply rooted in it’s Leninist traditions, would ever see itself and it’s self interest handing over power to a more open elected political entity. The Chinese communist party calls this the theory of “Peaceful transitionism,” and it’s something which the communist party regards internally, as political enemy number one. So yes, I saw China becoming more open, but always with a big doubt in my mind having been in Tiananmen way back when, 30 years ago now, that it would every voluntarily open it’s politics to sort of transitions we’ve seen elsewhere.

Eric Schmidt (49:46):

Thank you Prime Minister Kevin Rudd, you’re incredibly insightful on all such matters. Thank you again.

Kevin Rudd (49:53):

Thanks for having me on your podcast, Eric.

Eric Schmidt (49:58):

Where are we now? The liberal world order is not as free, global, or organized as it could be 75 years after the democratic nations created it. COVID has deepened fissures in the international system and accelerated our slide toward anti-democracy. In a pandemic, we have not seen tremendous leadership out of the largest democracies. Instead, we’ve seen compromise, and in compromise comes death. Because they have not figured out how to collectively manage both health and economic growth. It’s a false choice to tell people to choose between heath and economic growth, you have to solve both at the same time.

Eric Schmidt (50:40):

75 years ago, not just the winner of the war, but the leader of the free world, the United States, set the global world order, set the rules, set the way that the institutions would work, and set a style of approach of solving problems. Today, the United States has relegated that role to others. That loss of leadership means that the world does not have a natural organizational point. It’s probable that the world will devolve a bit, becoming a little bit more confusing. And during a pandemic, you need strong centralized leadership as opposed to confusion and lack of leadership. The most important thing now with democracies, is to recognize that democracies have a certain shape and a certain set of values and to restate them, and to call out behaviors that are inconsistent with democratic values, and strengthen those democratic values.

Eric Schmidt (51:35):

I’m quite convinced that democracies with strong values and a lot of voter participation, will do just fine. The most important thing in our democracy is to increase voter participation so that people have a share in the outcome. Study after study indicates that generations that don’t participate, don’t buy into the leadership, they don’t buy into the decisions, they don’t have a shared sense of the outcome, and they ultimately become troublemakers. Over and over again, we want very high participation and I think we’re going to get it this time.

Eric Schmidt (52:08):

Secretary Albright and Prime Minister Rudd, have helped us understand some of the major past, present, and future forces shaping the story, but thankfully, the story’s not over. We must reimagine democracy in global leadership for a hyper connected and technological world. We must reaffirm liberal democracy as the most fair and effective form of governance. And we must call on the nations that uphold these human values and rights to steer the international system through this century and beyond. We’ve done this before, I know we can do it again.

Eric Schmidt (52:40):

On the next episode of Reimagine, we’ll finish our season by reimagining our lives, planet, and universe with astrophysicist, Neil deGrasse Tyson.

The post Reimagine Podcast with Eric Schmidt: Democracy After the Pandemic appeared first on Kevin Rudd.

Kevin RuddNikkei Asia: China, Japan and South Korea have good news for planet Earth

Published in Nikkei Asia on 4 November 2020

No matter who is declared the winner of the U.S. presidential election, Asia’s pathway to becoming a carbon-neutral continent is now increasingly clear.

Six months ago, Asia was lagging desperately behind the rest of the world, including South America and Africa, in its commitment to achieving net zero emissions by midcentury. Only the governments of New Zealand, plus the Marshall Islands and Fiji as the usual vanguards of international climate leadership, had made such a commitment and — importantly — also enshrined it in domestic legislation.

The recent groundbreaking commitments by China, Japan, and South Korea, mean the three largest economies in East Asia now have clear pathways to decarbonization by mid-century. In terms of Asia’s G-20 membership, only India, Australia and Indonesia now lag behind.

Importantly, Japan and Korea’s announcements will also help put pressure on China to hopefully reach carbon neutrality closer to 2050 — around the time of the 100th anniversary of the founding of the People’s Republic of China — and to achieve net zero greenhouse gas emissions a decade later. These pathways remain an open debate in Beijing’s political circles, including in the wake of last week’s Fifth Plenum, and as preparations continue toward their next Five Year Plan.

We may see further signals from China on this by the time of the UN Secretary-General’s event to celebrate the 5-year anniversary of the signing of the Paris Agreement on December 12, and the world will certainly be watching closely. More so if Joe Biden is set to move into the White House the following month, meaning close to 60% of the world’s carbon emissions will then be from countries committed to net zero emissions.

Beyond the symbolism of these political commitments, they are first and foremost massive market signals. This is especially the case for China, Japan and Korea’s major trading partners, including their largest import markets for coal. But, it is important to note, they are not out of step with the direction of Asia’s biggest companies in recent years.

For example, the Thai conglomerate C.P. Group, one of the world’s largest agri-food producers, had already committed to net zero emissions. In recent days, Malaysia’s Petronas — the region’s largest oil and gas producer — joined them. Even BHP Billiton in my own country — one of the world’s largest mining companies and no fan of climate action — has adopted the same goal.

These announcements reflect what is happening at a subnational level in Asia. In the last year, Asia has outpaced the rest of the world in terms of commitments by cities and regions to net zero emissions with Tokyo, Wuhan, Hong Kong and eight Australian states and territories all joining the list. Taken together, they alone represent over 223 million people, or 10% of the region’s population. This leadership has been a key part of why the approach of these three national governments has now shifted. The challenge now for the region is threefold.

First, at a political level, to holistically embrace the vision of becoming a carbon-neutral continent in the same way Europe has done. This will be a much harder enterprise than it has been in Europe, even with some of their coal-dependent economies and right-wing governments, but it is not impossible.

Key to this will be driving consideration of more national-level commitments to net zero emissions, especially among the 10-member Association of Southeast Asian Nations, which represent more of a mixed bag in this regard. This could be a key area for cooperative regional leadership between China’s President Xi Jinping, Japan’s Prime Minister Yoshihide Suga, and South Korea’s President Moon Jae-in, including in the lead-up to next year’s COP26 Climate Conference in Glasgow.

Second, these governments must put their money where their mouths are and stop underpinning the development of carbon-intensive infrastructure — especially coal-fired power plants — across the rest of the region. Japan and Korea have taken important steps in this regard recently, but there is more still to be done. China’s Belt and Road Initiative is obviously a particular concern in that context, especially when Chinese investment, development finance, and support via equipment or personnel is taken together.

Third, these governments should align their short-term actions with their long-term vision. In China, Japan and South Korea, the challenges to do so may be different, but the problem is the same. Unless each of these three countries can also enhance their Paris targets for 2030 by the time they get to COP26, the depth and sincerity of their long-term commitments will come increasingly under the spotlight. For China, this must mean peaking emissions by 2025, accelerating action in other areas that they committed to do in Paris, and getting on a pathway to phase out coal by 2040. For Japan and Korea, it must mean phasing out coal even sooner — by 2030 — and seriously ramping up the share of renewables in their energy mix.

There is clearly a new wave of climate leadership emerging across Asia. The main question now for the region is whether it is able to ride that wave successfully, or whether its own actions in the short term, or lack of wider regional momentum, risks bringing it to a shuddering halt as the rest of the world moves forward.

The post Nikkei Asia: China, Japan and South Korea have good news for planet Earth appeared first on Kevin Rudd.

Worse Than FailureAnnouncements: What The Fun Holiday Activity?

The holidays are a time of traditions, but traditions do change. For example, classic holiday specials have gone from getting cut down for commercials, to getting snapped up by streaming services. Well, perhaps it's time for a new holiday tradition. A holiday tradition which includes a minor dose of… WTF.

We're happy to announce our "Worse Than Failure Holiday Special" Contest. This is your chance to submit your own take on a very special holiday story. Not only is it your chance to get your place in history secured for all eternity, but also win some valuable prizes.

What We Want

We want your best holiday story. Any holiday is valid, though given the time of year, we're expecting one of the many solstice-adjacent holidays. This story can be based on real experiences, or it can be entirely fictional, because what we really want is a new holiday tradition.

The best submissions will:

  • Contain a core WTF, whether it's a bad boss, bad technology decisions, or incompetent team members
  • Prominently feature your chosen holiday
  • End with a valuable moral lesson, that leave us feeling full of holiday cheer

Are you going to write a traditional story? Or maybe a Dr. Seussian rhyme? A long letter to Santa? That's up to you.

How We Want It

Submissions are open from now until December 11th. Use our submission form. Check the "Story" box, and set the subject to WTF Holiday Special. Make sure to fill out the email address field, so we can contact you if you win!

What You Get

The best story will be a feature on our site, and also receive some of our new swag: a brand new TDWTF hoodie, a TDWTF mug, and a variety of stickers and other small swag.

The 2 runners up will also get a mug, stickers and other small swag.

Get writing, and let's create a new holiday tradition which helps us remember the true meaning of WTFs.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

Worse Than FailureCodeSOD: When All You Have Is .Sort, Every Problem Looks Like a List(of String)

When it comes to backwards compatibility, Microsoft is one of those vendors that really commits to it. It’s not that they won’t make breaking changes once in awhile, but they recognize that they need to be cautious about it, and give customers a long window to transition.

This was true back when Microsoft made it clear that .NET was the future, and that COM was going away. To make the transition easier, they created a COM Interop system which let COM code call .NET code, and vice versa. The idea was that you would never need to rewrite everything from scratch, you could just transition module by module until all the COM code was gone and just .NET remained. This also meant you could freely mix Visual Basic and Visual Basic.Net, which never caused any problems.

Well Moritz sends us some .NET code that gets called by COM code, and presents us with the rare case where we probably should just rewrite everything from scratch.

    ''' <summary>
    ''' Order the customer list alphabetically
    ''' </summary>
    ''' <returns></returns>
    ''' <remarks></remarks>
    Public Function orderCustomerAZ() As Boolean
      Try
        Dim tmpStrList As New List(Of String)
        Dim tmpCustomerList As New List(Of Customer)
        ' We create a list of ID strings and order it      
        For i = 0 To CustomerList.Count - 1
          tmpStrList.Add(CustomerList(i).ID)
        Next i
        tmpStrList.Sort()
        ' We create the new tmp list of customers
        For i = 0 To tmpStrList.Count - 1
          For j = 0 To CustomerList.Count - 1
            If CustomerList(j).ID = tmpStrList(i) Then
              tmpCustomerList.Add(CustomerList(j).Clone)
              Exit For
            End If
          Next j
        Next i
        ' We update the list of customers
        CustomerList.Clear()
        CustomerList = tmpCustomerList
        Return True
      Catch ex As Exception
        CompanyName.Logging.ErrorLog.LogException(ex)
        Return False
      End Try
    End Function

As the name implies, our goal is to sort a list of customers… by ID. That’s not really implied by the name. The developer responsible knew how to sort a list of strings, and didn’t feel any need to learn what the correct way to sort a list of objects were.

So first, they build a tmpStrList which holds all their IDs. Then they Sort() that.

Now that the IDs are sorted, they need to organize the original data in that order. So they compare each element of the sorted list to each element of the unsorted list, and if there’s a match, copy the element into tmpCustomerList, ensuring that list holds the elements in the sorted order.

Finally, we clear out the original list and replace it with the sorted version. Return True on success, return False on failure. This last bit makes the most sense: chucking exceptions across COM Interop is fraught, so it’s easier to just return status codes.

Everything else though is a clear case of someone who didn’t want to read the documentation. They knew that a list had a Sort method which would sort things like numbers or strings, so boom. Why look at all the other ways you can sort lists? What’s a “comparator” or a lambda? Seems like useless extra classes.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

,

Krebs on SecurityTwo Charged in SIM Swapping, Vishing Scams

Two young men from the eastern United States have been hit with identity theft and conspiracy charges for allegedly stealing bitcoin and social media accounts by tricking employees at wireless phone companies into giving away credentials needed to remotely access and modify customer account information.

Prosecutors say Jordan K. Milleson, 21 of Timonium, Md. and 19-year-old Kingston, Pa. resident Kyell A. Bryan hijacked social media and bitcoin accounts using a mix of voice phishing or “vishing” attacks and “SIM swapping,” a form of fraud that involves bribing or tricking employees at mobile phone companies.

Investigators allege the duo set up phishing websites that mimicked legitimate employee portals belonging to wireless providers, and then emailed and/or called employees at these providers in a bid to trick them into logging in at these fake portals.

According to the indictment (PDF), Milleson and Bryan used their phished access to wireless company employee tools to reassign the subscriber identity module (SIM) tied to a target’s mobile device. A SIM card is a small, removable smart chip in mobile phones that links the device to the customer’s phone number, and their purloined access to employee tools meant they could reassign any customer’s phone number to a SIM card in a mobile device they controlled.

That allowed them to seize control over a target’s incoming phone calls and text messages, which were used to reset the password for email, social media and cryptocurrency accounts tied to those numbers.

Interestingly, the conspiracy appears to have unraveled over a business dispute between the two men. Prosecutors say on June 26, 2019, “Bryan called the Baltimore County Police Department and falsely reported that he, purporting to be a resident of the Milleson family residence, had shot his father at the residence.”

“During the call, Bryan, posing as the purported shooter, threatened to shoot himself and to shoot at police officers if they attempted to confront him,” reads a statement from the U.S. Attorney’s Office for the District of Maryland. “The call was a ‘swatting’ attack, a criminal harassment tactic in which a person places a false call to authorities that will trigger a police or special weapons and tactics (SWAT) team response — thereby causing a life-threatening situation.”

The indictment alleges Bryan swatted his alleged partner in retaliation for Milleson failing to share the proceeds of a digital currency theft. Milleson and Bryan are facing charges of wire fraud, unauthorized access to protected computers, aggravated identity theft and wire fraud conspiracy.

The indictment doesn’t specify the wireless companies targeted by the phishing and vishing schemes, but sources close to the investigation tell KrebsOnSecurity the two men were active members of OGusers, an online forum that caters to people selling access to hijacked social media accounts.

Bryan allegedly used the nickname “Champagne” on OGusers. On at least two occasions in the past few years, the OGusers forum was hacked and its user database — including private messages between forum members — were posted online. In a private message dated Nov. 15, 2019, Champagne can be seen asking another OGusers member to create a phishing site mimicking T-Mobile’s employee login page (t-mobileupdates[.]com).

Sources tell KrebsOnSecurity the two men are part of a larger conspiracy involving individuals from the United States and United Kingdom who’ve used vishing and phishing to trick work-at-home employees into giving away credentials needed to remotely access their employers’ networks.

LongNowExplorers Discover Pinnacle of Coral Taller Than Empire State Building in Great Barrier Reef

Even now, even in shallow waters, the sea continues to surprise us with new wonders (many of them rich in “living fossils” like the chambered nautilus and various sharks).

Reefs are themselves fabulous living examples of multitudinous pace layers, not unlike the structural layers of a house Stewart Brand details in How Buildings Learn—only these buildings literally do learn, as scaffolded colonial organisms with their own inarguable (and manifold) agencies:

Explorers of the Great Barrier Reef have discovered a giant pinnacle of coral taller than the Empire State Building.

Mariners long ago charted seven pinnacle reefs off the cape that, by definition, lie apart from the main barrier system. Bathed in clear waters, the detached reefs swarm with sponges, corals and brightly colored fish — as well as sharks — and are oases for migrating sea life. Their remoteness makes the pinnacles little-studied, and Australia’s Great Barrier Reef Marine Park Authority has assigned them its highest levels of protection, which limit such activities as commercial fishing. One detached reef at Raine Island is the world’s most important nesting area for green sea turtles.

The new pinnacle was found a mile and a half from a known detached reef. Dr. Beaman, who formerly served in the Royal Australian Navy as a hydrographic surveyor, said he and his team were certain it was previously unknown. Its seven relatives, he added, were all charted in the 1880s, more than 120 years ago.

Charles StrossEditorial Entanglements

A young editor once asked me what was the biggest secret to editing a fiction magazine. My answer was "confidence." I have to be confident that the stories I choose will fit together, that people will read them and enjoy them, and most importantly, that each month I'll receive enough publishable material to fill the pages of the magazine.

Asimov's Science Fiction comes out as combined monthly issues six times a year. A typical issue contains ten to twelve stories. That means I buy about 65 stories a year. Roughly speaking, I need to buy five to six stories per month--although I may actually buy two one month and ten the next. That I will receive these stories should seem inevitable. I get to choose them from about eight hundred submissions per month. Yet, since I know that I will have to reject over 99 percent of the stories that wing their way to me, there is always a slight concern that that someday 100 percent of the submissions won't be right for the magazine.

Luckily, this anxiety is strongly offset by a lifetime of experience. For sixteen years as the editor-in-chief, and far longer as a staff member, I've seen that each issue of the magazine has been filled with wonderful stories. Asimov's tales are balanced, they are long and short, amusing and tragic, near- and distant-future explorations of hard SF, far-flung space opera, time travel, surreal tales and a little fantasy. They're by well-known names and brand new authors. I have confidence these stories will show up and that I'll know them when I see them.

I have edited or co-edited more than two-dozen reprint anthologies. These books consisted of stories that previously appeared in genre magazines. Pulling them together mostly required sifting through years and years of published fiction. The tales have been united by a common theme such as Robots or Ghosts or The Solar System.

Editing my first original anthology was not like editing these earlier books or like editing an issue of the magazine. Entanglements: Tomorrows Lovers, Families, and Friends, which I edited as part of the Twelve Tomorrow Series, has just come out from MIT Press. The tales are connected by a theme--the effect of emerging technologies on relationships--but the stories are brand new. Instead of waiting for eight hundred stories to come to me, I asked specific authors for their tales. I approached prominent authors like Nancy Kress (who is also profiled in the book by Lisa Yaszek), Annalee Newitz, James Patrick Kelly, and Mary Robinette Kowal, as well as up and coming authors like Sam J. Miller, Cadwell Turnbull, and Rich Larson. I was working with some writers for the first time. Others, like Suzanne Palmer and Nick Wolven, were people I'd published on several occasions.

I deliberately chose authors who I felt were capable of writing the sort of hard science fiction that the Twelve Tomorrows series is famous for. I was also pretty sure that I was contacting people who were good at making deadlines! I knew I enjoyed the work of Chinese author Xia Jia and I was delighted to have an opportunity to work with her translator, Ken Liu. I was also thrilled to get artwork from Tatiana Plakhova.

Once I commissioned the stories, I had to wait with fingers crossed. What if an author went off in the wrong direction? What if an author failed to get inspired? What if they all missed their deadlines? It turned out that I had no need to worry. Each author came through with a story that perfectly fit the anthology's theme. The material was diverse, with stories ranging from tales about lovers and mentors and friends to stories populated with children and grandparents. The book includes charming and amusing tales, heart-rending stories, and exciting thrillers.

I learned so much from editing Entanglements. The next time I edit an original anthology, I expect to approach it with a self-assurance akin to the confidence I feel when I read through a month of submissions to Asimov's.

Worse Than FailureSweet Release

READ FASTER READ BETTER

Release Notes: October 31, 2019

  • Added auto-save feature every five minutes. Auto-saves can be found in C:\Users\[username]\Documents\TheApp\autosaves.
  • Added ability to format text with bold, underline, and italics.
  • Removed confusing About page. Terms and conditions can now be found under Help.

"And ... send." Mark sent the weekly release notes to the distribution list, copying them from where the app itself would display them on boot. "Now everyone should be on the same page, and I can get to work on my next big feature."

Two hours later, Janine, the product manager, stopped by his cube. "Hey, Mark. I was thinking. You know that About page? I keep getting complaints. What would it take to just axe it?"

"Already done in the latest version," he replied, not even looking up from the code.

"So that's, what, three hours of work?"

Mark had to tear his eyes away from the screen to look at Janine, baffled. "Huh? No, it's done. Already. It's gone. Didn't you update this morning?"

"Oh! Already! Okay, thanks. Good work." She vanished, leaving him to reload his train of thought and focus on the refactor he was doing.

Half an hour later, just as he was in the middle of something, one of the users, Roger, dropped in. "Hey, Mark! I know this should go through Janine, but I have a great idea, and I wanted to see if it was feasible."

"Hang on ... okay ... shoot." Mark hit Ctrl-S and focused on Roger. Remember, think customer service.

"Listen," Roger said. "Every once in a while, right, I'm working on something, and someone comes by to interrupt, right?"

"Okay?" began Mark, unclear where this was going.

"And you know how it goes. One thing leads to another, and so on, and eventually, I forget what I was doing, and I close out the program."

"Sure." Mark risked a glance at his IDE, wondering if he had time to start compiling or not.

"So, what if the program saved automatically, like, when I exit or something?" Roger asked.

"Oh, actually, as of this morning it auto-saves every five minutes," Mark said.

"Okay, cool, cool, but like, it should save when I exit."

"Um, I think it asks if you want to save, but I could maybe put that—"

"Or," Roger interrupted, "better yet, it should know when I get distracted, and save then, so I don't lose anything."

"It should ... know? How would it know?"

"Eh, you're right. Maybe it should just save every ten minutes."

Mark pinched the bridge of his nose. "I can do that. What about every five?"

"Perfect! Get right on that," Roger declared, striding away. "Good man."

He'll figure it out eventually, Mark decided, going back to his IDE.

He compiled, ran the software, and was in the middle of testing when Janine came by in a panic, carrying her open laptop. "Mark! We have to roll back the release!"

He didn't wait for auto-save, but exited his debugger, immediately pulling up the release console. "What, what's wrong? What happened?"

"You know how you killed the About page?" she demanded, eyes wide with horror.

"Yeah?"

"Well the Terms and conditions were in there! Legal says we can't ship without terms and conditions! This is a huge priority-one bug, I don't know how you missed it!"

Mark's shoulders slumped as he stopped logging into the release console. "Oh. I put them under Help."

"But I told you to put them under About!"

"And then you told me to kill the About page but keep the Terms and Conditions, so I moved them under Help. Didn't you read the release notes?"

"Oh, right, right, hang on, let me just pull it up here ... oh, never mind, it's under Help. False alarm! Carry on."

So Mark carried on, one eye on the time. I barely got anything done, as usual for a Monday. I really don't want to stay late tonight ... Still, he managed to get into the flow of things, and was just refactoring a critical class when Sue, Mark's boss, stopped by. Mark of course pulled his attention away from the code to talk to the boss, though already he was beginning to resent the constant interruptions.

"Hey, Marky Mark, how's it going?" asked Sue.

"Fine."

"Good, good. Listen, I know you're busy, so I'll get right to it: we have a request from the CEO, so it'll need to get into next week's release for sure."

Feeling his odds of getting the refactor committed evaporating, Mark nodded. "All right, I'm on it. What is it?"

"So, you know how the product can send email, right?"

My least favorite feature. "Yup. What about it?"

"Well the CEO was thinking, he can do stuff in Gmail that you can't do in our product, and he wants to know why."

He wants me to replicate all of Gmail in the product?! "What things, specifically?" Mark managed to ask calmly.

"He's not super technical, but he's talking about things like bold, italics, and underlines. Those are the big three."

Mark smashed his forehead into the keyboard for a moment before lifting his head to mutter: "Why do I even send release notes?"

"What?"

"We released that feature this morning!"

"Oh. Good show! Thanks Mark, you're the best."

Just as he was packing up for the day, Janine stopped by again, knocking on the edge of his cubicle, a phone to her ear. "Mark! Listen, I've got the CEO on the phone, he wants to know where we find the autosaves, and I can't figure it out. Do you know?"

Mark looked at the clock: 5:10. "Nope!" he said cheerily. "Check the release notes, I'm sure it's in there somewhere."

"I looked, I didn't see it."

"Shame, but I'm already logged out of everything. Tell him to do a real save and we'll get back to him in the morning."

"Oh, never mind, he found it! Turns out it was in the release notes. Thanks Mark, you're a lifesaver!"

If you say so. Mark walked out the door, not bothering to reply, and headed directly across the street to the pub for his weekly Monday Evening Beer.

Six days until we start from the top, he thought.

[Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!

,

Cryptogram New Windows Zero-Day

Google’s Project Zero has discovered and published a buffer overflow vulnerability in the Windows Kernel Cryptography Driver. The exploit doesn’t affect the cryptography, but allows attackers to escalate system privileges:

Attackers were combining an exploit for it with a separate one targeting a recently fixed flaw in Chrome. The former allowed the latter to escape a security sandbox so the latter could execute code on vulnerable machines.

The vulnerability is being exploited in the wild, although Microsoft says it’s not being exploited widely. Everyone expects a fix in the next Patch Tuesday cycle.