Planet Russell

,

Sam VargheseLions’ coach Ackermann asleep at the wheel again

Last year, Johan Ackermann, the coach of South Africa’s Lions super rugby team was literally asleep during the final against the Hurricanes. His team lost to the Hurricanes 3-20.

This year, he appeared to be dozing again as his team lost, only to a different New Zealand team, the Crusaders.

The Lions lost a player to a red card about a minute before half-time but given the inherent advantages they had — they were playing at home, at altitude which made the visitors prone to running out of gas, and in dry weather which has always suited them — they could still have won.

The Lions were trailing 3-15 at half-time and this being a game where the winner would end up taking all, they should have used the kickable penalties they were awarded in the second half to move closer on the scoreboard. But for some mysterious reason, they kept going for touch and aiming for a try instead. At least two kickable penalties were wasted in this manner; a score of 9-15 would have given the home team that much more fire in their bellies in the final run home.

The Lions lost loose forward Kwagga Smith a minute before half-time when he collided with Crusaders’ full-back David Havili who had gone up to take a high kick. Smith had no chance of taking the ball and did not go up in the air to contest it either, but just stood there like a water buffalo; it resulted in Havili’s tripping over him and taking a very dangerous toss. Referee Jaco Peyper had no option but to send Smith off.

(As an aside, it is interesting to note the difference in the way that referees react to the likelihood of head and neck injuries these days. I remember a Test match in 2003, when Australia’s Wendell Sailor tackled All Black Mils Muliaina while the latter was in the air. It was much more dangerous than what Smith did but Sailor only got a yellow card.)

The Lions failed to learn from their previous win, against the Waikato Chiefs in the semi-finals. In that game, the Chiefs were terribly tired towards the latter half of the game and, after leading by a big margin at half-time, were beaten 44-29. The trip from New Zealand to Johannesburg and playing at altitude really took its toll.

Thus Ackermann should have told his men to keep the gap between the two teams on the scoreboard as small as possible and go for broke in the last 10 minutes when the Crusaders would be feeling the effects of altitude and the long flight. But by the 62nd minute, when the Lions got their first try, the score had blown out to 3-25. It could well have been 9-25.

(It must be noted that the Crusaders’ coach Scott Robertson displayed a great deal of intelligence in his substitutions, bringing on players off the bench to ensure that those who took the field at the start were not exhausted before they were replaced.)

Given that the Lions also scored with about seven minutes left, taking those two kickable penalties would have put them within two points. And that would have no doubt given them additional energy to fight it out, especially in front of a vociferous home crowd that filled the stadium to its maximum.

Alas, poor instructions from Ackermann again played the Lions false. This is his last game as coach; maybe the man who replaces him will realise that a coach can do a great deal to help a team win.

,

Planet DebianBits from Debian: Google Platinum Sponsor of DebConf17

Googlelogo

We are very pleased to announce that Google has committed support to DebConf17 as a Platinum sponsor.

Google is one of the largest technology companies in the world, providing a wide range of Internet-related services and products as online advertising technologies, search, cloud computing, software, and hardware.

Google has been supporting Debian by sponsoring DebConf since more than ten years, and at gold level since DebConf12.

With this additional commitment as Platinum Sponsor for DebConf17, Google contributes to make possible our annual conference, and directly supports the progress of Debian and Free Software helping to strengthen the community that continues to collaborate on Debian projects throughout the rest of the year.

Thank you very much Google, for your support of DebConf17!

DebConf17 is starting!

Many Debian contributors are already taking advantage of DebCamp and the Open Day to work individually or in groups developing and improving Debian. DebConf17 will officially start on August 6, 2017. Visit the DebConf17 website at https://debconf17.debconf.org to know the schedule, live streaming and other details.

Planet DebianLars Wirzenius: Enabling TRIM/DISCARD on Debian, ext4, luks, and lvm

I realised recently that my laptop isn't set up to send TRIM or DISCARD commands to its SSD. That means the SSD firmware has a harder time doing garbage collection (see whe linked Wikipedia page for more details.)

After some searching, I found two articles by Christopher Smart: one, update. Those, plus some addition reading of documentation, and a little experimentation, allowed me to do this. Since the information is a bit scattered, here's the details, for Debian stretch, as much for my own memory as to make sure this is collected into one place.

  • Append ,discard to the fourth column on relevant lines in /etc/crypttab. For me, this means the fourth column should be luks,discard.
  • Change in /etc/lvm/lvm.conf that says issue_discards to enable it (assign 1 instead of 0).
  • Append rd.luks.options=discard to the GRUB_CMDLINE_LINUX_DEFAULT value in /etc/default/grub
  • Run sudo update-grub
  • Run sudo update-initramfs -u
  • Reboot.
  • Run sudo fstrim -av - if this works, you're good! If it gives you errors, then you get to debug. I have no idea what I'm talking about.
  • Copy /usr/share/doc/util-linux/examples/fstrim.* to /etc/systemd/system and run sudo systemctl enable fstrim.timer. This will tell systemd to run fstrim every week. (If you don't use systemd you'll have to adapt the systemd bits mentioned here. I've no idea how.)

Note that it seems to be a possible information leak to TRIM encryped devices. I don't know the details, but if that bothers you, don't do it.

I don't know of any harmful effects for enabling TRIM for everything, except the crypto bit above, so I wonder if it wouldn't make sense for the Debian installer to do this by default.

Planet DebianDaniel Silverstone: USB Device Stacks, on RTFM, part 2

Previously we talked about all the different kinds of descriptors which USB devices use to communicate their capability. This is important stuff because to write any useful USB device firmware we need to be able to determine how to populate our descriptors. However, having that data on the device is entirely worthless without an understanding of how it gets from the device to the host so that it can be acted upon. To understand that, let's look at the USB wire protocol.

Note, I'll again be talking mostly about USB2.0 low- and full-speed. I believe that high speed is approximately the same but with faster wires, except not quite that simple.

Down to the wire

I don't intend to talk about the actual electrical signalling, though it's not un-reasonable for you to know that USB is a pair of wires forming a differentially signalled bidirectional serial communications link. The host is responsible for managing all the framing and timing on the link, and for formatting the communications into packets.

There are a number of packet types which can appear on the USB link:

Packet type Purpose
Token Packet When the host wishes to send a message to the Control endpoint to configure the device, read data IN, or write data OUT, it uses this to start the transaction.
Data(0/1) Packet Following a Setup, In, or Out token, a Data packet is a transfer of data (in either direction). The 0 and 1 alternate to provide a measure of confidence against lost packets.
Handshake Packet Following a data packet of some kind, the other end may ACK the packet (all was well), NAK the packet (report that the device cannot, temporarily, send/receive data, or that an interrupt endpoint isn't triggered), or STALL the bus in which case the host needs to intervene.
Start of Frame Every 1ms (full-speed) the host will send a SOF packet which carries a frame number. This can be used to help keep time on very simple devices. It also divides the bus into frames within which bandwidth is allocated.

As an example, when the host wishes to perform a control transfer, the following packets are transacted in turn:

  1. Setup Token - The host addresses the device and endpoint (OUT0)
  2. Data0 Packet - The host transmits a GET_DESCRIPTOR for the device descriptor
  3. Ack Packet - The device acknowledges receipt of the request

This marks the end of the first transaction. The device decodes the GET_DESCRIPTOR request and prepares the device descriptor for transmission. The transmission occurs as the next transaction on the bus. In this example, we're assuming 8 byte maximum transmission sizes, for illustrative purposes.

  1. In Token - The host addresses the device and endpoint (IN0)
  2. Data1 Packet - The device transmits the first 8 bytes of the descriptor
  3. Ack Packet - The host acknowledges the data packet
  4. In Token - The host addresses the device and endpoint (IN0)
  5. Data0 Packet - The device transmits the remaining 4 bytes of the descriptor (padded)
  6. Ack Packet - The host acknowledges the data packet

The second transaction is now complete, and the host has all the data it needs to proceed. Finally a status transaction occurs in which:

  1. Out Token - The host addresses the device and endpoint (OUT0)
  2. Data1 Packet - The host transmits a 0 byte data packet to indicate successful completion
  3. Ack Packet - The device acknowledges the completion, indicating its own satisfaction

And thus ends the full control transaction in which the host retrieves the device descriptor.

From a high level, we need only consider the activity which occurs at the point of the acknowledgement packets. In the above example:

  1. On the first ACK the device prepares IN0 to transmit the descriptor, readying whatever low level device stack there is with a pointer to the descriptor and its length in bytes.
  2. On the second ACK the low levels are still thinking.
  3. On the third ACK the transmission from IN0 is complete and the endpoint no longer expects to transfer data.
  4. On the fourth ACK the control transaction is entirely complete.

Thinking at the low levels of the control interface

Before we can build a high level USB stack, we need to consider the activity which might occur at the lower levels. At the low levels, particularly of the device control interface, work has to be done at each and every packet. The hardware likely deals with the token packet for us, leaving the data packets for us to process, and the resultant handshake packets will be likely handled by the hardware in response to our processing the data packets.

Since every control transaction is initiated by a setup token, let's look at the setup requests which can come our way...

Setup Packet (Data) Format
Field Name Byte start Byte length Encoding Meaning
bmRequestType 0 1 Bitmap Describes the kind of request, and the target of it. See below.
bRequest 1 1 Code The request code itself, meanings of the rest of the fields vary by bRequest
wValue 2 2 Number A 16 bit value whose meaning varies by request type
wIndex 4 2 Number A 16 bit value whose meaning varies by request type but typically encodes an interface number or endpoint.
wLength 6 2 Number A 16 bit value indicating the length of the transfer to come.

Since bRequest is essentially a switch against which multiple kinds of setup packet are selected between, here's the meanings of a few...

GET_DESCRIPTOR (Device) setup packet
Field Name Value Meaning
bmRequestType 0x08 Data direction is IN (from device to host), recipient is the device
bRequest 0x06 GET_DESCRIPTOR (in this instance, the device descriptor is requested)
wValue 0x0001 This means the device descriptor
wIndex 0x0000 Irrelevant, there's only 1 device descriptor anyway
wLength 12 This is the length of a device descriptor (12 bytes)
SET_ADDRESS to set a device's USB address
Field Name Value Meaning
bmRequestType 0x00 Data direction is OUT (from host to device), recipient is the device
bRequest 0x05 SET_ADDRESS (Set the device's USB address)
wValue 0x00nn The address for the device to adopt (max 127)
wIndex 0x0000 Irrelevant for address setting
wLength 0 There's no data transfer expected for this setup operation

Most hardware blocks will implement an interrupt at the point that the Data packet following the Setup packet has been receive. This is typically called receiving a 'Setup' packet and then it's up to the device stack low levels to determine what to do and dispatch a handler. Otherwise an interrupt will fire for the IN or OUT tokens and if the endpoint is zero, the low level stack will handle it once more.

One final thing worth noting about SET_ADDRESS is that it doesn't take effect until the completion of the zero-length "status" transaction following the setup transaction. As such, the status request from the host will still be sent to address zero (the default for new devices).

A very basic early "packet trace"

This is an example, and is not guaranteed to be the packet sequence in all cases. It's a good indication of the relative complexity involved in getting a fresh USB device onto the bus though...

When a device first attaches to the bus, the bus is in RESET state and so the first event a device sees is a RESET which causes it to set its address to zero, clear any endpoints, clear the configuration, and become ready for control transfers. Shortly after this, the device will become suspended.

Next, the host kicks in and sends a port reset of around 30ms. After this, the host is ready to interrogate the device.

The host sends a GET_DESCRIPTOR to the device, whose address at this point is zero. Using the information it receives from this, it can set up the host-side memory buffers since the device descriptor contains the maximum transfer size which the device supports.

The host is now ready to actually 'address' the device, and so it sends another reset to the device, again around 30ms in length.

The host sends a SET_ADDRESS control request to the device, telling it that its new address is nn. Once the acknowledgement has been sent from the host for the zero-data status update from the device, the device sets its internal address to the value supplied in the request. From now on, the device shall respond only to requests to nn rather than to zero.

At this point, the host will begin interrogating further descriptors, looking at the configuration descriptors and the strings, to build its host-side representation of the device. These will be GET_DESCRIPTOR and GET_STRING_DESCRIPTOR requests and may continue for some time.

Once the host has satisfied itself that it knows everything it needs to about the device, it will issue a SET_CONFIGURATION request which basically starts everything up in the device. Once the configuration is set, interrupt endpoints will be polled, bulk traffic will be transferred, Isochronous streams begin to run, etc.

Okay, but how do we make this concrete?

So far, everything we've spoken about has been fairly abstract, or at least "soft". But to transfer data over USB does require some hardware. (Okay, okay, we could do it all virtualised, but there's no fun in that). The hardware I'm going to be using for the duration of this series is the STM32 on the blue-pill development board. This is a very simple development board which does (in theory at least) support USB device mode.

If we view the schematic for the blue-pill, we can see a very "lightweight" USB interface which has a pullup resistor for D+. This is the way that a device signals to the host that it is present, and that it wants to speak at full-speed. If the pullup were on D- then it would be a low-speed device. High speed devices need a little more complexity which I'm not going to go into for today.

The USB lines connect to pins PA11 and PA12 which are the USB pins on the STM32 on the board. Since USB is quite finicky, the STM32 doesn't let you remap that function elsewhere, so this is all looking quite good for us so far.

The specific STM32 on the blue-pill is the STM32F103C8T6. By viewing its product page on ST's website we can find the reference manual for the part. Jumping to section 23 we learn that this STM32 supports full-speed USB2.0 which is convenient given the past article and a half. We also learn it supports up to eight endpoints active at any one time, and offers double-buffering for our bulk and isochronous transfers. It has some internal memory for packet buffering, so it won't use our RAM bandwidth while performing transfers, which is lovely.

I'm not going to distill the rest of that section here, because there's a large amount of data which explains how the USB macrocell operates. However useful things to note are:

  • How IN OUT and SETUP transfers work.
  • How the endpoint buffer memory is configured.
  • That all bus-powered devices MUST respond to suspend/resume properly
  • That the hardware will prioritise endpoint interrupts for us so that we only need deal with the most pressing item at any given time.
  • There is an 'Enable Function' bit in the address register which must be set or we won't see any transactions at all.
  • How the endpoint registers signal events to the device firmware.

Next time, we're going to begin the process of writing a very hacky setup routine to try and initialise the USB device macrocell so that we can see incoming transactions through the ITM. It should be quite exciting, but given how complex this will be for me to learn, it might be a little while before it comes through.

Cory DoctorowWalkaway is a finalist for the Dragon Awards and is #1 on Locus’s hardcover bestseller list

Dragon Con’s Dragon Award ballot was just published and I’m delighted to learn that my novel Walkaway is a finalist in the “Best Apocalyptic Novel” category, along with Daniel Humphreys’ A Place Outside the Wild, Omar El Akkad’s American War, Declan Finn and Allan Yoskowitz’s Codename: Unsub, N.K. Jemisin’s The Obelisk Gate, Rick Heinz’s The Seventh Age: Dawn, and J.F. Holmes’s ZK: Falling.


I’m also delighted to note that Walkaway is currently Locus Magazine’s #1 top-selling hardcover at science fiction and fantasy bookstores in the USA and Canada.

Many thanks to all those who nominated Walkaway for the Dragon Award, and everyone who shopped for a copy at their friendly neighborhood sf store!

Planet DebianBits from Debian: DebConf17 Open Day

Today, the day preceeding the official start of the annual Debian Conference, is the Open Day at DebConf17, at Collège Maisonneuve in Montreal (Canada).

This day is open to the public with events of interest to a wide audience.

The schedule of today's events include, among others:

  • A Newbie's Newbie Guide to Debian
  • Ask Anything About Debian
  • Debian Packaging 101
  • Debian InstallFest
  • Presentations or workshops related to free software projects and local organizations.

Everyone is welcome to attend! It is a great possibility for interested users to meet our community and for Debian to widen our community.

See the full schedule for today's events at https://debconf17.debconf.org/schedule/open-day/.

If you want to engage remotely, you can watch the video streaming of the Open Day events happening in the "Rex" room, or join the conversation in the channels #debconf17-rex, #debconf17-potato and #debconf17-woody in the OFTC IRC network.

DebConf is committed to a safe and welcome environment for all participants. See the DebConf Code of Conduct and the Debian Code of Conduct for more details on this.

Debian thanks the commitment of numerous sponsors to support DebConf17, particularly our Platinum Sponsors Savoir-Faire Linux, Hewlett Packard Enterprise, and Google.

DebConf17 logo

Planet DebianSteinar H. Gunderson: Dear conference organizers

Dear conference organizers,

In this day and age, people stream conferences and other events over the Internet. Most of the Internet happens to be in a different timezone from yours (it's crazy, I know!). This means that if you publish a schedule, please say which timezone it's in. We've even got this thing called JavaScript now, which allows you to also convert times to the user's local timezone (the future is now!), so you might want to consider using it. :-)

(Yes, this goes for you, DebConf, and also for you, Assembly.)

Don MartiHey kids, favicon!

Finally fixed those 404s from browsers looking for favicon.ico on this blog.

  1. Google image search for images where "reuse with modification" is allowed.

  2. Found this high-quality lab mouse SVG image.

  3. Opened it in GNU Image Manipulation Program, posterized, cropped to a square. Kept the transparent background.

  4. Just went to realfavicongenerator.net and did what it says, and added the resulting images and markup to the site.

That's about it. Now there's a little mouse in the browser tab (and it should do the right thing with the icons if someone pins it to their home screen on mobile.)

Planet DebianGunnar Wolf: DebConf17 Key Signing Party: You are here↓

I ran my little analysis program written last year to provide a nice map on the DebConf17 key signing party, based on the . What will you find if you go there?

  • A list of all the people that will take part of the KSP
  • Your key's situation relative to the KSP keyring

As an example, here is my location on the map (click on the graph to enlarge):

Its main use? It will help you find what clusters are you better linked with - And who you have not cross-signed with. Some people have signed you but you didn't sign them? Or the other way around? Whom should you approach to make the keyring better connected? Can you spot some attendees who are islands and can get some help getting better connected to our keyring? Please go ahead and do it!

PS— There are four keys that are mentioned in the DebConf17 Keysigning Party Names file I used to build this from: 0xE8446B4AC8C77261, 0x485E1BD3AE76CB72, 0x4618E4C700000173, E267B052364F028D. The public keyserver network does not know about them. If you control one of those keys and you want me to run my script again to include it, please send it to the keyservers and mail me. If your key is not in the keyservers, nobody will be able to sign it!

,

CryptogramFriday Squid Blogging: Squid Fake News

I never imagined that there would be fake news about squid. (That website lets you write your own stories.)

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Read my blog posting guidelines here.

Planet DebianDaniel Silverstone: USB Device Stacks, on RTFM

I have been spending time with Jorge Aparicio's RTFM for Cortex M3 framework for writing Rust to target Cortex-M3 devices from Arm (and particularly the STM32F103 from ST Microelectronics). Jorge's work in this area has been of interest to me ever since I discovered him working on this stuff a while ago. I am very tempted by the idea of being able to implement code for the STM32 with the guarantees of Rust and the language features which I have come to love such as the trait system.

I have been thinking to myself that, while I admire and appreciate the work done on the GNUK, I would like to, personally, have a go at implementing some kind of security token on an STM32 as a USB device. And with the advent of the RTFM for M3 work, and Jorge's magical tooling to make it easier to access and control the registers on an M3 microcontroller, I figured it'd be super-nice to do this in Rust, with all the advantages that entails in terms of isolating unsafe behaviour and generally having the potential to be more easily verified as not misbehaving.

To do this though, means that I need a USB device stack which will work in the RTFM framework. Sadly it seems that, thus-far, only Jorge has been working on drivers for any of the M3 devices his framework supports. And one person can only do so much. So, in my infinite madness, I decided I should investigate the complexity of writing a USB device stack in Rust for the RTFM/M3 framework. (Why I thought this was a good idea is lost to the mists of late night Googling, but hey, it might make a good talk at the next conference I go to). As such, this blog post, and further ones along these lines, will serve as a partial tour of what I'm up to, and a partial aide-memoir for me about learning USB. If I get something horribly wrong, please DO contact me to correct me, otherwise I'll just continue to be wrong. If I've simplified something but it's still strictly correct, just let me know if it's an oversimplification since in a lot of cases there's no point in me putting the full details into a blog posting. I will mostly be considering USB2.0 protocol details but only really for low and full speed devices. (The hardware I'm targetting does low-speed and full-speed, but not high-speed. Though some similar HW does high-speed too, I don't have any to hand right now)

A brief introduction to USB

In order to go much further, I needed a grounding in USB. It's a multi-layer protocol as you might expect, though we can probably ignore the actual electrical layer since any device we might hope to support will have to have a hardware block to deal with that. We will however need to consider the packet layer (since that will inform how the hardware block is implemented and thus its interface) and then the higher level protocols on top.

USB is a deliberately asymmetric protocol. Devices are meant to be significantly easier to implement, both in terms of hardware and software, as compared with hosts. As such, despite some STM32s having OTG ports, I have no intention of supporting host mode at this time.

USB is arranged into a set of busses which are, at least in the USB1.1 case, broadcast domains. As such, each device has an address assigned to it by the host during an early phase called 'configuration'. Once the address is assigned, the device is expected to only ever respond to messages addressed to it. Note that since everything is asymmetric in USB, the device can't send messages on its own, but has to be asked for them by the host, and as such the addressing is always from host toward device.

USB devices then expose a number of endpoints through which communication can flow IN to the host or OUT to the device. Endpoints are not bidirectional, but the in and out endpoints do overlap in numbering. There is a special pair of endpoints, IN0 and OUT0 which, between them, form what I will call the device control endpoints. The device control endpoints are important since every USB device MUST implement them, and there are a number of well defined messages which pass over them to control the USB device. In theory a bare minimum USB device would implement only the device control endpoints.

Configurations, and Classes, and Interfaces, Oh My!

In order for the host to understand what the USB device is, and what it is capable of, part of the device control endpoints' responsibility is to provide a set of descriptors which describe the device. These descriptors form a heirarchy and are then glommed together into a big lump of data which the host can download from the device in order to decide what it is and how to use it. Because of various historical reasons, where a multi-byte value is used, they are defined to be little-endian, though there are some BCD fields. Descriptors always start with a length byte and a type byte because that way the host can parse/skip as necessary, with ease.

The first descriptor is the device descriptor, is a big one, and looks like this:

Device Descriptor
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (18)
bDescriptorType 1 1 Constant Device Descriptor (0x01)
bcdUSB 2 2 BCD USB spec version compiled with
bDeviceClass 4 1 Class Code, assigned by USB org (0 means "Look at interface descriptors", common value is 2 for CDC)
bDeviceSubClass 5 1 SubClass Code, assigned by USB org (usually 0)
bDeviceProtocol 6 1 Protocol Code, assigned by USB org (usually 0)
bMaxPacketSize 7 1 Number Max packet size for IN0/OUT0 (Valid are 8, 16, 32, 64)
idVendor 8 2 ID 16bit Vendor ID (Assigned by USB org)
idProduct 10 2 ID 16bit Product ID (Assigned by manufacturer)
bcdDevice 12 2 BCD Device version number (same encoding as bcdUSB)
iManufacturer 14 1 Index String index of manufacturer name (0 if unavailable)
iProduct 15 1 Index String index of product name (0 if unavailable)
iSerialNumber 16 1 Index String index of device serial number (0 if unavailable)
bNumConfigurations 17 1 Number Count of configurations the device has.

This looks quite complex, but breaks down into a relatively simple two halves. The first eight bytes carries everything necessary for the host to be able to configure itself and the device control endpoints properly in order to communicate effectively. Since eight bytes is the bare minimum a device must be able to transmit in one go, the host can guarantee to get those, and they tell it what kind of device it is, what USB protocol it supports, and what the maximum transfer size is for its device control endpoints.

The encoding of the bcdUSB and bcdDevice fields is interesting too. It is of the form 0xMMmm where MM is the major number, mm the minor. So USB2.0 is encoded as 0x0200, USB1.1 as 0x0110 etc. If the device version is 17.36 then that'd be 0x1736.

Other fields of note are bDeviceClass which can be 0 meaning that interfaces will specify their classes, and idVendor/idProduct which between them form the primary way for the specific USB device to be identified. The Index fields are indices into a string table which we'll look at later. For now it's enough to know that wherever a string index is needed, 0 can be provided to mean "no string here".

The last field is bNumConfigurations and this indicates the number of ways in which this device might function. A USB device can provide any number of these configurations, though typically only one is provided. If the host wishes to switch between configurations then it will have to effectively entirely quiesce and reset the device.

The next kind of descriptor is the configuration descriptor. This one is much shorter, but starts with the same two fields:

Configuration Descriptor
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (9)
bDescriptorType 1 1 Constant Configuration Descriptor (0x02)
wTotalLength 2 2 Number Size of the configuration in bytes, in total
bNumInterfaces 4 1 Number The number of interfaces in this configuration
bConfigurationValue 5 1 Number The value to use to select this configuration
iConfiguration 6 1 Index The name of this configuration (0 for unavailable)
bmAttributes 7 1 Bitmap Attributes field (see below)
bMaxPower 8 1 Number Maximum bus power this configuration will draw (in 2mA increments)

An important field to consider here is the bmAttributes field which tells the host some useful information. Bit 7 must be set, bit 6 is set if the device would be self-powered in this configuration, bit 5 indicates that the device would like to be able to wake the host from sleep mode, and bits 4 to 0 must be unset.

The bMaxPower field is interesting because it encodes the power draw of the device (when set to this configuration). USB allows for up to 100mA of draw per device when it isn't yet configured, and up to 500mA when configured. The value may be used to decide if it's sensible to configure a device if the host is in a low power situation. Typically this field will be set to 50 to indicate the nominal 100mA is fine, or 250 to request the full 500mA.

Finally, the wTotalLength field is interesting because it tells the host the total length of this configuration, including all the interface and endpoint descriptors which make it up. With this field, the host can allocate enough RAM to fetch the entire configuration descriptor block at once, simplifying matters dramatically for it.

Each configuration has one or more interfaces. The interfaces group some endpoints together into a logical function. For example a configuration for a multifunction scanner/fax/printer might have an interface for the scanner function, one for the fax, and one for the printer. Endpoints are not shared among interfaces, so when building this table, be careful.

Next, logically, come the interface descriptors:

Interface Descriptor
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (9)
bDescriptorType 1 1 Constant Interface Descriptor (0x04)
bInterfaceNumber 2 1 Number The number of the interface
bAlternateSetting 3 1 Number The interface alternate index
bNumEndpoints 4 1 Number The number of endpoints in this interface
bInterfaceClass 5 1 Class The interface class (USB Org defined)
bInterfaceSubClass 6 1 SubClass The interface subclass (USB Org defined)
bInterfaceProtocol 7 1 Protocol The interface protocol (USB Org defined)
iInterface 8 1 Index The name of the interface (or 0 if not provided)

The important values here are the class/subclass/protocol fields which provide a lot of information to the host about what the interface is. If the class is a USB Org defined one (e.g. 0x02 for Communications Device Class) then the host may already have drivers designed to work with the interface meaning that the device manufacturer doesn't have to provide host drivers.

The bInterfaceNumber is used by the host to indicate this interface when sending messages, and the bAlternateSetting is a way to vary interfaces. Two interfaces with the came bInterfaceNumber but different bAlternateSettings can be switched between (like configurations, but) without resetting the device.

Hopefully the rest of this descriptor is self-evident by now.

The next descriptor kind is endpoint descriptors:

Endpoint Descriptor
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (7)
bDescriptorType 1 1 Constant Endpoint Descriptor (0x05)
bEndpointAddress 2 1 Endpoint Endpoint address (see below)
bmAttributes 3 1 Bitmap Endpoint attributes (see below)
wMaxPacketSize 4 2 Number Maximum packet size this endpoint can send/receive
bInterval 6 1 Number Interval for polling endpoint (in frames)

The bEndpointAddress is a 4 bit endpoint number (so there're 16 endpoint indices) and a bit to indicate IN vs. OUT. Bit 7 is the direction marker and bits 3 to 0 are the endpoint number. This means there are 32 endpoints in total, 16 in each direction, 2 of which are reserved (IN0 and OUT0) giving 30 endpoints available for interfaces to use in any given configuration. The bmAttributes bitmap covers the transfer type of the endpoint (more below), and the bInterval is an interval measured in frames (1ms for low or full speed, 125µs in high speed). bInterval is only valid for some endpoint types.

The final descriptor kind is for the strings which we've seen indices for throughout the above. String descriptors have two forms:

String Descriptor (index zero)
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (variable)
bDescriptorType 1 1 Constant String Descriptor (0x03)
wLangID[0] 2 2 Number Language code zero (e.g. 0x0409 for en_US)
wLangID[n] 4.. 2 Number Language code n ...

This form (for descriptor 0) is that of a series of language IDs supported by the device. The device may support any number of languages. When the host requests a string descriptor, it will supply both the index of the string and also the language id it desires (from the list available in string descriptor zero). The host can tell how many language IDs are available simply by dividing bLength by 2 and subtracting 1 for the two header bytes.

And for string descriptors of an index greater than zero:

String Descriptor (index greater than zero)
Field Name Byte start Byte length Encoding Meaning
bLength 0 1 Number Size of the descriptor in bytes (variable)
bDescriptorType 1 1 Constant String Descriptor (0x03)
bString 2.. .. Unicode The string, in "unicode" format

This second form of the string descriptor is simply the the string is in what the USB spec calls 'Unicode' format which is, as of 2005, defined to be UTF16-LE without a BOM or terminator.

Since string descriptors are of a variable length, the host must request strings in two transactions. First a request for 2 bytes is sent, retrieving the bLength and bDescriptorType fields which can be checked and memory allocated. Then a request for bLength bytes can be sent to retrieve the entire string descriptor.

Putting that all together

Phew, this is getting to be quite a long posting, so I'm going to leave this here and in my next post I'll talk about how the host and device pass packets to get all that information to the host, and how it gets used.

CryptogramFriday Squid Blogging: Giant Squids Have Small Brains

New research:

In this study, the optic lobe of a giant squid (Architeuthis dux, male, mantle length 89 cm), which was caught by local fishermen off the northeastern coast of Taiwan, was scanned using high-resolution magnetic resonance imaging in order to examine its internal structure. It was evident that the volume ratio of the optic lobe to the eye in the giant squid is much smaller than that in the oval squid (Sepioteuthis lessoniana) and the cuttlefish (Sepia pharaonis). Furthermore, the cell density in the cortex of the optic lobe is significantly higher in the giant squid than in oval squids and cuttlefish, with the relative thickness of the cortex being much larger in Architeuthis optic lobe than in cuttlefish. This indicates that the relative size of the medulla of the optic lobe in the giant squid is disproportionally smaller compared with these two cephalopod species.

From the New York Times:

A recent, lucky opportunity to study part of a giant squid brain up close in Taiwan suggests that, compared with cephalopods that live in shallow waters, giant squids have a small optic lobe relative to their eye size.

Furthermore, the region in their optic lobes that integrates visual information with motor tasks is reduced, implying that giant squids don't rely on visually guided behavior like camouflage and body patterning to communicate with one another, as other cephalopods do.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Read my blog posting guidelines here.

CryptogramPenetrating a Casino's Network through an Internet-Connected Fish Tank

Attackers used a vulnerability in an Internet-connected fish tank to successfully penetrate a casino's network.

BoingBoing post.

Planet DebianMichal Čihař: Changes to Docker container for Weblate

I've made several changes to the Weblate Docker container which are worth mentioning today.

First of all if you are still using nijel/weblate, you should switch to weblate/weblate. They both currently share same configuration, but it might happen that some future updates will go to the weblate owned container only.

Now back to the container changes. Since beginning we were using Django built in server. That's fine for development purposes, but it really doesn't work that well in production as it can handle only one request at time. Therefore we've switched to more robust approach using nginx + uwsgi + supervisor.

Thanks to this, the docker-compose no longer needs separate nginx server as everything is now sanely handled within the weblate container itself.

Filed under: Debian English Gammu phpMyAdmin SUSE Weblate

Worse Than FailureError'd: A Test-imonial

"You know, usually these statements are just marketing B.S., but I think this guy's got the right idea," wrote Philip K.

 

"Windows 10 forgot it is 2017 when it decided my USB stick was in fact, a floppy drive," writes Joshua R.

 

"Sydney Ferry Service's really uses Vista's 'overlapping WTF' technology effectively," Matthias writes.

 

Hans wrote, "So, let me see if I understand this - my password strength is weak though it's 64 fully random chars and clearly I should've used fewer chars to make it more secure?"

 

"Isn't there a saying that goes 'null news is good news'?" writes Bob S.

 

Walton H. wrote, "I've never heard of 'Lua Error' before but they did an amazing job!"

 

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Planet DebianDirk Eddelbuettel: R for System Adminstration

Just getting back from the most fun meetup I have been to in quite some time: episode 23 (by their count) of Open Source Open Mic hosted by Matt Godbolt and Joe Walnes here in Chicago. Nothing but a sequence of lightning talks. Plus beer and pizza. Sounds awesome? It was!

We had fantastic talks across at least half a dozen languages, covering both new-ish (Pony) and interesting ones such (Rust, Go, ...) plus of course some Javascript and some Python, no Java (yay!) and a few batshit crazy things like a self-hosting database in its own (shell) code, a terminal gif viewer (!!), and more. And it gave me an opportunity to quickly (one evening and morning commute) jam out a presentation about what is in the title: R for system administration.

And I am only half-joking. I had used R a couple of years ago when I needed to select, subset, modify, ... a large number of image files given some timestamp and filename patterns. And given how well R works in a vectorised manner with both regular expressions and timestamps, as well as on top of essentially all standard POSIX-style operating system / file-system functions, I picked up that thread again on the problem of ... cleaning up the file storage underlying CRANberries which by now has well over fifty-seven thousand (!!) tarballs of CRAN packages based on now ten years of CRANberries. So I showed how to prune this in essentially half a dozen lines of R (and data.table code), plus some motivation---all just right for a lightning talk. Seemingly the talk went well enough as quite a few folks gave a thumbs up and compliments over beers afterwards.

But see for yourself as the slides are now uploaded to my standard talks page.

My thanks to Matt and Joe for organizing the meetup. I think I will be back.

,

Planet DebianJoey Hess: home power monitoring

For years I've recorded solar panel data by hand. Filled two notebooks with columns of figures. My new charge controller, an EPsolar Tracer-BN, finally let me automate it.

morning activity; by 8 am the sun is still behind the hill but, 16 watts are being produced, and by 11:30 am, the battery bank is full

You can explore my home power data here: http://homepower.joeyh.name/
(click and drag to zoom)

The web interface loads the RRD files into a web browser using javascriptRRD. I wrote a haskell program that drives the epsolar-tracer python library to poll for data, and stores it in RRD files. Could have used collectd or something, but the interface to the charge controller is currently a bit flakey and I have to be careful about retries and polling frequencies. Also I wanted full control over how much data is stored in the RRD files.

Full source code

Planet DebianDaniel Silverstone: Gitano 1.1

Today marks the release of Gitano 1.1. Richard(s) and I have spent quite a lot of time and effort on this release, and there's plenty of good stuff in it. We also released new versions of Lace, Supple, Luxio, and Gall to go alongside it, with bugfixes and improvements.

At this point, I intend to take a short break from Gitano to investigate some Rust-on-STM32 stuff, and then perhaps do some NetSurf work too.

Planet DebianJeremy Bicha: Link: Ubuntu @ GUADEC 2017 and plans for GNOME Shell migration

Since Didier Roche’s blog is not on Planet GNOME or Planet Debian and I think his post is of widespread interest, I’m linking to it here. Enjoy!

Ubuntu @ GUADEC 2017 and plans for GNOME Shell migration

TEDTEDGlobal 2017: Announcing the speaker lineup for our Arusha conference

TEDGlobal 2017 kicks off August 27–30, 2017, in Arusha, Tanzania. Ten years after the last TEDGlobal in Arusha, we’ll again gather a community from across the continent and around the world to explore ideas that may propel Africa’s next leap — in business, politics and justice, creativity and entrepreneurship, science and tech.

Today, we’re thrilled to announce our speaker lineup for TEDGlobal 2017! It’s a powerful list you can skim here — to dive into speaker bios and learn about the 8 themed sessions of TEDGlobal 2017, visit our full Program Guide.

OluTimehin Adegbeye, Writer and activist: Writing on gender justice, sexual and reproductive rights, urban poverty and media OluTimehin Adegbeye shares her (often very strong) opinions on Twitter and in long-form work. @OhTimehin

Oshiorenoya Agabi, Neurotechnology entrepreneur: Oshiorenoya Agabi is engineering neurons to express synthetic receptors which give them an unprecedented ability to become aware of surroundings. koniku.io

Nabila Alibhai, Place-maker: Nabila Alibhai leads inCOMMONS, a new organization focused on civic engagement, public spaces, and building collective responsibility for our shared places.@NabilaAlibhai

Bibi Bakare-Yusuf, Publisher: Bibi Bakare-Yusuf is co-founder and publishing director of one of Africa’s leading publishing houses, Cassava Republic Press. cassavarepublic.biz

Christian Benimana, Architect: Christian Benimana is co-founder of the African Design Center, a training program for young architects. massdesigngroup.org

Gus Casely-Hayford, Cultural historian: Gus Casely-Hayford writes, lectures, curates and broadcasts widely about African culture.

In Session 5, Repatterning, speakers will talk about the worlds we create — in fiction, fashion, design, music.

Natsai Audrey Chieza, Designer: Natsai Audrey Chieza is a design researcher whose fascinating work crosses boundaries between technology, biology, design and cultural studies. @natsaiaudrey

Tania Douglas, Biomedical engineer: Tania Douglas imagines how biomedical engineering can help address some of Africa’s health challenges. @tania_douglas

Touria El Glaoui, Art fair curator: To showcase vital new art from African nations and the diaspora, Touria El Glaoui founded the powerhouse 1:54 Contemporary African Art Fair. @154artfair

Meron Estefanos, Refugee activist: Meron Estefanos is the executive director of the Eritrean Initiative on Refugee Rights, advocating for refugees and victims of trafficking and torture. @meronina

Chika Ezeanya-Esiobu, Indigenous knowledge expert: Working across disciplines, Chika Ezeanya-Esiobu explores indigenous knowledge, homegrown and grassroots approaches to the sustainable advancement of Sub-Saharan Africa. chikaforafrica.com

Kamau Gachigi, Technologist: At Gearbox, Kamau Gachigi empowers Kenya’s next generation of creators to prototype and fabricate their visions. @kamaufablab

Ameenah Gurib-Fakim: President of Mauritius: Ameenah Gurib-Fakim is the 6th president of the island of Mauritius. As a biodiversity scientist as well, she explores the medical and nutrition secrets of her home. @aguribfakim

Leo Igwe, Human rights activist: Leo Igwe works to end a variety of human rights violations that are rooted in superstition, including witchcraft accusations, anti-gay hate, caste discrimination and ritual killing. @leoigwe

Joel Jackson, Transport entrepreneur: Joel Jackson is the founder and CEO of Mobius Motors, set to launch a durable, low-cost SUV made in Africa. mobiusmotors.com

Tunde Jegede, Composer, cellist, kora virtuoso: TED Fellow Tunde Jegede combines musical traditions to preserve classical forms and create new ones. tundejegede.com

Paul Kagame, President of the Republic of Rwanda: As president of Rwanda, Paul Kagame has received recognition for his leadership in peace-building, development, good governance, promotion of human rights and women’s empowerment, and advancement of education and ICT. @PaulKagame

Zachariah Mampilly, Political scientist: Zachariah Mampilly is an expert on the politics of both violent and non-violent resistance. He is the author of “Rebel Rulers: Insurgent Governance and Civilian Life during War” and “Africa Uprising: Popular Protest and Political Change.” @Ras_Karya

Vivek Maru, Legal empowerment advocate: Vivek Maru is the founder of Namati, a movement for legal empowerment around the world powered by cadres of grassroots legal advocates. Global Legal Empowerment Network

In Session 6: A Hard Look, these speakers will confront myths and hard facts about the continent, from the lens of politics and human rights as well as the reality of life as a small farmer.

Kola Masha, Agricultural leader: Kola Masha is the managing director of Babban Gona, an award-winning, high-impact, financially sustainable and highly scalable social enterprise, part-owned by the farmers they serve. @BabbanGona

Clapperton Chakanetsa Mavhunga, MIT professor, grassroots thinker-doer, author: Clapperton Chakanetsa Mavhunga studies the history, theory, and practice of science, technology, innovation, and entrepreneurship in the international context, with a focus on Africa. sts-program.mit.edu/people/sts-faculty/c-clapperton-mavhunga/

Thandiswa Mazwai, Singer: Thandiswa is one of the most influential South African musicians of this generation. @thandiswamazwai

Yvonne Chioma Mbanefo, Digital learning advocate: After searching for an Igbo language learning tool for her kids, digital strategist Yvonne Mbanefo helped create the first illustrated Igbo dictionary for children. Now she’s working on Yoruba, Hausa, Gikuyu and more. @yvonnembanefo

Sara Menker, Technology entrepreneur: Sara Menker is founder and CEO of Gro Intelligence, a tech company that marries the application of machine learning with domain expertise and enables users to understand and predict global food and agriculture markets. @SaraMenker

Eric Mibuari, Computer scientist: Eric Mibuari studies the blockchain at IBM Research, and is the founder of the Laare Community Technology Centre in Meru, Kenya. laare.csail.mit.edu

Kingsley Moghalu, Political economist: Kingsley Moghalu is a global leader who has made contributions to the stability, progress and wealth of nations, societies and individuals across such domains as academia, economic policy, banking and finance, entrepreneurship, law and diplomacy. kingsleycmoghalu.com

Sethembile Msezane, Artist: Sethembile Msezane the act of public commemoration — how it creates myths, constructs histories, includes some and excludes others. @sthemse

Kisilu Musya, Farmer and filmmaker: For six years, Kisilu Musya has filmed his life on a small farm in South East Kenya, to make the documentary “Thank You for the Rain.” thankyoufortherain.com

Robert Neuwirth, Author: To research his book “Stealth of Nations,” Robert Neuwirth spent four years among street vendors, smugglers and “informal” import/export firms. @RobertNeuwirth

Kevin Njabo, Biodiversity scientist: Kevin Njabo is coordinating the development of UCLA’s newly established Congo Basin Institute (CBI) in Yaoundé, Cameroon.

Alsarah and the Nubatones, East African retro-popsters: Inspired by both the golden age of Sudanese pop music of the ’70s and the New York effervescence, Alsarah & the Nubatones have built a repertoire where an exhilarating oud plays electric melodies on beautiful jazz-soul bass lines, and where sharp and modern percussions breathe new life to age-old rhythms. alsarah.com

Ndidi Nwuneli, Social innovation expert: Through her work in food and agriculture, and as a leadership development mentor, Ndidi Okonkwo Nwuneli commits to building economies in West Africa. @ndidiNwuneli

Dayo Ogunyemi, Cultural media builder: Dayo Ogunyemi is the founder of 234 Media, which makes principal investments in the media, entertainment and technology sectors. @AfricaMET

Nnedi Okorafor, Science fiction writer: Nnedi Okorafor weaves African cultures into the evocative settings and memorable characters of her science fiction work for kids and adults. @Nnedi

Fredros Okumu, Mosquito scientist: Fredros Okumu studies human-mosquito interactions, hoping to understand how to keep people from getting malaria. ihi.or.tz

Qudus Onikeku, Dancer, choreographer: With a background as an acrobat and dancer, Qudus Onikeku is one of the preeminent Nigerian choreographers working today. @qudusonikeku

DK Osseo-Asare, Designer: DK Osseo-Asare is a designer who makes buildings, landscapes, cities, objects and digital tools. @dkoa

Keller Rinaudo, Robotics entrepreneur: Keller Rinaudo is CEO and co-founder of Zipline, building drone delivery for global public health customers. @kellerrinaudo

Reeta Roy, President and CEO, The Mastercard Foundation: A thoughtful leader and an advocate for the world’s most vulnerable, Reeta Roy has worked tirelessly to build a foundation that is collaborative and known for its lasting impact. mastercardfdn.org

Chris Sheldrick, Co-founder & CEO, what3words: With what3words, Chris Sheldrick is providing a precise and simple way to talk about location, by dividing the world into a grid of 3m x 3m squares and assigning each one a unique 3 word address. what3words.com

George Steinmetz, Aerial photographer: Best known f­or his exploration photography, George Steinmetz has a restless curiosity for the unknown: remote deserts, obscure cultures, the ­mysteries of science and technology. georgesteinmetz.com

Olúfẹ́mi Táíwò, Historian and philosopher: Drawing on a rich cultural and personal history, Olúfẹ́mi Táíwò studies philosophy of law, social and political philosophy, Marxism, and African and Africana philosophy. africana.cornell.edu/

Pierre Thiam, Chef: Pierre Thiam shares the cuisine of his home in Senegal through global restaurants and highly praised cookbooks. pierrethiam.com

Iké Udé, Artist: The work of Nigerian-born Iké Udé explores a world of dualities: photographer/performance artist, artist/spectator, African/postnationalist, mainstream/marginal, individual/everyman and fashion/art. ikeude.com

Washington Wachira, Wildlife ecologist and nature photographer: Birder and ecologist Washington Wachira started the Youth Conservation Awareness Programme (YCAP) to nurture young environmental enthusiasts in Kenya. washingtonwachira.com

Ghada Wali, Designer: A pioneering graphic designer in Egypt, Ghada Wali has designed fonts, brands and design-driven art projects. ghadawali.com


CryptogramSplitting the NSA and US Cyber Command

Rumor is that the Trump administration will separate the NSA and US Cyber Command. I have long thought this was a good idea. Here's a good discussion of what it does and doesn't mean.

Worse Than FailureNature In Its Volatility

About two years ago, we took a little trip to the Galapagos- a tiny, isolated island where processes and coding practices evolved… a bit differently. Calvin, as an invasive species, brought in new ways of doing things- like source control, automated builds, and continuous integration- and changed the landscape of the island forever.

Geospiza parvula

Or so it seemed, until the first hiccup. Shortly after putting all of the code into source control and automating the builds, the application started failing in production. Specifically, the web service calls out to a third party web service for a few operations, and those calls universally failed in production.

“Now,” Hank, the previous developer and now Calvin’s supervisor, “I thought you said this should make our deployments more reliable. Now, we got all these extra servers, and it just plumb don’t work.”

“We’re changing processes,” Calvin said, “so a glitch could happen easily. I’ll look into it.”

“Looking into it” was a bit more of a challenge than it should have been. The code was a pasta-golem: a gigantic monolith of spaghetti. It had no automated tests, and wasn’t structured in a way that made it easy to test. Logging was nonexistent.

Still, Calvin’s changes to the organization helped. For starters, there was a brand new test server he could use to replicate the issue. He fired up his testing scripts, ran them against the test server, and… everything worked just fine.

Calvin checked the build logs, to confirm that both test and production had the same version, and they did. So next, he pulled a copy of the code down to his machine, and ran it. Everything worked again. Twiddling the config files didn’t accomplish anything. He build a version of the service configured for remote debugging, and chucked it up to the production server… and the error went away. Everything suddenly started working fine.

Quickly, he reverted production. On his local machine, he did something he’d never really had call to do- he flipped the build flag from “Debug” to “Release” and recompiled. The service hung. When built in “Release” mode, the resulting DLL had a bug that caused a hang, but it was something that never appeared when built in “Debug” mode.

“I reckon you’re still workin’ on this,” Hank asked, as he ambled by Calvin’s office, thumbs hooked in his belt loops. “I’m sure you’ve got a smart solution, and I ain’t one to gloat, but this ain’t never happened the old way.”

“Well, I can get a temporary fix up into production,” Calvin said. He quickly threw a debug build up onto production, which wouldn’t have the bug. “But I have to hunt for the underlying cause.”

“I guess I just don’t see why we can’t build right on the shared folder, is all.”

“This problem would have cropped up there,” Calvin said. “Once we build for Release, the problem crops up. It’s probably a preprocessor directive.”

“A what now?”

Hank’s ignorance about preprocessor directives was quickly confirmed by a search through the code- there was absolutely no #if statements in there. Calvin spent the next few hours staring at this block of code, which is where the application seemed to hang:

public class ServiceWrapper
{
    bool thingIsDone = false;
    //a bunch of other state variables

    public string InvokeSoap(methodArgs args)
    {
        //blah blah blah
        soapClient client = new Client();
        client.doThingCompleted += new doThingEventHandler(MyCompletionMethod);
        client.doThingAsync(args);

        do
        {
            string busyWork = "";
        }
        while (thingIsDone == false)

        return "SUCCESS!" //seriously, this is what it returns
    }

    private void MyCompletionMethod(object sender, completedEventArgs e)
    {
        //do some other stuff
        thingIsDone = true;
    }
}

Specifically, it was in the busyWork loop where the thing hung. He stared and stared at this code, trying to figure out why thingIsDone never seemed to become true, but only when built in Release. Obviously, it had to be a compiler optimization- and that’s when the lightbulb went off.

The C# compiler, when building for release, will look for variables whose values don’t appear to change, and replace them with in-lined constants. In serial code, this can be handled with some pretty straightforward static analysis, but in multi-threaded code, the compiler can make “mistakes”. There’s no way for the compiler to see that thingIsDone ever changes, since the change happens in an external thread. The fix is simple: chuck volatile on the variable declaration to disable that optimization.

volatile bool thingIsDone = false solved the problem. Well, it solved the immediate problem. Having seen the awfulness of that code, Calvin couldn’t sleep that night. Nightmares about the busyWork loop and the return "SUCCESS!" kept him up. The next day, the very first thing he did was refactor the code to actually properly handle multiple threads.

[Advertisement] Atalasoft’s imaging SDKs come with APIs & pre-built controls for web viewing, browser scanning, annotating, & OCR/barcode capture. Try it for 30 days with included support.

Planet DebianElena 'valhalla' Grandi: Debian Day in Varese

Debian Day in Varese

I'm stuck home instead of being able to go to DebConf, but that doesn't mean that Debian Day will be left uncelebrated!

Since many of the locals are away for the holidays, we of @Gruppo Linux Como and @LIFO aren't going to organize a full day of celebrations, but at the very least we are meeting for a dinner in Varese, at some restaurant that will be open on that date.

Everybody is welcome: to join us please add your name (nickname or identifier of any kind, as long as it fits in the box) on dudle.inf.tu-dresden.de/debday before thursday, August 10th, so that we can
get a reservation at the restaurant.

Planet DebianMichal Čihař: Going to DebConf17

After fours years, I will again make it to DebConf, I'm looking forward to meet many great people, so if you want to meet and happen to be in Montreal next week come and say hello to me :-).

It seems I've settled down on four year schedule - I've attended DebConf09 and DebConf13 so far. Let's see if next one will come in 2021 or earlier.

Filed under: Debian English Gammu phpMyAdmin Weblate

,

Planet DebianMarkus Koschany: My Free Software Activities in July 2017

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you’re interested in  Java, Games and LTS topics, this might be interesting for you.

Debian Games

  • I backported freeciv, freeorion and minetest to stretch-backports.
  • The bug fix (#866378) for 3dchess also landed in Stretch and Jessie.
  • I sponsored Lugaru for Vincent Prat and Martin Erik Werner, a really cool 3D fighting game featuring a rabbit. The game is dfsg-free now and will replace openlugaru.
  • I uploaded fifechan to unstable and packaged new upstream versions of fife, unknown-horizons, adonthell-data and hyperrogue.
  • I fixed bugs in bloboats (#864534), lordsawar (RC #866988), kraptor (#826423), pathogen (#845991), fretsonfire (#866426), blockout2 (#826416), boswars (#827112), kanatest (RC #868315, fix also backported to Stretch), overgod (#827114), morris (#829948, #721834, #862224), mousetrap (#726842), alsoft-conf (#784052, #562898) and nikwi (#835625)
  • I uploaded a new revision of clanlib and teg fixing Perl transition bugs. The patches were provided by gregor herrmann. I added myself to Uploaders in case of teg because the package was missing a human maintainer.
  • I adopted trackballs after I discovered #868983 where Henrique de Moraes Holschuh called attention to a new fork of Trackballs. The current version was broken and unplayable and it was only a matter of time before the game was removed from Debian. I could fix a couple of bugs, forwarded some issues upstream and I believe a nice game was saved.
  • I uploaded Bullet 2.86.1 to unstable and completed another Bullet transition.

Debian Java

Debian LTS

This was my seventeenth month as a paid contributor and I have been paid to work 23,5 hours on Debian LTS, a project started by Raphaël Hertzog. In that time I did the following:

  • From 24. July until 31. July I was in charge of our LTS frontdesk. I triaged bugs in tinyproxy, varnish, freerdp, ghostscript, gcc-4.6, gcc-4.7, fontforge, teamspeak-server, teamspeak-client, qpdf, nvidia-graphics-drivers and sipcrack. I also pinged Diego Biurrun for more information about the next libav update and replied to questions on the debian-lts mailing list and LTS IRC channel.
  • DLA 1034-1. Issued a security update for php5 fixing 5 CVE. I discussed CVE-2017-11362 with the security team. We came to the conclusion that it was no security issue but just a normal bug.
  • DLA 1036-1. Issued a security update for gsoap fixing 1 CVE.
  • DLA 1037-1. Issued a security update for catdoc fixing 1 CVE.
  • DLA 613-2. Issued a regression update for roundcube.
  • DLA 1045-1. Issued a security update for graphicsmagick fixing 10 CVE.
  • DLA 1047-1. Issued a security update for supervisor fixing 1 CVE.
  • DLA-1048-1.  Issued a security update for ghostscript fixing 8 CVE.

Non-maintainer upload

  • I uploaded the security fix for spice to unstable which was already fixed in Stretch and earlier versions.

Thanks for reading and see you next time.

Planet DebianSteve Kemp: So I did a thing, then another thing.

So I did start a project, to write a puppet-dashboard, it is functionally complete, but the next step is to allow me to raise alerts based on failing runs of puppet - in real-time.

(i.e. Now that I have a dashboard I wish to not use it. I want to be alerted to failures, without having to remember to go look for them. Something puppet-dashboard can't do ..)

In other news a while back I slipped in a casual note about having a brain scan done, here in sunny Helsinki.

One of the cool things about that experience, in addition to being told I wasn't going to drop dead that particular day, was that the radiologist told me that I could pay €25 to get a copy of my brain data in DICOM format.

I've not yet played with this very much, but I couldn't resist a brief animation:

  • See my brain.
    • Not the best quality, or the best detail, but damn. It is my brain.
    • I shall do better with more experimentation I think.
    • After I posted it my wife, a doctor, corrected me: That wasn't a gif of my brain, instead it was a gif of my skull. D'oh!

CryptogramVoting Machine Security

Last week, DefCon hosted a "Voter Hacker Village" event. Every single voting machine there was easily hackable.

Here are detailed details. There should be a summary report soon; I'll add it to this post when it's published.

Planet DebianMarkus Koschany: PDFsam: How to upgrade a Maven application for Debian

In the coming weeks and months I intend to write a mini series about packaging Java software for Debian. The following article basically starts in the middle of this journey because the PDFsam upgrade is still fresh in my mind. It requires some preexisting knowledge about build tools like Maven and some Java terminology. But do not fear. Hopefully it will make sense in the end when all pieces fall into place.

A month ago I decided to upgrade PDFsam, a Java application to split, merge, extract, mix and rotate PDF documents. The current version 1.1.4 is already seven years old and uses Ant as its build system. Unfortunately up to now nobody was interested enough to invest the time to upgrade it to the latest version. A quick internet search unveils that the current sources can be found on github.com. Another brief look reveals we are dealing with a Maven project here because we can find a pom.xml file in the root directory and there is no sign of Ant’s typical build.xml file anymore. Here are some general tips how to proceed from this point by using the PDFsam upgrade as an example.

Find out how many new dependencies you really need

The pom.xml file declares its dependencies in the <dependencies> section. It is good practice to inspect the pom.xml file and determine how much work will be required to upgrade the package. A seasoned Java packager will quickly find common dependencies like Hibernate or the Apache Commons libraries. Fortunately for you they are already packaged in Debian because a lot of projects depend on them. If you are unsure what is and what is not packaged for Debian, tracker.debian.org and codesearch.debian.net are useful tools to search for those packages. If in doubt just ask on debian-java@lists.debian.org. There is no automagical tool (yet) to find out what dependencies are really new (we talk about mh_make soon) but if you use the aforementioned tools and websites you will notice that in June 2017 one could not find the following artifacts: fontawesomefx, eventstudio, sejda-* and jackson-jr-objects. There are also jdepend and testFx but notice they are marked as <scope>test</scope> meaning they are only required if you would like to run upstream’s test suite as well. For the sake of simplicity, it is best to ignore them for now and to focus on packaging only dependencies which are really needed to compile the application. Test dependencies can always be added later.

This pom.xml investigation leads us to the following conclusion: PDFsam depends on Sejda, a PDF library. Basically Sejda is the product of a major refactoring that happened years ago and allows upstream to develop PDFsam faster and in multiple directions. For Debian packagers it is quite clear now that the “upgrade” of PDFsam is in reality more like packaging a completely new application. The inspection of Sejda’s pom.xml file (another Maven project) reveals we also have to package imgscalr, Twelvemonkeys and SAMBox. We continue with these pom.xml analyses and end up with these new source packages: jackson-jr, libimgscalr-java, libsambox-java, libsejda-java, libsejda-injector-java, libsejda-io-java, libsejda-eventstudio-java, libtwelvemonkeys-java, fontawesomefx and libpdfbox2-java. Later I discovered that gettext-maven-plugin was also required.

This was not obvious at first glance if you only check the pom.xml in the root directory but PDFsam and Sejda are multi-module projects! In this case every subdirectory (module) contains another pom.xml with additional information, so ideally you should check those too before you decide to start with your packaging. But don’t worry it is often possible to ignore modules with a simple –ignore  rule inside your debian/*.poms file. The package will have less functionality but it can be still useful if you only need a subset of the modules. Of course in this case ignoring the gettext-maven-plugin artifact would result in a runtime error. C’est la vie.

A brief remark about Java package names: Java library packages must be named like libXXX-java. This is important for binary packages to avoid naming collisions. We are more tolerant when it comes to source package names but in general we recommend to use the exact same name as for the binary package. There are exceptions like prefixing source packages with their well known project name like jackson-XXX or jboss-XXX but this should only be used when there are already existing packages that use such a naming scheme. If in doubt, talk to us.

mh_make or how to quickly generate an initial debian directory

Packaging a Maven library is usually not very difficult even if it consists of multiple modules. The tricky part is to get the maven.rules, maven.IgnoreRules and your *.poms file right but debian/rules often only consists of a single dh line and the rest is finding the build-dependencies and adding them to debian/control.

A small tool called mh_make, which is included in maven-debian-helper, can lend you a helping hand. The tool is not perfect yet. It requires that most build-dependencies are already installed on your local system, otherwise it won’t create the initial debian directory and will only produce some unfinished (but in some cases still useful) files.

A rule of thumb is to start with a package that does not depend on any other new dependency and requires the fewest build-dependencies.  I have chosen libtwelvemonkeys-java because it was the simplest package and met the aforementioned criteria.

Here is how mh_make looks like in action. (The animated GIF was created with Byzanz) First of all download the release tarball, unpack it and run mh_make inside the root directory.

Ok, what is happening here? First you can choose a source and binary package name. Then disable the tests and don’t run javadoc to create the documentation. This will simplify things a little.  Tests and javadoc settings can be added later. Choose the version you want to package and then you can basically follow the default recommendations and confirm them by hitting the Enter key. Throughout the project we choose to transform the upstream version with the symbolic “debian” version. Remember that Java/Maven is version-centric. This will ensure that our Maven dependencies are always satisfied later and we can simply upgrade our Maven libraries and don’t have to change the versions by hand in various pom.xml files; maven-debian-helper will automatically transform them for us to “debian”. Enable all modules. If you choose not to, you can select each module individually. Note that later on some of the required build-dependencies cannot be found because they are either not installed (libjmagick6-java) or they cannot be found in Debian’s Maven repository under /usr/share/maven-repo.  You can fix this by entering a substitution rule or, as I did in this case, you can just ignore these artifacts for now. They will be added to maven.IgnoreRules. In order to successfully compile your program you have to remove them from this file later again, create the correct substitution rule in maven.rules and add the missing build-dependencies to debian/control. For now we just want to quickly create our initial debian directory.

If everything went as planned a complete debian directory should be visible in your root directory. The only thing left is to fix the substitution rule for the Servlet API 3.1. Add libservlet3.1-java to Build-Depends and the following rule to maven.rules:

javax.servlet s/servlet-api/javax.servlet-api/ * s/.*/3.1/ * *
s/javax.servlet/javax.servlet.jsp/ s/jsp-api/javax.servlet.jsp-api/ * s/.*/2.3/ * *

The maven.rules file consists of multiple rows separated by six columns. The values represent groupId, artifactId, type, version number and two fields which I never use. 🙂 You can just use an asterisk to match any value. Every value can be substituted. This is necessary when the value of upstream’s pom.xml file differs from Debian’s system packages. This happens frequently for API packages which are uploaded to Maven Central multiple times under a different groupId/artifactId but provide the same features. In this case the Twelvemonkeys’ pom requires an older API version but Debian is already at version 3.1. Note that we require a strict version number in this case because libservlet3.1-java does not use a symbolic debian version since we provide more than one Servlet API in the archive and this measure prevents conflicts.

Thanks for reading this far. More articles about Java packaging will follow in the near future and hopefully they will clarify some terms and topics which could only be briefly mentioned in this post.

before

and after

 

 

 

Krebs on SecurityFlash Player is Dead, Long Live Flash Player!

Adobe last week detailed plans to retire its Flash Player software, a cross-platform browser plugin so powerful and so packed with security holes that it has become the favorite target of malware developers. To help eradicate this ubiquitous liability, Adobe is enlisting the help of Apple, Facebook, Google, Microsoft and Mozilla. But don’t break out the bubbly just yet: Adobe says Flash won’t be put down officially until 2020.

brokenflash-aIn a blog post about the move, Adobe said more sites are turning away from proprietary code like Flash toward open standards like HTML5, WebGL and WebAssembly, and that these components now provide many of the capabilities and functionalities that plugins pioneered.

“Over time, we’ve seen helper apps evolve to become plugins, and more recently, have seen many of these plugin capabilities get incorporated into open web standards,” Adobe said. “Today, most browser vendors are integrating capabilities once provided by plugins directly into browsers and deprecating plugins.”

It’s remarkable how quickly Flash has seen a decline in both use and favor, particularly among the top browser makers. Just three years ago, at least 80 percent of desktop Chrome users visited a site with Flash each day, according to Google. Today, usage of Flash among Chrome users stands at just 17 percent and continues to decline (see Google graphic below).

For Mac users, the turning away from Flash began in 2010, when Apple co-founder Steve Jobs famously penned his “Thoughts on Flash” memo that outlined the reasons why the technology would not be allowed on the company’s iOS products. Apple stopped pre-installing the plugin that same year.

The percentage of Chrome users over time that have used Flash on a Web site. Image: Google.

The percentage of Chrome users over time that have used Flash on a Web site. Image: Google.

“Today, if users install Flash, it remains off by default,” a post by Apple’s WebKit Team explains. “Safari requires explicit approval on each website before running the Flash plugin.”

Mozilla said that starting this month Firefox users will choose which websites are able to run the Flash plugin.

“Flash will be disabled by default for most users in 2019, and only users running the Firefox Extended Support Release will be able to continue using Flash through the final end-of-life at the end of 2020,” writes Benjamin Smedberg for Mozilla. “In order to preserve user security, once Flash is no longer supported by Adobe security patches, no version of Firefox will load the plugin.”

Facebook has long hosted plenty of games that invoke Flash, but over time more Facebook apps and games turned to HTML5, the company said.

“Today, more than 200 HTML5 games are live on our platform, most of which launched within the last year,” wrote Facebook’s Jakub Pudelek. “Many of the largest developers on the platform…migrated at least one Flash game to HTML5 on the Facebook platform with minimal impact to their existing customers.”

Finally, Microsoft said it has begun phasing out Flash from Microsoft Edge and Internet Explorer, culminating in the removal of Flash from Windows entirely by the end of 2020. For now, Microsoft Edge, the default browser on newer versions of Windows, will continue to ask users for permission to run Flash on most sites the first time the site is visited, remembering the user’s preference on any subsequent visits.

By mid- to late 2018, Microsoft says, Edge will require permission for Flash to be run each browser session. But by mid 2018, Microsoft will disable Flash by default in both Edge and Internet Explorer. Read more about Microsoft’s timeline for Flash elimination here.

For years, unpatched vulnerabilities in Flash plugins have been the top moneymaker for users of various commercial “exploit kits,” crimeware designed to be stitched into the fabric of hacked or malicious sites and exploit browser plugin flaws.

An analysis of exploit kit activity  by Arlington, Va.-based security firm Recorded Future showed that Flash Player vulnerabilities provided six of the top 10 vulnerabilities used by exploit kits in 2016 [full disclosure: Recorded Future is an advertiser on this blog].

Image: Recorded Future

Image: Recorded Future

I look forward to a time when Flash Player is in the rearview mirror entirely. Until then, KrebsOnSecurity will continue to call attention to new security updates for Flash Player and other widely used Adobe products.

Even so, I’ll also continue to encourage readers to remove or hobble Flash Player unless and until it is needed for a specific site or purpose. More on that approach (as well as slightly less radical solutions ) can be found in A Month Without Adobe Flash Player. The short version is that you can probably get by without Flash installed and not miss it at all.

For readers still unwilling to cut the cord, there are half-measures that work almost as well. Fortunately, disabling Flash in Chrome is simple enough. Paste “chrome://settings/content” into a Chrome browser bar and then select “Flash” from the list of items. By default it should be set to “Ask first” before running Flash, although users also can disable Flash entirely here or whitelist and blacklist specific sites.

Another, perhaps less elegant, solution is to keep Flash installed in a browser that you don’t normally use, and then to only use that browser on sites that require it.

Planet DebianHideki Yamane: I'm going to DebConf17


... No, you're not, my cat.

CryptogramDetecting Stingrays

Researchers are developing technologies that can detect IMSI-catchers: those fake cell phone towers that can be used to surveil people in the area.

This is good work, but it's unclear to me whether these devices can detect all the newer IMSI-catchers that are being sold to governments worldwide.

News article.

Worse Than FailureCodeSOD: Synchronized Threads

Tim was debugging one of those multithreading bugs, where there appeared to be a race condition of some kind. The developer who had initially written the code denied that such a thing could exist: “It’s impossible, I used locks to synchronize the threads!”

Well, he did use locks at the very least.

/// <summary>
/// Performs the synchronisation
/// </summary>
/// <param name="state">Current state</param>
private void Synchronize(object state)
{
    // Take care that this can only run in one thread at a time
    var lockThis = new Object();
    lock (lockThis)
    {
        //…code…
    }
}

There is of course, one problem. The object you use for the lock needs to be shared across threads. This is less a “lock” in the sense of an “air lock” and more a lock in the sense of a “complete hull breach”.

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Planet DebianJonathan Dowland: Debian on the Raspberry Pi3

Back in November, Michael Stapelberg blogged about running (pure) Debian on the Raspberry Pi 3. This is pretty exciting because Raspbian still provide 32 bit packages, so this means you can run a true ARM64 OS on the Pi. Unfortunately, one of the major missing pieces with Debian on the Pi3 at this time is broken video support.

A helpful person known as "SandPox" wrote to me in June to explain that they had working video for a custom kernel build on top of pure Debian on the Pi, and they achieved this simply by enabling CONFIG_FB_SIMPLE in the kernel configuration. On request, this has since been enabled for official Debian kernel builds.

Michael and I explored this and eventually figured out that this does work when building the kernel using the upstream build instructions, but it doesn't work when building using the Debian kernel package's build instructions.

I've since ran out of time to look at this more, so I wrote to request help from the debian-kernel mailing list, alas, nobody has replied yet.

I've put up the dmesg.txt for a boot with the failing kernel, which might offer some clues. Can anyone help figure out what's wrong?

Thanks to Michael for driving efforts for Debian on the Pi, and to SandPox for getting in touch to make their first contribution to Debian. Thanks also to Daniel Silverstone who loaned me an ARM64 VM (from Scaleway) upon which I performed some of my kernel builds.

,

Krebs on SecurityNew Bill Seeks Basic IoT Security Standards

Lawmakers in the U.S. Senate today introduced a bill that would set baseline security standards for the government’s purchase and use of a broad range of Internet-connected devices, including computers, routers and security cameras. The legislation, which also seeks to remedy some widely-perceived shortcomings in existing cybercrime law, was developed in direct response to a series of massive cyber attacks in 2016 that were fueled for the most part by poorly-secured “Internet of Things” (IoT) devices.

iotc

The IoT Cybersecurity Improvement Act of 2017 seeks to use the government’s buying power to signal the basic level of security that IoT devices sold to Uncle Sam will need to have. For example, the bill would require vendors of Internet-connected devices purchased by the federal government make sure the devices can be patched when security updates are available; that the devices do not use hard-coded (unchangeable) passwords; and that vendors ensure the devices are free from known vulnerabilities when sold.

The bill, introduced by Sens. Steve Daines (R-Mont.), Cory Gardner (R-Colo.), Mark Warner (D-Va.) and Ron Wyden (D-Ore.), directs the White House Office of Management and Budget (OMB) to develop alternative network-level security requirements for devices with limited data processing and software functionality. In addition, it requires each executive agency to inventory all Internet-connected devices in use by the agency.

The bill’s provisions would seem to apply to virtually any device that has an Internet connection and can transmit data. Under the proposal, an IoT device has a fairly broad definition, being described as “a physical object that is capable of connecting to and is in regular connection with the Internet;” and one that “has computer processing capabilities that can collect, send or receive data.”

According to the bill’s core sponsors, the measure already has the support of several key legislative technology groups, including the Center for Democracy & Technology (CDT), Mozilla, and the Berklett Cybersecurity Project at Harvard University’s Berkman Klein Center for Internet & Society.

Those advocates were no doubt involved in shaping other aspects of this legislation, including one that exempts cybersecurity researchers engaging in good-faith research from liability under the Computer Fraud and Abuse Act (CFAA), a dated anti-cybercrime law that many critics say has been abused by government prosecutors and companies to intimidate and silence security researchers.

Perhaps the most infamous example of prosecutorial overreach under the CFAA comes in Aaron Swartz, a Harvard research fellow who committed suicide after being hounded by multiple CFAA fraud charges by state and federal prosecutors for downloading a large number of academic journals.

Specifically, the bill would “exempt cybersecurity researchers engaging in good-faith research from liability under the Computer Fraud and Abuse Act and the Digital Millennium Copyright Act when in engaged in research pursuant to adopted coordinated vulnerability disclosure guidelines,” according to a statement released by Sen. Warner (link added).

The measure also directs the Department of Homeland Security to issue guidelines regarding cybersecurity coordinated vulnerability disclosure policies to be required by contractors providing connected devices to the U.S. government.

Last fall, Sens. Warner and others pinged federal regulators at the U.S. Federal Trade Commission (FTC) and the U.S. Federal Communications Commission (FCC) to see if something more could be done about the proliferating threat from poorly-secured IoT devices.

At the time, the world had just witnessed two of the largest cyberattacks the Internet had ever seen (including one against this Web site). Those attacks were launched with the help of IoT devices — mostly cheap security cameras and Internet routers — that were hacked thanks largely to user accounts which could not be removed and which were configured to be remotely accessible over the Internet.

A full text of the Senate proposal is available here.

Update, 3:49 p.m. ET: Corrected abbreviation for Sen. Wyden’s home state.

Planet DebianPaul Wise: FLOSS Activities July 2017

Changes

Issues

Review

Administration

  • Debian: fsck/reboot a buildd, reboot a segfaulting buildd, report/fix broken hoster contact, ping hoster about down machines, forcibly reset backup machine, merged cache patch for network-test.d.o, do some samhain dances, fix two stunnel services, update an IP address in LDAP, fix /etc/aliases on one host, reboot 1 non-responsive VM
  • Debian mentors: security updates, reboot
  • Debian wiki: whitelist several email addresses
  • Debian build log scanner: deploy my changes
  • Debian PTS: deploy my changes
  • Openmoko: security updates & reboots

Communication

  • Ping Advogato users on Planet Debian about updating/removing their feeds since it shut down
  • Invite deepin to the Debian derivatives census
  • Welcome Deepin to the Debian derivatives census
  • Inquire about the status of GreenboneOS, HandyLinux

Sponsors

All work was done on a volunteer basis.

Planet DebianThorsten Alteholz: My Debian Activities in July 2017

FTP assistant

This month I am back to normal numbers and accepted 319 packages. I also kept the promise from last month and rejected 26 uploads.

Debian LTS

This was my thirty-seventh month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.

This month my all in all workload went up to 23.5h. During that time I did LTS uploads of:

  • [DLA 1025-1] bind9 security update for two CVEs
  • [DLA 1038-1] libtasn1-3 security update for one CVE
  • [DLA 1025-2] bind9 regression update
  • [DLA 1039-1] rkhunter security update for one CVE
  • [DLA 1040-1] resiprocate security update for one CVE
  • [DLA 1041-1] nasm security update for two CVEs
  • [DLA 1042-1] libquicktime security update for seven CVEs

I could also remove libtorrent-rasterbar and pspp from dla-needed.txt as the affected code was not in the Wheezy version or it was just a simple bug.

Last but not least I also had a few days of frontdesk duties.

Other stuff

This month I uploaded a new version of entropybroker with a revised set of systemd service files. At the moment there is public instance of entropybroker running at eb.debian.net. Its entropy is fed by several Entropy Keys made by Simtec Electronics. Though it is public, it is not yet anonymous, so if you need some entropy please drop me a line. At the moment there are two consumers, but the buffers are still filled.

I also uploaded several new packages, orcania, yder, hoel and ulfius. If everything works as expected, there will be soon an oauth2 server available in Debian.

Last but not least my DOPOM of this month has been ptunnel.

Planet DebianReproducible builds folks: Reproducible Builds: Weekly report #118

Here's what happened in the Reproducible Builds effort between Sunday July 23 and Saturday July 29 2017:

Toolchain development and fixes

  • Chris Lamb sent an experimental patch to apt to make the output of apt-ftparchive reproducible. Thanks to David Kalnischkies for reworking the result. (#869557)

Packages reviewed and fixed, and bugs filed

Reviews of unreproducible packages

4 package reviews have been added, 2 have been updated and 24 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Aaron M. Ucko (1)
  • Adrian Bunk (35)
  • Helmut Grohne (4)
  • Stefan Tatschner (1)

diffoscope development

Misc.

This week's edition was written by Chris Lamb, Mattia Rizzolo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

CryptogramNSA Collects MS Windows Error Information

Back in 2013, Der Spiegel reported that the NSA intercepts and collects Windows bug reports:

One example of the sheer creativity with which the TAO spies approach their work can be seen in a hacking method they use that exploits the error-proneness of Microsoft's Windows. Every user of the operating system is familiar with the annoying window that occasionally pops up on screen when an internal problem is detected, an automatic message that prompts the user to report the bug to the manufacturer and to restart the program. These crash reports offer TAO specialists a welcome opportunity to spy on computers.

When TAO selects a computer somewhere in the world as a target and enters its unique identifiers (an IP address, for example) into the corresponding database, intelligence agents are then automatically notified any time the operating system of that computer crashes and its user receives the prompt to report the problem to Microsoft. An internal presentation suggests it is NSA's powerful XKeyscore spying tool that is used to fish these crash reports out of the massive sea of Internet traffic.

The automated crash reports are a "neat way" to gain "passive access" to a machine, the presentation continues. Passive access means that, initially, only data the computer sends out into the Internet is captured and saved, but the computer itself is not yet manipulated. Still, even this passive access to error messages provides valuable insights into problems with a targeted person's computer and, thus, information on security holes that might be exploitable for planting malware or spyware on the unwitting victim's computer.

Although the method appears to have little importance in practical terms, the NSA's agents still seem to enjoy it because it allows them to have a bit of a laugh at the expense of the Seattle-based software giant. In one internal graphic, they replaced the text of Microsoft's original error message with one of their own reading, "This information may be intercepted by a foreign sigint system to gather detailed information and better exploit your machine." ("Sigint" stands for "signals intelligence.")

The article talks about the (limited) value of this information with regard to specific target computers, but I have another question: how valuable would this database be for finding new zero-day Windows vulnerabilities to exploit? Microsoft won't have the incentive to examine and fix problems until they happen broadly among its user base. The NSA has a completely different incentive structure.

I don't remember this being discussed back in 2013.

CryptogramVulnerabilities in Car Washes

Articles about serious vulnerabilities in IoT devices and embedded systems are now dime-a-dozen. This one concerns Internet-connected car washes:

A group of security researchers have found vulnerabilities in internet-connected drive-through car washes that would let hackers remotely hijack the systems to physically attack vehicles and their occupants. The vulnerabilities would let an attacker open and close the bay doors on a car wash to trap vehicles inside the chamber, or strike them with the doors, damaging them and possibly injuring occupants.

Worse Than FailureTales from the Interview: The 5% Candidate

Exams Start... Now

There are many kinds of jackasses in this world, from the pretentious prick to the smug cynic. Each has their own flavor of awfulness, their own way of making you hate not only them but the entire world that gave birth to them. This story is about one kind of jackass in particular, perhaps the most classic flavor: the man so sure of his own greatness that he becomes enraged at the world whenever it fails to bow before his massive intellect.

You see these people a lot on Twitter these days. With self-righteous fury, they demand that you get with the program and acknowledge their clear superiority. But as obnoxious as they are online, they're worse in person ... especially if they turn up at your job interview.

Today's candidate applied for a job at a government IT department. Unlike stories you've seen on this site before, this government shop was actually fairly efficient and pleasant to work for. They were hiring Java developers, preferably ones that also had UI and database skills. As such, they had over 100 CVs to skim through for their first 2 positions. After removing those written in crayon, with massive coffee rings obscuring the text, or which had return addresses in prison, they were able to narrow the field to a mere 30, but it was still far more candidates than they wanted to interview in a few short days.

But interview they did. At 10 candidates a day, they barely had time to weed through people; however, it didn't take long to eliminate most of the candidates. Some lacked a basic understanding of computers, such as how to launch applications when they're not strewn across the desktop. Others lacked a basic understanding of programming, being entirely unable to tell Java apart from Microsoft Word. Still others—disturbingly many others—lacked a basic understanding of hygiene.

For Round 2, they decided only to work with agencies they'd had firsthand experience with, either from that office or from previous companies. They also put together a quick "sniff test" to filter the wheat from the chaff. This 30-minute test checked for basic logic skills, including some open-ended CS questions and Java code to debug. They were looking more for the explanations behind the answers than the answers themselves, hoping to get some idea of how these people reasoned.

It worked like a charm. Those who scored under 50% were always appalling in the interview, and those who scored highly were always at worst acceptable. They quickly found their candidates. When it came time to fill the next junior opening, the decision was unanimous: they would use the sniff test as a screen, refusing to interview anyone who failed.

Enter The Architect, our aforementioned jackass. This guy seemed pretty good on paper: "10 years experience in infrastructure architecture, design patterns, certifications, and software development practices" according to his cover letter. Applying for a junior role was a bit odd for this veteran, to be sure, but they gave him the test anyway.

And boy, did he fail. His final score was a mere 5%. Every answer included a tirade about how the question was wrong. Every. Single. One.

Some of you may not believe this man exists. But some of you have met him, or one of his many counterparts the world over. This is the man who, when faced with a question like:

Linked List, Binary Tree, Stack and Queue - describe a simple program to read in a million names and output them in reverse order using one of the above structures.

Writes an answer like:

Seriously??? I wouldn't use any data structures. I'd use a database. Thats what there there for. Man you need a rethink!!!

Or when faced with this simple logic test:

What's the missing sequence: 2, 4, 8, __, 32 1, 3, 9, 27, __

Replies:

2, 4, 8, 10, 32 You've missed out 6, 12, 14, 16, 18 etc. This is unacceptable for a test at this level. Are you sure you want people of my caliber here? Sort it out please!!!

Those who've had the misfortune of meeting someone like this know what comes next, but I'll relate it anyway.

The exam was graded and laughed at. The interviewer went into the room to tell the man he just "wasn't the right fit."

The man exploded with rage: screaming obscenities, wishing death and destruction upon the interviewer, the business, the whole city. He refused to leave until they offered him the job. It took 3 people plus the security team to escort him out of the building, and even then he wouldn't go until they threatened to call the police.

Somewhere out there, there is a blog in which this agency is lambasted up and down for its poor hiring practices. It probably goes on a scathing rant, estimating (too highly) how much of "MY TAXES!!!!" this man pays to support these "incompetent" developers who "wasted MY time!" with their "bullsh!t interview". Maybe it even theorizes that taxes themselves are illegal, as the man proudly declares himself a "sovereign citizen".

Thankfully, you are reading The Daily WTF and not this man's blog. In fact, I'd dare say nobody is visiting this man's blog. That's probably why he's so very angry in the first place.

[Advertisement] Incrementally adopt DevOps best practices with BuildMaster, ProGet and Otter, creating a robust, secure, scalable, and reliable DevOps toolchain.

Planet DebianRussell Coker: QEMU for ARM Processes

I’m currently doing some embedded work on ARM systems. Having a virtual ARM environment is of course helpful. For the i586 class embedded systems that I run it’s very easy to setup a virtual environment, I just have a chroot run from systemd-nspawn with the --personality=x86 option. I run it on my laptop for my own development and on a server my client owns so that they can deal with the “hit by a bus” scenario. I also occasionally run KVM virtual machines to test the boot image of i586 embedded systems (they use GRUB etc and are just like any other 32bit Intel system).

ARM systems have a different boot setup, there is a uBoot loader that is fairly tightly coupled with the kernel. ARM systems also tend to have more unusual hardware choices. While the i586 embedded systems I support turned out to work well with standard Debian kernels (even though the reference OS for the hardware has a custom kernel) the ARM systems need a special kernel. I spent a reasonable amount of time playing with QEMU and was unable to make it boot from a uBoot ARM image. The Google searches I performed didn’t turn up anything that helped me. If anyone has good references for getting QEMU to work for an ARM system image on an AMD64 platform then please let me know in the comments. While I am currently surviving without that facility it would be a handy thing to have if it was relatively easy to do (my client isn’t going to pay me to spend a week working on this and I’m not inclined to devote that much of my hobby time to it).

QEMU for Process Emulation

I’ve given up on emulating an entire system and now I’m using a chroot environment with systemd-nspawn.

The package qemu-user-static has staticly linked programs for emulating various CPUs on a per-process basis. You can run this as “/usr/bin/qemu-arm-static ./staticly-linked-arm-program“. The Debian package qemu-user-static uses the binfmt_misc support in the kernel to automatically run /usr/bin/qemu-arm-static when an ARM binary is executed. So if you have copied the image of an ARM system to /chroot/arm you can run the following commands like the following to enter the chroot:

cp /usr/bin/qemu-arm-static /chroot/arm/usr/bin/qemu-arm-static
chroot /chroot/arm bin/bash

Then you can create a full virtual environment with “/usr/bin/systemd-nspawn -D /chroot/arm” if you have systemd-container installed.

Selecting the CPU Type

There is a huge range of ARM CPUs with different capabilities. How this compares to the range of x86 and AMD64 CPUs depends on how you are counting (the i5 system I’m using now has 76 CPU capability flags). The default CPU type for qemu-arm-static is armv7l and I need to emulate a system with a armv5tejl. Setting the environment variable QEMU_CPU=pxa250 gives me armv5tel emulation.

The ARM Architecture Wikipedia page [2] says that in armv5tejl the T stands for Thumb instructions (which I don’t think Debian uses), the E stands for DSP enhancements (which probably isn’t relevant for me as I’m only doing integer maths), the J stands for supporting special Java instructions (which I definitely don’t need) and I’m still trying to work out what L means (comments appreciated).

So it seems clear that the armv5tel emulation provided by QEMU_CPU=pxa250 will do everything I need for building and testing ARM embedded software. The issue is how to enable it. For a user shell I can just put export QEMU_CPU=pxa250 in .login or something, but I want to emulate an entire system (cron jobs, ssh logins, etc).

I’ve filed Debian bug #870329 requesting a configuration file for this [1]. If I put such a configuration file in the chroot everything would work as desired.

To get things working in the meantime I wrote the below wrapper for /usr/bin/qemu-arm-static that calls /usr/bin/qemu-arm-static.orig (the renamed version of the original program). It’s ugly (I would use a config file if I needed to support more than one type of CPU) but it works.

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
  if(setenv("QEMU_CPU", "pxa250", 1))
  {
    printf("Can't set $QEMU_CPU\n");
    return 1;
  }
  execv("/usr/bin/qemu-arm-static.orig", argv);
  printf("Can't execute \"%s\" because of qemu failure\n", argv[0]);
  return 1;
}

Don MartiWhy surveillance marketers don't worry about GDPR (but privacy nerds should)

A lot of privacy people these days sound like a little kid arguing with a sibling. You're going to be in big trouble when Dad gets home!

Dad, here, is the European Union, who's going to put the General Data Protection Regulation foot down, and then, oh, boy, those naughty surveillance marketers are going to catch it, and wish that they had been listening to us about privacy all along.

Right?

But Internet politics never works like that. Sure, European politicians don't want to hand over power to the right-wing factions who are better at surveillance marketing than they are. And foreign agents use Facebook (and other US-based companies) to attack legit political systems. But that stuff is not going to be enough to save GDPR.

The problem is that perfectly normal businesses are using GDPR-violating sneaky tracking pixels and other surveillance marketing as part of their daily marketing routine.

As the GDPR deadline approaches, surveillance marketers in Europe are going to sigh and painstakingly explain to European politicians that of course this GDPR thing isn't going to work. "You see, politicians, it's an example of political overreach that completely conflicts with technical reality." European surveillance marketers will use the same kind of language about GDPR that the freedom-loving side used when we talked about the proposed CBDTPA. It's just going to Break the Internet! People will lose their jobs!

The result is predictable. GDPR will be delayed, festooned with exceptions, or both, and the hoped-for top-down solution to privacy problems will not come. There's no shortcut. We'll only get a replacement for surveillance marketing when we build the tools, the networks, the business processes, the customer/voter norms, and then the political power.

Planet DebianRuss Allbery: Learning Rust

I'm obviously not spending much time writing here. It's been a rather busy month at work, and I've been doing other things on the weekend that aren't particularly interesting to write about.

This past week, though, I took advantage of our semi-annual Hack Week to finally learn Rust. I have several co-workers who love the language and have been wanting to stretch my programming language knowledge a bit. I was also profoundly disappointed by Go, which has been touted as the new C-style systems language but which I think is awful. All the reasons why is a topic for another post, but the obnoxiously verbose error handling is probably my biggest complaint. (This is the worst property of C; why would you copy it?) Rust was a favorite of a few people who felt the same way I did about Go, which seemed promising.

I made it through the first thirteen chapters of the second edition Rust book and wrote a not-entirely-trivial program (a tool to filter and search trace logs a Dropbox client) with a co-worker, and I think I'm in love with this language. It reminds me of everything I liked about Perl, except with all the weird bolted-on bits of Perl cleaned up and done properly, and with types. Despite having spent most of my career writing Perl and Python (and C, which is typed but not very well), I love strongly-typed languages. I just usually don't like the rest of the syntax of languages like Java and Go. Rust avoids the garbage collection nonsense (and huge performance issues), gives me the level of fine control that I am used to with C, but gets rid of memory allocation errors and provides a much richer type system and type matching. It feels a bit like an approachable Haskell, and I quickly found myself chaining iterators and pushing myself to write in a more functional style.

The lifetime stuff in Rust can be frustrating, and there are a few limitations that can be hard to deal with (like iterating over one field of a struct while modifying another field of a struct, which comes up a lot and which is the sort of thing you have to avoid in Rust). But I like a language with a very picky compiler. And I love programming in a language where my first attempt is clunky and verbose, and then I think about the problem a bit and rewrite it in half as many lines of code, and then I sleep on it and come back and can delete half of the code again.

I sadly don't have a lot of work projects right now where Rust is the right answer. I'm mostly maintaining existing code bases in Python, and Python is more accessible and more maintainable in most situations. But I now would love to find the time to rewrite a bunch of my personal C projects in Rust, and I'm watching for any new opportunity to use Rust.

If you like new programming languages, but you don't have the time or inclination to live on the bleeding edge, Rust has gotten more stable and is at a good point to start. The documentation is fantastic, support for generating documentation is built into the language, the Rust book is a great teaching research, and Rust is available as Debian packages (so you don't have to do the horrifying curl | bash nonsense in the official Rust documentation).

Recommended, particularly for people who love Perl or functional languages (or functional Perl heavy on map and grep), want a more modern language with fewer odd corners, and want low-level control and native speed.

Planet DebianRuss Allbery: Review: The Fifth Season

Review: The Fifth Season, by N.K. Jemisin

Series: The Broken Earth #1
Publisher: Orbit
Copyright: 2015
ISBN: 0-316-22930-X
Format: Kindle
Pages: 497

The world of The Fifth Season is one of near-constant seismic activity. Volcanoes, massive earthquakes, and all the catastrophes that follow them are a constant threat. Civilization barely survives the turmoil, and only because of two things: strict cultural rules about how to handle a "fifth season" of heavy seismic activity and its aftermath (called stonelore), and the orogenes.

Orogenes are humans (well, there is some debate about that) who have an organ that others don't, a biological ability to manipulate the seismic activity and the earth itself. They can protect others by damping down activity, smoothing faults, and redirecting seismic shock waves, but they can also destroy: pull earth out of shape, set off quakes, and create paths for magma to surface. And, to gather the power to manipulate the earth, they draw energy from everything around them, including from other people, often fatally. Orogenes are feared and hated by the typical person.

The Stillness, the ironically-named continent on which this book is set, is very old and has had numerous civilizations destroyed by some seismic catastrophe. The landscape is scattered with useless or dangerous remnants of previous forgotten civilizations; the history, likewise, with only the stonelore and some muddled mythology available to most people. The current rulers have kept their empire for a surprising length of time, however, due mostly to the stable ground beneath their centrally-located capital. That stability comes from Fulcrum-trained orogenes, who are taken from their family as children and trained harshly to serve their society by suppressing or fixing dangerous seismic events. Fulcrum orogenes don't have an awful life (well, most of them; for some, it is pure torture), but they're effectively slaves, kept under the watchful eye of Guardians who have mysterious powers of their own.

Against this background, The Fifth Season tells three interwoven stories. Essun lives in a small village (comm) at the start of the book, leading a quiet life, until one of her children is beaten to death by her husband following a seismic event that he thinks the child stopped. He's taken their other child and left. Essun, severely traumatized, heads after him to attempt a rescue, or at least revenge. Damaya is a child from another comm who is sold to the Guardians by her parents when she demonstrates orogenic ability, and who goes through Fulcrum training. And Syenite is a Fulcrum orogene, assigned to a field mission with a difficult but very senior orogene named Alabaster.

All of these stories eventually interweave, and eventually reveal where they fit in the somewhat unobvious chronology of the story, but it takes some time to get there. It also takes some time for the primary characters to have much in the way of agency. Essun starts with the most, once she recovers her senses enough to start her hunt for revenge. Syenite is ambitious but junior, and Damaya is a child, trying to navigate an unknown world of student politics and strict rules. And all three of the main characters are orogenes, rogga when one is being insulting, and this world does not like orogenes. At all.

The Fifth Season starts with an unusual narrative style: a conversational narrator who begin with some of the world background and some mysterious scenes that didn't make sense until much later in the book (late enough that I didn't remember them or make sense of them until I re-read them for this review). The book then focuses on Essun, whose scenes are written in second person present. Normally I think second person feels weirdly intrusive and off-putting, but once I got used to it here, I think it works as well as I've seen it work anywhere. I also see why Jemisin did it: Essun starts the story so traumatized that she's partly disassociating. First person wouldn't have worked, and the second-person voice gives that trauma some immediacy and emotional heft that would have been hard to achieve in third person.

The story starts slowly, and builds slowly, as the world is introduced and Jemisin lays down the texture and history of the world. The world-building is ambitious in tracing down the ramifications of the seismic chaos and the implications of orogene ability (although it's best to think of it as pure magic, despite the minor science fiction trappings). But through that world-building, what this story is building is a deep, powerful, frustrated rage. The Fifth Season is an angry book. It's a book about outcasts, about slaves. About people who, even if they're succeeding within the parameters they're given, are channeled and stymied and controlled. It's a story about smiling, kind paternalism hiding lies, control, and abuse, about how hard it is to find enough space from the smothering destructiveness of a totalitarian culture to let yourself relax. It's a story about the horrible things people are willing to do to those they don't consider fully human, and all the ways in which safety, expediency, tradition, culture, and established social roles conspire to keep people within the box where they belong. And it's a story about how being constantly on edge, constantly dreading the next abuse, breaking under it, and being left wanting to burn the whole world to the ground.

I struggled at the start of this book, but it grew on me, and by about halfway through it had me hooked completely. At first, Syenite's part of the story (the most traditionally told) was my favorite, but the coming-of-age stories of her and Damaya were overtaken by Essun's far more complex, cautious, and battle-weary tale. And I loved Jemisin's world-building. There's a lot of depth here, a lot of things going on that are unexplained by clearly important, and a restraint and maturity in how the world is revealed that makes it feel older and more layered than Jemisin's The Hundred Thousand Kingdoms.

The major drawback of this book is that it is very much the first book of a series, and it doesn't so much have an ending as a hard stop. It's not quite a cliff-hanger, but it's nearly as unsatisfying as one. Most of the major questions of the book — who the stone eaters are and what they want, and the fate of Essun's husband and child, just to name two — are still unresolved at the end of the story. There is a bit of emotional closure, but not a true moment of catharsis for all of the rage. Hopefully that will be coming in a future book.

This is a very unusual story, mixing fantasy and a sort of magic (orogeny) with some science fiction elements and a deep history. It's gritty, textured, emotional, and furious, and very much worth reading. I'm looking forward to the next book in the series.

Followed by The Obelisk Gate.

Rating: 8 out of 10

Planet Linux AustraliaGabriel Noronha: NBN FTTN

Unfortunate for us our home only got FTTN NBN connection. but like others I thought I would share the speed improvement results from cleaning up wiring inside your own home. we have 2 phone sockets 1 in the bedroom and one in the kitchen. by removing the cable from the kitchen to the bedroom, we managed to increase our maximum line rate from 14.2Mbps upload and 35.21Mbps download to 20Mbps upload and 47 Mbps download.

Bedroom Phone Line connected.
Line Statistics Post Wiring clean up

we’ve also put a speed change request from the 12/5 plan to the 50/20 plan so next month we should be enjoying a bit more of an NBN.

To think that with FTTH you could of had up to 4 100/40 connections. and you wouldn’t of had to pay someone to rewire your phone sockets.

Update:

speed change has gone through

NBN ModemModem statistics on 50/20 speed

Planet DebianJunichi Uekawa: Playing with appengine python ndb.

Playing with appengine python ndb. Some interfaces changed from the old interface and I don't think I quite got the hang of it. The last time I was actively using it is when I was doing tokyodebian reservation system that I haven't touched for 3 years and started 8 years ago.

,

TED5 stellar mini-docs that will make you rethink time

Five mini-documentary films captivated the TEDWomen 2016 audience — directed, written and produced by female filmmakers whose work embodies today’s best and most innovative storytelling. In a partnership between Lifetime and Chicken & Egg Pictures, these short films are artful in the ways their storytelling catalyzes social change and the TEDWomen 2016 theme, “It’s About Time.”

Watch the selected films below and learn more about the award-winning filmmakers behind them.

Lyari Girl Boxing

About this film: In Lyari, Pakistan—called “the Colombia of Karachi” because of the tightening grip of rival gangs and widespread drug culture—a group of female boxers are taking ownership of their fate.

About the filmmaker: Sharmeen Obaid Chinoy is a two-time Academy Award and Emmy-winning documentary filmmaker. In the past 15 years, she has made more than a dozen multi-award-winning films in over 10 countries around the world. Her films include A Girl in the River, Song of Lahore, Peacekeepers: A Journey of a Thousand Miles and Saving Face. In 2012, Time Magazine included Sharmeen in its annual list of the 100 most influential people in the world. In 2013, the Canadian government awarded her a Queen Elizabeth II Diamond Jubilee Medal for her work in the field of documentary films, and the World Economic Forum honored her with a Crystal Award at their annual summit in Davos. She is a TED Senior Fellow.

How Much Is Enough?

About this film: Several American mothers reflect on two key questions: How much extra time would you like in a day? What would you do with that extra time?

About the filmmaker: Grace Lee directed the Peabody-winning documentary American Revolutionary: The Evolution of Grace Lee Boggs, which Hollywood Reporter called “an entertainingly revealing portrait of the power of a single individual to effect change.” The film premiered at the 2013 Los Angeles Film Festival and was broadcast on the PBS series “POV.” Her previous documentary The Grace Lee Project was broadcast on Sundance Channel and was called “ridiculously entertaining” by New York magazine. She recently produced two documentaries for PBS: the Emmy-nominated Makers: Women in Politics and Off the Menu: Asian America. As a Women at Sundance Fellow, she is developing a social issue comedy series.

A Mother’s Dream

About this film: An intimate portrait of a day in the life of Collette Flanagan, a mother who lost a child to police violence and now empowers others to demand constructive and concrete systemic change in their communities.

About the filmmaker: Filmmaker, artist and author Michèle Stephenson pulls from her Panamanian and Haitian roots and experience as a human rights attorney to tell compelling, personal stories that resonate beyond the margins. Her most recent film, American Promise, was nominated for three Emmys, won the Jury Prize at Sundance, and was selected for the New York Film Festival’s Main Slate Program. Shewas recently awarded the Chicken & Egg Pictures Filmmaker Breakthrough Award and is a 2016 Guggenheim Fellow and a Sundance Skoll Storytellers for Change Fellow. Her recent book, Promises Kept, written along with co-authors Joe Brewster and Hilary Beard, won an NAACP Image Award for Outstanding Literary Work.

 

BeeLove

About this film: This film captures the unlikely story of Sweet Beginnings, a company that employs ex-offenders by teaching them how to be beekeepers and harvest honey.

About the filmmaker: Kristi Jacobson is an award-winning filmmaker and founder of Catalyst Films. Her latest film, Solitary, an immersive look at life inside a supermax prison, premiered at the 2016 Tribeca Film Festival and will be released on HBO in 2017. She has created films for HBO, PBS, ESPN, ABC, the Sundance Channel, A&E, Lifetime and Channel 4/UK. Her films, including American Standoff, Toots and A Place at the Table, reveal her passion for capturing nuanced, intimate and provocative portrayals of individuals and communities. She’s a 2016 recipient of Chicken & Egg Pictures’ Breakthrough Filmmaker Award, awarded to 5 nonfiction filmmakers whose artful and innovative storytelling catalyzes social change.

 

The Experience of Time

About this film: This short film explores the history of humans’ complicated relationship with time, deconstructs our obsession with controlling it, and contemplates how to be more mindful of this valuable resource.

About the filmmaker: Elaine McMillion Sheldon is a Peabody-winning documentary filmmaker and media artist. She’s the creative director of the Emmy-nominated interactive documentary Hollow and runs “She Does,” a weekly podcast that documents creative women’s journeys. In 2016, she was awarded the Breakthrough Filmmaker award from Chicken & Egg Pictures. Sheldon has been named one of 50 People Changing The South by Southern Living Magazine, a 2013 Future of Storytelling Fellow, and one of the 25 New Faces of Independent Film by Filmmaker magazine. She’s a founding member of All Y’all Southern Documentary Collective.


Sociological ImagesI argued that men avoid ball-kicking to protect the myth of masculinity; in secret, they agreed

In 2015 I wrote an essay in which I speculated about why we don’t see men kicking each other in the balls more often. We leave no stones unturned here at SocImages, folks.

I argued that men don’t kick each other in the balls because it would reveal to everyone an inherent and undeniable biological weakness in every man, not just the man getting kicked.  In other words, it’s a secret pact to protect the myth of masculine superiority.

I expected a reaction, but I was genuinely surprised at what transpired. In public — in the comments — men debated strategy, arguing that men don’t kick each other in the balls because it’s actually a difficult blow to land or would escalate the fight. But in private — in my email inbox — men sent me hushed messages of you-are-so-right-though.

This is interesting because people rarely bother to go to the trouble of googling me, finding my email address, and writing me a note. The comments thread is right there and there’s a link to my twitter account at the end of the post. Most people criticize or compliment me publicly. Moreover, the emails have never stopped coming. I get one now every couple months — almost two years later — which I think means that ball kicking is something men (and it’s always men) are quietly seeking information about.

So, what do they say in private to me?

The one I received today was characteristic and the guy who wrote it gave me permission to share some of it. I’ll call him “Guy.”

First, Guy agreed that the vulnerability of having testicles is distressing to him specifically because he has been taught that boys and men are supposed to be stronger than girls and women.

Boys usually think of themselves as being tough and we want to be tough and tougher than girls especially. The idea that a girl could hurt a big strong boy like me is ridiculous right. But then I got older and learned about testicles and that girls didnt have them and i was embarrassed that I had a weak spot and they didn’t.

Second, he acknowledged that knowing that other people know about this vulnerability adds to the stress of having it.

I always hate in movies when a guy gets hit in the balls and drops especially if a woman did the kicking and if I am watching it with women. I don’t want anyone to know I have a weak spot or to acknowledge it. I still try to workout and be big and strong but I always feel vulnerable down there. My older sister and i used to play fight and i started getting bigger than her and winning. Then one time she faked a kick to my groin and i jumped back and covered myself. She had this self satisfied smurk on her face like ya dont mess with me and i never did again.

This vulnerability, Guy emphasizes, isn’t just a trivial thing; it’s everything. It affects how he feels about his whole body (“your only as strong as your weakest link”) and it’s psychologically consuming (“I hate knowing this”).

Your only as strong as your weakest link and guys have the weakest link on the body. I hate knowing this and I’m afraid women realize this and I think alot of guys feel the same even if they dont admit it.

“They dont admit it,” Guy writes, which means it’s a secret shame. And, like many of the men who’ve emailed me, he thanks me for putting it out there in public and says that it’s a relief to actually talk about it.

Anyway I think you really hit a nerve with this article and I think its kinda therapeutic to talk about it cause I usually keep it to myself. Keep up the good work and Take Care!

I think this is amazing.

I’m touched, first of all, by the emotional vulnerability that Guy and the other (mostly young) men who’ve emailed me have shown. Behind all of the pretending like they’re a “big strong boy,” these guys are nervous, worried that their front is going to be exposed and everyone is going to see them as a fraud and a failure. Not a Real Man at all.

In fact, they worry that everyone already sees them that way. The sister’s smirk tells Guy, in no uncertain terms, that his front is transparent. “I won’t expose you,” it says. “Not today. But I can and we both know it.” No matter how hard he tries — no matter how big his biceps or bank account, no matter how corner his office is or how hot his wife — he’s got those goddamn testicles and they’re right there.

Guy explains that it makes him want to compensate. He works out to be “big and strong.” But it’ll never be enough. He says, “I always feel vulnerable down there.” He feels vulnerable anyway. There’s really nothing he can do.

This is telling us something profound about what it feels like to be a man in America today. Told to live up to an impossible standard of invulnerability; they inevitably feel like failures. Told specifically to be more invulnerable than (and not vulnerable to) women, by biological accident, they’re not. What a cruel twist of the testicles. It hurts.

And I wonder how much of what men do in their lives is a response to this psychic injury. How many of Donald Trump’s shenanigans, for example, have to do with the fact that he knows, and he knows that everyone knows, that someone could just drop him with a kick to the balls at any time? It sounds absurd to blame the risk of nuclear war on Trump’s testicles, but these young men are telling me that, right around puberty — as they are graduating from boys to men, doubling down on their difference from girls and women, and being told that to earn others’ esteem they have to be bigger and stronger — they have a disturbing revelation that compels them to embark on a lifetime of proving they’re not weak.

Until we all agree to let men be human, they’re going to keep living lives of quiet desperation. And the rest of us have to keep fearing what they will do to avoid being exposed.

Lisa Wade, PhD is a professor at Occidental College. She is the author of American Hookup, a book about college sexual culture, and a textbook about gender. You can follow her on Twitter, Facebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

Planet DebianJonathan McDowell: How to make a keyring

Every month or two keyring-maint gets a comment about how a key update we say we’ve performed hasn’t actually made it to the active keyring, or a query about why the keyring is so out of date, or told that although a key has been sent to the HKP interface and that is showing the update as received it isn’t working when trying to upload to the Debian archive. It’s frustrating to have to deal with these queries, but the confusion is understandable. There are multiple public interfaces to the Debian keyrings and they’re not all equal. This post attempts to explain the interactions between them, and how I go about working with them as part of the keyring-maint team.

First, a diagram to show the different interfaces to the keyring and how they connect to each other:

keyring-maint workflow

Public interfaces

rsync: keyring.debian.org::keyrings

This is the most important public interface; it’s the one that the Debian infrastructure uses. It’s the canonical location of the active set of Debian keyrings and is what you should be using if you want the most up to date copy. The validity of the keyrings can be checked using the included sha512sums.txt file, which will be signed by whoever in keyring-maint did the last keyring update.

HKP interface: hkp://keyring.debian.org/

What you talk to with gpg --keyserver keyring.debian.org. Serves out the current keyrings, and accepts updates to any key it already knows about (allowing, for example, expiry updates, new subkeys + uids or new signatures without the need to file a ticket in RT or otherwise explicitly request it). Updates sent to this interface will be available via it within a few hours, but must be manually folded into the active keyring. This in general happens about once a month when preparing for a general update of the keyring; for example b490c1d5f075951e80b22641b2a133c725adaab8.

Why not do this automatically? Even though the site uses GnuPG to verify incoming updates there are still occasions we’ve seen bugs (such as #787046, where GnuPG would always import subkeys it didn’t understand, even when that subkey was already present). Also we don’t want to allow just any UID to be part of the keyring. It is thus useful to retain a final set of human based sanity checking for any update before it becomes part of the keyring proper.

Alioth/anonscm: https://anonscm.debian.org/git/keyring/keyring/

A public mirror of the git repository the keyring-maint team use to maintain the keyring. Every action is recorded here, and in general each commit should be a single action (such as adding a new key, doing a key replacement or moving a key between keyrings). Note that pulling in the updates sent via HKP count as a single action, rather than having a commit per key updated. This mirror is updated whenever a new keyring is made active (i.e. made available via the rsync interface). Until that point pending changes are kept private; we sometimes deal with information such as the fact someone has potentially had a key compromised that we don’t want to be public until we’ve actually disabled it. Every “keyring push” (as we refer to the process of making a new keyring active) is tagged with the date it was performed. Releases are also tagged with their codenames, to make it easy to do comparisons over time.

Debian archive

This is actually the least important public interface to the keyring, at least from the perspective of the keyring-maint team. No infrastructure makes use of it and while it’s mostly updated when a new keyring is made active we only make a concerted effort to do so when it is coming up to release. It’s provided as a convenience package rather than something which should be utilised for active verification of which keys are and aren’t currently part of the keyring.

Team interface

Master repository: kaufmann.debian.org:/srv/keyring.debian.org/master-keyring.git

The master git repository for keyring maintenance is stored on kaufmann.debian.org AKA keyring.debian.org. This system is centrally managed by DSA, with only DSA and keyring-maint having login rights to it. None of the actual maintenance work takes place here; it is a bare repo providing a central point for the members of keyring-maint to collaborate around.

Private interface

Private working clone

This is where all of the actual keyring work happens. I have a local clone of the repository from kaufmann on a personal machine. The key additions / changes I perform all happen here, and are then pushed to the master repository so that they’re visible to the rest of the team. When preparing to make a new keyring active the changes that have been sent to the HKP interface are copied from kaufmann via scp and folded in using the pull-updates script. The tree is assembled into keyrings with a simple make and some sanity tests performed using make test. If these are successful the sha512sums.txt file is signed using gpg --clearsign and the output copied over to kaufmann. update-keyrings is then called to update the active keyrings (both rsync + HKP). A git push public pushes the changes to the public repository on anonscm. Finally gbp buildpackage --git-builder='sbuild -d sid' tells git-buildpackage to use sbuild to build a package ready to be uploaded to the archive.

Hopefully that helps explain the different stages and outputs of keyring maintenance; I’m aware that it would be a good idea for this to exist somewhere on keyring.debian.org as well and will look at doing so.

Planet DebianDaniel Silverstone: F/LOSS activity, July 2017

Once again, my focus was on Gitano, which we're working toward a 1.1 for. We had another one of our Gitano developer days which was attended by Richard maw and myself. You are invited to read the wiki page but a summary of what happened, which directly involved me, is:

  • Once again, we reviewed our current task state
  • We had a good discussion about our code of conduct including adopting a small change from upstream to improve matters
  • I worked on, and submitted a patch for, improving nested error message reports in Lace.
  • I reviewed and merged some work from Richard about pattern centralisation
  • I responded to comments on a number of in-flight series Richard had reviewed for me.
  • We discussed our plans for 1.1 and agreed that we'll be skipping a developer day in August because so much of it is consumed by DebConf and so on.

Other than that, related to Gitano during July I:

  • Submitted some code series before the developer day covering Gall cleanups and hook support in Gitano.
  • Reviewed and merged some more Makefile updates from Richard Ipsum
  • Reviewed and merged a Supple fix for environment cleardown from Richard Ipsum
  • Fixed an issue in one of the Makefiles which made it harder to build with dh-lua
  • I began work in earnest on Gitano CI, preparing a lot of scripts and support to sit around Jenkins (for now) for CIing packaging etc for Gitano and Debian
  • I began work on a system branch concept for Gitano CI which will let us handle the CI of branches in the repos, even if they cross repos.

I don't think I've done much non-Gitano F/LOSS work in July, but I am now in Montréal for debconf 2017 so hopefully more to say next month.

Planet DebianChris Lamb: Free software activities in July 2017

Here is my monthly update covering what I have been doing in the free software world during July 2017 (previous month):

  • Updated travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds:
    • Moved the default mirror from ftp.de.debian.org to deb.debian.org. []
    • Create a sensible debian/changelog file if one does not exist. []
  • Updated django-slack, my library to easily post messages to the Slack group-messaging utility:
    • Merged a PR to clarify the error message when a channel could not be found. []
    • Reviewed and merged a suggestion to add a TestBackend. []
  • Added Pascal support to Louis Taylor's anyprint hack to add support for "print" statements from other languages into Python. []
  • Filed a PR against Julien Danjou's daiquiri Python logging helper to clarify an issue in the documentation. []
  • Merged a PR to Strava Enhancement Suite — my Chrome extension that improves and fixes annoyances in the web interface of the Strava cycling and running tracker — to remove Zwift activities with maps. []
  • Submitted a pull request for Redis key-value database store to fix a spelling mistake in a binary. []
  • Sent patches upstream to the authors of the OpenSVC cloud engine and the Argyll Color Management System to fix some "1204" typos.
  • Fixed a number of Python and deployment issues in my stravabot IRC bot. []
  • Correct a "1204" typo in Facebook's RocksDB key-value store. []
  • Corrected =+ typos in the Calibre e-book reader software. []
  • Filed a PR against the diaspy Python interface to the DIASPORA social network to correct the number of seconds in a day. []
  • Sent a pull request to remedy a =+ typo in sparqlwrapper, a SPARQL endpoint interface for Python. []
  • Filed a PR against Postfix Admin to fix some =+ typos. []
  • Fixed a "1042" typo in ImageJ, a Java image processing library. []
  • On a less-serious note, I filed an issue for Brad Abraham's bot for the Reddit sub-reddit to add some missing "hit the gym" advice. []

I also blogged about my recent lintian hacking and installation-birthday package.


Reproducible builds


Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

(I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.)

This month I:

  • Assisted Mattia with a draft of an extensive status update to the debian-devel-announce mailing list. There were interesting follow-up discussions on Hacker News and Reddit.
  • Submitted the following patches to fix reproducibility-related toolchain issues within Debian:
  • I also submitted 5 patches to fix specific reproducibility issues in autopep8, castle-game-engine, grep, libcdio & tinymux.
  • Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
  • Worked on publishing our weekly reports. (#114 #115, #116 & #117)

I also made the following changes to our tooling:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • comparators.xml:
    • Fix EPUB "missing file" tests; they ship a META-INF/container.xml file. []
    • Misc style fixups. []
  • APK files can also be identified as "DOS/MBR boot sector". (#868486)
  • comparators.sqlite: Simplify file detection by rewriting manual recognizes call with a Sqlite3Database.RE_FILE_TYPE definition. []
  • comparators.directory:
    • Revert the removal of a try-except. (#868534)
    • Tidy module. []

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Add missing File::Temp imports in the JAR and PNG handlers. This appears to have been exposed by lazily-loading handlers in #867982. (#868077)

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Avoid a race condition between check-and-creation of Buildinfo instances. []


Debian

My activities as the current Debian Project Leader are covered in my "Bits from the DPL emails to the debian-devel-announce mailing list.

Patches contributed

  • obs-studio: Remove annoying "click wrapper" on first startup. (#867756)
  • vim: Syntax highlighting for debian/copyright files. (#869965)
  • moin: Incorrect timezone offset applied due to "84600" typo. (#868463)
  • ssss: Add a simple autopkgtest. (#869645)
  • dch: Please bump $latest_bpo_dist to current stable release. (#867662)
  • python-kaitaistruct: Remove Markdown and homepage references from package long descriptions. (#869265)
  • album-data: Correct invalid Vcs-Git URI. (#869822)
  • pytest-sourceorder: Update Homepage field. (#869125)

I also made a very large number of contributions to the Lintian static analysis tool. To avoid duplication here, I have outlined them in a separate post.


Debian LTS


This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 1014-1 for libclamunrar, a library to add unrar support to the Clam anti-virus software to fix an arbitrary code execution vulnerability.
  • Issued DLA 1015-1 for the libgcrypt11 crypto library to fix a "sliding windows" information leak.
  • Issued DLA 1016-1 for radare2 (a reverse-engineering framework) to prevent a remote denial-of-service attack.
  • Issued DLA 1017-1 to fix a heap-based buffer over-read in the mpg123 audio library.
  • Issued DLA 1018-1 for the sqlite3 database engine to prevent a vulnerability that could be exploited via a specially-crafted database file.
  • Issued DLA 1019-1 to patch a cross-site scripting (XSS) exploit in phpldapadmin, a web-based interface for administering LDAP servers.
  • Issued DLA 1024-1 to prevent an information leak in nginx via a specially-crafted HTTP range.
  • Issued DLA 1028-1 for apache2 to prevent the leakage of potentially confidential information via providing Authorization Digest headers.
  • Issued DLA 1033-1 for the memcached in-memory object caching server to prevent a remote denial-of-service attack.

Uploads

  • redis:
    • 4:4.0.0-1 — Upload new major upstream release to unstable.
    • 4:4.0.0-2 — Make /usr/bin/redis-server in the primary package a symlink to /usr/bin/redis-check-rdb in the redis-tools package to prevent duplicate debug symbols that result in a package file collision. (#868551)
    • 4:4.0.0-3 — Add -latomic to LDFLAGS to avoid a FTBFS on the mips & mipsel architectures.
    • 4:4.0.1-1 — New upstream version. Install 00-RELEASENOTES as the upstream changelog.
    • 4:4.0.1-2 — Skip non-deterministic tests that rely on timing. (#857855)
  • python-django:
    • 1:1.11.3-1 — New upstream bugfix release. Check DEB_BUILD_PROFILES consistently, not DEB_BUILD_OPTIONS.
  • bfs:
    • 1.0.2-2 & 1.0.2-3 — Use help2man to generate a manpage.
    • 1.0.2-4 — Set hardening=+all for bindnow, etc.
    • 1.0.2-5 & 1.0.2-6 — Don't use upstream's release target as it overrides our CFLAGS & install RELEASES.md as the upstream changelog.
    • 1.1-1 — New upstream release.
  • libfiu:
    • 0.95-4 — Apply patch from Steve Langasek to fix autopkgtests. (#869709)
  • python-daiquiri:
    • 1.0.1-1 — Initial upload. (ITP)
    • 1.1.0-1 — New upstream release.
    • 1.1.0-2 — Tidy package long description.
    • 1.2.1-1 — New upstream release.

I also reviewed and sponsored the uploads of gtts-token 1.1.1-1 and nlopt 2.4.2+dfsg-3.


Debian bugs filed

  • ITP: python-daiquiri — Python library to easily setup basic logging functionality. (#867322)
  • twittering-mode: Correct incorrect time formatting due to "84600" typo. (#868479)

CryptogramRobot Safecracking

Robots can crack safes faster than humans -- and differently:

So Seidle started looking for shortcuts. First he found that, like many safes, his SentrySafe had some tolerance for error. If the combination includes a 12, for instance, 11 or 13 would work, too. That simple convenience measure meant his bot could try every third number instead of every single number, immediately paring down the total test time to just over four days. Seidle also realized that the bot didn't actually need to return the dial to its original position before trying every combination. By making attempts in a certain careful order, it could keep two of the three rotors in place, while trying new numbers on just the last, vastly cutting the time to try new combinations to a maximum of four seconds per try. That reduced the maximum bruteforcing time to about one day and 16 hours, or under a day on average.

But Seidle found one more clever trick, this time taking advantage of a design quirk in the safe intended to prevent traditional safecracking. Because the safe has a rod that slips into slots in the three rotors when they're aligned to the combination's numbers, a human safecracker can apply light pressure to the safe's handle, turn its dial, and listen or feel for the moment when that rod slips into those slots. To block that technique, the third rotor of Seidle's SentrySafe is indented with twelve notches that catch the rod if someone turns the dial while pulling the handle.

Seidle took apart the safe he and his wife had owned for years, and measured those twelve notches. To his surprise, he discovered the one that contained the slot for the correct combination was about a hundredth of an inch narrower than the other eleven. That's not a difference any human can feel or listen for, but his robot can easily detect it with a few automated measurements that take seconds. That discovery defeated an entire rotor's worth of combinations, dividing the possible solutions by a factor of 33, and reducing the total cracking time to the robot's current hour-and-13 minute max.

We're going to have to start thinking about robot adversaries as we design our security systems.

Planet DebianRussell Coker: Running a Tor Relay

I previously wrote about running my SE Linux Play Machine over Tor [1] which involved configuring ssh to use Tor.

Since then I have installed a Tor hidden service for ssh on many systems I run for clients. The reason is that it is fairly common for them to allow a server to get a new IP address by DHCP or accidentally set their firewall to deny inbound connections. Without some sort of VPN this results in difficult phone calls talking non-technical people through the process of setting up a tunnel or discovering an IP address. While I can run my own VPN for them I don’t want their infrastructure tied to mine and they don’t want to pay for a 3rd party VPN service. Tor provides a free VPN service and works really well for this purpose.

As I believe in giving back to the community I decided to run my own Tor relay. I have no plans to ever run a Tor Exit Node because that involves more legal problems than I am willing or able to deal with. A good overview of how Tor works is the EFF page about it [2]. The main point of a “Middle Relay” (or just “Relay”) is that it only sends and receives encrypted data from other systems. As the Relay software (and the sysadmin if they choose to examine traffic) only sees encrypted data without any knowledge of the source or final destination the legal risk is negligible.

Running a Tor relay is quite easy to do. The Tor project has a document on running relays [3], which basically involves changing 4 lines in the torrc file and restarting Tor.

If you are running on Debian you should install the package tor-geoipdb to allow Tor to determine where connections come from (and to not whinge in the log files).

ORPort [IPV6ADDR]:9001

If you want to use IPv6 then you need a line like the above with IPV6ADDR replaced by the address you want to use. Currently Tor only supports IPv6 for connections between Tor servers and only for the data transfer not the directory services.

Data Transfer

I currently have 2 systems running as Tor relays, both of them are well connected in a European DC and they are each transferring about 10GB of data per day which isn’t a lot by server standards. I don’t know if there is a sufficient number of relays around the world that the share of the load is small or if there is some geographic dispersion algorithm which determined that there are too many relays in operation in that region.

CryptogramMeasuring Vulnerability Rediscovery

New paper: "Taking Stock: Estimating Vulnerability Rediscovery," by Trey Herr, Bruce Schneier, and Christopher Morris:

Abstract: How often do multiple, independent, parties discover the same vulnerability? There are ample models of vulnerability discovery, but little academic work on this issue of rediscovery. The immature state of this research and subsequent debate is a problem for the policy community, where the government's decision to disclose a given vulnerability hinges in part on that vulnerability's likelihood of being discovered and used maliciously by another party. Research into the behavior of malicious software markets and the efficacy of bug bounty programs would similarly benefit from an accurate baseline estimate for how often vulnerabilities are discovered by multiple independent parties.

This paper presents a new dataset of more than 4,300 vulnerabilities, and estimates vulnerability rediscovery across different vendors and software types. It concludes that rediscovery happens more than twice as often as the 1-9% range previously reported. For our dataset, 15% to 20% of vulnerabilities are discovered independently at least twice within a year. For just Android, 13.9% of vulnerabilities are rediscovered within 60 days, rising to 20% within 90 days, and above 21% within 120 days. For the Chrome browser we found 12.57% rediscovery within 60 days; and the aggregate rate for our entire dataset generally rises over the eight-year span, topping out at 19.6% in 2016. We believe that the actual rate is even higher for certain types of software.

When combined with an estimate of the total count of vulnerabilities in use by the NSA, these rates suggest that rediscovery of vulnerabilities kept secret by the U.S. government may be the source of up to one-third of all zero-day vulnerabilities detected in use each year. These results indicate that the information security community needs to map the impact of rediscovery on the efficacy of bug bounty programs and policymakers should more rigorously evaluate the costs of non-disclosure of software vulnerabilities.

We wrote a blog post on the paper, and another when we issued a revised version.

Comments on the original paper by Dave Aitel. News articles.

Worse Than FailureRepresentative Line: Groovy Typing, Man

Groovy was one of those programming languages that spent about six months as the trendy language du jour, and I haven’t heard much about it since. If I were to learn it, I’d want to learn by example- going through real-world Groovy code and seeing how it works.

An anonymous submitter has provided one sample for me to learn from:

List<String> items = new ArrayList<String>(Arrays.asList(data.split(",")))
String itemOne = items[2].toString()

It reminds me of those Family Circus comics where little Billy would wander the entire city to get from the front yard to the back yard.

It does indeed. And certainly, the type conversions are definitely the long way around: String -> String[] -> List<String> -> String -> String. But more than anything else, it’s the second statement that really gets me.

String itemOne = items[2].toString()


[Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!

Planet DebianNorbert Preining: Calibre in Debian

Some news about Calibre in Debian: I have been added to the list of maintainers, thanks Martin, and the recent release of Calibre 3.4 into Debian/unstable brought some fixes concerning the desktop integration. Now I am working on Calibre 3.5.

Calibre 3.5 separates out one module, html5-parser, into a separate package which needs to be included into Debian first. I have prepared and uploaded a version, but NEW processing will keep this package from entering Debian for a while. Other things I am currently doing is going over the list of bugs and try to fix or close as many as possible.

Finally the endless Rar support story still continues. I still don’t have any response from the maintainer of unrar-nonfree in Debian, so I am contemplating to package my own version of libunrar. As I wrote in the previous post, Calibre now checks whether the Python module unrardll is available, and if, uses it to decode rar-packed ebooks. I have a package for unrardll ready to be uploaded, but it needs the shared library version, and thus I am stuck waiting for unrar-nonfree.

Anyway, to help all those wanting to play with the latest Calibre, my archive nowadays contains:

  • preliminary packages for Calibre 3.5
  • updated package for unrar-nonfree that ship the shared library
  • the new pacakge unrardll to be uploaded after unrar-nonfree is updated
  • the new package html5-parser which is in NEW

Together these packages provide the newest Calibre with Rar support.

deb http://www.preining.info/debian/ calibre main
deb-src http://www.preining.info/debian/ calibre main

The releases are signed with my Debian key 0x6CACA448860CDC13

Enjoy

,

Planet Linux AustraliaOpenSTEM: This Week in HASS – term 3, week 4

This week younger students start investigating how we can find out about the past. This investigation will be conducted over the next 3 weeks and will culminate in a Scientific Report. Older students are considering different sources of historical information and how they will use these sources in their research.

Foundation/Prep/Kindy to Year 3

Students in stand-alone Foundation/Prep/Kindy classes (Unit F.3), as well as those in integrated classes (Unit F-1.3) and Years 1 (Unit 1.3), 2 (Unit 2.3) and 3 (Unit 3.3) are all starting to think about how we can find out about the past. This is a great opportunity for teachers to encourage students to think about how we know about the past and brainstorm ideas, as well as coming up with their own avenues of inquiry. Teachers may wish to hold a Question and Answer session in class to help guide students to examine many different aspects of this topic. The resource Finding Out About The Past contains core information to help the teacher guide the discussion to cover different ways of examining the past. This discussion can be tailored to the level and individual circumstances of each class. Foundation/Prep/Kindy students are just starting to think about the past as a time before the present and how this affects what we know about past events. The discussion can be developed in higher years, and the teacher can start to introduce the notion of sources of information, including texts and material culture. This investigation forms the basis for the Method section of the Scientific Report, which is included in the Student Workbook.

Years 3 to 6

Students in Years 3 (Unit 3.7), 4 (Unit 4.3), 5 (Unit 5.3) and 6 (Unit 6.3) are following a similar line of investigation this week, but examining Historical Sources specifically. As well as Primary and Secondary Sources, students are encouraged to think about Oral Sources, Textual Sources and Material Culture (artefacts such as stone tools or historical items). This discussion forms the basis for students completing the Method section of their Scientific Report, where they will list the sources of information and how these contributed to their research. Older students might be able to self-direct this process, although teachers may wish to guide the process through an initial class discussion. Teachers may wish to take the class through a discussion of the sources they are using for their research and discuss how students will use and report on these sources in their report for their topic.

Planet DebianJose M. Calhariz: Crossgrading a complex Desktop and Debian Developer machine running Debian 9, for real.

After sometime without looking into this problem, I decided to do another try. I do not found a way to do a complex crossgrade of my desktop without massively removing packages. And there are bug and bug that require to edit the config scripts of the packages.

So here is my another try to do a crossgrade of my desktop, this time for real.

apt-get update
apt-get upgrade
apt-get autoremove
apt-get clean
dpkg --list > original.dpkg
apt-get --download-only install dpkg:amd64 tar:amd64 apt:amd64 bash:amd64 dash:amd64 init:amd64 mawk:amd64
for pack32 in $(grep i386 original.dpkg | egrep "^ii " | awk '{print $2}' ) ; do 
  echo $pack32 ; 
  apt-get --download-only install -y --allow-remove-essential ${pack32%:i386}:amd64 ; 
done
cd /var/cache/apt/archives/
dpkg --install libacl1_*amd64.deb libattr1_*_amd64.deb libapt-pkg5.0_*amd64.deb libbz2-1.0_*amd64.deb dpkg_*amd64.deb tar_*amd64.deb apt_*amd64.deb bash_*amd64.deb dash_*amd64.deb 
dpkg --install --skip-same-version *.deb
dpkg --configure --pending
dpkg --install --skip-same-version *.deb
dpkg --remove libcurl4-openssl-dev:i386
dpkg --configure --pending
dpkg --remove libkdesu5 kde-runtime
apt-get --fix-broken install
apt-get install  $(egrep "^ii"  ~/original.dpkg | grep -v ":i386" | grep -v "all" | grep -v "aiccu" | grep -v "acroread" | grep -v "flashplayer-mozilla" | grep -v "flash-player-properties" | awk '{print $2}')

Reboot.

Then the system failed to boot, missing lvm2 package.

Boot with a live CD.

sudo -i
mount /dev/sdc2         /mnt
mount /dev/vg100/usr /mnt/usr
mount /dev/vg100/var /mnt/var
mount -o bind /proc    /mnt/proc
mount -o bind /sys     /mnt/sys
mount -o bind /dev/    /mnt/dev
mount -o bind /dev/pts  /mnt/dev/pts
chroot /mnt /bin/su -
apt-get install lvm2
exit
reboot

Still somethings do not work, like command fakeroot.

for pack32 in $(grep i386 original.dpkg  | egrep "^ii " | awk '{print $2}' ) ; do     
  echo $pack32 ;     
  if dpkg --status $pack32 | grep -q "Multi-Arch: same" ; then       
    apt-get -y install ${pack32%:i386}:amd64 ;     
  fi ;   
done

for pack32 in $(grep i386 original.dpkg  | egrep "^ii " | awk '{print $2}' ) ; do     
  echo $pack32 ;     
  apt-get -y install ${pack32%:i386}:amd64 ;     
done

Now is time to find what still does not work and how to solve it.

Planet DebianNiels Thykier: Introducing the debhelper buildlabel prototype for multi-building packages

For most packages, the “dh” short-hand rules (possibly with a few overrides) work great.  It can often auto-detect the buildsystem and handle all the trivial parts.

With one notably exception: What if you need to compile the upstream code twice (or more) with different flags?  This is the case for all source packages building both regular debs and udebs.

In that case, you would previously need to override about 5-6 helpers for this to work at all.  The five dh_auto_* helpers and usually also dh_install (to call it with different –sourcedir for different packages).  This gets even more complex if you want to support Build-Profiles such as “noudeb” and “nodoc”.

The best way to support “nodoc” in debhelper is to move documentation out of dh_install’s config files and use dh_installman, dh_installdocs, and dh_installexamples instead (NB: wait for compat 11 before doing this).  This in turn will mean more overrides with –sourcedir and -p/-N.

And then there is “noudeb”, which currently requires manual handling in debian/rules.  Basically, you need to use make or shell if-statements to conditionally skip the udeb part of the builds.

All of this is needlessly complex.

Improving the situation

In an attempt to make things better, I have made a new prototype feature in debhelper called “buildlabels” in experimental.  The current prototype is designed to deal with part (but not all) of the above problems:

  • It will remove the need for littering your rules file for supporting “noudeb” (and in some cases also other “noX” profiles).
  • It will remove the need for overriding the dh_install* tools just to juggle with –sourcedir and -p/-N.

However, it currently not solve the need for overriding the dh_auto_* tools and I am not sure when/if it will.

The feature relies on being able to relate packages to a given series of calls to dh_auto_*.  In the following example, I will use udebs for the secondary build.  However, this feature is not tied to udebs in any way and can be used any source package that needs to do two or more upstream builds for different packages.

Assume our example source builds the following binary packages:

  • foo
  • libfoo1
  • libfoo-dev
  • foo-udeb
  • libfoo1-udeb

And in the rules file, we would have something like:

[...]

override_dh_auto_configure:
    dh_auto_configure -B build-deb -- --with-feature1 --with-feature2
    dh_auto_configure -B build-udeb -- --without-feature1 --without-feature2

[...]

What is somewhat obvious to a human is that the first configure line is related to the regular debs and the second configure line is for the udebs.  However, debhelper does not know how to infer this and this is where buildlabels come in.  With buildlabels, you can let debhelper know which packages and builds that belong together.

How to use buildlabels

To use buildlabels, you have to do three things:

  1. Pick a reasonable label name for the secondary build.  In the example, I will use “udeb”.
  2. Add “–buildlabel=$LABEL” to all dh_auto_* calls related to your secondary build.
  3. Tag all packages related to “my-label” with “X-DH-Buildlabel: $LABEL” in debian/control.  (For udeb packages, you may want to add “Build-Profiles: <!noudeb>” while you are at it).

For the example package, we would change the debian/rules snippet to:

[...]

override_dh_auto_configure:
    dh_auto_configure -B build-deb -- --with-feature1 --with-feature2
    dh_auto_configure --buildlabel=udeb -B build-udeb -- --without-feature1 --without-feature2

[...]

(Remember to update *all* calls to dh_auto_* helpers; the above only lists dh_auto_configure to keep the example short.)  And then add “X-DH-Buildlabel: udeb” in the stanzas for foo-udeb + libfoo1-udeb.

With those two minor changes:

  • debhelper will skip the calls to dh_auto_* with –buildlabel=udeb if the udeb packages are skipped.
  • dh_auto_install will automatically pick a separate destination directory by default for the udeb build (assuming you do not explicitly override it with –destdir).
  • dh_install will now automatically pick up files from the destination directory.that dh_auto_install used for the given package (even if you overwrote it with –destdir).  Note that you have to remove any use of “–sourcedir” first as this disables the auto-detection.  This also works for other dh_install* tools supporting –sourcedir in compat 11 or later.

Real example

Thanks to Michael Biebl, I was able to make an branch in the systemd git repository to play with this feature.  Therefore I have an real example to use as a show case.  The gist of it is in the following three commits:

Full branch can be seen at: https://anonscm.debian.org/git/pkg-systemd/systemd.git/log/?h=wip-dh-prototype-smarter-multi-builds

Request for comments / call for testing

This prototype is now in experimental (debhelper/10.7+exp.buildlabels) and you are very welcome to take it for a spin.  Please let me know if you find the idea useful and feel free to file bugs or feature requests.  If deemed useful, I will merge into master and include in a future release.

If you have any questions or comments about the feature or need help with trying it out, you are also very welcome to mail the debhelper-devel mailing list.

Known issues / the fine print:

  • It is experimental feature and may change without notice.
  • The current prototype may break existing packages as it is not guarded by a compat bump to ease your testing.  I am still very curious to hear about any issues you may experience.
  • The default build directory is re-used even with different buildlabels, so you still need to use explicit build dirs for buildsystems that prefer building in a separate directory (e.g. meson).
  • udebs are not automatically tagged with an “udeb” buildlabel.  This is partly by design as some source packages only build udebs (and no regular debs).  If they were automatically tagged, the existing packages would break.
  • Label names are used in path names, so you may want to refrain from using “too exciting” label names.
  • It is experimental feature and may change without notice. (Yes, I thought it deserved repeating)

Filed under: Debhelper, Debian

,

Planet Linux AustraliaDavid Rowe: QSO Today Podcast

Eric, 4Z1UG, has kindly interviewed me for his fine QSO Today Podcast.

Planet DebianAntoine Beaupré: My free software activities, July 2017

Debian Long Term Support (LTS)

This is my monthly working on Debian LTS. This time I worked on various hairy issues surrounding ca-certificates, unattended-upgrades, apache2 regressions, libmtp, tcpdump and ipsec-tools.

ca-certificates updates

I've been working on the removal of the Wosign and StartCom certificates (Debian bug #858539) and, in general, the synchronisation of ca-certificates across suites (Debian bug #867461) since at least last march. I have made an attempt at summarizing the issue which led to a productive discussion and it seems that, in the end, the maintainer will take care of synchronizing information across suites.

Guido was right in again raising the question of synchronizing NSS across all suites (Debian bug #824872) which itself raised the other question of how to test reverse dependencies. This brings me back to Debian bug #817286 which, basically proposed the idea of having "proposed updates" for security issues. The problem is while we can upload test packages to stable proposed-updates, we can't do the same in LTS because the suite is closed and we operate only on security packages. This issue came up before in other security upload and we need to think better about how to solve this.

unattended-upgrades

Speaking of security upgrades brings me to the question of a bug (Debian bug #867169) that was filed against the wheezy version of unattended-upgrades, which showed that the package simply stopped working since the latest stable release, because wheezy became "oldoldstable". I first suggested using the "codename" but that appears to have been introduced only after wheezy.

In the end, I proposed a simple update that would fix the configuration files and uploaded this as DLA-1032-1. This is thankfully fixed in later releases and will not require such hackery when jessie becomes LTS as well.

libmtp

Next up is the work on the libmtp vulnerabilities (CVE-2017-9831 and CVE-2017-9832). As I described in my announcement, the work to backport the patch was huge, as upstream basically backported a whole library from the gphoto2 package to fix those issues (and probably many more). The lack of a test suite made it difficult to trust my own work, but given that I had no (negative) feedback, I figured it was okay to simply upload the result and that became DLA-1029-1.

tcpdump

I then looked at reproducing CVE-2017-11108, a heap overflow triggered tcpdump would parse specifically STP packets. In Debian bug #867718, I described how to reproduce the issue across all suites and opened an issue upstream, given that the upstream maintainers hadn't responded responded in weeks according to notes in the RedHat Bugzilla issue. I eventually worked on a patch which I shared upstream, but that was rejected as they were already working on it in their embargoed repository.

I can explain this confusion and duplication of work with:

  1. the original submitter didn't really contact security@tcpdump.org
  2. he did and they didn't reply, being just too busy
  3. they replied and he didn't relay that information back

I think #2 is most likely: the tcpdump.org folks are probably very busy with tons of reports like this. Still, I should probably have contacted security@tcpdump.org directly before starting my work, even though no harm was done because I didn't divulge issues that were already public.

Since then, tcpdump has released 4.9.1 which fixes the issue, but then new CVEs came out that will require more work and probably another release. People looking into this issue must be certain to coordinate with the tcpdump security team before fixing the actual issues.

ipsec-tools

Another package that didn't quite have a working solution is the ipsec-tools suite, in which the racoon daemon was vulnerable to a remotely-triggered DOS attack (CVE-2016-10396). I reviewed and fixed the upstream patch which introduced a regression. Unfortunately, there is no test suite or proof of concept to control the results.

The reality is that ipsec-tools is really old, and should maybe simply be removed from Debian, in favor of strongswan. Upstream hasn't done a release in years and various distributions have patched up forks of those to keep it alive... I was happy, however, to know that a maintainer will take care of updating the various suites, including LTS, with my improved patch. So this fixes the issue for now, but I would strongly encourage users to switch away from ipsec-tools in the future.

apache2

Finally, I was bitten by the old DLA-841-1 upload I did all the way back in February, as it introduced a regression (Debian bug #858373). It turns out it was possible to segfault Apache workers with a trivial HTTP request, in certain (rather exotic, I might add) configurations (ErrorDocument 400 directive pointing to a cgid script in worker mode).

Still, it was a serious regression and I found a part of the nasty long patch we worked on back then that was faulty, and introduced a small fix to correct that. The proposed package unfortunately didn't yield any feedback, and I can only assume it will work okay for people. The result is the DLA-841-2 upload which fixes the regression. I unfortunately didn't have time to work on the remaining CVEs affecting apache2 in LTS at the time of writing.

Triage

I also did some miscellaneous triage by filing Debian bug #867477 for poppler in an effort to document better the pending issue.

Next up was some minor work on eglibc issues. CVE-2017-8804 has a patch, but it's been disputed. since the main victim of this and the core of the vulnerability (rpcbind) has already been fixed, I am not sure this vulnerability is still a thing in LTS at all.

I also looked at CVE-2014-9984, but the code is so different in wheezy that I wonder if LTS is affected at all. Unfortunately, the eglibc gymnastics are a little beyond me and I do not feel confident enough to just push those issues aside for now and let them open for others to look at.

Other free software work

And of course, there's my usual monthly volunteer work. My ratio is a little better this time, having reached an about even ratio between paid and volunteer work, whereas this was 60% volunteer work in march.

Announcing ecdysis

I recently published ecdysis, a set of template and code samples that I frequently reuse across project. This is probably the least pronounceable project name I have ever chosen, but this is somewhat on purpose. The goal of this project is not collaboration or to become a library: it's just a personal project which I share with the world as a curiosity.

To quote the README file:

The name comes from what snakes and other animals do to "create a new snake": they shed their skin. This is not so appropriate for snakes, as it's just a way to rejuvenate their skin, but is especially relevant for anthropods since then "ecdysis" may be associated with a metamorphosis:

Ecdysis is the moulting of the cuticle in many invertebrates of the clade Ecdysozoa. Since the cuticle of these animals typically forms a largely inelastic exoskeleton, it is shed during growth and a new, larger covering is formed. The remnants of the old, empty exoskeleton are called exuviae. — Wikipedia

So this project is metamorphosed into others when the documentation templates, code examples and so on are reused elsewhere. For that reason, the license is an unusally liberal (for me) MIT/Expat license.

The name also has the nice property of being absolutely unpronounceable, which makes it unlikely to be copied but easy to search online.

It was an interesting exercise to go back into older projects and factor out interesting code. The process is not complete yet, as there are older projects I'm still curious in reviewing. A bunch of that code could also be factored into upstream project and maybe even the Python standard library.

In short, this is stuff I keep on forgetting how to do: a proper setup.py config, some fancy argparse extensions and so on. Instead of having to remember where I had written that clever piece of code, I now shove it in the crazy chaotic project where I can find it again in the future.

Beets experiments

Since I started using Subsonic (or Libresonic) to manage the music on my phone, album covers are suddenly way more interesting. But my collection so far has had limited album covers: my other media player (gmpc) would download those on the fly on its own and store them in its own database - not on the filesystem. I guess this could be considered to be a limitation of Subsonic, but I actually appreciate the separation of duty here. Garbage in, garbage out: the quality of Subsonic's rendering depends largely on how well setup your library and tags are.

It turns out there is an amazing tool called beets to do exactly that kind of stuff. I originally discarded that "media library management system for obsessive-compulsive [OC] music geeks", trying to convince myself i was not an "OC music geek". Turns out I am. Oh well.

Thanks to beets, I was able to download album covers for a lot of the albums in my collection. The only covers that are missing now are albums that are not correctly tagged and that beets couldn't automatically fix up. I still need to go through those and fix all those tags, but the first run did an impressive job at getting album covers.

Then I got the next crazy idea: after a camping trip where we forgot (again) the lyrics to Georges Brassens, I figured I could start putting some lyrics on my ebook reader. "How hard can that be?" of course, being the start of another crazy project. A pull request and 3 days later, I had something that could turn a beets lyrics database into a Sphinx document which, in turn, can be turned into an ePUB. In the process, I probably got blocked from MusixMatch a hundred times, but it's done. Phew!

The resulting e-book is about 8000 pages long, but is still surprisingly responsive. In the process, I also happened to do a partial benchmark of Python's bloom filter libraries. The biggest surprise there was the performance of the set builtin: for small items, it is basically as fast as a bloom filter. Of course, when the item size grows larger, its memory usage explodes, but in this case it turned out to be sufficient and bloom filter completely overkill and confusing.

Oh, and thanks to those efforts, I got admitted in the beetbox organization on GitHub! I am not sure what I will do with that newfound power: I was just scratching an itch, really. But hopefully I'll be able to help here and there in the future as well.

Debian package maintenance

I did some normal upkeep on a bunch of my packages this month, that were long overdue:

  • uploaded slop 6.3.47-1: major new upstream release
  • uploaded an NMU for maim 5.4.64-1.1: maim was broken by the slop release
  • uploaded pv 1.6.6-1: new upstream release
  • uploaded kedpm 1.0+deb8u1 to jessie (oldstable): one last security fix (Debian bug #860817, CVE-2017-8296) for that derelict password manager
  • uploaded charybdis 3.5.5-1: new minor upstream release, with optional support for mbedtls
  • filed Debian bug #866786 against cryptsetup to make the remote initramfs SSH-based unlocking support multiple devices: thanks to the maintainer, this now works flawlessly in buster and may be backported to stretch
  • expanded on Debian bug #805414 against gdm3 and Debian bug #845938 against pulseaudio, because I had trouble connecting my computer to this new Bluetooth speaker. turns out this is a known issue in Pulseaudio: whereas it releases ALSA devices, it doesn't release Bluetooth devices properly. Documented this more clearly in the wiki page
  • filed Debian bug #866790 regarding old stray Apparmor profiles that were lying around my system after an upgrade, which got me interested in Debian bug #830502 in turn
  • filed Debian bug #868728 against cups regarding a weird behavior I had interacting with a network printer. turns out the other workstation was misconfigured... why are printers still so hard?
  • filed Debian bug #870102 to automate sbuild schroots upgrades
  • after playing around with rash tried to complete the packaging (Debian bug #754972) of percol with this pull request upstream. this ended up to be way too much overhead and I reverted to my old normal history habits.

Planet DebianDirk Eddelbuettel: Updated overbought/oversold plot function

A good six years ago I blogged about plotOBOS() which charts a moving average (from one of several available variants) along with shaded standard deviation bands. That post has a bit more background on the why/how and motivation, but as a teaser here is the resulting chart of the SP500 index (with ticker ^GSCP):

Example chart of overbought/oversold levels from plotOBOS() function 

The code uses a few standard finance packages for R (with most of them maintained by Joshua Ulrich given that Jeff Ryan, who co-wrote chunks of these, is effectively retired from public life). Among these, xts had a recent release reflecting changes which occurred during the four (!!) years since the previous release, and covering at least two GSoC projects. With that came subtle API changes: something we all generally try to avoid but which is at times the only way forward. In this case, the shading code I used (via polygon() from base R) no longer cooperated with the beefed-up functionality of plot.xts(). Luckily, Ross Bennett incorporated that same functionality into a new function addPolygon --- which even credits this same post of mine.

With that, the updated code becomes

## plotOBOS -- displaying overbough/oversold as eg in Bespoke's plots
##
## Copyright (C) 2010 - 2017  Dirk Eddelbuettel
##
## This is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 2 of the License, or
## (at your option) any later version.

suppressMessages(library(quantmod))     # for getSymbols(), brings in xts too
suppressMessages(library(TTR))          # for various moving averages

plotOBOS <- function(symbol, n=50, type=c("sma", "ema", "zlema"),
                     years=1, blue=TRUE, current=TRUE, title=symbol,
                     ticks=TRUE, axes=TRUE) {

    today <- Sys.Date()
    if (class(symbol) == "character") {
        X <- getSymbols(symbol, from=format(today-365*years-2*n), auto.assign=FALSE)
        x <- X[,6]                          # use Adjusted
    } else if (inherits(symbol, "zoo")) {
        x <- X <- as.xts(symbol)
        current <- FALSE                # don't expand the supplied data
    }

    n <- min(nrow(x)/3, 50)             # as we may not have 50 days

    sub <- ""
    if (current) {
        xx <- getQuote(symbol)
        xt <- xts(xx$Last, order.by=as.Date(xx$`Trade Time`))
        colnames(xt) <- paste(symbol, "Adjusted", sep=".")
        x <- rbind(x, xt)
        sub <- paste("Last price: ", xx$Last, " at ",
                     format(as.POSIXct(xx$`Trade Time`), "%H:%M"), sep="")
    }

    type <- match.arg(type)
    xd <- switch(type,                  # compute xd as the central location via selected MA smoother
                 sma = SMA(x,n),
                 ema = EMA(x,n),
                 zlema = ZLEMA(x,n))
    xv <- runSD(x, n)                   # compute xv as the rolling volatility

    strt <- paste(format(today-365*years), "::", sep="")
    x  <- x[strt]                       # subset plotting range using xts' nice functionality
    xd <- xd[strt]
    xv <- xv[strt]

    xyd <- xy.coords(.index(xd),xd[,1]) # xy coordinates for direct plot commands
    xyv <- xy.coords(.index(xv),xv[,1])

    n <- length(xyd$x)
    xx <- xyd$x[c(1,1:n,n:1)]           # for polygon(): from first point to last and back

    if (blue) {
        blues5 <- c("#EFF3FF", "#BDD7E7", "#6BAED6", "#3182BD", "#08519C") # cf brewer.pal(5, "Blues")
        fairlylight <<- rgb(189/255, 215/255, 231/255, alpha=0.625) # aka blues5[2]
        verylight <<- rgb(239/255, 243/255, 255/255, alpha=0.625) # aka blues5[1]
        dark <<- rgb(8/255, 81/255, 156/255, alpha=0.625) # aka blues5[5]
        ## buglet in xts 0.10-0 requires the <<- here
    } else {
        fairlylight <<- rgb(204/255, 204/255, 204/255, alpha=0.5)  # two suitable grays, alpha-blending at 50%
        verylight <<- rgb(242/255, 242/255, 242/255, alpha=0.5)
        dark <<- 'black'
    }

    plot(x, ylim=range(range(x, xd+2*xv, xd-2*xv, na.rm=TRUE)), main=title, sub=sub, 
         major.ticks=ticks, minor.ticks=ticks, axes=axes) # basic xts plot setup
    addPolygon(xts(cbind(xyd$y+xyv$y, xyd$y+2*xyv$y), order.by=index(x)), on=1, col=fairlylight)  # upper
    addPolygon(xts(cbind(xyd$y-xyv$y, xyd$y+1*xyv$y), order.by=index(x)), on=1, col=verylight)    # center
    addPolygon(xts(cbind(xyd$y-xyv$y, xyd$y-2*xyv$y), order.by=index(x)), on=1, col=fairlylight)  # lower
    lines(xd, lwd=2, col=fairlylight)   # central smooted location
    lines(x, lwd=3, col=dark)           # actual price, thicker
}

and the main change are the three calls to addPolygon. To illustrate, we call plotOBOS("SPY", years=2) with an updated plot of the ETF representing the SP500 over the last two years:

Updated example chart of overbought/oversold levels from plotOBOS() function 

Comments and further enhancements welcome!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Planet DebianRobert McQueen: Welcome, Flathub!

Alex Larsson talks about Flathub at GUADEC 2017At the Gtk+ hackfest in London earlier this year, we stole an afternoon from the toolkit folks (sorry!) to talk about Flatpak, and how we could establish a “critical mass” behind the Flatpak format. Bringing Linux container and sandboxing technology together with ostree, we’ve got a technology which solves real world distribution, technical and security problems which have arguably held back the Linux desktop space and frustrated ISVs and app developers for nearly 20 years. The problem we need to solve, like any ecosystem, is one of users and developers – without stuff you can easily get in Flatpak format, there won’t be many users, and without many users, we won’t have a strong or compelling incentive for developers to take their precious time to understand a new format and a new technology.

As Alex Larsson said in his GUADEC talk yesterday: Decentralisation is good. Flatpak is a tool that is totally agnostic of who is publishing the software and where it comes from. For software freedom, that’s an important thing because we want technology to empower users, rather than tell them what they can or can’t do. Unfortunately, decentralisation makes for a terrible user experience. At present, the Flatpak webpage has a manually curated list of links to 10s of places where you can find different Flatpaks and add them to your system. You can’t easily search and browse to find apps to try out – so it’s clear that if the current situation remains we’re not going to be able to get a critical mass of users and developers around Flatpak.

Enter Flathub. The idea is that by creating an obvious “center of gravity” for the Flatpak community to contribute and build their apps, users will have one place to go and find the best that the Linux app ecosystem has to offer. We can take care of the boring stuff like running a build service and empower Linux application developers to choose how and when their app gets out to their users. After the London hackfest we sketched out a minimum viable system – Github, Buildbot and a few workers – and got it going over the past few months, culminating in a mini-fundraiser to pay for the hosting of a production-ready setup. Thanks to the 20 individuals who supported our fundraiser, to Mythic Beasts who provided a server along with management, monitoring and heaps of bandwidth, and to Codethink and Scaleway who provide our ARM and Intel workers respectively.

We inherit our core principles from the Flatpak project – we want the Flatpak technology to succeed at alleviating the issues faced by app developers in targeting a diverse set of Linux platforms. None of this stops you from building and hosting your own Flatpak repos and we look forward to this being a wide and open playing field. We care about the success of the Linux desktop as a platform, so we are open to proprietary applications through Flatpak’s “extra data” feature where the client machine downloads 3rd party binaries. They are correctly labeled as such in the AppStream, so will only be shown if you or your OS has configured GNOME Software to show you apps with proprietary licenses, respecting the user’s preference.

The new infrastructure is up and running and I put it into production on Thursday. We rebuilt the whole repository on the new system over the course of the week, signing everything with our new 4096-bit key stored on a Yubikey smartcard USB key. We have 66 apps at the moment, although Alex is working on bringing in the GNOME apps at present – we hope those will be joined soon by the KDE apps, and Endless is planning to move over as many of our 3rd party Flatpaks as possible over the coming months.

So, thanks again to Alex and the whole Flatpak community, and the individuals and the companies who supported making this a reality. You can add the repository and get downloading right away. Welcome to Flathub! Go forth and flatten… 🙂

Flathub logo

Cory DoctorowA Hopeful Look At The Apocalypse: interview with Innovation Hub

I’m on the latest episode of Innovation Hub (MP3):

Science-fiction is a genre that imagines the future. It doesn’t necessarily predict the future (after all, where are flying cars?), but it grapples with the technological and societal changes happening today to better understand our world and where it’s heading.

So, what does it mean when so much of our most popular science-fiction – The Handmaid’s Tale, The Walking Dead, and The Hunger Games – present bleak, depressing futures? Cory Doctorow might just have an answer. He’s a blogger, writer, activist, and author of the new book Walkaway, an optimistic disaster novel.

Three Takeaways

* Doctorow thinks that science-fiction can give people “ideas for what to do if the future turns out in different ways.” Like how William Gibson’s Neuromancer didn’t just predict the internet, it predicted the intermingling of corporations and the state.

* When you have story after story about how people turn on each other after disaster, Doctorow believes it gives us the largely false impression that people act like jerks in crises. When in fact, people usually rise to the occasion.

* With Walkaway, his “optimistic” disaster novel, Doctorow wanted to present a new narrative about resolving differences between people who are mostly on the same side.

CryptogramRoombas will Spy on You

The company that sells the Roomba autonomous vacuum wants to sell the data about your home that it collects.

Some questions:

What happens if a Roomba user consents to the data collection and later sells his or her home -- especially furnished -- and now the buyers of the data have a map of a home that belongs to someone who didn't consent, Mr. Gidari asked. How long is the data kept? If the house burns down, can the insurance company obtain the data and use it to identify possible causes? Can the police use it after a robbery?

EDITED TO ADD (6/29): Roomba is backtracking -- for now.

Planet DebianRobert McQueen: Hello again!

Like all good blog posts, this one starts with an apology about not blogging for ages – in my case it looks like it’s been about 7 years which is definitely a new personal best (perhaps the equally or more remarkable thing is that I have diligently kept WordPress running in the meantime). In that time, as you might expect, a few things have happened, like I met a wonderful woman and fell in love and we have two wonderful children. I also decided to part ways with my “first baby” and leave my role as CTO & co-founder of Collabora. This was obviously a very tough decision – it’s a fantastic team where I met and made many life-long friends, and they are still going strong and doing awesome things with Open Source. However, shortly after that, in February last year, I was lucky enough to land what is basically a dream job working at Endless Computers as the VP of Deployment.

As I’m sure most readers know, Endless is building an OS to bring personal computing to millions of new users across the world. It’s based on Free Software like GNOME, Debian, ostree and Flatpak, and the more successful Endless is, the more people who get access to education, technology and opportunity – and the more FOSS users and developers there will be in the world. But in my role, I get to help define the product, understand our users and how technology might help them, take Open Source out to new people, solve commercial problems to get Endless OS out into the world, manage a fantastic team and work with bright people, learn from great managers and mentors, and still find time to squash bugs, review and write more code than I used to. Like any startup, we have a lot to do and not enough time to do it, so although there aren’t quite enough days in the week, I’m really happy!

In any case, the main point of this blog post is that I’m at GUADEC in Manchester right now, and I’d like to blog about Flathub, but I thought it would be weird to just show up and say that after 7 years of silence without saying hello again. 🙂

Planet DebianRussell Coker: Apache Mesos on Debian

I decided to try packaging Mesos for Debian/Stretch. I had a spare system with a i7-930 CPU, 48G of RAM, and SSDs to use for building. The i7-930 isn’t really fast by today’s standards, but 48G of RAM and SSD storage mean that overall it’s a decent build system – faster than most systems I run (for myself and for clients) and probably faster than most systems used by Debian Developers for build purposes.

There’s a github issue about the lack of an upstream package for Debian/Stretch [1]. That upstream issue could probably be worked around by adding Jessie sources to the APT sources.list file, but a package for Stretch is what is needed anyway.

Here is the documentation on building for Debian [2]. The list of packages it gives as build dependencies is incomplete, it also needs zlib1g-dev libapr1-dev libcurl4-nss-dev openjdk-8-jdk maven libsasl2-dev libsvn-dev. So BUILDING this software requires Java + Maven, Ruby, and Python along with autoconf, libtool, and all the usual Unix build tools. It also requires the FPM (Fucking Package Management) tool, I take the choice of name as an indication of the professionalism of the author.

Building the software on my i7 system took 79 minutes which includes 76 minutes of CPU time (I didn’t use the -j option to make). At the end of the build it turned out that I had mistakenly failed to install the Fucking Package Management “gem” and it aborted. At this stage I gave up on Mesos, the pain involved exceeds my interest in trying it out.

How to do it Better

One of the aims of Free Software is that bugs are more likely to get solved if many people look at them. There aren’t many people who will devote 76 minutes of CPU time on a moderately fast system to investigate a single bug. To deal with this software should be prepared as components. An example of this is the SE Linux project which has 13 source modules in the latest release [3]. Of those 13 only 5 are really required. So anyone who wants to start on SE Linux from source (without considering a distribution like Debian or Fedora that has it packaged) can build the 5 most important ones. Also anyone who has an issue with SE Linux on their system can find the one source package that is relevant and study it with a short compile time. As an aside I’ve been working on SE Linux since long before it was split into so many separate source packages and know the code well, but I still find the separation convenient – I rarely need to work on more than a small subset of the code at one time.

The requirement of Java, Ruby, and Python to build Mesos could be partly due to language interfaces to call Mesos interfaces from Ruby and Python. Ohe solution to that is to have the C libraries and header files to call Mesos and have separate packages that depend on those libraries and headers to provide the bindings for other languages. Another solution is to have autoconf detect that some languages aren’t installed and just not try to compile bindings for them (this is one of the purposes of autoconf).

The use of a tool like Fucking Package Management means that you don’t get help from experts in the various distributions in making better packages. When there is a FOSS project with a debian subdirectory that makes barely functional packages then you will be likely to have an experienced Debian Developer offer a patch to improve it (I’ve offered patches for such things on many occasions). When there is a FOSS project that uses a tool that is never used by Debian developers (or developers of Fedora and other distributions) then the only patches you will get will be from inexperienced people.

A software build process should not download anything from the Internet. The source archive should contain everything that is needed and there should be dependencies for external software. Any downloads from the Internet need to be protected from MITM attacks which means that a responsible software developer has to read through the build system and make sure that appropriate PGP signature checks etc are performed. It could be that the files that the Mesos build downloaded from the Apache site had appropriate PGP checks performed – but it would take me extra time and effort to verify this and I can’t distribute software without being sure of this. Also reproducible builds are one of the latest things we aim for in the Debian project, this means we can’t just download files from web sites because the next build might get a different version.

Finally the fpm (Fucking Package Management) tool is a Ruby Gem that has to be installed with the “gem install” command. Any time you specify a gem install command you should include the -v option to ensure that everyone is using the same version of that gem, otherwise there is no guarantee that people who follow your documentation will get the same results. Also a quick Google search didn’t indicate whether gem install checks PGP keys or verifies data integrity in other ways. If I’m going to compile software for other people to use I’m concerned about getting unexpected results with such things. A Google search indicates that Ruby people were worried about such things in 2013 but doesn’t indicate whether they solved the problem properly.

Planet DebianChris Lamb: More Lintian hacking

Lintian is static analysis tool for Debian packages, reporting on various errors, omissions and quality-assurance issues to the maintainer.

I seem to have found myself hacking on it a bit more recently (see my previous installment). In particular, here's the code of mine — which made for a total of 20 bugs closed — that made it into the recent 2.5.52 release:

New tags

  • Check for the presence of an .asc signature in a .changes file if an upstream signing key is present. (#833585, tag)
  • Warn when dpkg-statoverride --add is called without a corresponding --list. (#652963, tag)
  • Check for years in debian/copyright that are later than the top entry in debian/changelog. (#807461, tag)
  • Trigger a warning when DEB_BUILD_OPTIONS is used instead of DEB_BUILD_MAINT_OPTIONS. (#833691, tag)
  • Look for "FIXME" and similar placeholders in various files in the debian directory. (#846009, tag)
  • Check for useless build-dependencies on dh-autoreconf or autotools-dev under Debhelper compatibility levels 10 or higher. (#844191, tag)
  • Emit a warning if GObject Introspection packages are missing dependencies on ${gir:Depends}. (#860801, tag)
  • Check packages do not contain upstart configuration under /etc/init. (#825348, tag)
  • Emit a classification tag if maintainer scripts such as debian/postinst is an ELF binary. (tag)
  • Check for overly-generic manual pages such as README.3pm.gz. (#792846, tag)
  • Ensure that (non-ELF) maintainer scripts begin with #!. (#843428, tag)

Regression fixes

  • Ensure r-data-without-readme-source checks the source package, not the binary; README.source files are not installed in the latter. (#866322, tag)
  • Don't emit source-contains-prebuilt-ms-help-file for files generated by Halibut. (#867673, tag)
  • Add .yml to the list of file extensions to avoid false positives when emitting extra-license-file. (#856137, tag)
  • Append a regression test for enumerated lists in the "a) b) c) …" style, which would previously trigger a "duplicate word" warning if the following paragraph began with an "a." (#844166, tag)

Documentation updates

  • Rename copyright-contains-dh-make-perl-boilerplate to copyright-contains-automatically-extracted-boilerplate as it can be generated by other tools such as dh-make-elpa. (#841832, tag)
  • Changes to new-package-should-not-package-python2-module (tag):
    • Upgrade from I: to W:. (#829744)
    • Clarify wording in description to make the justification clearer.
  • Clarify justification in debian-rules-parses-dpkg-parsechangelog. (#865882, tag)
  • Expand the rationale for the latest-debian-changelog-entry-without-new-date tag to mention possible implications for reproducible builds. (tag)
  • Update the source-contains-prebuilt-ms-help-file description; there exists free software to generate .chm files. (tag)
  • Append an example shell snippet to explain how to prevent init.d-script-sourcing-without-test. (tag)
  • Add a missing "contains" verb to the description of the debhelper-autoscript-in-maintainer-scripts tag. (tag)
  • Consistently use the same "Debian style" RFC822 date format for both "Mirror timestamp" and "Last updated" on the Lintian index page. (#828720)

Misc

  • Allow the use of suppress-tags=<tag>[,<tag>[,<tag>]] in ~/.lintianrc. (#764486)
  • Improve the support for "3.0 (git)" packages. However, they remain marked as unsupported-source-format as they are not accepted by the Debian archive. (#605999)
  • Apply patch from Dylan Aïssi to also check for .RData files (not just .Rdata) when checking for the copyright status of R Project data files. (#868178, tag)
  • Match more Lena Söderberg images. (#827941, tag)
  • Refactor a hard-coded list of possible upstream key locations to the common/signing-key-filenames Lintian::Data resource.

Don MartiExtracting just the audio from big video files

Got a big video, and want a copy of just the audio for listening on a device with limited storage? Use Soundconverter.

soundconverter -b -m mp3 -s .mp3 long-video.webm

(MP3 patents are expired now, hooray! I'm just using MP3 here because if I get a rental car that lets me plug in a USB stick for listening, the MP3 format is most likely to be supported.)

Soundconverter has a GUI but you can use -b for batch mode from the shell. soundconverter --help for help. You do need to set both the MIME type, with -m, and the file suffix, with -s.

Planet DebianNorbert Preining: Gaming: The Long Dark

I normally don’t play survival games or walking simulators, but The Long Dark by Hinterland Games, which I purchased back then when it was still in early access on Steam, took me into new realms. You are tossed out into the Canadian wilderness with hardly anything, and your only aim is to survive, find shelter, food, craft tools, hunt for food, explore. And while everything by now is Sandbox mode, on August 1st the first episode of Story mode is released. Best time to get the game!

You will be greeted with some icy nights, but also with great vistas, relaxed evenings at a fire place, you will try to survive on moldy food and rotten energy bars, but also feast on the fireside while reading a good book. A real treat this game!

Sandbox mode features five different areas to explore. Each one is large enough to spend weeks (in game time) wandering around. The easiest area to start with is Mystery Lake, with plenty of shelter (several huts) and abundance of resources. And just in case you are getting bored, all the areas are connected via tunnels or caves and one can wander of into the neighboring places. My home in Mystery Lake was always the Camp Office, the usual suspect. Nice views, fishing huts nearby to get fresh fish, lots of space.

After managing to get from your starting place (which is arbitrary as far as I see) to one of the shelters, one starts collecting food, wood, savaging every accessible place for tools, weapons, burning material. And soon the backpack becomes to heavy, and one needs to store stuff and decide what to take.

This is a very well done part of the game. The backpack is not limited by number of items, but you are limited in weight you can carry. That includes clothes (which can get quite heavy) and all the items in your backpack. In addition, the longer the day and the more tired you become, the less weight you can carry. And if the backpack starts getting too heavy you crawl to a very slow movement.

There are many influences of the outside world on the player’s condition: temperature, the wetness of your clothes, hunger, thirst, exhaustion, but also infections and bruises, all need to be taken care of, otherwise the end is coming faster than one wishes for.

I have only two things to complain: First, if one walks outside, or runs outside, the own body temperature does not rise. This is unrealistically and should have been taken into account. The other thing is the difficulty: I have played weeks in game time in the easiest level, without any problem. But the moment I switched to the second level of difficulty (of 5!), I not even manage it for 2(!) days. Wolves, starvation, thirst, any of those kills me within an instant. I don’t want to know how the hardest level feels, but it has a certain steep step here.

The game takes a very realistic view onto the weather: Every day is different, sunny, foggy, blizzard, windy, often changing very quickly. It is wise to plan one’s activities according to the weather, as it is very unforgiving.

With beautifully crafted landscapes, loads of areas to explore, your own pride to survive for at least a few weeks, and lots of tools to find and craft and try out, this games, even while it is still in Sandbox mode, is a real feat. My absolute favorite since I have finished the Talos Principle and Portal series, absolutely recommendable!

,

Krebs on SecuritySuspended Sentence for Mirai Botmaster Daniel Kaye

Last month, KrebsOnSecurity identified U.K. citizen Daniel Kaye as the likely real-life identity behind a hacker responsible for clumsily wielding a powerful botnet built on Mirai, a malware strain that enslaves poorly secured Internet of Things (IoT) devices for use in large-scale online attacks. Today, a German court issued a suspended sentence for Kaye, who now faces cybercrime charges in the United Kingdom.

Daniel Kaye's Facebook profile page.

Daniel Kaye’s Facebook profile page.

In February 2017, authorities in the United Kingdom arrested a 29-year-old U.K. man on suspicion of knocking more than 900,000 Germans offline in a Mirai attack in November 2016. Shortly after that 2016 attack, a hacker using the nickname “Bestbuy” told reporters he was responsible for the outage, apologizing for the incident.

Prosecutors in Europe had withheld Kaye’s name from the media throughout the trial. But a court in Germany today confirmed Kaye’s identity as it handed down a suspended sentence on charges stemming from several failed attacks from his Mirai botnet — which nevertheless caused extensive internet outages for ISPs in the U.K., Germany and Liberia last year.

On July 5, KrebsOnSecurity published Who is the GovRAT Author and Mirai Botmaster BestBuy. The story followed clues from reports produced by a half-dozen security firms that traced common clues between this BestBuy nickname and an alter-ego, “Spiderman.”

Both identities were connected to the sale of an espionage tool called GovRAT, which is documented to have been used in numerous cyber espionage campaigns against governments, financial institutions, defense contractors and more than 100 corporations.

That July 5 story traced a trail of digital clues left over 10 years back to Daniel Kaye, a 29-year-old man who had dual U.K. and Israeli citizenship and who was engaged to be married to a U.K. woman.

A “mind map” tracing some of the research mentioned in this post.

Last week, a 29-year-old identified by media only as “Daniel K” pleaded guilty in a German court for launching the attacks that knocked 900,000 Deutsche Telekom customers offline. Prosecutors said Daniel K sold access to his Mirai botnet as an attack-for-hire service.

The defendant reportedly told the court that the incident was the biggest mistake of his life, and that he took money in exchange for launching attacks in order to help start a new life with his fiancee.

Today, the regional court in the western city of Cologne said it would suspend the sentence of one year and eight months against Kaye, according to a report from Agence France Presse.

While it may seem that Kaye was given a pass by the German court, he is still facing criminal charges in Britain, where authorities have already requested his extradition.

As loyal readers here no doubt know, KrebsOnSecurity last year was massively attacked by the first-ever Mirai botnet — an attack which knocked this site offline for almost four days before it came back online under the protection of Google’s Project Shield service.

In January 2017, this blog published the results of a four-month investigation into who was likely responsible for not only for writing Mirai, but for leaking the source code for the malware — spawning dozens of competing Mirai botnets like the one that Kaye built. To my knowledge, no charges have yet been filed against any of the individuals named in that story.

Planet DebianSteve Kemp: So I'm considering a new project

In the past there used to be a puppet-labs project called puppet-dashboard, which would let you see the state of your managed-nodes. Having even a very basic and simple "report user-interface" is pretty neat when you're pushing out a change, and you want to see it be applied across your fleet of hosts.

There are some other neat features, such as allowing you to identify failures easily, and see nodes that haven't reported-in recently.

This was spun out into a community-supported project which is largely stale:

Having a dashboard is nice, but the current state of the software is less good. It turns out that the implementation is pretty simple though:

  • Puppet runs on a node.
  • The node reports back to the puppet-master what happened.
  • The puppet-master can optionally HTTP-post that report to the reporting node.

The reporting node can thus receive real-time updates, and do what it wants with them. You can even sidestep the extra server if you wish:

  • The puppet-master can archive the reports locally.

For example on my puppet-master I have this:

  root@master /var/lib/puppet/reports # ls | tail -n4
  smaug.dh.bytemark.co.uk
  ssh.steve.org.uk
  www.dns-api.com
  www.steve.org.uk

Inside each directory is a bunch of YAML files which describe the state of the host, and the recipes that were applied. Parsing those is pretty simple, the hardest part would be making a useful/attractive GUI. But happily we have the existing one to "inspire" us.

I think I just need to write down a list of assumptions and see if they make sense. After all the existing installation(s) won't break, it's just a matter of deciding whether it is useful/worthwhile way to spend some time.

  • Assume you have 100+ hosts running puppet 4.x
  • Assume you want a broad overview:
    • All the nodes you're managing.
    • Whether their last run triggered a change, resulted in an error, or logged anything.
    • If so what changed/failed/was output?
  • For each individual run you want to see:
    • Rough overview.
  • Assume you don't want to keep history indefinitely, just the last 50 runs or so of each host.

Beyond that you might want to export data about the managed-nodes themselves. For example you might want a list of all the hosts which have "bash" installed on them. Or "All nodes with local user "steve"." I've written that stuff already, as it is very useful for auditing & etc.

The hard part about that is that to get the extra data you'll need to include a puppet module to collect it. I suspect a new dashboard would be broadly interesting/useful but unless you have that extra detail it might not be so useful. You can't point to a slightly more modern installation and say "Yes this is worth migrating to". But if you have extra meta-data you can say:

  • Give me a list of all hosts running wheezy.
  • Give me a list of all hosts running exim4 version 4.84.2-2+deb8u4.

And that facility is very useful when you have shellshock, or similar knocking at your door.

Anyway as a hacky start I wrote some code to parse reports, avoiding the magic object-fu that the YAML would usually invoke. The end result is this:

 root@master ~# dump-run www.steve.org.uk
 www.steve.org.uk
    Puppet Version: 4.8.2
    /var/lib/puppet/reports/www.steve.org.uk/201707291813.yaml
    Runtime: 2.16
    Status:changed
    Time:2017-07-29 18:13:04 +0000
    Resources
            total -> 176
            skipped -> 2
            failed -> 0
            changed -> 3
            out_of_sync -> 3
            scheduled -> 0
            corrective_change -> 3
    Changed Resources
            Ssh_authorized_key[skx@shelob-s-fi] /etc/puppet/code/environments/production/modules/ssh_keys/manifests/init.pp:17
            Ssh_authorized_key[skx@deagol-s-fi] /etc/puppet/code/environments/production/modules/ssh_keys/manifests/init.pp:22
            Ssh_authorized_key[steve@ssh.steve.org.uk-s-fi] /etc/puppet/code/environments/production/modules/ssh_keys/manifests/init.pp:27
    Skipped Resources
            Exec[clone sysadmin utils]
            Exec[update sysadmin utils]

CryptogramMe on Restaurant Surveillance Technology

I attended the National Restaurant Association exposition in Chicago earlier this year, and looked at all the ways modern restaurant IT is spying on people.

But there's also a fundamentally creepy aspect to much of this. One of the prime ways to increase value for your brand is to use the Internet to practice surveillance of both your customers and employees. The customer side feels less invasive: Loyalty apps are pretty nice, if in fact you generally go to the same place, as is the ability to place orders electronically or make reservations with a click. The question, Schneier asks, is "who owns the data?" There's value to collecting data on spending habits, as we've seen across e-commerce. Are restaurants fully aware of what they are giving away? Schneier, a critic of data mining, points out that it becomes especially invasive through "secondary uses," when the "data is correlated with other data and sold to third parties." For example, perhaps you've entered your name, gender, and age into a taco loyalty app (12th taco free!). Later, the vendors of that app sell your data to other merchants who know where and when you eat, whether you are a vegetarian, and lots of other data that you have accidentally shed. Is that what customers really want?

Planet DebianOsamu Aoki: exim4 configuration for Desktop (better gmail support)

For most of our Desktop PC running with stock exim4 and mutt, I think sending out mail is becoming a bit rough since using random smarthost causes lots of trouble due to the measures taken to prevent spams.

As mentioned in Exim4 user FAQ , /etc/hosts should have FQDN with external DNS resolvable domain name listed instead of localdomain to get the correct EHLO/HELO line.  That's the first step.

The stock configuration of exim4 only allows you to use single smarthost for all your mails.  I use one address for my personal use which is checked by my smartphone too.  The other account is for subscribing to the mailing list.  So I needed to tweak ...

Usually, mutt is smart enough to set the From address since my .muttrc has

# Set default for From: for replyes for alternates.
set reverse_name

So how can I teach exim4 to send mails depending on the  mail accounts listed in the From header.

For my gmail accounts, each mail should be sent to the account specific SMTP connection matching your From header to get all the modern SPAM protection data in right state.  DKIM, SPF, DMARC...  (Besides, they overwrite From: header anyway if you use wrong connection.)

For my debian.org mails, mails should be sent from my shell account on people.debian.org so it is very unlikely to be blocked.  Sometimes, I wasn't sure some of these debian.org mails sent through my ISP's smarthost are really getting to the intended person.

To these ends, I have created small patches to the /etc/exim4/conf.d files and reported it to Debian BTS: #869480 Support multiple smarthosts (gmail support).  These patches are for the source package.

To use my configuration tweak idea, you have easier route no matter which exim version you are using.  Please copy and read pertinent edited files from my github site to your installed /etc/exim4/conf.d files and get the benefits.
If you really wish to keep envelope address etc. to match From: header, please rewite agressively using the From: header using eddited rewrite/31_exim4-config_rewriting as follows:

.ifndef NO_EAA_REWRITE_REWRITE
*@+local_domains "${lookup{${local_part}}lsearch{/etc/email-addresses}\
                   {$value}fail}" f
# identical rewriting rule for /etc/mailname
*@ETC_MAILNAME "${lookup{${local_part}}lsearch{/etc/email-addresses}\
                   {$value}fail}" f
.endif
* "$h_from:" Frs

So far its working fine for me but if you find bug, let me know.

Osamu

CryptogramZero-Day Vulnerabilities against Windows in the NSA Tools Released by the Shadow Brokers

In April, the Shadow Brokers -- presumably Russia -- released a batch of Windows exploits from what is presumably the NSA. Included in that release were eight different Windows vulnerabilities. Given a presumed theft date of the data as sometime between 2012 and 2013 -- based on timestamps of the documents and the limited Windows 8 support of the tools:

  • Three were already patched by Microsoft. That is, they were not zero days, and could only be used against unpatched targets. They are EMERALDTHREAD, EDUCATEDSCHOLAR, and ECLIPSEDWING.

  • One was discovered to have been used in the wild and patched in 2014: ESKIMOROLL.

  • Four were only patched when the NSA informed Microsoft about them in early 2017: ETERNALBLUE, ETERNALSYNERGY, ETERNALROMANCE, and ETERNALCHAMPION.

So of the five serious zero-day vulnerabilities against Windows in the NSA's pocket, four were never independently discovered. This isn't new news, but I haven't seen this summary before.

Worse Than FailureError'd: The Things That Should Not Be

"I tried to export my game to HTML5, but I guess it just wasn't meant to be," Edward W. writes.

 

Tom H. wrote, "I guess the build server never saw that memo."

 

"I love going out to dinner with my friend null null," writes Adam R., "She never steals any of my food!"

 

Mike C. wrote, "Sorry JIRA, all the keys on my keyboard are defined."

 

"You guys! I caught an error! 🎣 🎣" writes Nick.

 

Hamakei asks, "Never mind who's watching the Watchmen...who helps the helpers?"

 

[Advertisement] High availability, Load-balanced or Basic – design your own Universal Package Manager, allow the enterprise to scale as you grow. Download and see for yourself!

Don MartiOnline ads don't matter to P&G

In the news: P&G Cuts More Than $100 Million in ‘Largely Ineffective’ Digital Ads

Not surprising.

Proctor & Gamble makes products that help you comply with widely held cleanliness norms.

Digital ads are micro-targeted to you as an individual.

That's the worst possible brand/medium fit. If you don't know that the people who expect you to keep your house or body clean are going to be aware of the same product, how do you know whether to buy it?

Bonus link from Bob Hoffman last year: Will The P&G Story Bring Down Ad Tech? Please?

Planet Linux AustraliaPia Waugh: RegTech – a primer for the uninitiated

Whilst working at AUSTRAC I wrote a brief about RegTech which was quite helpful. I was given permission to blog the generically useful parts of it for general consumption :) Thanks Leanne!

Overview – This brief is the most important thing you will read in planning transformation! Government can’t regulate in the way we have traditionally done. Traditional approaches are too small, too slow and too ineffective. We need to explore new ways to regulate and achieve the goal of a stronger financial sector resistance to abuse that leverages data, automation, machine learning, technology and collaboration. We are here to help!

The key here is to put technology at the heart of the business strategy, rather than as simply an implementation mechanism. By embracing technology thinking, which means getting geeks into the strategy and policy rooms, we can build the foundation of a modern, responsive, agile, proactive and interactive regulator that can properly scale.

The automation of compliance with RegTech has the potential to overcome individual foibles and human error in a way that provides the quantum leap in culture and compliance that our regulators, customers, policy makers and the community are increasingly demanding… The Holy Grail is when we start to actually write regulation and legislation in code. Imagine the productivity gains and compliance savings of instantaneous certified compliance… We are now in one of the most exciting phases in the development of FinTech since the inception of e-banking.Treasurer Morrison, FinTech Australia Summit, Nov 2016

On the back of the FinTech boom, there is a growth in companies focused on “RegTech” solutions and services to merge technology and regulation/compliance needs for a more 21st century approach to the problem space. It is seen as a logical next step to the FinTech boom, given the high costs and complexity of regulation in the financial sector, but the implications for the broader regulatory sector are significant. The term only started being widely used in 2015. Other governments have started exploring this space, with the UK Government investing significantly.

Core themes of RegTech can be summarised as: data; automation; security; disruption; and enabling collaboration. There is also an overall drive towards everything being closer to real-time, with new data or information informing models, responses and risk in an ongoing self-adjusting fashion.

  • Data driven regulation – better monitoring, better use of available big and small data holdings to inform modelling and analysis (rather than always asking a human to give new information), assessment on the fly, shared data and modelling, trends and forecasting, data analytics for forward looking projections rather than just retrospective analysis, data driven risk and adaptive modelling, programmatic delivery of regulations (regulation as a platform).
  • Automation – reporting, compliance, risk modelling of transactions to determine what should be reported as “suspicious”, system to system registration and escalation, use of machine learning and AI, a more blended approach to work combining humans and machines.
  • Security – biometrics, customer checks, new approaches to KYC, digital identification and assurance, sharing of identity information for greater validation and integrity checking.
  • Disruptive technologies – blockchain, cloud, machine learning, APIs, cryptography, augmented reality and crypto-currencies just to start!
  • Enabling collaboration – for-profit regulation activities, regulation/compliance services and products built on the back of government rules/systems/data, access to distributed ledgers, distributed risk models and shared data/systems, broader private sector innovation on the back of regulator open data and systems.

Some useful references for the more curious:

Planet DebianJoachim Breitner: How is coinduction the dual of induction?

Earlier today, I demonstrated how to work with coinduction in the theorem provers Isabelle, Coq and Agda, with a very simple example. This reminded me of a discussion I had in Karlsruhe with my then colleague Denis Lohner: If coinduction is the dual of induction, why do the induction principles look so different? I like what we observed there, so I’d like to share this.

The following is mostly based on my naive understanding of coinduction based on what I observe in the implementation in Isabelle. I am sure that a different, more categorial presentation of datatypes (as initial resp. terminal objects in some category of algebras) makes the duality more obvious, but that does not necessarily help the working Isabelle user who wants to make sense of coninduction.

Inductive lists

I will use the usual polymorphic list data type as an example. So on the one hand, we have normal, finite inductive lists:

datatype 'a list = nil | cons (hd : 'a) (tl : "'a list")

with the well-known induction principle that many of my readers know by heart (syntax slightly un-isabellized):

P nil → (∀x xs. P xs → P (cons x xs)) → ∀ xs. P xs

Coinductive lists

In contrast, if we define our lists coinductively to get possibly infinite, Haskell-style lists, by writing

codatatype 'a llist = lnil | lcons (hd : 'a)  (tl : "'a llist")

we get the following coinduction principle:

(∀ xs ys.
    R xs ys' → (xs = lnil) = (ys = lnil) ∧
               (xs ≠ lnil ⟶ ys' ≠ lnil ⟶
	         hd xs = hd ys ∧ R (tl xs) (tl ys))) →
→ (∀ xs ys. R xs ys → xs = ys)

This is less scary that it looks at first. It tell you “if you give me a relation R between lists which implies that either both lists are empty or both lists are nonempty, and furthermore if both are non-empty, that they have the same head and tails related by R, then any two lists related by R are actually equal.”

If you think of the infinte list as a series of states of a computer program, then this is nothing else than a bisimulation.

So we have two proof principles, both of which make intuitive sense. But how are they related? They look very different! In one, we have a predicate P, in the other a relation R, to point out just one difference.

Relation induction

To see how they are dual to each other, we have to recognize that both these theorems are actually specializations of a more general (co)induction principle.

The datatype declaration automatically creates a relator:

rel_list :: ('a → 'b → bool) → 'a list → 'b list → bool

The definition of rel_list R xs ys is that xs and ys have the same shape (i.e. length), and that the corresponding elements are pairwise related by R. You might have defined this relation yourself at some time, and if so, you probably introduced it as an inductive predicate. So it is not surprising that the following induction principle characterizes this relation:

Q nil nil →
(∀x xs y ys. R x y → Q xs ys → Q (cons x xs) (cons y ys)) →
(∀xs ys → rel_list R xs ys → Q xs ys)

Note how how similar this lemma is in shape to the normal induction for lists above! And indeed, if we choose Q xs ys ↔ (P xs ∧ xs = ys) and R x y ↔ (x = y), then we obtain exactly that. In that sense, the relation induction is a generalization of the normal induction.

Relation coinduction

The same observation can be made in the coinductive world. Here, as well, the codatatype declaration introduces a function

rel_llist :: ('a → 'b → bool) → 'a llist → 'b llist → bool

which relates lists of the same shape with related elements – only that this one also relates infinite lists, and therefore is a coinductive relation. The corresponding rule for proof by coinduction is not surprising and should remind you of bisimulation, too:

(∀xs ys.
    R xs ys → (xs = lnil) = (ys = lnil) ∧
              (xs ≠ lnil ⟶ ys ≠ lnil ⟶
	        Q (hd xs) (hd ys) ∧ R (tl xs) (tl ys))) →
(∀ xs ys → R xs ys → rel_llist Q xs ys)

It is even more obvious that this is a generalization of the standard coinduction principle shown above: Just instantiate Q with equality, which turns rel_llist Q into equality on the lists, and you have the theorem above.

The duality

With our induction and coinduction principle generalized to relations, suddenly a duality emerges: If you turn around the implication in the conclusion of one you get the conclusion of the other one. This is an example of “cosomething is something with arrows reversed”.

But what about the premise(s) of the rules? What happens if we turn around the arrow here? Although slighty less immediate, it turns out that they are the same as well. To see that, we start with the premise of the coinduction rule, reverse the implication and then show that to be equivalent to the two premises of the induction rule:

(∀xs ys.
    R xs ys ← (xs = lnil) = (ys = lnil) ∧
              (xs ≠ lnil ⟶ ys ≠ lnil ⟶
	        Q (hd xs) (hd ys) ∧ R (tl xs) (tl ys)))
= { case analysis (the other two cases are vacuously true) }
  (∀xs ys.
    xs = lnil → ys = lnil →
    R xs ys ← (xs = lnil) = (ys = lnil) ∧
              (xs ≠ lnil ⟶ ys ≠ lnil ⟶
	        Q (hd xs) (hd ys) ∧ R (tl xs) (tl ys)))
∧ (∀xs ys.
    xs ≠ lnil ⟶ ys ≠ lnil
    R xs ys ← (xs = lnil) = (ys = lnil) ∧
              (xs ≠ lnil ⟶ ys ≠ lnil ⟶
	        Q (hd xs) (hd ys) ∧ R (tl xs) (tl ys)))
= { simplification }
  (∀xs ys.  xs = lnil → ys = lnil → R xs ys
∧ (∀x xs y ys.  R (cons x xs) (cons y ys) ← (Q x y ∧ R xs ys))
= { more rewriting }
  R nil nil
∧ (∀x xs y ys. Q x y → R xs ys → R (cons x xs) (cons y ys))

Conclusion

The coinduction rule is not the direct dual of the induction rule, but both are specializations of more general, relational proof methods, where the duality is clearly present.

More generally, this little excursion shows that it is often beneficial to think of types less as sets, and more as relations – this way of thinking is surprisingly fruitful, and led to proofs of parametricity and free theorems and other nice things.

Planet DebianNOKUBI Takatsugu: uim package integraion is working in progress

Now I am helping to maintain uim-package. It is splited into too many packages (over 40, including dbgsym).

Working status is written as the debian wiki entry https://wiki.debian.org/JapaneseEnvironment/Uim.

Currentry it is not finished, when I’ve done all of them, I’ll upload them to experimental.

,

Cory DoctorowHey, Little Rock, AR: there’s a special stage performance of Little Brother coming your way for Banned Books Week!

Adapted by Josh Costello from the novel by Cory Doctorow
September 15, 16, 22, 23, 24, 28, 29, 30, 2017
Directed by Ryan Whitfield and Jason Green

SYNOPSIS
While skipping school and playing an alternate reality game, San Francisco teenager Marcus Yallow ends up in the middle of a terrorist attack and on the wrong side of the Department of Homeland Security. This play asks “What is the right thing to do when authorities become oppressors?”

CAST
LITTLE BROTHER CAST LIST
Marcus – Jeffrey Oakley
Ange – Kayley Shettles
Jolu – Yusuf Richardson
Daryl – Jack Clay

ENSEMBLE
Severe Haircut – Madison McMichael
Benson/Sutherland – Robert Gatlin
Guard – Essence Robinson
Mom – Isabelle Marchese
Dad – Max Green
Turk/CHP Officer – Braden Hammock
Ms. Galvez – Anais Moore
Charles – Elijah White
Police Officer 1 – Kyndall Jackson
Police Officer 2- Mia Simone Parker
Trudy Doo – Emily Shull
NPR Announcer – Allison Boggs
Concertgoer – Rachel Worthington
Reporter – Hannah Livingston
Fox Commentator – Katie Rasure
BBC Reporter – Olivia Ward
Pirate Queen – Abigail Harris
On stage light/sound/projection tech – Trenton Gorman, Claire Green

TICKETS & TIMES
$16— Adults
$12— Students & Seniors
Thursday, Friday and Saturday night curtain time is 7:30 pm.
Sunday afternoon curtain time is 2:30 pm.

The Box Office and the theater open one (1) hour prior to curtain.
The House opens 30 minutes prior to curtain.
Please arrive promptly. There will be no late admission.

Planet DebianJoachim Breitner: Coinduction in Coq and Isabelle

The DeepSpec Summer School is almost over, and I have had a few good discussions. One revolved around coinduction: What is it, how does it differ from induction, and how do you actually prove something. In the course of the discussion, I came up with a very simple coinductive exercise, and solved it both in Coq and Isabelle

The task

Define the extended natural numbers coinductively. Define the min function and the  ≤  relation. Show that min(n, m) ≤ n holds.

Coq

The definitions are straight forward. Note that in Coq, we use the same command to define a coinductive data type and a coinductively defined relation:

CoInductive ENat :=
  | N : ENat
  | S : ENat -> ENat.

CoFixpoint min (n : ENat) (m : ENat)
  :=match n, m with | S n', S m' => S (min n' m')
                    | _, _       => N end.

CoInductive le : ENat -> ENat -> Prop :=
  | leN : forall m, le N m
  | leS : forall n m, le n m -> le (S n) (S m).

The lemma is specified as

Lemma min_le: forall n m, le (min n m) n.

and the proof method of choice to show that some coinductive relation holds, is cofix. One would wish that the following proof would work:

Lemma min_le: forall n m, le (min n m) n.
Proof.
  cofix.
  destruct n, m.
  * apply leN.
  * apply leN.
  * apply leN.
  * apply leS.
    apply min_le.
Qed.

but we get the error message

Error:
In environment
min_le : forall n m : ENat, le (min n m) n
Unable to unify "le N ?M170" with "le (min N N) N

Effectively, as Coq is trying to figure out whether our proof is correct, i.e. type-checks, it stumbled on the equation min N N = N, and like a kid scared of coinduction, it did not dare to “run” the min function. The reason it does not just “run” a CoFixpoint is that doing so too daringly might simply not terminate. So, as Adam explains in a chapter of his book, Coq reduces a cofixpoint only when it is the scrutinee of a match statement.

So we need to get a match statement in place. We can do so with a helper function:

Definition evalN (n : ENat) :=
  match n with | N => N
               | S n => S n end.

Lemma evalN_eq : forall n, evalN n = n.
Proof. intros. destruct n; reflexivity. Qed.

This function does not really do anything besides nudging Coq to actually evaluate its argument to a constructor (N or S _). We can use it in the proof to guide Coq, and the following goes through:

Lemma min_le: forall n m, le (min n m) n.
Proof.
  cofix.
  destruct n, m; rewrite <- evalN_eq with (n := min _ _).
  * apply leN.
  * apply leN.
  * apply leN.
  * apply leS.
    apply min_le.
Qed.

Isabelle

In Isabelle, definitions and types are very different things, so we use different commands to define ENat and le:

theory ENat imports  Main begin

codatatype ENat =  N | S  ENat

primcorec min where
   "min n m = (case n of
       N ⇒ N
     | S n' ⇒ (case m of
        N ⇒ N
      | S m' ⇒ S (min n' m')))"

coinductive le where
  leN: "le N m"
| leS: "le n m ⟹ le (S n) (S m)"

There are actually many ways of defining min; I chose the one most similar to the one above. For more details, see the corec tutorial.

Now to the proof:

lemma min_le: "le (min n m) n"
proof (coinduction arbitrary: n m)
  case le
  show ?case
  proof(cases n)
    case N then show ?thesis by simp
  next
    case (S n') then show ?thesis
    proof(cases m)
      case N then show ?thesis by simp
    next
      case (S m')  with ‹n = _› show ?thesis
        unfolding min.code[where n = n and m = m]
        by auto
    qed
  qed
qed

The coinduction proof methods produces this goal:

proof (state)
goal (1 subgoal):
 1. ⋀n m. (∃m'. min n m = N ∧ n = m') ∨
          (∃n' m'.
               min n m = S n' ∧
               n = S m' ∧
	       ((∃n m. n' = min n m ∧ m' = n) ∨ le n' m'))

I chose to spell the proof out in the Isar proof language, where the outermost proof structure is done relatively explicity, and I proceed by case analysis mimiking the min function definition.

In the cases where one argument of min is N, Isabelle’s simplifier (a term rewriting tactic, so to say), can solve the goal automatically. This is because the primcorec command produces a bunch of lemmas, one of which states n = N ∨ m = N ⟹ min n m = N.

In the other case, we need to help Isabelle a bit to reduce the call to min (S n) (S m) using the unfolding methods, where min.code contains exactly the equation that we used to specify min. Using just unfolding min.code would send this method into a loop, so we restrict it to the concrete arguments n and m. Then auto can solve the remaining goal (despite all the existential quantifiers).

Summary

Both theorem provers are able to prove the desired result. To me it seems that it is slightly more convenient in Isabelle because a lot of Coq infrastructure relies on the type checker being able to effectively evaluate expressions, which is tricky with cofixpoints, wheras evaluation plays a much less central role in Isabelle, where rewriting is the crucial technique, and while one still cannot simply throw min.code into the simpset, so working with objects that do not evaluate easily or completely is less strange.

Agda

I was challenged to do it in Agda. Here it is:

module ENat where

open import Coinduction

data ENat : Set where
  N : ENat
  S : ∞ ENat → ENat

min : ENat → ENat → ENat
min (S n') (S m') = S (♯ (min (♭ n') (♭ m')))
min _ _ = N

data le : ENat → ENat → Set where
  leN : ∀ {m} → le N m
  leS : ∀ {n m} → ∞ (le (♭ n) (♭ m)) → le (S n) (S m)

min_le : ∀ {n m} → le (min n m) n
min_le {S n'} {S m'} = leS (♯ min_le)
min_le {N}    {S m'} = leN
min_le {S n'} {N} = leN
min_le {N}    {N} = leN

I will refrain from commenting it, because I do not really know what I have been doing here, but it typechecks, and refer you to the official documentation on coinduction in Agda. But let me note that I wrote this using plain inductive types and recursion, and added , and until it worked.

TEDAnonymous ideas worth spreading — and the surprising discoveries behind their curation

The intimacy of listening: Producer Cloe Shasha shares what she and her team learned while producing TED and Audible’s original audio series “Sincerely, X.”

In the spring of 2016, we put out a call for submissions for anonymous talks from around the world for the first season of our new podcast, Sincerely, X. We received hundreds of ideas — stories touching on a broad range of topics. As we read through them, we found ourselves flooded by tragedy, comedy, intrigue and surprise. Stories of victims of abuse, struggles with mental health, lessons from prison, insider secrets within companies and governmental organizations, and so much more.

>> Sincerely, X was co-produced with Audible. Episode 1, “Dr. Burnout,” is available now on Apple Podcasts and the TED Android app. <<

The premise of the podcast Sincerely, X felt simple at first: sharing important ideas, anonymously. The episodes would include speakers who need to separate their professional ideas from their personal lives; those who want to share an idea, but fear it would hurt someone in their family if they did so publicly; and quiet idealists whose solutions could transform lives. Why anonymous? Our theory was that inviting people to share ideas without having to reveal their identity might allow for an entirely new category of talks.

We dove into this pool of submissions to figure out who would make a great speaker for the show, and started interviewing people by phone. We were looking for compelling stories that had a strong need for anonymity while also considering them through the lens that we use for TED Talk submissions. In other words, did each story have an idea worth spreading?

Throughout the process of creating Sincerely, X season 1, we realized that we had to think about these talks quite differently from TED Talks on a stage, and we adapted along the way.

Signposting in an audio talk

When you’re watching a speaker on a stage, context and sentiment are communicated through the speaker’s body language, facial expressions and images (if they have slides). In audio, with only one of our senses engaged, a lot more information has to be transmitted through a speaker’s voice alone.

This came up when we worked with the speaker in episode 2, “Pepper Spray.” It’s the story of a woman who lived a normal-seeming life — until one day she lashed out in a department store and began pepper-spraying strangers. There are a lot of details that she shares about her life in that episode — both before and after the pepper spray incident. If she were telling this story on a stage, the audience would experience visual cues that would indicate whether she were reflecting on the far past versus the recent past, or whether she felt shameful or justified in her actions. (Watch a TED Talk with the sound off sometime, and you’ll be surprised at how much context you can pick up!) But when we shared the audio with colleagues for their feedback, they were at times confused by the sequence of events in the story. So we worked with the speaker to help her find places to include signposting sentences such as, “But I want to come back to the hero of the story.” In other words, phrases that could ground the listener in what’s about to come.  

The intimacy of listening

In the same way that hearing a ghost story around a campfire conjures up scary visualizations, hearing a difficult story on a podcast can build intense images in your mind. Drawing the line between deeply moving content and manipulative content can be tricky and nuanced.

In the case of some Sincerely, X episodes, a few of the early drafts of talks contained details that felt disturbingly intimate — details that might have packed an emotional punch from the distance of a stage, but that felt too intimate coming out of earbuds. We had to learn how to mitigate that intensity by listening to the content and getting feedback from early screeners who shared honest reactions.

This was a relevant dynamic for several of our speakers, including our speaker in episode 6, “Rescued by Ritual.” This speaker talks about a private ritual she invented in order to cope with the horror of her abusive marriage before she left her ex-husband. In the earliest draft, in order to provide context for the purpose of her ritual, the leadup to the reenactment of the ritual involved details that were difficult to hear for some early listeners. So we worked with the speaker to figure out which details she felt were most needed in order to paint an accurate picture of that time in her life.

To read or to memorize?

When it comes to our TED speakers on the stage, we typically encourage two ways of preparing for a talk: either memorizing their content so thoroughly that they can recite it seamlessly while standing on one foot with the television blaring, or memorizing an outline and riffing off that rehearsed structure once onstage. As Chris Anderson says, partially memorizing a talk produces an “uncanny valley” effect — a seemingly robotic or artificial performance. It’s hard to appear authentic while devoting a fair amount of energy to the process of recall. So if someone is not a great memorizer, we encourage improvising the sentences based on a solid outline of the concepts. Both of these forms of preparation are aimed at fostering an authentic delivery from the speaker, which cultivates a powerful connection between the speaker and the audience.

In the context of Sincerely, X, we thought about how to foster that authentic delivery, and considered that preparing speakers to read their talks might be a lower-stress way to record speakers in the studio. But it soon became clear that unless a speaker had acting experience, reading a talk sounded like… reading. So we experimented with having speakers memorize their talks extremely thoroughly before coming into the studio. And this worked for some speakers; when we recorded the speaker in episode 1, “Dr. Burnout,” she delivered her talk beautifully once she had fully committed it to memory.

Sincerely, X was co-produced by TED and Audible. The team was led by executive producers Collin Campbell, Deron Triff and June Cohen (who is also the host). Episode 1, “Dr. Burnout,” is available now on Apple Podcasts and the TED Android app. We’ll be releasing new episodes every Thursday for the next ten weeks.

We’ll be releasing new episodes every Thursday for the next ten weeks.


Planet Linux AustraliaLinux Users of Victoria (LUV) Announce: LUV Beginners August Meeting: TBD

Aug 19 2017 12:30
Aug 19 2017 16:30
Aug 19 2017 12:30
Aug 19 2017 16:30
Location: 
Infoxchange, 33 Elizabeth St. Richmond

Workshop to be announced.

There will also be the usual casual hands-on workshop, Linux installation, configuration and assistance and advice. Bring your laptop if you need help with a particular issue. This will now occur BEFORE the talks from 12:30 to 14:00. The talks will commence at 14:00 (2pm) so there is time for people to have lunch nearby.

The meeting will be held at Infoxchange, 33 Elizabeth St. Richmond 3121 (enter via the garage on Jonas St.) Late arrivals, please call (0421) 775 358 for access to the venue.

LUV would like to acknowledge Infoxchange for the venue.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

August 19, 2017 - 12:30

Planet Linux AustraliaLinux Users of Victoria (LUV) Announce: LUV Main August 2017 Meeting

Aug 1 2017 18:30
Aug 1 2017 20:30
Aug 1 2017 18:30
Aug 1 2017 20:30
Location: 
The Dan O'Connell Hotel, 225 Canning Street, Carlton VIC 3053

Tuesday, August 1, 2017

6:30 PM to 8:30 PM
The Dan O'Connell Hotel
225 Canning Street, Carlton VIC 3053

Speakers:

  • Tony Cree, CEO Aboriginal Literacy Foundation (to be confirmed)
  • Russell Coker, QEMU and ARM on AMD64

Russell Coker will demonstrate how to use QEMU to run software for ARM CPUs on an x86 family CPU.

The Dan O'Connell Hotel, 225 Canning Street, Carlton VIC 3053

Food and drinks will be available on premises.

Before and/or after each meeting those who are interested are welcome to join other members for dinner.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

August 1, 2017 - 18:30

CryptogramFiring a Locked Smart Gun

The Armatix IP1 "smart gun" can only be fired by someone who is wearing a special watch. Unfortunately, this security measure is easily hackable.

Krebs on SecurityGas Pump Skimmer Sends Card Data Via Text

Skimming devices that crooks install inside fuel station gas pumps frequently rely on an embedded Bluetooth component allowing thieves to collect stolen credit card data from the pumps wirelessly with any mobile device. The downside of this approach is that Bluetooth-based skimmers can be detected by anyone else with a mobile device. Now, investigators in the New York say they are starting to see pump skimmers that use cannibalized cell phone components to send stolen card data via text message.

Skimmers that transmit stolen card data wirelessly via GSM text messages and other mobile-based communications methods are not new; they have been present — if not prevalent — in ATM skimming devices for ages.

But this is the first instance KrebsOnSecurity is aware of in which such SMS skimmers have been found inside gas pumps, and that matches the experience of several states hardest hit by pump skimming activity.

The beauty of the GSM-based skimmer is that it can transmit stolen card data wirelessly via text message, meaning thieves can receive real-time transmissions of the card data anywhere in the world — never needing to return to the scene of the crime. That data can then be turned into counterfeit physical copies of the cards.

Here’s a look at a new skimmer pulled from compromised gas pumps at three different filling stations in New York this month. Like other pump skimmers, this device was hooked up to the pump’s internal power, allowing it to operate indefinitely without relying on batteries.

A GSM-based card skimmer found embedded in a gas pump in the northeastern United States.

A GSM-based card skimmer found embedded in a gas pump in the northeastern United States.

It may be difficult to see from the picture above, but the skimmer includes a GSM-based device with a SIM card produced by cellular operator T-Mobile. The image below shows the other side of the pump skimmer, with the SIM card visible in the upper right corner of the circuitboard:

The reverse side of this GSM-based pump skimmer shows a SIM card from T-Mobile.

The reverse side of this GSM-based pump skimmer shows a SIM card from T-Mobile.

It’s not clear what type of mobile device was used in this skimmer, and the police officer who shared these images with KrebsOnSecurity said the forensic analysis of the device was ongoing.

Here’s a close-up of the area around the SIM card:

GSMpumpskimcloseup2

The officer, who shared these photos on condition of anonymity, said this was thought to be the first time fraud investigators in New York had ever encountered a GSM-based pump skimmer.

Skimmers used at all three New York filling stations impacted by the scheme included T-Mobile SIM cards, but the investigator said analysis so far showed the cards held no other data other than the SIM’s card’s unique serial number (ICCID).

KrebsOnSecurity reached out to weights and measures officials in several states most heavily hit by pump skimming activity, including Arizona, California and Florida.

Officials in all three states said they’ve yet to find a GSM-based skimmer attached to any of their pumps.

Skimmers at the pump are most often the work of organized crime rings that traffic in everything from stolen credit and debit cards to the wholesale theft and commercial resale of fuel — in some cases from (and back to) the very fuel stations that have been compromised with the gang’s skimming devices.

Investigators say skimming gangs typically gain access to station pumps by using a handful of master keys that still open a great many pumps in use today. In a common scenario, one person will distract the station attendant as fuel thieves pull up alongside the pump in a van with doors that obscure the machine on both sides. For an in-depth look at the work on one fuel-theft gang working out of San Diego, check out this piece.

There are generally no outward signs when a pump has been compromised by a skimmer, but a study KrebsOnSecurity published last year about a surge in pump skimming activity in Arizona suggests that skimmer gangs can spot the signs of a good mark.

Fraud patterns show fuel theft gangs tend to target stations that are close to major highway arteries; those with older pumps; and those without security cameras, and/or a regular schedule for inspecting security tape placed on the pumps.

Many filling stations are upgrading their pumps to include more physical security — such as custom locks and security cameras. In addition, newer pumps can accommodate more secure chip-based payment cards that are already in use by all other G20 nations.

But these upgrades are disruptive and expensive, and some stations are taking advantage of recent moves by Visa to delay adding much-needed security improvements, such as chip-capable readers.

Until late 2016, fuel station owners in the United States had until October 1, 2017 to install chip-capable readers at their pumps. Under previous Visa rules, station owners that didn’t have chip-ready readers in place by then would have been on the hook to absorb 100 percent of the costs of fraud associated with transactions in which the customer presented a chip-based card yet was not asked or able to dip the chip (currently, card-issuing banks and consumers eat most of the fraud costs from fuel skimming).

But in December 2016, Visa delayed the requirements, saying fuel station owners would now have until October 1, 2020 to meet the liability shift deadline.

The best advice one can give to avoid pump skimmers is to frequent stations that appear to place an emphasis on physical security. More importantly, some pump skimming devices are capable of stealing debit card PINs as wellso it’s good idea to avoid paying with a debit card at the pump.

Armed with your PIN and debit card data, thieves can clone the card and pull money out of your account at an ATM. Having your checking account emptied of cash while your bank sorts out the situation can be a huge hassle and create secondary problems (bounced checks, for instance).

Worse Than FailureTable 12

We've all encountered database tables that look like this:

  ID    Data
  ----- --------------------------------------------
  00001 TRUE, FALSE, FILE_NOT_FOUND
  00002 MALE|FEMALE|TRANS|EUNUCH|OTHER|M|Q|female|Female|male|Male|$
  00003 <?xml version="1.0" encoding="UTF-8"?><item id="1234"><name "Widget"/>...</item>
  00004 1234|Fred,Lena,Dana||||||||||||1.3DEp42|

Oh the joy of figuring out what each field of each row represents. The fun of deciphering the code that writes and reads/parses each row of data. In a moment, you will fondly look back on that experience as the Good-Old-Days.

People waving the Canadian Flag

The task of administering elections in the Great White North is handled by the appropriately-named agency Elections Canada. As part of their mandate, they provide the results of past elections in granular detail, both as nicely formatted web pages and as downloadable raw files. The latter are meant to be used by researchers for studying how turnout varies across provinces, ages, races, etc., as well as arguing about the merits of proportional representation versus single transferable votes; and so forth.

One of the more comprehensive data files is descriptively known as Table-Twelve, and it contains a record for every candidate who ran in the election. Each record contains how many votes they got, the riding (electoral district) in which they competed, their affiliated party, home town, occupation, and hundreds of other details about the candidate. This file has been published for every election since the 38th general in 2004. Vicki was charged with creating a new parser for this data.

Table-Twelve is a CSV file in the same way that managers describe their new agile process as <details of waterfall here>. While parsing a CSV file in general is no big deal, writing a function to parse this data was far harder than she expected. For one thing, the column titles change from year to year. One might think Who cares, as long as the data is in the same sequence. One would be wrong. As an example, depending upon the year, the identifier for the electoral district might be in a column named "Electoral District Name", "Electoral District" or "District", and might contain a string representing the district name, or a numeric district identifier, either of which may or may not be enclosed in single or double quotes. Just to make it interesting, some of the quoted strings have commas, and some of the numbers are commafied as well.

Further inspection revealed that the columns are not only inconsistently named, but named so as to be completely misleading. There's a column labeled "Majority". If you're thinking that it contains a boolean to indicate whether the candidate got a majority, or 50%+1 of the number of cast votes (i.e.: "How many votes do you need for a majority?"), you'd be mistaken. Nor is it even a slight misuse (where it should have been "Plurality"). Instead, it's the delta between the winning candidate and the second-place candidate in that riding. They also helpfully give you the quotient of this delta to the total cast votes as the "Majority Percentage".

Canada has a parliamentary system; it's also important to know how many candidates of each party won, so the party designation is obviously going to be easy to access, right? Or maybe you'd like to sort by surname? Well, it turns out that the party is appended to the field containing the candidate's name, delimited with a single space (and possibly an asterisk if they were incumbent). But the candidate's name and the party are already each a variable number of words (some have middle names or two surnames) delimited by single spaces. The party name, however, must be given in both English and French, separated by a forward slash. Of course, some parties already have a slash in their name! Oh, and if the candidate didn't run as a member of a party, they might be listed as "Independent" or as "No affiliation"; both are used in any given file.

Above and beyond the call of making something difficult to parse, the files are full of French accented text, so the encoding changes from file to file, here ISO-8859, there UTF-8, over there a BOM or two.

Don't get me wrong, I've written parsers for this sort of garbage by creating a bunch of routines to do trivial parsing and using them for larger logical parsers, and so on until you can parse all of the fields in an entire row, and all the special cases that spew forth. But the files they were supposed to parse were consistent from one day to the next.

Vicki is considering pulling out all of her hair, braiding it together and using it to hang the person who designed Table-Twelve.

[Advertisement] High availability, Load-balanced or Basic – design your own Universal Package Manager, allow the enterprise to scale as you grow. Download and see for yourself!

Planet DebianMichal Čihař: Weblate 2.16: Call for translations

Weblate 2.16 is almost ready (I expect no further code changes), so it's really great time to contribute to it's translations! Weblate 2.16 will be probably released during my presence at DebConf 17.

As you might expect, Weblate is translated using Weblate, so the contributions should be really easy. In case there is something unclear, you can look into Weblate documentation.

I'd especially like to see improvements in the Italian translation which was one of the first in Weblate beginnings, but hasn't received much love in past years.

Filed under: Debian English SUSE Weblate

,

Planet DebianNorbert Preining: Software Development as mathematician in academia – everyone bites the dust

Is it possible to do software development, mathematical or not, as mathematician in academics? This is a question I was asking myself recently a lot, seeing my own development from logician at a state university getting rid of foreigners to software developer. And then, a friend pointed me to this very depressing document: The origins of SageMath by William Stein, the main developer of SageMath. And I realized that it seems to be a global phenomenon that mathematicians who are interested in software development have to leave academics. What a sad affair.

SageMath has a clear mission:

Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab.

All the “Ma”-software packages are commercial, and expensive. On the other hand they often have very good algorithms implemented. The Sage developers invested lots of time, energy, and brain power to develop excellent algorithm in an open source project for the mathematical researcher, but this investment wasn’t honored in academic life. To quote from the presentation:

Issues with software dev in academia

  • Hard money for software development is virtually nonexistent: I can’t think of anyone I know who got tenured based on his or her software.
  • Researchers on soft money are systematically discriminated against in favor of tenure-track and tenured faculty.
  • Researchers are increasingly evaluated solely on bibliometric counts rather than an informed assessment of their overall portfolio of papers, code, software, industry engagement, or student supervision.

The origins of SageMath, p.31

I can fully agree to this. Both from my own experience as well as from those around me. The presentation slides are full of other examples, from the developers of NumPy, Jupyter, as well as statements by Stephen Wolfram from Mathematica about this issue. A textbook how to not setup academia.

My assumption was that this hits only on non-tenured staff, the academic precariat. It is shocking to see that even William Stein with a tenure position is leaving academics. It seems the times are not ready 🙁

Every great open source math library is built on the ashes of someone’s academic career.
The origins of SageMath, p.32

Rondam RamblingsThe definition of dishonorable

Donald Trump during the campaign: Donald Trump in office: I wonder if he even knows what the T in LGBT stands for. The bigotry and ignorance behind this decision are truly staggering.  The implication that a transgender person imposes "tremendous medical costs and disruption" which impedes "decisive and overwhelming victory" when they serve "in any capacity" (emphasis mine) is

Planet DebianThomas Lange: Building a Debian Live CD with FAI

In this wiki entry, I describe how to extend a FAI nfsroot, so it can be used as a file system for a diskless client or a Live CD. A host can mount it via NFS when booting via PXE. You can create a Live CD easily by using the command fai-cd.

This works also nicely with a Xfce desktop, and I've prepared a ISO image for easy testing.

You can log in as user demo, and the password is fai.

The next thing is to check, if we can use FAI's dirinstall or install method for creating the same environment, so it will be easy to create customized Live images.

FAI

LongNowWhy Do Some Forms of Knowledge Go Extinct?

The History of Art and Architecture slide library at Trinity College, Dublin. Via the Department of Ultimology.


Fiona Hallinan is an artist and researcher based at Trinity College, Dublin. She’s co-founder of a project along with curator Kate Strain called the Department of Ultimology. Ultimology is the study of that which is dead or dying in a series or process. When applied to academic disciplines, it becomes the study of extinct or endangered subjects, theories, and tools of learning. Long Now recently spoke with Hallinan when she visited The Interval. What follows is a transcript of our conversation, edited for length and clarity.

LONG NOW: What was the inspiration for a department studying extinct or endangered subjects and theories?

Fiona Hallinan: It began back when Kate and I were both alumni of the History of Art and Architecture Department at Trinity University College, Dublin. We learned everything we studied from a rather limited slide library. And we were speculating how in the last ten years those slides probably had been digitized, and students now probably had access to an infinite number of images compared to our limited selection. We wondered how that had impacted how people learned the discipline, and therefore how that had actually evolved the discipline of art history itself. So we came up with an idea for a department within the university that would examine all the other disciplines and departments from that perspective.

Via the Department of Ultimology.


We had encountered the term “ultimology” in the context of the study of endangered languages and thought that that could be expanded to become a general discipline across the university that looked at that which was dead or dying. In 02014 we applied for and won the Trinity Creative Challenge, which was a provost’s award for artistic projects that would explore the university and present the knowledge being produced there to the general public. We spent the next year conducting interviews with different heads of departments and disciplines about what was ultimological in their disciplines. Based off of our findings, we organized the First International Conference of Ultimology, a public event that presented a mix of artistic commissions, presentations and real academic papers. Through that we were invited to be hosted as the Department of Ultimology in residence at CONNECT, which is the center for future networks at Trinity.

LN: What is your methodology when approaching a given academic discipline? Are you reaching out to specific fields and subjects that you suspect as having ultimological potential?

FH: At the beginning we just wanted to get as wide as scope as possible; we had a particular narrative that we expected to encounter, namely, that there was an increasing commercialization of the university because certain disciplines could receive funding that perhaps other modes of knowledge production could not on account of phasing out of interest and activity. We thought that a subject like, say, medieval architecture might be virtually impossible to get funding for nowadays versus something like computational linguistics. And as a result, this was causing a shift or change in the structure of the university.

The Illusion of Infinite Resources,” by the Department of Ultimology.


While we did find that that was true to an extent, we also found that as a term, “ultimology” was really exciting for lots of the academics that we spoke to, and there was a sense of relief that finally there was somewhere they could put all of this endangered or extinct knowledge. Often, we would go into a meeting and people would be prepared with heaps of examples, whereas other times people would be interested but say that ultimology wasn’t really that relevant to their discipline, only to realize through inquiry that it was.

One example of that was in Trinity’s Department of Psychology, where the department head, Dr. Jean Quigley, said that psychology didn’t really have anything ultimological because ideas and tools were added all the time instead of being taken away. We asked her for an example of something that had been recently added, and she described the concept of personality. From that, we asked what would the set of qualities we call “personality” been described as before. And she said that people would have spoken about the soul. So from that conversation we started to think about different methodologies, and we described that methodology as negative space—the space that the concept would have occupied before.

A second methodology we developed was the idea of ultimology as a service. We hold clinics where academics come to us and speak to us, and the ultimological becomes a service akin to therapy where people can get things off their chest or they can talk about their research papers that didn’t go anywhere. It becomes a repository for the burden of the recent past.

Another methodology we began to utilize was the idea of embodiment, where we embody the Department of Ultimology through commissioning artists to make us the accessories or trappings of a real department, like bureaucratic forms.

Lanyards designed by Dennis McNulty for the First International Conference of Ultimology. Via the Department of Ultimology.


For our conference, we found a company in Dublin that had a hundred remaining lanyards with mobile phone loops on them, which would have been used in the pre-smartphone age. We commissioned an artist, Dennis McNulty, to riff on these lanyards with a poetic piece of text on them about the designer of the iPhone. The lanyard itself looked like an iPhone. And so there was this potential in an object like a lanyard that connoted a certain context and space of knowledge production, and I think there’s scope there to work with artists to consider those objects and what they mean and what their associations are for us. The bureaucratic questionnaire fulfills a similar function: it asks what research is, and talks about the idea of a person’s practice. While it looks very bureaucratic, its purpose is to get people to go deeply into reflecting on what they actually do.

The performativity of being a “department”  is essential. By doing it, it becomes real. While the Department of Ultimology is technically an art project, it’s not about just a specific outcome or a specific object coming out of it;  it’s more about using an artistic process to re-evaluate everything critically.

LN: What role does nostalgia play in the Department of Ultimology? Do the academics you interview bemoan a lost discipline or practice?  

FH: We try to be careful to avoid nostalgia, to avoid people being sad for something just because of a kind of fondness for it. While I’m not against nostalgia personally, I think it’s less interesting to fetishize the past, and more interesting to look at how these things actually affect the future.

Glassware blown by Trinity’s resident glassblower John Kelly.


For example, we met with Dr. Sylvia Draper, Head of the School of Chemistry at Trinity, and asked her what had changed in the discipline of Chemistry. She spoke about how glassware used to be an essential part of research. If you were a student of chemistry, you might actually design a piece of glassware that goes with your research. Draper told us that Trinity College had a glassblowing workshop on site with a glassblower named John Kelly, but that he was going to retire in two years and would not be replaced. It ties back to the commercialization of the university: the reason he’s not being replaced is because he’s salaried and a salaried employee is a high cost for the university. And so he and his work become expendable because in theory the department can just bring in cheaper, standard glassware from abroad.

However, if you’re a student and you’re planning your experiment and it requires an intricate, strange, unique piece of glass, it might now be much more expensive for you to get it, which might impact how you look at your research. You might be less willing or able to do something weirder, essentially. I picture it like these tiny little cracks that maybe can’t be explored in a discipline as people are funnelled down into a more particular standard route.

John Kelly at work in his lab at Trinity College, Dublin. Via the Department of Ultimology.


So while there’s a sense of nostalgia thinking about John Kelly in his lab and his beautiful glassware, it’s less about trying to preserve what he’s doing for the sake of it; there’s an actual reason behind it that’s important to know about. It’s also very short-term thinking. Say his salary is 50,000 Euro a year, and a piece of special glassware costs 1,000 Euro to ship in. it’s really quickly not going to add up, and is a short-sighted view of saving money now without much thought to the future.

LN: Looking to the future, what’s next for the Department of Ultimology?

Kate Strain and Fiona Hallinan, founders of the Department of Ultimology.


We’re hoping to publish a journal in December. We’re treating the journey of making it all as part of the project as well. So it won’t be a roll-out of a finished product, and I think that we might think of the field of peer review as potential for a public event.  

Ultimately, we would like to start a Department of Ultimology in every time zone. We say “time zones” because  it’s a way of dividing the world that is perhaps more timeless than countries or nation-states. There’s an instability to those, particularly at the moment, whereas time zones have a celestial, larger-than-us quality.

Keep up with the Department of Ultimology by heading to its website or following it on Twitter.

Worse Than FailureCodeSOD: The Nuclear Option

About a decade ago, Gerald worked at a European nuclear plant. There was a “minor” issue where a controller connected to a high-voltage power supply would start missing out on status messages. “Minor”, because it didn’t really pose a risk to life and limb- but still, any malfunction with a controller attached to a high-voltage power supply in a nuclear power plant needs to be addressed.

So Gerald went off and got the code. It was on a file share, in a file called final.zip. Or, wait, was it in the file called real-final.zip? Or installed.zip? Or, finalnew.zip?

It took a few tries, but eventually he picked out the correct one. To his surprise, in addition to the .c and .h files he expected to see, there was also a mysterious .xls. And that’s where things went bad.

Pause for a moment to consider a problem: you receive a byte containing an set of flags to represent an error code. So, you need to check each individual bit to understand what the exact error is. At this point, you’re probably reaching for a bitshift operator, because that’s the easiest way to do it.

I want you to imagine, for a moment, however, that you don’t really know C, or bitwise operations, or even what a bit is. Instead, you know two things: that there are 255 possible error codes, and how to use Excel. With those gaps in knowledge, you might perhaps, just manually write an Excel spreadsheet with every possible option, using Excel's range-drag operation to fill in the columns with easily predictable values. You might do this for 254 rows of data. Which, as a note, the range of possible values is 255, so guess what was causing the error?

if (variable==   0       ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   1       ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   2       ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   3       ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   4       ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   5       ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   6       ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   7       ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   8       ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   9       ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   10      ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   11      ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   12      ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   13      ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   14      ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   15      ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   16      ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   17      ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   18      ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   19      ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   20      ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   21      ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   22      ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   23      ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   24      ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   25      ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   26      ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   27      ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   28      ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   29      ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   30      ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   31      ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     0       ;}
if (variable==   32      ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   33      ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   34      ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   35      ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   36      ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   37      ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   38      ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   39      ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   40      ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   41      ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   42      ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   43      ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   44      ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   45      ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   46      ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   47      ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   48      ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   49      ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   50      ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   51      ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   52      ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   53      ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   54      ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   55      ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   56      ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   57      ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   58      ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   59      ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   60      ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   61      ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   62      ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   63      ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     0       ;}
if (variable==   64      ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   65      ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   66      ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   67      ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   68      ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   69      ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   70      ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   71      ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   72      ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   73      ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   74      ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   75      ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   76      ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   77      ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   78      ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   79      ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   80      ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   81      ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   82      ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   83      ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   84      ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   85      ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   86      ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   87      ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   88      ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   89      ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   90      ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   91      ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   92      ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   93      ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   94      ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   95      ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     0       ;}
if (variable==   96      ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   97      ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   98      ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   99      ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   100     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   101     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   102     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   103     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   104     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   105     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   106     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   107     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   108     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   109     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   110     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   111     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   112     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   113     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   114     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   115     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   116     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   117     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   118     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   119     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   120     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   121     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   122     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   123     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   124     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   125     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   126     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   127     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     0       ;}
if (variable==   128     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   129     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   130     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   131     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   132     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   133     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   134     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   135     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   136     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   137     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   138     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   139     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   140     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   141     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   142     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   143     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   144     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   145     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   146     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   147     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   148     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   149     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   150     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   151     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   152     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   153     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   154     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   155     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   156     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   157     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   158     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   159     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     0       ;h=     1       ;}
if (variable==   160     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   161     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   162     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   163     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   164     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   165     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   166     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   167     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   168     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   169     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   170     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   171     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   172     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   173     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   174     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   175     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   176     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   177     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   178     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   179     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   180     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   181     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   182     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   183     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   184     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   185     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   186     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   187     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   188     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   189     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   190     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   191     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     0       ;h=     1       ;}
if (variable==   192     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   193     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   194     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   195     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   196     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   197     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   198     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   199     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   200     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   201     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   202     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   203     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   204     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   205     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   206     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   207     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   208     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   209     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   210     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   211     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   212     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   213     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   214     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   215     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   216     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   217     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   218     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   219     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   220     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   221     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   222     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   223     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     0       ;g=     1       ;h=     1       ;}
if (variable==   224     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   225     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   226     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   227     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   228     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   229     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   230     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   231     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   232     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   233     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   234     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   235     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   236     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   237     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   238     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   239     ) {     a=      1       ; b=    1       ; c=    1       ; d=    1       ;e=      0       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   240     ) {     a=      0       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   241     ) {     a=      1       ; b=    0       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   242     ) {     a=      0       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   243     ) {     a=      1       ; b=    1       ; c=    0       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   244     ) {     a=      0       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   245     ) {     a=      1       ; b=    0       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   246     ) {     a=      0       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   247     ) {     a=      1       ; b=    1       ; c=    1       ; d=    0       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   248     ) {     a=      0       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   249     ) {     a=      1       ; b=    0       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   250     ) {     a=      0       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   251     ) {     a=      1       ; b=    1       ; c=    0       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   252     ) {     a=      0       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   253     ) {     a=      1       ; b=    0       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
if (variable==   254     ) {     a=      0       ; b=    1       ; c=    1       ; d=    1       ;e=      1       ;f=     1       ;g=     1       ;h=     1       ;}
[Advertisement] Infrastructure as Code built from the start with first-class Windows functionality and an intuitive, visual user interface. Download Otter today!

Don MartiIncentivizing production of information goods

Just thinking about approaches to incentivizing production of information goods, and where futures markets might fit in.

Artificial property

Article 1, Section 8, of the US Constitution still covers this one best.

To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries;

We know about the problems with this one. It encourages all kinds of rent-seeking and freedom-menacing behavior by the holders of property interests in information. And the transaction costs are too high to incentivize the production of some useful kinds of information.

Commoditize the complement

Joel Spolsky explained it best, in Strategy Letter V. Smart companies try to commoditize their products’ complements. (See also: the list of business models in the Some Easily Rebutted Objections to GNU's Goals section of the GNU Manifesto)

This one has been shown to work for some categories of information goods but not others. (We have Free world-class browsers and OS kernels because search engines and hardware are complements. We don't have free world-class software in categories such as CAD.)

Signaling

Release a free information good as a way to signal competence in performing a service, or at least a large investment by the author in persuading others that the author is competent. Works at the level of the individual labor market and in consulting. Don't know if this works in other areas.

Game and market mechanisms

With "gamified crowdsourcing" you can earn play rewards for very low transaction costs, and contribute very small tasks.

Common Voice

Higher transaction costs are associated with "crowdfunding" which sounds similar but requires more collaboration and administration.

In the middle, between crowdsourcing and crowdfunding, is a niche for a mechanism with lower transaction costs than crowdfunding but more rewards than crowdsourcing.

By using the existing bug tracker to resolve contracts, a bug futures market keeps transaction costs low. By connecting to an existing cryptocurrency, a bug futures market enables a kind of reward that is more liquid, and transferrable among projects.

We don't know how wide the bug futures niche is. Is it a tiny space between increasingly complex tasks that can be resolved by crowdsourcing and increasingly finer-grained crowdfunding campaigns?

Or are bug futures capable of achieving low enough transaction costs to be an attractive incentivization mechanism for a lot of tasks that go into a variety of information goods?

Don MartiGot a reply from Twitter

I thought it would be fun to try Twitter ads, and, not surprisingly, I started getting fake followers pretty quickly after I started a Twitter follower campaign.

Since I'm paying nine cents a head for these followers, I don't want to get ripped off. So naturally I put in a support ticket to Twitter, and just heard back.

Thanks for writing in about the quality of followers and engagements. One of the advantages of the Twitter Ads platform is that any RTs of your promoted ads are sent to the retweeting account's followers as an organic tweet. Any engagements that result are not charged, however followers gained may not align with the original campaign's targeting criteria. These earned followers or engagements do show in the campaign dashboard and are used to calculate cost per engagement, however you are not charged for them directly.

Twitter also passes all promoted engagements through a filtering mechanism to avoid charging advertisers for any low-quality or invalid engagements. These filters run on a set schedule so the engagements may show in the campaign dashboard, but will be deducted from the amount outstanding and will not be charged to your credit card.

If you have any further questions, please don't hesitate to reply.

That's pretty dense San Francisco speak, so let me see if I can translate to the equivalent for a normal product.

Hey, what are these rat turds doing in my raisin bran?

Thanks for writing in about the quality of your raisin bran eating experience. One of the advantages of the raisin bran platform is that during the production process, your raisin bran is made available to our rodent partners as an organic asset.

I paid for raisin bran, so why are you selling me raisin-plus-rat-turds bran?

Any ingredients that result from rodent engagement are not charged, however ingredients gained may not align with your original raisin-eating criteria.

Can I have my money back?

We pass all raisin bran sales through a filtering mechanism to avoid charging you for invalid ingredients. The total weight of the product, as printed on the box, includes these ingredients, but the weight of invalid ingredients will be deducted from the amount charged to your credit card.

So how can I tell which rat turds are "organic" so I'm not paying for them, and which are the ones that you just didn't catch and are charging me for?

(?)

Buying Twitter followers: Fiverr or Twitter?

On Fiverr, Twitter followers are about half a cent each ($5/1000). On Twitter, I'm gettting followers for about 9 cents each. The Twitter price is about 18x the Fiverr price.

But every follower that someone else buys on Fiverr has to be "aged" and disguised in order to look realistic enough not to get banned. The bot-herders have to follow legit follower campaigns such as mine and not just their paying customers.

If Twitter is selling those "follow" actions to me for nine cents each, and the bot-herder is only making half a cent, how is Twitter not making more from bogus Twitter followers than the bot-herders are?

If you're verified on Twitter, you may not be seeing how much of a shitshow their ad business is. Maybe the're going to have to sell Twitter to me sooner than I thought.

,

Planet DebianNorbert Preining: Debian/TeX Live 2017.20170724-1

Yesterday I uploaded the first update of the TeX Live packages in Debian after TeX Live 2017 has entered Debian/unstable. The packages should by now have reached most mirrors. Nothing spectacular here besides a lot of updates and new packages.

If I have to pick one update it would be the one of algorithm2e, a package that has seen lots of use and some bugs due to two years of inactivity. Good to see a new release.

Enjoy.

New packages

algolrevived, invoice2, jfmutil, maker, marginfit, pst-geometrictools, pst-rputover, pxufont, shobhika, tikzcodeblocks, zebra-goodies.

Updated packages

acmart, adobemapping, algorithm2e, arabluatex, archaeologie, babel, babel-french, bangorexam, beamer, beebe, biblatex-gb7714-2015, bibleref, br-lex, bxjscls, combofont, computational-complexity, dozenal, draftfigure, elzcards, embrac, esami, factura, fancyhdr, fei, fithesis, fmtcount, fontspec, fonttable, forest, fvextra, genealogytree, gotoh, GS1, l3build, l3experimental, l3kernel, l3packages, latexindent, limap, luapackageloader, lwarp, mcf2graph, microtype, minted, mptopdf, pdfpages, polynom, powerdot, probsoln, pxbase, pxchfon, pythontex, reledmac, siunitx, struktex, tcolorbox, tetex, texdirflatten, uowthesistitlepage, uptex-fonts, xcharter.

Planet DebianPetter Reinholdtsen: Norwegian Bokmål edition of Debian Administrator's Handbook is now available

I finally received a copy of the Norwegian Bokmål edition of "The Debian Administrator's Handbook". This test copy arrived in the mail a few days ago, and I am very happy to hold the result in my hand. We spent around one and a half year translating it. This paperbook edition is available from lulu.com. If you buy it quickly, you save 25% on the list price. The book is also available for download in electronic form as PDF, EPUB and Mobipocket, as can be read online as a web page.

This is the second book I publish (the first was the book "Free Culture" by Lawrence Lessig in English, French and Norwegian Bokmål), and I am very excited to finally wrap up this project. I hope "Håndbok for Debian-administratoren" will be well received.

Krebs on SecurityHow a Citadel Trojan Developer Got Busted

A U.S. District Court judge in Atlanta last week handed a five year prison sentence to Mark Vartanyan, a Russian hacker who helped develop and sell the once infamous and widespread Citadel banking trojan. This fact has been reported by countless media outlets, but far less well known is the fascinating backstory about how Vartanyan got caught.

For several years, Citadel ruled the malware scene for criminals engaged in stealing online banking passwords and emptying bank accounts. U.S. prosecutors say Citadel infected more than 11 million computers worldwide, causing financial losses of at least a half billion dollars.

Like most complex banking trojans, Citadel was marketed and sold in secluded, underground cybercrime markets. Often the most time-consuming and costly aspect of malware sales and development is helping customers with any tech support problems they may have in using the crimeware.

In light of that, one innovation that Citadel brought to the table was to crowdsource some of this support work, easing the burden on the malware’s developers and freeing them up to spend more time improving their creations and adding new features.

Citadel users discuss the merits of including a module to remove other parasites from host PCs.

Citadel users discuss the merits of including a module to remove other parasites from host PCs.

Citadel boasted an online tech support system for customers designed to let them file bug reports, suggest and vote on new features in upcoming malware versions, and track trouble tickets that could be worked on by the malware developers and fellow Citadel users alike. Citadel customers also could use the system to chat and compare notes with fellow users of the malware.

It was this very interactive nature of Citadel’s support infrastructure that FBI agents would ultimately use to locate and identify Vartanyan, who went by the nickname “Kolypto.” The nickname of the core seller of Citadel was “Aquabox,” and the FBI was keen to identify Aquabox and any programmers he’d hired to help develop Citadel.

In June 2012, FBI agents bought several licenses of Citadel from Aquabox, and soon the agents were suggesting tweaks to the malware that they could use to their advantage. Posing as an active user of the malware, FBI agents informed the Citadel developers that they’d discovered a security vulnerability in the Web-based interface that Citadel customers used to keep track of and collect passwords from infected systems (see screenshot below).

A screenshot of the Citadel botnet panel.

A screenshot of the Web-based Citadel botnet control panel.

Aquabox took the bait, and asked the FBI agents to upload a screen shot of the bug they’d found. As noted in this September 2015 story, the FBI agents uploaded the image to file-sharing giant Sendspace.com and then subpoenaed the logs from Sendspace to learn the Internet address of the user that later viewed and downloaded the file.

The IP address came back as the same one they had previously tied to Aquabox. The other address that accessed the file was in Ukraine and tied to Vartanyan. Prosecutors said Vartanyan’s address soon after was seen uploading to Sendspace a patched version of Citadel that supposedly fixed the vulnerability identified by the agents posing as Citadel users.

Mark Vartanyan. Source: Twitter.

Mark Vartanyan. Source: Twitter.

“In the period August 2012 to January 2013, there were in total 48 files uploaded from Marks IP to Sendspace,” reads a story in the Norwegian daily VG that KrebsOnSecurity had translated into English here (PDF). “Those files were downloaded by ‘Aquabox’ with 2 IPs (193.105.134.50 and 149.154.155.81).”

Investigators would learn that Vartanyan was a Russian citizen who’d grown up in Ukraine. At the time of his arrest, Mark was living in Norway, which later extradited him to the United States for prosecution. In March 2017, Vartanyan pleaded guilty to one count of computer fraud, and was sentenced on July 19 to five years in federal prison.

Another Citadel developer, Dimitry Belorossov (a.k.a. “Rainerfox”), was arrested and sentenced in 2015 to four years and six months in prison after pleading guilty to distributing Citadel.

Early in its heydey, some text strings were added to the Citadel Trojan which named Yours Truly as the real author of Citadel (see screenshot below). While I obviously had no involvement in writing the trojan, I have written a great deal about its core victims — mainly dozens of small businesses here in the United States who saw their bank accounts drained of hundreds of thousands or millions of dollars after a Citadel infection.

A text string inside of the Citadel trojan. Source: AhnLab

A text string inside of the Citadel trojan. Source: AhnLab

CryptogramAlternatives to Government-Mandated Encryption Backdoors

Policy essay: "Encryption Substitutes," by Andrew Keane Woods:

In this short essay, I make a few simple assumptions that bear mentioning at the outset. First, I assume that governments have good and legitimate reasons for getting access to personal data. These include things like controlling crime, fighting terrorism, and regulating territorial borders. Second, I assume that people have a right to expect privacy in their personal data. Therefore, policymakers should seek to satisfy both law enforcement and privacy concerns without unduly burdening one or the other. Of course, much of the debate over government access to data is about how to respect both of these assumptions. Different actors will make different trade-offs. My aim in this short essay is merely to show that regardless of where one draws this line -- whether one is more concerned with ensuring privacy of personal information or ensuring that the government has access to crucial evidence -- it would be shortsighted and counterproductive to draw that line with regard to one particular privacy technique and without regard to possible substitutes. The first part of the paper briefly characterizes the encryption debate two ways: first, as it is typically discussed, in stark, uncompromising terms; and second, as a subset of a broader problem. The second part summarizes several avenues available to law enforcement and intelligence agencies seeking access to data. The third part outlines the alternative avenues available to privacy-seekers. The availability of substitutes is relevant to the regulators but also to the regulated. If the encryption debate is one tool in a game of cat and mouse, the cat has other tools at his disposal to catch the mouse -- and the mouse has other tools to evade the cat. The fourth part offers some initial thoughts on implications for the privacy debate.

Blog post.

Worse Than FailureThe Logs Don't Lie

She'd resisted the call for years. As a senior developer, Makoto knew how the story ended: one day, she'd be drafted into the ranks of the manager, forswearing her true love webdev. When her boss was sacked unexpectedly, mere weeks after the most senior dev quit, she looked around and realized she was holding the short straw. She was the most senior. This is her story.

As she settled into her new responsibilities, Makoto started coming in earlier and earlier in the hopes of getting some development work done. As such, she started to get accustomed to the rhythm of the morning shift, before most devs had rolled out of bed, but after the night shift ops guys had gone home.

Bad sign number 1: the CEO wandering past, looking a bit lost and vaguely concerned.

"Can I help you?" Makoto asked, putting down her breakfast pastry.

Bad sign number 2 was his reply: "Does the Internet look down to you?"

Makoto quickly pulled up her favorite Internet test site, /r/aww, to verify that she still had connectivity. "Seems all right to me."

"Well, I can't get online."

Webdev-Makoto would've shrugged and thought, Not my circus. Manager-Makoto forced a grin onto her face and said, "I'll get my guys on that."

"Thanks, you're a real champ." Satisfied, the CEO wandered back to whatever it was he did all day, leaving Makoto to explain a problem she wasn't experiencing to guys way more qualified to work on this than she was.

Hoping to explain the discrepancy, she unplugged her laptop. This time, the adorable kittens failed to load.

"Success!" she told the empty office. "This is officially some weird wi-fi problem."

She drafted up a notice to that effect, sent it to the office mailing list, and assigned her teammate Sven to find and fix the problem. By 9:00 AM, all was well, and her team had sent out an update to that effect.

Now well into her daily routine, Makoto put the incident behind her. After all, it was resolved, wasn't it?

4:00 PM rolled around, and Makoto was somehow the recipient for an angry email from Greg in Sales. Is the internet still out? I need to close out my sales!!! Why hasn't your team fixed this yet! We could lose $300,000 if I can't close out my sales by 5PM!!!!!

Makoto rolled her eyes at the unnecessary number of exclamation points and checked the sales pipeline. Sure enough, there was nothing preventing her from accessing Greg's queue and verifying that all $100 worth of sales were present and accounted for.

Makoto cracked her knuckles and crafted the most polite response she could muster: As per my update at 9am, the Internet is back online and you should be able to perform any and all job duties at this time.

The reply came 2 minutes later: I cannot close my opportunities!!!

Makoto forwarded the email chain to Sven before rolling over to his desk. "Greg's being a drama llama again. Can you pull the firewall logs and prove he's got Internet?"

"'Course."

10 minutes and 4 raised eyebrows later, Sven replied to the ticket, copying Greg's boss and attaching a screenshot of the logs. As Makoto stated, we are online at this time. Is it possible your computer received a virus from browsing PornHub since 9:30 this morning?

Greg spent the next day in meetings with HR, and the next week on unpaid leave to think about what he'd done. To this day, he cannot look Sven or Makoto in the eye as they pass each other in the hallway. Makoto suspects he won't suffer long—only as long as it takes him to find another job. Maybe one with IT people who don't know what search keywords he uses.

[Advertisement] Scale your release pipelines, creating secure, reliable, reusable deployments with one click. Download and learn more today!

,

TEDWhat if? … and other questions that lead to big ideas: The talks of TED@UPS

Hosts Bryn Freedman and Kelly Stoetzel welcome us to the show at TED@UPS, July 20, 2017, at SCADshow in Atlanta, Georgia. (Photo: Mary Anne Morgan / TED)

What if one person could change the world? What if we could harness our collective talent, insight and wisdom? And what if, together, we could spark a movement with positive impact far into the future?

For a third year, UPS has partnered with TED to bring experts in business, logistics, design and technology to the stage to share ideas from the forefront of innovation. At this year’s TED@UPS — held on July 20, 2017, at SCADShow in Atlanta, Georgia — 18 speakers and performers showed how daring human imagination can solve our most difficult problems. 

After opening remarks from Juan Perez, UPS’s chief information and engineering officer, the talks in Session 1

Why protectionism isn’t a good deal. We’ve heard a lot of rhetoric lately suggesting that importers, like the US, are losing valuable manufacturing jobs to exporters like China, Mexico and Vietnam. In reality, those manufacturing jobs haven’t disappeared for the reasons you may think, says border and logistics specialist Augie Picado. Automation, not offshoring, is really to blame, he says; in fact, of the 5.7 million manufacturing jobs lost in the US between 2000 and 2010, 87 percent of them were lost to automation. If that trend continues, it means that future protectionist policies would save 1 in 10 manufacturing jobs, at best — but, more likely, they’d lead to tariffs and trade wars. And with the nature of modern manufacturing inexorably trending toward shared production, in which individual products are manufactured using materials produced in many different countries, protectionist policies make even less sense. Shared production allows us to manufacture higher-quality products at prices we can afford, but it’s impossible without efficient cross-border movement of materials and products. As Picado asks: “Does it make more sense to drive up prices to the point where we can’t afford basic goods, for the sake of protecting a job that might be eliminated by automation in a few years anyway?” 

Christine Thach shares her experience growing up in a refugee community — and the lessons it taught her about life and business — at TED@UPS. (Photo: Mary Anne Morgan / TED)

Capitalism for the collective. Christine Thach was raised within a tight-knit community of Cambodian refugees in the United States. Time after time, she witnessed the triumphs of community-first thinking through her own family’s hardships, steadfast relationships and continuous investment in refugee-owned businesses. “This collective-success mindset we’ve seen in refugees can actually improve the way we do business,” she says. “The self-interested foundations of capitalism, and the refugee collectivist mindset, are not in direct conflict with each other. They’re actually complementary.” Thach thinks an all-for-one, one-for-all mentality may just be able to shake up capitalism in a way that benefits everyone — if companies shift away from the individual and rally for group prosperity.

In defense of perfectionism. Some people think perfectionism is a bad thing, that it only leaves us disappointed. Jon Bowers disagrees; he sees perfectionism as “a willingness to do what is difficult to achieve what is right.” Bowers manages a facility where he trains professional delivery drivers. The stakes are high — 100 people in the US die every day in car accidents. So he’s a fan of striving to get as close to perfect as possible. We shouldn’t lower our standards because we’re afraid to fail, Bowers says. “We need to fail … failure is a natural stepping stone toward perfection.”

Uma Adwani shares the joys of teaching math at TED@UPS. (Photo: Mary Anne Morgan / TED)

Math’s hidden messages. “I hated math until it saved my life,” says Uma Adwani. As a young woman, Adwani left her small hometown of Akola, India, to start a career and life for herself in an unfamiliar city on her own. For months, she scraped by on three dollars a day — until a primary school hired her to teach the subject she loathed the most: math. But as Uma worked to prepare her lessons (and keep her job!), she started to discover “the magic of even and odd numbers, the poetry, the symmetry.” She shares the secret wisdom she found in the multiplication tables, like this one: if I am even to myself, no matter what I am multiplied with or what I go through in life, the result will always be even!

Truck driver turned activist John McKown tells sobering stories of human trafficking at TED@UPS. (Photo: Mary Anne Morgan / TED)

Activism on the road. As a small-town police officer, John McKown dealt with his share of prostitution cases. But after he left the force and became a truck driver, he faced prostitution in a new light — at truck stops. After first brushing them off as an annoyance, Bowers came to realize that the many prostitutes who go from truck to truck offering “dates” at truck stops weren’t just stuck, they were enslaved. According to the FBI, 293,000 American children are at risk of enslavement, McKown says, and now he sees it as a moral imperative to help. When he pulls into a truck stop, he’s not just looking for a parking spot; he’s looking for a way to help — and he encourages others not to turn a blind eye to this problem.

A life of awe. For artist Jennifer Allison, getting dressed can feel like rubbing against a cactus, the lights at the grocery store seem more like strobes at a disco, and the number four is always royal blue. It wasn’t until Allison was an adult that she was given a name for the strange, and often painful, way her brain processes information — Sensory Processing Disorder (SPD). Allison shares the many ways she tried to cope with her condition — from stealing cars (and returning them) to self-medication and eventually an overdose — before returning to her childhood love: art. In an intimate talk, Allison shares how art saved her life, transforming her world “from pain and chaos to mesmerizing awe and wonder.” She urges us to find what transforms our own worlds, “whether it’s through art or science, nature or religion.” Because, she explains, it’s this sense of awe that connects us to the bigger picture and each other, grounding us and making life worth living.

Johnny Staats grew up singing gospel in church and his family band. Now a UPS driver and bluegrass virtuoso, he plays music with people along his route and at Carnegie Hall. Joined by multi-instrumentalist Davey Vaughn, Staats closes out Session 1 of TED@UPS with a performance of his original song, “His Love Has Got a Hold on Me.”

Singer Stella Stevenson and pianist Danny Bauer open Session 2 by transforming the TED@UPS stage into a jazz lounge with a bold, smoky cover of “Our Day Will Come.”

What’s the point of living in the city? Leading organizations predict that by 2050, 66 percent of the population will live in cities with worsening crime, congestion and inequality. Julio Gil believes the opposite. Trends come and go, he says, and city living will eventually go, as people realize we can now get the same benefits of city while living in the countryside. With the delivery innovations and ubiquitous technology of modern life, there’s no reason not to settle outside the city for a bigger piece of land. Soon enough, he says, “city life” will able to be lived anywhere with the help of drones, social media and augmented reality. Gil challenges the TED@UPS audience to think outside big-city walls to consider the advantages of greener pastures.

Sebastian Guo heralds the arrival of the Chinese millennials — the biggest emerging consumer demographic in the world — at TED@UPS. (Photo: Mary Anne Morgan / TED)

Pay attention to Chinese millennials. The business world is obsessed with American millennials, but Sebastian Guo suggests that a different group is about to take over the world: Chinese millennials. If they were their own country, Chinese millennials would be the world’s third largest. They’re well-educated and super motivated — 57 percent have a bachelor’s degree and 23 percent have a master’s, and they’re choosing majors that give them a competitive edge, specifically STEM and business management. As the biggest emerging consumer demographic on the planet, Chinese millennials spend four times more on mobile purchases than their American counterparts. And then there are the intangibles. The Chinese are big-picture people whose thinking starts from the overview and makes its way to the specific, Guo says, which means they focus on growth and the future in the workplace. And 10 years of smartphones hasn’t erased thousands of years of Confucian ideals, which emphasize a sense of hierarchy in social relations and suggest that a Chinese millennial might be more deferential to their managers at work. The world is tilted towards China now, Guo says, and Chinese millennials are ready to be explorers in this new adventure.

Robot-proof our jobs. “Driver” is the most common job in 29 of the 50 states — and with self-driving cars on the horizon, this could quickly turn into a big problem. To keep robots from taking our jobs, innovation architect David Lee says that we should stop asking people to work like robots and let work feel like … the weekend! “Human beings are amazing on weekends,” Lee says. They’re artists, carpenters, chefs and athletes. The key is to start asking people what problems they are inspired to solve and what talents they want to bring to work. Let them lead the way. “When you invite people to be more, they can amaze us with how much more they can be,” Lee says.

Back with a welcomed musical interlude, Johnny Staats and Davey Vaughn return to the TED@UPS stage to perform an original song, “The West Virginia Coal Miner.”

How drones are revolutionizing healthcare. Partnering across disciplines, UPS joined with Zipline, Gavi and the Rwandan government to create the world’s first drone-based medical delivery system. The scalable system transports emergency medical supplies to remote villages in Rwanda. On track to its goal of saving thousands of lives a year, it could help transform how we deliver medical resources in the future as populations outgrow aging infrastructure. Learn more about this unique partnership in the mini-doc “Collaboration Lifeline,” shown for the first time at TED@UPS.

Planning happiness. City planners are already busy designing futures full of bike paths and LED-certified buildings. But are they designing for our happiness? It’s hard to define, and even harder to plan for, but urban planner Thomas Madrecki has a simple solution: Ask the public. “Our quality of life improves most when we feel engaged and empowered,” he explains, and one of the best ways planners can do this is by making public participation a priority. He calls for an “overhaul of the planning process” through public engagement, clear communication, and meetings the public actually want to attend. It’s not enough for urban planners to be trained in zoning regulations, data methods and planning history — they need to be trained in people, says Madrecki. After all, happiness and health are not engineering problems; they’re people problems.

Innovators don’t see different things; they see things differently. As a Colonel in the Air Force Reserve and an MD-11 Captain at UPS, Jeff Kozak thinks a lot about fuel, and for good reason. For his airline, fuel is by far the largest expense, at over $1.3 billion a year. Kozak tells the story of a counterintuitive idea he had to optimize fuel efficiency and cut carbon emissions by focusing on finding the exact amount of fuel needed for each plane to get to each leg of its journey. Initially met with resistance by an industry that believed more fuel was always better, the plan worked — after just ten days the airline saved $500,000 and eliminated 1,300 tons of CO2 emissions. “Let’s all continue to strive to see things differently and stay open to ideas that go against conventional thinking,” Kozak says. “Despite the resistance this type of thinking can often bring, embracing the counterintuitive can make all the difference.”

Former professional wrestler Mike Kinney encourages us all to turn ourselves up at TED@UPS. (Photo: Mary Anne Morgan / TED)

That’s me … in the chaps. How do you go from a typical high school senior to a sweaty wild man in chaps and a cowboy hat? “You turn yourself up!” says retired professional wrestler and UPS sales supervisor Mike Kinney. For years Kinney was a professional wrestler with the stage name Cowboy Gator Magraw, a persona he invented for the ring by amplifying the best parts of himself, the things about him that made him unique. In a talk equal parts funny and smart, Kinney taps into some locker-room wisdom to show us how we can all turn up to reach our full potential.

To close out the show, violinist Jessica Cambron and flutist Paige James play a moving rendition of the goodnight waltz (and Ken Burns fan favorite) “Ashokan Farewell,” accompanied by Johnny Staats and Davey Vaughn.


TEDOur podcast “Sincerely, X” co-produced with Audible now available free worldwide

Last year, TED and Audible co-produced a new audio series that invited speakers to share ideas—anonymously. Our goal was to make room for an entirely new trove of ideas: those that could only be broadcast publicly if the speaker’s identity remained private.

The series debuted with a number of powerful stories, and we learned a lot in the process (read about producer Cloe Shasha’s personal experience here).

Now, we’re bringing that first season for free to Apple Podcasts, the TED Android app, or wherever you get your podcasts.

We begin with our first episode, “Dr. Burnout,” featuring a doctor who says she committed a fatal mistake with a patient, leading her to a disturbing diagnosis: the medical field pushes for professional burnout. She unveils a powerful perspective on how doctors must deepen their self-awareness.

We’ll be releasing new episodes every Thursday for the next 10 weeks.

Fans can also access all the episodes today at audible.com/sincerelyx

 


CryptogramUS Army Researching Bot Swarms

The US Army Research Agency is funding research into autonomous bot swarms. From the announcement:

The objective of this CRA is to perform enabling basic and applied research to extend the reach, situational awareness, and operational effectiveness of large heterogeneous teams of intelligent systems and Soldiers against dynamic threats in complex and contested environments and provide technical and operational superiority through fast, intelligent, resilient and collaborative behaviors. To achieve this, ARL is requesting proposals that address three key Research Areas (RAs):

RA1: Distributed Intelligence: Establish the theoretical foundations of multi-faceted distributed networked intelligent systems combining autonomous agents, sensors, tactical super-computing, knowledge bases in the tactical cloud, and human experts to acquire and apply knowledge to affect and inform decisions of the collective team.

RA2: Heterogeneous Group Control: Develop theory and algorithms for control of large autonomous teams with varying levels of heterogeneity and modularity across sensing, computing, platforms, and degree of autonomy.

RA3: Adaptive and Resilient Behaviors: Develop theory and experimental methods for heterogeneous teams to carry out tasks under the dynamic and varying conditions in the physical world.

Slashdot thread.

And while we're on the subject, this is an excellent report on AI and national security.

Worse Than FailureCodeSOD: This or That

Processing financial transactions is not the kind of software you want to make mistakes in. If something is supposed to happen, it is definitely supposed to happen. Not partially happen. Not maybe happen.

Thus, a company like Charles R’s uses a vendor-supplied accounting package. That vendor has a professional services team, so when the behavior needs to be customized, Charles’s company outsources that development to the vendor.

Of course, years later, that code needs to get audited, and it’s about then that you find out that the vendor outsourced their “professional services” to the lowest bidder, creating a less-than-professional service result.

If you want to make sure than when the country code is equal to "HND", you want to be really sure.

if(transaction.country == config.country_code.HND || transaction.country == config.country_code.HND)
    parts[0] = parts[0].replace(/\B(?=(\d{3})+(?!\d))/g, ",");
else
    parts[0] = parts[0].replace(/\B(?=(\d{3})+(?!\d))/g, ".");
[Advertisement] Application Release Automation for DevOps – integrating with best of breed development tools. Free for teams with up to 5 users. Download and learn more today!

,

Planet Linux AustraliaOpenSTEM: This Week in HASS – term 3, week 3

This week our youngest students are playing games from different places around the world, in the past. Slightly older students are completing the Timeline Activity. Students in Years 4, 5 and 6 are starting to sink their teeth into their research project for the term, using the Scientific Process.

Foundation/Prep/Kindy to Year 3

Playing hoopsThis week students in stand-alone Foundation/Prep/Kindy classes (Unit F.3) and those integrated with Year 1 (Unit F-1.3) are examining games from the past. The teacher can choose to match these to the stories from Week 1 of the unit, as games are listed matching each of the places and time periods included in those stories. However, some games are more practical to play than others, and some require running around, so the teacher may wish to choose games which suit the circumstances of each class. Teachers can discuss how different places have different types of games and why these games might be chosen in those places (e.g. dragons in China and lions in Africa).

Students in Years 1 (Unit 1.3), 2 (Unit 2.3) and 3 (Unit 3.3) have this week to finish off the Timeline Activity. The Timeline activity requires some investment of time, which can be done as 2 half hour sessions or one longer session. Some flexible timing is built into the unit for teachers who want to match this activity to the number line in Maths, and other revise or cover the number line in more depth as a complement to this activity.

Years 3 to 6

Arthur Phillip

Last week students in Years 3 to 6 chose a research topic, related to a theme in Australian History. Different themes are studied by different year levels. Students in Year 3 (Unit 3.7) study a topic in the history of their capital city or local community. Students in Year 4 (Unit 4.3) study a topic from Australian history in the precolonial or early colonial periods. Students in Year 5 (Unit 5.3) study a topic from Australian colonial history and students in Year 6 (Unit 6.3) study a topic related to Federation or 20th century Australian history. These research topics are undertaken as a Scientific Investigation. This week the focus is on defining a Research Question and undertaking Background Research. Student workbooks will guide students through the process of choosing a research question within their chosen topic, and then how to start the Background Research. These sections will be included in the Scientific Report each student produces at the end of this unit. OpenSTEM resources available with each unit provide a starting point for this Background Research.

 

Rondam RamblingsDonald Trump shows that democracy is working. Alas.

I must confess to indulging in a certain amount of schadenfreude watching Donald Trump squirm.  I have been an unwavering never-Trumper since before he announced he was running for president.  And yet I am mindful of the fact that nearly all of the predictions I have made about Trump's political fortunes have been wrong.  In fact, while researching links for this post I realized that I wrote

Planet Linux AustraliaGabriel Noronha: test post

test posting from wordpress.com

01 – [Jul-24 13:35 API] Volley error on https://public-api.wordpress.com/rest/v1.1/sites/4046490/posts/366/?context=edit&locale=en_AU – exception: null
02 – [Jul-24 13:35 API] StackTrace: com.android.volley.ServerError
at com.android.volley.toolbox.BasicNetwork.performRequest(BasicNetwork.java:179)
at com.android.volley.NetworkDispatcher.run(NetworkDispatcher.java:114)

03 – [Jul-24 13:35 API] Dispatching action: PostAction-PUSHED_POST
04 – [Jul-24 13:35 POSTS] Post upload failed. GENERIC_ERROR: The Jetpack site is inaccessible or returned an error: transport error – HTTP status code was not 200 (403) [-32300]
05 – [Jul-24 13:35 POSTS] updateNotificationError: Error while uploading the post: The Jetpack site is inaccessible or returned an error: transport error – HTTP status code was not 200 (403) [-32300]
06 – [Jul-24 13:35 EDITOR] Focus out callback received

Don MartiMy bot parsed 12,387 RSS feeds and all I got were these links.

Bryan Alexander has a good description of an "open web" reading pipeline in I defy the world and go back to RSS. I'm all for the open web, but 40 separate folders for 400 feeds? That would drive me nuts. I'm a lumper, not a splitter. I have one folder for 12,387 feeds.

My chosen way to use RSS (and one of the great things about RSS is you can choose UX independently of information sources) is a "scored river". Something like Dave Winer's River of News concept, that you can navigate by just scrolling, but not exactly a river of news.

  • with full text if available, but without images. I can click through if I want the images.

  • items grouped by score, not feed. (Scores assigned managed by a dirt-simple algorithm where a feed "invests" a percentage of its points in every link, and the investments pay out in a higher score for that feed if the user likes a link.)

I also put the byline at the bottom of each item. Anyway, one thing I have found out about manipulating my own filter bubble is that linklog feeds and blogrolls are great inputs. So here's a linklog feed. (It's mirrored from the live site, which annoys everyone except me.)

Here are some actual links.

This might look funny: How I ran my kids like an Atlassian team for a month. But think about it for a minute. Someone at every app or site your kids use is doing the same thing, and their goals don't include "Dignity and Respect" or "Hard Work Smart Work".

Global network of 'hunters' aim to take down terrorists on the internet It took me a few days to figure things out and after a few weeks I was dropping accounts like flies…

Google's been running a secret test to detect bogus ads — and its findings should make the industry nervous. (This is a hella good idea. Legit publishers could borrow it: just go ad-free for a few minutes at random, unannounced, a couple of times a week, then send the times straight to CMOs. Did you buy ads that someone claimed ran on our site at these times? Well, you got played.)

For an Inclusive Culture, Try Working Less As I said, to this day, my team at J.D. Edwards was the most diverse I’ve ever worked on....Still, I just couldn’t get over that damned tie.

The Al Capone theory of sexual harassment Initially, the connection eluded us: why would the same person who made unwanted sexual advances also fake expense reports, plagiarize, or take credit for other people’s work?

Jon Tennant - The Cost of Knowledge But there’s something much more sinister to consider; recently a group of researchers saw fit to publish Ebola research in a ‘glamour magazine’ behind a paywall; they cared more about brand association than the content. This could be life-saving research, why did they not at least educate themselves on the preprint procedure....

Twitter Is Still Dismissing Harassment Reports And Frustrating Victims

This Is How Your Fear and Outrage Are Being Sold for Profit (Profit? What about TEH LULZ??!?!1?)

Fine, have some cute animal photos, I was done with the other stuff anyway: Photographer Spends Years Taking Adorable Photos of Rats to Break the Stigma of Rodents

,

Cory DoctorowCome see me at San Diego Comic-Con!


There are three more stops on my tour for Walkaway: tomorrow at San Diego Comic-Con, next weekend at Defcon 25 in Las Vegas, and August 10th at the Burbank Public Library.


My Comic-Con day is tomorrow/Sunday, July 23: first, a 10AM signing at the Tor Books booth (#2701); then a panel, The Future is Bleak, with Annalee Newitz, Scott Westerfeld, Scott Reintgen and Alex R. Kahler; and finally a 1:15PM signing at autographic area AA06.


(Image: Gage Skidmore, CC-BY-SA)

Don Martithe other dude

Making the rounds, this is a fun one: A computer was asked to predict which start-ups would be successful. The results were astonishing.

  • 2014: When there's no other dude in the car, the cost of taking an Uber anywhere becomes cheaper than owning a vehicle. So the magic there is, you basically bring the cost below the cost of ownership for everybody, and then car ownership goes away.

  • 2018 (?): When there's no other dude in the fund, the cost of financing innovation anywhere becomes cheaper than owning a portfolio of public company stock. So the magic there is, you basically bring the transaction costs of venture capital below the cost of public company ownership for everybody, and then public companies go away.

Could be a thing for software/service companies faster than we might think. Futures contracts on bugs→equity crowdfunding and pre-sales of tokens→bot-managed follow-on fund for large investors.

,

TEDProsthetics that feel more natural, how mushrooms may help save bees, and more

Please enjoy your roundup of TED-related news:

Prosthetics that feel more natural. A study in Science Robotics lays out a surgical technique developed by Shriya Srinivasan, Hugh Herr and others that may help prosthetics feel more like natural limbs. During an amputation, the muscle pairs that allow our brains to sense how much force is applied to a limb and where it is in space are severed, halting sensory feedback to and from the brain and affecting one’s ability to balance, handle objects and move. But nerves that send signals to the amputated limb remain intact in many amputees. Using rats, the scientists connected these nerves with muscles grafted from other parts of the body — a technique that successfully restored the muscle pair relationship and sensory feedback being sent to the brain. Combined with other research on translating nerve signals into instructions for moving the prosthetic limb, the technique could help amputees regain the ability to sense where the prosthetic is in space and the forces applied to it. They plan to begin implementing this technique in human amputees. (Watch Herr’s TED Talk)

From mathematician to politician. Emmanuel Macron wants France to be at the forefront of science, and science to be incorporated in global politics, but this is easier said than done. The election of Cédric Villani to the French National Assembly—a mathematician, Fields medalist, and TED speaker—provides a reason for optimism. “Currently, scientific knowledge within French political circles is close to zero,” Villani said in an interview with Science. “It’s important that some scientific expertise is present in the National Assembly.” Villani’s election is a step in that direction. (Watch Villani’s TED Talk)

A digital upgrade for the US government. The United States Digital Services, of which Matt Cutts is acting administrator, released its July Report to Congress. Since 2014, the USDS has worked with Silicon Valley engineers and experienced government employees to streamline federal websites and online services. Currently, the USDS is working with seven federal agencies, including the Department of Defense, the Department of Health and Human Services and the Department of Education. Ultimately, the USDS’ digital intervention is not just about reducing cost and increasing efficiency– it’s about restoring people’s trust in government. (Watch Cutts’ TED Talk)

Can mushrooms help save bees? Bee populations have been in decline for the past decade, and the consequences could be dire. But in a video for Biographic, produced by Louie Schwartzberg and including mycologist Paul Stamets, scientists discuss an unexpected solution: mushrooms. The spores and extract from Metarhizium anisopliae, a common species of mushroom, are toxic to varroa mites, the vampiric parasite which sucks blood from bees and causes colony collapse disorder. However, bees can tolerate low doses free of harm. Metarhizium anisopliae has even been shown to promote beehive longevity. This could be a step forward in curbing the mortality rate of nature’s most prolific pollinator. (Watch Schwartzberg’s TED Talk and Stamets’ TED Talk)

Support for women entrepreneurs. The World Bank Group announced its creation of The Women Entrepreneurs Finance Initiative (We-Fi), a facility that will create a $1 billion fund to support and encourage female entrepreneurship. Initiated by the U.S. and Germany, it quickly received support from other nations including Canada, Japan, Saudi Arabia and South Korea. Nearly 70% of small and medium-sized enterprises owned by women in developing countries are denied or unable to receive adequate financial services. We-Fi aims to overcome these and many other obstacles by providing early support, networking opportunities and access to markets. “Women’s economic empowerment is critical to achieve the inclusive economic growth required to end extreme poverty, which is why it has been such a longstanding priority for us,” World Bank Group President Jim Yong Kim said. “This new facility offers an unprecedented opportunity to harness both the public and private sectors to open new doors of opportunity for women entrepreneurs and women-owned firms in developing countries around the globe.” (Watch Kim’s TED Talk)

Daring to drive. Getting behind the wheel of a car is something many of us take for granted. However, as Manal al-Sharif details in her new memoir, Daring to Drive: A Saudi Woman’s Awakening, it’s not that way for everybody. The daughter of a taxi driver, al-Sharif got an education and landed a good job. The real challenge was simply getting to work—as a rule, Saudi women are not allowed to drive. Daring to Drive tells the story of her activism in the face of adversity. (Watch al-Sharif’s TED Talk)

Have a news item to share? Write us at contact@ted.com and you may see it included in this biweekly round-up.


CryptogramFriday Squid Blogging: Giant Squid Caught Off the Coast of Ireland

It's the second in two months. Video.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Read my blog posting guidelines here.

CryptogramHacking a Segway

The Segway has a mobile app. It is hackable:

While analyzing the communication between the app and the Segway scooter itself, Kilbride noticed that a user PIN number meant to protect the Bluetooth communication from unauthorized access wasn't being used for authentication at every level of the system. As a result, Kilbride could send arbitrary commands to the scooter without needing the user-chosen PIN.

He also discovered that the hoverboard's software update platform didn't have a mechanism in place to confirm that firmware updates sent to the device were really from Segway (often called an "integrity check"). This meant that in addition to sending the scooter commands, an attacker could easily trick the device into installing a malicious firmware update that could override its fundamental programming. In this way an attacker would be able to nullify built-in safety mechanisms that prevented the app from remote-controlling or shutting off the vehicle while someone was on it.

"The app allows you to do things like change LED colors, it allows you to remote-control the hoverboard and also apply firmware updates, which is the interesting part," Kilbride says. "Under the right circumstances, if somebody applies a malicious firmware update, any attacker who knows the right assembly language could then leverage this to basically do as they wish with the hoverboard."

Worse Than FailureError'd: No Thanks Necessary

"I guess we're not allowed to thank the postal carriers?!" Brian writes.

 

"So, does the CPU time mean that Microsoft has been listening to every noise I have made since before I was born?" writes Shaun F.

 

"No problem. I will not attempt to re-use your error message without permission," wrote Alex K.

 

Mark B. writes, "Ah, if only we could have this in real life."

 

"Good work Google! Another perfect translation into German," Kolja wrote.

 

"I was searching for an Atmel MCU, so I naturally opened Atmel's Product Finder. I kind of wish that I didn't," writes Michael B.,

 

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

,

Krebs on SecurityExclusive: Dutch Cops on AlphaBay ‘Refugees’

Following today’s breaking news about U.S. and international authorities taking down the competing Dark Web drug bazaars AlphaBay and Hansa Market, KrebsOnSecurity caught up with the Dutch investigators who took over Hansa on June 20, 2017. When U.S. authorities shuttered AlphaBay on July 5, police in The Netherlands saw a massive influx of AlphaBay refugees who were unwittingly fleeing directly into the arms of investigators. What follows are snippets from an exclusive interview with Petra Haandrikman, team leader of the Dutch police unit that infiltrated Hansa.

Vendors on both AlphaBay and Hansa sold a range of black market items — most especially controlled substances like heroin. According to the U.S. Justice Department, AlphaBay alone had some 40,000 vendors who marketed a quarter-million sales listings for illegal drugs to more than 200,000 customers. The DOJ said that as of earlier this year, AlphaBay had 238 vendors selling heroin. Another 122 vendors advertised Fentanyl, an extremely potent synthetic opioid that has been linked to countless overdoses and deaths.

In our interview, Haandrikman detailed the dual challenges of simultaneously dealing with the exodus of AlphaBay users to Hansa and keeping tabs on the giant increase in new illicit drug orders that were coming in daily as a result.

The profile and feedback of a top AlphaBay vendor.

The profile and feedback of a top AlphaBay vendor. Image: ShadowDragon.io

KrebsOnSecurity (K): Talk a bit about how your team was able to seize control over Hansa.

Haandrikman (H): When we knew the FBI was working on AlphaBay, we thought ‘What’s better than if they come to us?’ The FBI wanted [the AlphaBay takedown] to look like an exit scam [where the proprietors of a dark web marketplace suddenly abscond with everyone’s money]. And we knew a lot of vendors on AlphaBay would probably come over to Hansa when AlphaBay was closed.

K: Where was Hansa physically based?

H: We knew the Hansa servers were in Lithuania, so we sent an MLAT (mutual legal assistance treaty) request to Lithuania and requested if we could proceed with our planned actions in their country. They were very willing to help us in our investigations.

K: So you made a copy of the Hansa servers?

H: We gained physical access to the machines in Lithuania, and were able to set up some clustering between the [Hansa] database servers in Lithuania and servers we were running in our country. With that, we were able to get a real time copy of the Hansa database, and then copy over the Web site code itself.

K: Did you have to take Hansa offline for a while during this process?

H: No, it didn’t really go offline. We were able to create our own copy of the site that was running on servers in the Netherlands. So there were two copies of the site running simultaneously.

The now-defunct Hansa Market.

The now-defunct Hansa Market.

K: At a press conference on this effort at the U.S. Justice Department in Washington, D.C. today, Rob Wainwright, director of the European law enforcement organization Europol, detailed how the closure of AlphaBay caused a virtual stampede of former AlphaBay buyers and sellers taking their business to Hansa Market. Tell us more about what that influx was like, and how you handled it.

H: Yes, we called them “AlphaBay refugees.” It wasn’t the technical challenge that caused problems. Because this was a police operation, we wanted to keep up with the orders to see if there were any large amounts [of drugs] being ordered to one place, [so that] we could share information with our law enforcement partners internationally.

K: How exactly did you deal with that? Were you able to somehow slow down the orders coming in?

H: We just closed registration on Hansa for new users for a few days. So there was a temporary restriction for being able to register on the site, which slowed down the orders each day to make sure that we could cope with the orders that were coming in.

K: Did anything unexpected happen as a result?

H: Some people started selling their Hansa accounts on Reddit. I read somewhere that one Hansa user sold his account for $40. The funny part about that was that sale happened about five minutes before we re-opened registration. There was a lot of frustration from ex-AlphaBay users that weren’t allowed to register on the site. But we also got defended by the Hansa community on social media, who said it was a great decision by us to educate certain AlphaBay users on Hansa etiquette, which doesn’t allow the sale of things permitted on AlphaBay and other dark markets, such as child pornography and firearms.

A message from Dutch authorities listing the top dark market vendors by nickname.

A message from Dutch authorities listing the top dark market vendors by nickname.

K: You mentioned earlier that the FBI wanted AlphaBay users to think that the reason for the closure of that marketplace was that its operators and administrators had conducted an ‘exit scam’ where they ran off with all of the Bitcoin and virtual currency that vendors and buyers had stored in their marketplace wallets temporarily. Why do you think they wanted this to look like an exit scam?

H: The idea was to hit the dark markets even harder when they think they’re just moving to another market and it turns to be law enforcement. Breaking the trust, so that [users] would not feel safe on a dark market.

K: It has been reported that just a few days ago the Hansa market administrators decided to ban the sale of Fentanyl. Were Dutch police involved in that at all?

H: It was a combination of things. One of the site’s employees or moderators started a discussion about this drug. We obviously also had our own opinion about it. It was a pretty good dialogue between us and the Hansa moderators to ban this from the site, and [that decision received] a lot of support from the community. But we didn’t instigate that discussion.

K: Have the Dutch police arrested anyone in connection with this investigation so far?

H: Yes, we identified several people in the Netherlands using the site, and there have already been several arrests made [tied to] Fentanyl.

K: Can you talk about whether your control over Hansa helped you identify users?

H: We did use some technical tricks to find out who people are, but we can’t go into that a lot because the investigation is still going on. But we did try to change the behavior [of some Hansa users] by asking for things that helped us to identify a lot of people and money.

K: What is your overall strategy in all of this?

H: Our strategy is that we want people to know that the Dark Web is not an anonymous place for criminals. Don’t think you can just buy or sell your drugs there without eventually getting caught by law enforcement. We want people to know you’re not safe on the Dark Web. Sooner or later we will come to get you.

Further reading: After AlphaBay’s Demise, Customers Flocked to Dark Market Run by Dutch Police

Krebs on SecurityAfter AlphaBay’s Demise, Customers Flocked to Dark Market Run by Dutch Police

Earlier this month, news broke that authorities had seized the Dark Web marketplace AlphaBay, an online black market that peddled everything from heroin to stolen identity and credit card data. But it wasn’t until today, when the U.S. Justice Department held a press conference to detail the AlphaBay takedown that the other shoe dropped: Police in The Netherlands for the past month have been operating Hansa Market, a competing Dark Web bazaar that enjoyed a massive influx of new customers immediately after the AlphaBay takedown.

The normal home page for the dark Web market Hansa has been replaced by this message from U.S. law enforcement authorities.

The normal home page for the dark Web market Hansa has been replaced by this message from U.S. law enforcement authorities.

U.S. Attorney General Jeff Sessions called the AlphaBay closure “the largest takedown in world history,” targeting some 40,000 vendors who marketed a quarter-million listings for illegal drugs to more than 200,000 customers.

“By far, most of this activity was in illegal drugs, pouring fuel on the fire of a national drug epidemic,” Sessions said. “As of earlier this year, 122 vendors advertised Fentanyl. 238 advertised heroin. We know of several Americans who were killed by drugs on AlphaBay.”

Andrew McCabe, acting director of the FBI, said AlphaBay was roughly 10 times the size of the Silk Road, a similar dark market that was shuttered in a global law enforcement sting in October 2013.

As impressive as those stats may be, the real coup in this law enforcement operation became evident when Rob Wainwright, director of the European law enforcement organization Europol, detailed how the closure of AlphaBay caused a virtual stampede of former AlphaBay buyers and sellers taking their business to Hansa Market, which had been quietly and completely taken over by Dutch police one month earlier — on June 20.

“What this meant…was that we could identify and disrupt the regular criminal activity that was happening on Hansa Market but also sweep up all of those new users that were displaced from AlphaBay and looking for a new trading plot form for their criminal activities,” Wainwright told the media at today’s press conference, which seemed more interested in asking Attorney General Sessions about a recent verbal thrashing from President Trump.

“In fact, they flocked to Hansa in droves,” Wainwright continued. “We recorded an eight times increase in the number of human users on Hansa immediately following the takedown of AlphaBay. Since the undercover operation to take over Hansa market by the Dutch Police, usernames and passwords of thousands of buyers and sellers of illicit commodities have been identified and are the subject of follow-up investigations by Europol and our partner agencies.”

On July 5, the same day that AlphaBay went offline, authorities in Thailand arrested Alexandre Cazes — a 25-year-old Canadian citizen living in Thailand — on suspicion of being the creator and administrator of AlphaBay. He was charged with racketeering, conspiracy to distribute narcotics, conspiracy to commit identity theft and money laundering, among other alleged crimes.

Alexandre Cazes, standing in front of one of four Lamborghini sports cars he owned. Image: Hanke.io.

Alexandre Cazes, standing in front of one of four Lamborghini sports cars he owned. Image: Hanke.io.

Law enforcement authorities in the US and abroad also seized millions of dollars worth of Bitcoin and other assets allegedly belonging to Cazes, including four Lamborghini cars and three properties.

However, law enforcement officials never got a chance to extradite Cazes to the United States to face trial. Cazes, who allegedly went by the nicknames “Alpha02” and “Admin,” reportedly committed suicide while still in custody in Thailand.

Online discussions dedicated to the demise of AlphaBay, Hansa and other Dark Web markets — such as this megathread over at Reddit — observe that law enforcement officials may have won this battle with their clever moves, but that another drug bazaar will simply step in to fill the vacuum.

But Ronnie Tokazowski, a senior analyst at New York City-based threat intelligence firm Flashpoint, said the actions by the Dutch and American authorities could make it more difficult for established vendors from AlphaBay and Hansa to build a presence using the same identities at alternative Dark Web marketplaces.

Vendors on Dark Web markets tend to re-use the same nickname across multiple marketplaces, partly so that other cybercriminals won’t try to assume and abuse their good names on other forums, but also because a reputation for quality customer service means everything on these marketplaces and is worth a pretty penny.

But Tokazowski said even if top vendors from AlphaBay/Hansa already have a solid reputation among buyers on other marketplaces, some of those vendors may choose to walk away from their former identities and start anew.

“One of the things [the Dutch Police and FBI] mentioned was they were going after other markets using some of the several thousand password credentials they had from AlphaBay and Hansa, as a way to get access to vendor accounts,” on other marketplaces, he said. “These actions are really going to have a lot of people asking who they can trust.”

A message from Dutch authorities listing the top dark market vendors by nickname.

A message from Dutch authorities listing the top dark market vendors by nickname.

“There are dozens of these Dark Web markets, people will start to scatter to them, and it will be interesting to see who steps up to become the next AlphaBay,” Tokazowski continued. “But if people were re-using usernames and passwords across dark markets, it’s going to be a bad day for them. And from a vendor perspective, [the takedowns] make it harder for sellers to transfer reputation to another market.”

For more on how the Dutch Police’s National High Tech Crimes Unit (NHTCU) quietly assumed control over the Hansa Market, check out this story.

This story may be updated throughout the day (as per usual, any updates will be noted with a timestamp). In the meantime, the Justice Department has released a redacted copy of the indictment against Cazes (PDF), as well as a forfeiture complaint (PDF).

Update, 4:00 p.m. ET: Added perspectives from Flashpoint, and link to exclusive interview with the leader of the Dutch police unit that infiltrated Hansa.

CryptogramEthereum Hacks

The press is reporting a $32M theft of the cryptocurrency Ethereum. Like all such thefts, they're not a result of a cryptographic failure in the currencies, but instead a software vulnerability in the software surrounding the currency -- in this case, digital wallets.

This is the second Ethereum hack this week. The first tricked people in sending their Ethereum to another address.

This is my concern about digital cash. The cryptography can be bulletproof, but the computer security will always be an issue.

Worse Than FailureFinding the Lowest Value

Max’s team moved into a new office, which brought with it the low-walled, “bee-hive” style cubicle partitions. Their project manager cheerfully explained that the new space “would optimize collaboration”, which in practice meant that every random conversation between any two developers turned into a work-stopping distraction for everyone else.

That, of course, wasn’t the only change their project manager instituted. The company had been around for a bit, and their original application architecture was a Java-based web application. At some point, someone added a little JavaScript to the front end. Then a bit more. This eventually segregated the team into two clear roles: back-end Java developers, and front-end JavaScript developers.

An open pit copper mine

“Silos,” the project manager explained, “are against the ethos of collaboration. We’re all going to be full stack developers now.” Thus everyone’s job description and responsibilities changed overnight.

Add an overly ambitious release schedule and some unclear requirements, and the end result is a lot of underqualified developers rushing to hit targets with tools that they don’t fully understand, in an environment that isn’t conducive to concentration in the first place.

Max was doing his best to tune out the background noise, when Mariella stopped into Dalton’s cube. Dalton, sitting straight across from Max, was the resident “front-end expert”, or at least, he had been before everyone was now a full-stack developer. Mariella was a long-time backend JEE developer who hadn’t done much of the web portion of their application at all, and was doing her best to adapt to the new world.

“Dalton, what’s the easiest way to get the minimum value of an array of numbers in JavaScript?” Mariella asked.

Max did his best to ignore the conversation. He was right in the middle of a particularly tricky ORM-related bug, and was trying to figure out why one fetch operation was generating just awful SQL.

“Hrmmmm…” Dalton said, tapping at his desk and adding to the distraction while he thought. “That’s a tough one. Oh! You should use a filter!”

“A filter, what would I filter on?”

Max combed through the JPA annotations that controlled their data access, cursing the “magic” that generated SQL queries, but as he started to piece it together, Dalton and Mariella continued their “instructional” session.

“In the filter callback, you’d just check to see if each value is the lowest one, and if it is, return true, otherwise return false.” Dalton knocked out a little drum solo on his desk, to celebrate his cleverness.

“But… I wouldn’t know which value is the lowest one, yet,” Mariella said.

“Oh, yeah… I see what you mean. Yeah, this is a tricky one.”

Max traced through the code. Okay, so the @JoinColumn is CUST_ID, so why is it generating a LIKE comparison instead of an equals? Wait, I think I’ve-

“Ah ha!” Dalton said, chucking Max’s train of thought off the rails and through an HO-scale village. “You just sort the array and take the first value!” *Thumpa thumpa tadatada* went Dalton’s little desk drum solo.

“I guess that makes sense,” Mariella said.

At this point, Max couldn’t stay out of the conversation. “No! Don’t do that. Use reduce. Sorting’s an n(lg n) operation.”

“Hunh?” Dalton said. His fingers nervously hovered over his desk, ready to play his next drum solo once he had a vague clue what Max was talking about. “In logs in? We’re not doing logging…”

Max tried again, in simple English. “Sorting is slow. The computer does a lot of extra work to sort all the elements.”

“No it won’t,” Dalton said. “It’ll just take the first element.”

“Ahem.” Max turned to discover the project manager looming over his cube. “We want to encourage collaboration,” the PM said, sternly, “but right now, Max, you’re being disruptive. Please be quiet and let the people around you work.”

And that was how Dalton’s Minimum Finding Algorithm got implemented, and released as part of their production code base.

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Planet Linux AustraliaOpenSTEM: New Dates for Earliest Archaeological Site in Aus!

Thylacine or Tasmanian Tiger.

This morning news was released of a date of 65,000 years for archaeological material at the site of Madjedbebe rock shelter in the Jabiluka mineral lease area, surrounded by Kakadu National Park. The site is on the land of the Mirarr people, who have partnered with archaeologists from the University of Queensland for this investigation. It has also produced evidence of the earliest use of ground-stone tool technology, the oldest seed-grinding tools in Australia and stone points, which may have been used as spears. Most fascinating of all, there is the jawbone of a Tasmanian Tiger or Thylacine (which was found across continental Australia during the Ice Age) coated in a red pigment, thought to be the reddish rock, ochre. There is much evidence of use of ochre at the site, with chucks and ground ochre found throughout the site. Ochre is often used for rock art and the area has much beautiful rock art, so we can deduce that these rock art traditions are as old as the occupation of people in Australia, i.e. at least 65,000 years old! The decoration of the jawbone hints at a complex realm of abstract thought, and possibly belief, amongst our distant ancestors – the direct forebears of modern Aboriginal people.

Kakadu view, NT Tourism.

Placing the finds from Madjebebe rock shelter within the larger context, the dating, undertaken by Professor Zenobia Jacobs from the University of Wollongong, shows that people were living at the site during the Ice Age, a time when many, now-extinct, giant animals roamed Australia; and the tiny Homo floresiensis was living in Indonesia. These finds show that the ancestors of Aboriginal people came to Australia with much of the toolkit of their rich, complex lives already in place. This technology, extremely advanced for the time, allowed them to populate the entire continent of Australia, first managing to survive in the hash Ice Age environment and then also managing to adapt to the enormous changes in sea level, climate and vegetation at the end of the Ice Age.

The team of archaeologists working at Madjebebe rock shelter, in conjunction with Mirarr traditional owners, are finding all sorts of wonderful archaeological material, from which they can deduce much rich, detailed information about the lives of the earliest people in Australia. We look forward to hearing more from them in the future. Students who are interested, especially those in Years 4, 5 and 6, can read more about these sites and the animals and lives of people in Ice Age Australia in our resources People Reach Australia, Early Australian Sites, Ice Age Animals and the Last Ice Age, which are covered in Units 4.1, 5.1 and 6.1.

TED10 books from TEDWomen for your summer reading list — and beyond

There’s no doubt that the speakers we invite to TEDWomen each year have amazing stories to tell. And many of them are published authors (or about to be!) whose work is worth exploring beyond their brief moments in the TED spotlight. So, if you’re looking for some inspiring, instructive and provocative books to add to your summer reading list, these recent books from 2016 TEDWomen speakers are worthy additions.

1. Beyond Respectability: The Intellectual Thought of Race Women by Brittney Cooper

Brittney Cooper wowed us at TEDWomen with her presentation on the racial politics of time. And in her new book, Beyond Respectability: The Intellectual Thought of Race Women, released in May, she doesn’t disappoint. Brittney says she got started studying black women intellectuals in graduate school. Although she learned a lot about the histories of black male intellectuals as an undergrad at Howard University, she “somehow managed not to learn anything about” the storied history of black women intellectuals in her four years there.

In her book, Brittney looks at the far-reaching intellectual achievements of female thinkers and activists like Ida B. Wells, Anna Julia Cooper, Mary Church Terrell, Fannie Barrier Williams, Pauli Murray and Toni Cade Bambara. NPR’s Genevieve Valentine writes that Brittney’s book is “a work of crucial cultural study … [that] lays out the complicated history of black woman as intellectual force, making clear how much work she has done simply to bring that category into existence.”

2. South of Forgiveness by Thordis Elva and Tom Stranger

One of the most intensely personal talks in San Francisco came from Thordis Elva and Tom Stranger. In 1996, 16-year-old Thordis shared a teenage romance with Tom, an exchange student from Australia. After a school dance, Tom raped Thordis. They didn’t speak for many years. Then, in her twenties, Thordis wrote to Tom, wanting to talk about what he did to her, and remarkably, he responded. For the first time, in front of the TEDWomen audience, Thordis and Tom talked openly about what happened and why she wanted to talk to him, and he to her.

South of Forgiveness: A True Story of Rape and Responsibility is a profoundly moving, open-chested and critical book. It is an exploration into sexual violence and self-knowledge that shines a healing light into the shrouded corners of our universal humanity. There is a disarming power in these pages that has the potential to change our language, shift our divisions, and invite us to be brave in discussing this pressing, global issue.

3. Girls & Sex by Peggy Orenstein

In a TED Talk that has already been viewed over 1.5 million times, author and journalist Peggy Orenstein, shared some of the things she learned about young girls and how they think about sex while researching her 2016 book, Girls & Sex: Navigating the Complicated New Landscape. In it, she explores the changing landscape of modern sexual expectations and its troubling impact on adolescents and especially young women. If you’re the parent of a young girl (or boy), it’s a must-read for understanding the “hidden truths, hard lessons, and important possibilities of girls’ sex lives in the modern world.”

4. Born Bright by C. Nicole Mason

At TEDWomen, C. Nicole Mason talked about what happens when we disrupt the path that society has paved for us based on where we were born, stereotypes and stigma. In her memoir, Born Bright: A Young Girl’s Journey from Nothing to Something in America, Nicole talks about how she did it in her own life, chronicling her own path out of poverty. In a beautifully written book, she examines “the conditions that make it nearly impossible to escape” and her own struggles with feeling like an outsider in academia and professional settings because of the way she talked, dressed and wore her hair.

5. The Gutsy Girl by Caroline Paul

Caroline Paul has a pretty amazing backstory. Once a young self-described “scaredy-cat,” Caroline grew up to fly planes, raft rivers, climb mountains, and fight fires. That’s right, she was one of the first women to work for the San Francisco Fire Department — a job that inspired her first work of nonfiction, Fighting Fire. In her most recent book, The Gutsy Girl: Escapades for Your Life of Epic Adventure, she expands on some of the stories she shared in her TED Talk, writing about “her greatest escapades — as well as those of other girls and women from throughout history.”

6. Marrow: A Love Story by Elizabeth Lesser

In a beautiful and surprisingly funny talk about strained family relationships and the death of a loved one, Elizabeth Lesser described the healing process of putting aside pride and defensiveness to make way for honest communication. “You don’t have to wait for a life-or-death situation to clean up the relationships that matter to you,” she says. “Be like a new kind of first responder … the one to take the first courageous step toward the other.”

In her courageous memoir, Marrow: A Love Story, the bestselling author of Broken Open shares the full story of her sister Maggie’s cancer and the difficult conversations they had during her illness as they healed their imperfect relationship and learned to love each other’s true selves.

7. I Know How She Does It by Laura Vanderkam

The theme of last year’s TEDWomen, as many of you will recall, was Time — all of us wrestle with how to be more productive, more engaged, more informed, to use our time wisely and well, to be more fully present in our lives. Writer and author Laura Vanderkam tackled the practical aspects of time management in her TED Talk. There are 168 hours in each week. How do we find time for what matters most?

In her book I Know How She Does It, Laura explains how successful women make the most of their time. With research, hard data and a lot of analysis, Laura “offers a framework for anyone who wants to thrive at work and life.”

8. Always Another Country by Sisonke Msimang

In her work, South African writer and activist Sisonke Msimang untangles the threads of race, class and gender that run through the fabric of African and global culture. In her popular TED Talk, she addressed the power of stories to promote change in our world and their “limitations, particularly for those of us who are interested in social justice.”

I am so pleased to report that after a very competitive bidding war, Sisonke will be publishing her first book, to be titled Always Another Country, in October.  The book, a memoir, will cover “her childhood in exile in Zambia and Kenya, her young adulthood and student years in North America and her return to South Africa during the euphoria of the 1990s.” I am so looking forward to reading her book and so should you.

9. When They Call You a Terrorist by Patrisse Cullors

Patrisse Cullors, one of the three co-founders of Black Lives Matter, is also working on a memoir due out in January 2018 titled When They Call You a Terrorist. Activist Eve Ensler writes that Patrisse “is a leading visionary and activist, feminist, civil rights leader who has literally changed the trajectory of politics and resistance in America.” Co-written with asha bandele, the memoir will recount the founding of the movement and serve as a reminder “that protest in the interest of the most vulnerable comes from love.”

10. On Intersectionality: Essential Writings by Kimberlé Crenshaw

Civil rights advocate Kimberlé Crenshaw had the TEDWomen audience on their feet during her passionate talk dissecting intersectionality, a term she coined 20 years ago that describes the double bind faced by victims of simultaneous racial and gender prejudice. “What do you call being impacted by multiple forces and then abandoned to fend for yourself?” she asked the audience. “Intersectionality seemed to do it for me.”

In a new collection of her writing, titled On Intersectionality: Essential Writings, due to be released next year, “readers will find the key essays and articles that have defined the concept of intersectionality and made Crenshaw a legal superstar.” Don’t miss it.

TEDWomen 2017

I also want to mention that registration for TEDWomen 2017 is open, so if you haven’t registered yet, please click this link and apply today — space is limited. This year, TEDWomen will be held November 1–3 in New Orleans. The theme is Bridges: We build them, we cross them, and sometimes we even burn them. We’ll explore the many aspects of this year’s theme through curated TED Talks, community dinners and activities.

Join us!
– Pat

Featured image: Reading a book at the beach (Simon Cocks, Flickr CC 2.0)


,

Cory DoctorowRudy Rucker on Walkaway



Walkaway is my first novel for adults since 2009 and I had extremely high hopes (and not a little anxiety) for it as it entered the world, back in April. Since then, I’ve been gratified by the kind words of many of my literary heroes, from William Gibson to Bruce Sterling to the kind cover quotes from Edward Snowden, Neal Stephenson and Kim Stanley Robinson.


Today I got a most welcome treat on those lines: a review by Rudy Rucker, lavishly illustrated with some of his excellent photos. Rucker really got the novel, got excited about the parts that excited me, and you can’t really ask for better than that.

“I’m groundhog daying again, aren’t I?”

Who’s saying this? It’s the character Dis. Her body is dead, but before she died, they managed (thanks to Dis’s work) to copy or transfer the brain processes into the cloud, that is, into a network of computers. And she can run as a sim in there. And she’s having trouble getting her sim to stabilize. It keeps freaking out and crashing. And each time she restarts the character Iceweasel sits there talking to the computer sim, trying to mellow it out, and Dis will realize she’s been rebooted, or restarted like Bill Murray in that towering cinematic SF masterpiece Groundhog Day. And Cory has the antic wit to make that verb.

The first half of the book is kind of a standard good young people against evil corporate rich people thing. But then, when Dis is talking about groundhog dayhing, it kicks into another gear. Cory pulls out a different stop on the mighty SF Wurlitzer organ: the software immortality trope. As I’m fond of saying, in my 1980 novel Software, I became one of the very first authors to write about the by-now-familiar notion of the mind as software. That is, your mind is in some sense like software running on your physical body. If we could create a sufficiently rich and flexible computer, the computer might be able to emulate a person.

There’s been a zillion movies, TV shows, SF stories and novels using this idea since then. What I liked so much about Walkaway is that Cory finds a way to make this (still fairly fantastic and unlikely) idea seem real and new.

Cory Doctorow’s WALKAWAY [Rudy Rucker]

LongNowInterview: Alexander Rose and Phil Libin on Long-Term Thinking

Long Now Executive Director Alexander Rose and former Evernote CEO Phil Libin recently spoke with the design agency Dialogue about the layers of civilization, the future of products, and the Clock of the Long Now.

The interview is wide-ranging, covering everything from the early tech, design and science fiction influences in Rose and Libin’s childhoods to how Long Now’s pace layers theory helps reconcile the tension between long-term planning and Silicon Valley’s fast-paced approach to entrepreneurship and product innovation.

The interview also provides a look at a little-known chapter in Long Now’s history, namely, how Alexander Rose left a career in video games and virtual world design after hearing about The Clock Project:

Stewart told me about The Clock Project. Back then the project was just a conversation between Danny Hillis, Brian Eno, and Stewart, but I just couldn’t get it out of my head when I heard about it. By strange luck, there was a Board meeting a week after where I met Danny for the first time. It was then that he told me he had a funder for the first prototype of the Clock and asked if I wanted to help build it. I immediately said, “Yes, this is what I want to do. I don’t want to work on video games anymore.”

Read Dialogue’s interview with Alexander Rose and Phil Libin in full (LINK).

Watch Stewart Brand and Long Now board member Paul Saffo discuss the Pace Layers of Civilization in a 02015 Conversation at The Interval (LINK).

Krebs on SecurityTrump Hotels Hit By 3rd Card Breach in 2 Years

Maybe some of you missed this amid all the breach news recently (I know I did), but Trump International Hotels Management LLC last week announced its third credit-card data breach in the past two years. I thought it might be useful to see these events plotted on a timeline, because it suggests that virtually anyone who used a credit card at a Trump property in the past two years likely has had their card data stolen and put on sale in the cybercrime underground as a result.

On May 2, 2017, KrebsOnSecurity broke the story that travel industry giant Sabre Corp. experienced a significant breach of its payment and customer data tied to bookings processed through a reservations system that serves more than 32,000 hotels and other lodging establishments. Last week, Trump International Hotels disclosed the SABRE breach impacted at least 13 Trump Hotel properties between August 2016 and March 2017. Trump Hotels said it was first notified of the breach on June 5.

A timeline of Trump Hotels’ credit card woes over the past two years. Click to enlarge.

According to Verizon‘s latest annual Data Breach Investigations Report (DBIR), malware attacks on point-of-sale systems used at front desk and hotel restaurant systems “are absolutely rampant” in the hospitality sector. Accommodation was the top industry for point-of-sale intrusions in this year’s data, with 87% of breaches within that pattern.

Other hotel chains that disclosed this past week getting hit in the Sabre breach include 11 Hard Rock properties (another chain hit by multiple card breach incidents); Four Seasons Hotels and Resorts; and at least two dozen Loews Hotels in the United States and Canada.

ANALYSIS/RANT

Given its abysmal record of failing to protect customer card data, you might think the hospitality industry would be anxious to assuage guests who may already be concerned that handing over their card at the hotel check-in desk also means consigning that card to cybercrooks (e.g. at underground carding shops like Trumps Dumps).

However, so far this year I’ve been hard-pressed to find any of the major hotel chains that accept more secure chip-based cards, which are designed to make card data stolen by point-of-sale malware and skimmers much more difficult to turn into counterfeit cards. I travel quite a bit — at least twice a month — and I have yet to experience a single U.S.-based hotel in the past year asking me to dip my chip-based card as opposed to swiping it.

A carding shop that sells stolen credit cards and invokes 45's likeness and name. No word yet on whether this cybercriminal store actually sold any cards stolen from Trump Hotel properties.

A carding shop that sells stolen credit cards and invokes 45’s likeness and name. No word yet on whether this cybercriminal store actually sold any cards stolen from Trump Hotel properties.

True, chip cards alone aren’t going to solve the whole problem. Hotels and other merchants that implement the ability to process chip cards still need to ensure the data is encrypted at every step of the transaction (known as “point-to-point” or “end-to-end” encryption). Investing in technology like tokenization — which allows merchants to store a code that represents the customer’s card data instead of the card data itself — also can help companies become less of a target.

Maybe it wouldn’t be so irksome if those of us concerned about security or annoyed enough at getting our cards replaced three or four times a year due to fraud could stay at a major hotel chain in the United States and simply pay with cash. But alas, we’re talking about an industry that essentially requires customers to pay by credit card.

Well, at least I’ll continue to accrue reward points on my credit card that I can use toward future rounds of Russian roulette with the hotel’s credit card systems.

It’s bad enough that cities and states routinely levy huge taxes on lodging establishments (the idea being the tax is disproportionately paid by people who don’t vote or live in the area); now we have the industry-wide “carder tax” conveniently added to every stay.

What’s the carder tax you ask? It’s the sense of dread and the incredulous “really?” that wells up when one watches his chip card being swiped yet again at the check-out counter.

It’s the time wasted on the phone with your bank trying to sort out whether you really made all those fraudulent purchases, and then having to enter your new card number at all those sites and services where the old one was stored. It’s that awkward moment when the waiter says in front of your date or guests that your card has been declined.

If you’re brave enough to pay for everything with a debit card (bad idea), it may be the time you spend without access to cash while your bank sorts things out. It may be the aggravation of dealing with bounced checks as a result of the fraud.

I can recall a recent stay wherein right next to the credit card machine at the hotel’s front desk was a stack of various daily newspapers, one of which had a very visible headline warning of an ongoing credit card breach at the same hotel that was getting ready to swipe my card yet again (by the way, I’m still kicking myself for not snapping a selfie right then).

After I checked out of that particular hotel, I descended to the parking garage to retrieve a rental car. The garage displayed large signs everywhere warning customers that the property was not responsible for any damage or thefts that may be inflicted on vehicles parked there. I recall thinking at the time that this same hotel probably should have been required to display a similar sign over their credit card machines (actually, they all should).

“The privacy and protection of our guests’ information is a matter we take very seriously.” This is from boilerplate text found in both the Trump Hotels and Loews Hotel statements. It sounds nice. Too bad it’s all hogwash. Once again, the timeline above speaks far more about the hospitality industry’s attitudes on credit card security than any platitudes offered in these all-too-common breach notifications.

Further reading:

Banks: Card Breach at Trump Hotel Properties
Trump Hotel Collection Confirms Card Breach
Sources: Trump Hotels Breached Again
Trump Hotels Settles Over Data Breach: To Pay $50,000 for 70,000 Stolen Cards
Breach at Sabre Corp.’s Hospitality Unit

CryptogramPassword Masking

Slashdot asks if password masking -- replacing password characters with asterisks as you type them -- is on the way out. I don't know if that's true, but I would be happy to see it go. Shoulder surfing, the threat is defends against, is largely nonexistent. And it is becoming harder to type in passwords on small screens and annoying interfaces. The IoT will only exacerbate this problem, and when passwords are harder to type in, users choose weaker ones.

Worse Than FailureCodeSOD: A Pre-Packaged Date

Microsoft’s SQL Server Integration Services is an ETL tool that attempts to mix visual programming (for designing data flows) with the reality that at some point, you’re just going to need to write some code. Your typical SSIS package starts as a straightforward process that quickly turns into a sprawling mix of spaghetti-fied .NET code, T-SQL stored procedures, and developer tears.

TJ L. inherited an SSIS package. This particular package contained a step where a C# sub-module needed to pass a date (but not a date-time) to the database. Now, this could be done easily by using C#’s date-handling objects, or even in the database by simply using the DATE type, instead of the DATETIME type.

Instead, TJ’s predecessor took this route instead:

CREATE PROC [dbo].[SetAsOfDate]
        @Date datetime = NULL
AS
SELECT @Date = CASE WHEN YEAR(@DATE) < 1950 THEN GETDATE()
                                        WHEN @Date IS NULL THEN GETDATE()
                                        ELSE @Date
                                END;

SELECT CAST(FLOOR(CAST(@Date AS FLOAT)) AS DATETIME) AS CurrentDate

The good about this code is that it checks its input parameters. That’s defensive programming. The ugly is the less-than 1950 check, which I can only assume is a relic of some Y2K bugfixes. The bad is the `CAST(FLOOR(CAST(@Date AS FLOAT)) as DATETIME).

[Advertisement] Otter, ProGet, BuildMaster – robust, powerful, scalable, and reliable additions to your existing DevOps toolchain.

,

Harald WelteVirtual Um interface between OsmoBTS and OsmocomBB

During the last couple of days, I've been working on completing, cleaning up and merging a Virtual Um interface (i.e. virtual radio layer) between OsmoBTS and OsmocomBB. After I started with the implementation and left it in an early stage in January 2016, Sebastian Stumpf has been completing it around early 2017, with now some subsequent fixes and improvements by me. The combined result allows us to run a complete GSM network with 1-N BTSs and 1-M MSs without any actual radio hardware, which is of course excellent for all kinds of testing scenarios.

The Virtual Um layer is based on sending L2 frames (blocks) encapsulated via GSMTAP UDP multicast packets. There are two separate multicast groups, one for uplink and one for downlink. The multicast nature simulates the shared medium and enables any simulated phone to receive the signal from multiple BTSs via the downlink multicast group.

/images/osmocom-virtum.png

In OsmoBTS, this is implemented via the new osmo-bts-virtual BTS model.

In OsmocomBB, this is realized by adding virtphy virtual L1, which speaks the same L1CTL protocol that is used between the real OsmcoomBB Layer1 and the Layer2/3 programs such as mobile and the like.

Now many people would argue that GSM without the radio and actual handsets is no fun. I tend to agree, as I'm a hardware person at heart and I am not a big fan of simulation.

Nevertheless, this forms the basis of all kinds of possibilities for automatized (regression) testing in a way and for layers/interfaces that osmo-gsm-tester cannot cover as it uses a black-box proprietary mobile phone (modem). It is also pretty useful if you're traveling a lot and don't want to carry around a BTS and phones all the time, or get some development done in airplanes or other places where operating a radio transmitter is not really a (viable) option.

If you're curious and want to give it a shot, I've put together some setup instructions at the Virtual Um page of the Osmocom Wiki.

Krebs on SecurityExperts in Lather Over ‘gSOAP’ Security Flaw

Axis Communications — a maker of high-end security cameras whose devices can be found in many high-security areas — recently patched a dangerous coding flaw in virtually all of its products that an attacker could use to remotely seize control over or crash the devices.

The problem wasn’t specific to Axis, which seems to have reacted far more quickly than competitors to quash the bug. Rather, the vulnerability resides in open-source, third-party computer code that has been used in countless products and technologies (including a great many security cameras), meaning it may be some time before most vulnerable vendors ship out a fix — and even longer before users install it.cam2cam

At issue is a flaw in a bundle of reusable code (often called a “code library“) known as gSOAP, a widely-used toolkit that software or device makers can use so that their creations can talk to the Internet (or “parse XML” for my geek readers). By some estimates, there are hundreds — if not thousands — of security camera types and other so-called “Internet of Things”(IoT) devices that rely upon the vulnerable gSOAP code.

By exploiting the bug, an attacker could force a vulnerable device to run malicious code, block the owner from viewing any video footage, or crash the system. Basically, lots of stuff you don’t want your pricey security camera system to be doing.

Genivia, the company that maintains gSOAP, released an update on June 21, 2017 that fixes the flaw. In short order, Axis released a patch to plug the gSOAP hole in nearly 250 of its products.

Genivia chief executive Robert Van Engelen said his company has already reached out to all of its customers about the issue. He said a majority of customers use the gSOAP software to develop products, but that mostly these are client-side applications or non-server applications that are not affected by this software crash issue.

“It’s a crash, not an exploit as far as we know,” Van Engelen said. “I estimate that over 85% of the applications are unlikely to be affected by this crash issue.”

Still, there are almost certainly dozens of other companies that use the vulnerable gSOAP code library and haven’t (or won’t) issue updates to fix this flaw, says Stephen Ridley, chief technology officer and founder of Senrio — the security company that discovered and reported the bug. What’s more, because the vulnerable code is embedded within device firmware (the built-in software that powers hardware), there is no easy way for end users to tell if the firmware is affected without word one way or the other from the device maker.

“It is likely that tens of millions of products — software products and connected devices — are affected by this,” Ridley said.

“Genivia claims to have more than 1 million downloads of gSOAP (most likely developers), and IBM, Microsoft, Adobe and Xerox as customers,” the Senrio report reads. “On Sourceforge, gSOAP was downloaded more than 1,000 times in one week, and 30,000 times in 2017. Once gSOAP is downloaded and added to a company’s repository, it’s likely used many times for different product lines.”

Anyone familiar with the stories published on this blog over the past year knows that most IoT devices — security cameras in particular — do not have a stellar history of shipping in a default-secure state (heck, many of these devices are running versions of Linux that date back more than a decade). Left connected to the Internet in an insecure state, these devices can quickly be infected with IoT threats like Mirai, which enslave them for use in high-impact denial-of-service attacks designed to knock people and Web sites offline.

When I heard about this bug I pinged the folks over at IPVM, a trade publication that tracks the video surveillance industry. IPVM Business Analyst Brian Karas said the type of flaw (known as a buffer overflow) in this case doesn’t expose the vulnerable systems to IoT worms like Mirai, which can spread to devices that are running under factory-default usernames and passwords.

IPVM polled almost a dozen top security camera makers, and said only two (including Axis) responded that they used the vulnerable gSOAP library in their products. Another three said they hadn’t yet determined whether any of their products were potentially vulnerable.

“You probably wouldn’t be able to make a universal, Mirai-style exploit for this flaw because it lacks the elements of simplicity and reproduceability,” Karas said, noting that the exploit requires that an attacker be able to upload at least a 2 GB file to the Web interface for a vulnerable device.

“In my experience, I don’t think it’s that common for embedded systems to accept a 2-gigabyte file upload,” Karas said. “Every device is going to respond slightly differently, and it would probably take a lot of time to research each device and put together some kind of universal attack tool. Yes, people should be aware of this and patch if they can, but this is nowhere near as bad as [the threat from] Mirai.”

Karas said similar to most other cyber security vulnerabilities in network devices, restricting network access to the unit will greatly reduce the chance of exploit.

“Cameras utilizing a VMS (video management system) or recorder for remote access, instead of being directly connected to the internet, are essentially immune from remote attack (though it is possible for the VMS itself to have vulnerabilities),” IPVM wrote in an analysis of the gSOAP bug. In addition, changing the factory default settings (e.g., picking decent administrator passwords) and updating the firmware on the devices to the latest version may go a long way toward sidestepping any vulnerabilities.

TEDSneak preview lineup unveiled for Africa’s next TED Conference

On August 27, an extraordinary group of people will gather in Arusha, Tanzania, for TEDGlobal 2017, a four-day TED Conference for “those with a genuine interest in the betterment of the continent,” says curator Emeka Okafor.

As Okafor puts it: “Africa has an opportunity to reframe the future of work, cultural production, entrepreneurship, agribusiness. We are witnessing the emergence of new educational and civic models. But there is, on the flip side, a set of looming challenges that include the youth bulge and under-/unemployment, a food crisis, a risky dependency on commodities, slow industrializations, fledgling and fragile political systems. There is a need for a greater sense of urgency.”

He hopes the speakers at TEDGlobal will catalyze discussion around “the need to recognize and amplify solutions from within the Africa and the global diaspora.”

Who are these TED speakers? A group of people with “fresh, unique perspectives in their initiatives, pronouncements and work,” Okafor says. “Doers as well as thinkers — and contrarians in some cases.” The curation team, which includes TED head curator Chris Anderson, went looking for speakers who take “a hands-on approach to solution implementation, with global-level thinking.”

Here’s the first sneak preview — a shortlist of speakers who, taken together, give a sense of the breadth and topics to expect, from tech to the arts to committed activism and leadership. Look for the long list of 35–40 speakers in upcoming weeks.

The TEDGlobal 2017 conference happens August 27–30, 2017, in Arusha, Tanzania. Apply to attend >>

Kamau Gachigi, Maker

“In five to ten years, Kenya will truly have a national innovation system, i.e. a system that by its design audits its population for talented makers and engineers and ensures that their skills become a boon to the economy and society.” — Kamau Gachigi on Engineering for Change

Dr. Kamau Gachigi is the executive director of Gearbox, Kenya’s first open makerspace for rapid prototyping, based in Nairobi. Before establishing Gearbox, Gachigi headed the University of Nairobi’s Science and Technology Park, where he founded a Fab Lab full of manufacturing and prototyping tools in 2009, then built another one at the Riruta Satellite in an impoverished neighborhood in the city. At Gearbox, he empowers Kenya’s next generation of creators to build their visions. @kamaufablab

Mohammed Dewji, Business leader

“My vision is to facilitate the development of a poverty-free Tanzania. A future where the opportunities for Tanzanians are limitless.” — Mohammed Dewji

Mohammed Dewji is a Tanzanian businessman, entrepreneur, philanthropist, and former politician. He serves as the President and CEO of MeTL Group, a Tanzanian conglomerate operating in 11 African countries. The Group operates in areas as diverse as trading, agriculture, manufacturing, energy and petroleum, financial services, mobile telephony, infrastructure and real estate, transport, logistics and distribution. He served as Member of Parliament for Singida-Urban from 2005 until his retirement in 2015. Dewji is also the Founder and Trustee of the Mo Dewji Foundation, focused on health, education and community development across Tanzania. @moodewji

Meron Estefanos, Refugee activist

“Q: What’s a project you would like to move forward at TEDGlobal?
A: Bringing change to Eritrea.” —Meron Estefanos

Meron Estefanos is an Eritrean human rights activist, and the host and presenter of Radio Erena’s weekly program “Voices of Eritrean Refugees,” aired from Paris. Estefanos is executive director of the Eritrean Initiative on Refugee Rights (EIRR), advocating for the rights of Eritrean refugees, victims of trafficking, and victims of torture. Ms Estefanos has been key in identifying victims throughout the world who have been blackmailed to pay ransom for kidnapped family members, and was a key witness in the first trial in Europe to target such blackmailers. She is co-author of Human Trafficking in the Sinai: Refugees between Life and Death and The Human Trafficking Cycle: Sinai and Beyond, and was featured in the film Sound of Torture. She was nominated for the 2014 Raoul Wallenberg Award for her work on human rights and victims of trafficking. @meronina

Touria El Glaoui, Art fair founder

“I’m looking forward to discussing the roles we play as leaders and tributaries in redressing disparities within arts ecosystems. The art fair is one model which has had a direct effect on the ways in which audiences engage with art, and its global outlook has contributed to a highly mobile and dynamic means of interaction.” — Touria El Glaoui

Touria El Glaoui is the founding director of the 1:54 Contemporary African Art Fair, which takes place in London and New York every year and, in 2018, launches in Marrakech. The fair highlights work from artists and galleries across Africa and the diaspora, bringing visibility in global art markets to vital upcoming visions. El Glaoui began her career in the banking industry before founding 1:54 in 2013. Parallel to her career, Touria has organised and co-curated exhibitions of her father’s work, the Moroccan artist Hassan El Glaoui, in London and Morocco. @154artfair

Gus Casely-Hayford, Historian

“Technological, demographic, economic and environmental change are recasting the world profoundly and rapidly. The sentiment that we are traveling through unprecedented times has left many feeling deeply unsettled, but there may well be lessons to learn from history — particularly African history — lessons that show how brilliant leadership and strategic intervention have galvanised and united peoples around inspirational ideas.” — Gus Casely-Hayford

Dr. Gus Casely-Hayford is a curator and cultural historian who writes, lectures and broadcasts widely on African culture. He has presented two series of The Lost Kingdoms of Africa for the BBC and has lectured widely on African art and culture, advising national and international bodies on heritage and culture. He is currently developing a National Portrait Gallery exhibition that will tell the story of abolition of slavery through 18th- and 19th-century portraits — an opportunity to bring many of the most important paintings of black figures together in Britain for the first time.

Oshiorenoya Agabi, Computational neuroscientist

“Koniku eventually aims to build a device that is capable of thinking in the biological sense, like a human being. We think we can do this in the next two to five years.” — Oshiorenoya Agabi on IndieBio.co

With his startup Koniku, Oshiorenoya Agabi is working to integrate biological neurons and silicon computer chips, to build computers that can think like humans can. Faster, cleverer computer chips are key to solving the next big batch of computing problems, like particle detection or sophisticated climate modeling — and to get there, we need to move beyond the limitations of silicon, Agabi believes. Born and raised in Lagos, Nigeria, Agabi is now based in the SF Bay Area, where he and his lab mates are working on the puzzle of connecting silicon to biological systems.

Natsai Audrey Chieza, Design researcher

Photo: Natsai Audrey Chieza

Natsai Audrey Chieza is a design researcher whose fascinating work crosses boundaries between technology, biology, design and cultural studies. She is founder and creative director of Faber Futures, a creative R&D studio that conceptualises, prototypes and evaluates the resilience of biomaterials emerging through the convergence of bio-fabrication, digital fabrication and traditional craft processes. As Resident Designer at the Department of Biochemical Engineering, University College London, she established a design-led microbiology protocol that replaces synthetic pigments with natural dyes excreted by bacteria — producing silk scarves dyed brilliant blues, reds and pinks. The process demands a rethink of the entire system of fashion and textile production — and is also a way to examine issues like resource scarcity, provenance and cultural specificity. @natsaiaudrey

Stay tuned for more amazing speakers, including leaders, creators, and more than a few truth-tellers … learn more >>


Valerie AuroraThe Al Capone theory of sexual harassment

This post was co-authored by Valerie Aurora and Leigh Honeywell and cross-posted on both of our blogs.

Mural of Al Capone, laughing and smoking a cigar
CC BY-SA 2.0 r2hox

We’re thrilled with the recent trend towards sexual harassment in the tech industry having actual consequences – for the perpetrator, not the target, for a change. We decided it was time to write a post explaining what we’ve been calling “the Al Capone Theory of Sexual Harassment.” (We can’t remember which of us came up with the name, Leigh or Valerie, so we’re taking joint credit for it.) We developed the Al Capone Theory over several years of researching and recording racism and sexism in computer security, open source software, venture capital, and other parts of the tech industry. To explain, we’ll need a brief historical detour – stick with us.

As you may already know, Al Capone was a famous Prohibition-era bootlegger who, among other things, ordered murders to expand his massively successful alcohol smuggling business. The U.S. government was having difficulty prosecuting him for either the murdering or the smuggling, so they instead convicted Capone for failing to pay taxes on the income from his illegal business. This technique is standard today – hence the importance of money-laundering for modern successful criminal enterprises – but at the time it was a novel approach.

The U.S. government recognized a pattern in the Al Capone case: smuggling goods was a crime often paired with failing to pay taxes on the proceeds of the smuggling. We noticed a similar pattern in reports of sexual harassment and assault: often people who engage in sexually predatory behavior also faked expense reports, plagiarized writing, or stole credit for other people’s work. Just three examples: Mark Hurd, the former CEO of HP, was accused of sexual harassment by a contractor, but resigned for falsifying expense reports to cover up the contractor’s unnecessary presence on his business trips. Jacob Appelbaum, the former Tor evangelist, left the Tor Foundation after he was accused of both sexual misconduct and plagiarism. And Randy Komisar, a general partner at venture capital firm KPCB, gave a book of erotic poetry to another partner at the firm, and accepted a board seat (and the credit for a successful IPO) at RPX that would ordinarily have gone to her.

Initially, the connection eluded us: why would the same person who made unwanted sexual advances also fake expense reports, plagiarize, or take credit for other people’s work? We remembered that people who will admit to attempting or committing sexual assault also disproportionately commit other types of violence and that “criminal versatility” is a hallmark of sexual predators. And we noted that taking credit for others’ work is a highly gendered behavior.

Then we realized what the connection was: all of these behaviors are the actions of someone who feels entitled to other people’s property – regardless of whether it’s someone else’s ideas, work, money, or body. Another common factor was the desire to dominate and control other people. In venture capital, you see the same people accused of sexual harassment and assault also doing things like blacklisting founders for objecting to abuse and calling people nasty epithets on stage at conferences. This connection between dominance and sexual harassment also shows up as overt, personal racism (that’s one reason why we track both racism and sexism in venture capital).

So what is the Al Capone theory of sexual harassment? It’s simple: people who engage in sexual harassment or assault are also likely to steal, plagiarize, embezzle, engage in overt racism, or otherwise harm their business. (Of course, sexual harassment and assault harms a business – and even entire fields of endeavor – but in ways that are often discounted or ignored.) Ask around about the person who gets handsy with the receptionist, or makes sex jokes when they get drunk, and you’ll often find out that they also violated the company expense policy, or exaggerated on their résumé, or took credit for a colleague’s project. More than likely, they’ve engaged in sexual misconduct multiple times, and a little research (such as calling previous employers) will show this, as we saw in the case of former Uber and Google employee Amit Singhal.

Organizations that understand the Al Capone theory of sexual harassment have an advantage: they know that reports or rumors of sexual misconduct are a sign they need to investigate for other incidents of misconduct, sexual or otherwise. Sometimes sexual misconduct is hard to verify because a careful perpetrator will make sure there aren’t any additional witnesses or records beyond the target and the target’s memory (although with the increase in use of text messaging in the United States over the past decade, we are seeing more and more cases where victims have substantial written evidence). But one of the implications of the Al Capone theory is that even if an organization can’t prove allegations of sexual misconduct, the allegations themselves are sign to also urgently investigate a wide range of aspects of an employee’s conduct.

Some questions you might ask: Can you verify their previous employment and degrees listed on their résumé? Do their expense reports fall within normal guidelines and include original receipts? Does their previous employer refuse to comment on why they left? When they give references, are there odd patterns of omission? For example, a manager who doesn’t give a single reference from a person who reported to them can be a hint that they have mistreated people they had power over.

Another implication of the Al Capone theory is that organizations should put more energy into screening potential employees or business partners for allegations of sexual misconduct before entering into a business relationship with them, as recently advocated by LinkedIn cofounder and Greylock partner Reid Hoffman. This is where tapping into the existing whisper network of targets of sexual harassment is incredibly valuable. The more marginalized a person is, the more likely they are to be the target of this kind of behavior and to be connected with other people who have experienced this behavior. People of color, queer people, people with working class jobs, disabled people, people with less money, and women are all more likely to know who sends creepy text messages after a business meeting. Being a member of more than one of these groups makes people even more vulnerable to this kind of harassment – we don’t think it was a coincidence that many of the victims of sexual harassment who spoke out last month were women of color.

What about people whose well-intentioned actions are unfairly misinterpreted, or people who make a single mistake and immediately regret it? The Al Capone theory of sexual harassment protects these people, because when the organization investigates their overall behavior, they won’t find a pattern of sexual harassment, plagiarism, or theft. A broad-ranging investigation in this kind of case will find only minor mistakes in expense reports or an ambiguous job title in a resume, not a pervasive pattern of deliberate deception, theft, or abuse. To be perfectly clear, it is possible for someone to sexually harass someone without engaging in other types of misconduct. In the absence of clear evidence, we always recommend erring on the side of believing accusers who have less power or privilege than the people they are accusing, to counteract the common unconscious bias against believing those with less structural power and to take into account the enormous risk of retaliation against the accuser.

Some people ask whether the Al Capone theory of sexual harassment will subject men to unfair scrutiny. It’s true, the majority of sexual harassment is committed by men. However, people of all genders commit sexual harassment. We personally know of two women who have sexually touched other people without consent at tech-related events, and we personally took action to stop these women from abusing other people. At the same time, abuse more often occurs when the abuser has more power than the target – and that imbalance of power is often the result of systemic oppression such as racism, sexism, cissexism, or heterosexism. That’s at least one reason why a typical sexual harasser is more likely to be one or all of straight, white, cis, or male.

What does the Al Capone theory of sexual harassment mean if you are a venture capitalist or a limited partner in a venture fund? Your first priority should be to carefully vet potential business partners for a history of unethical behavior, whether it is sexual misconduct, lying about qualifications, plagiarism, or financial misdeeds. If you find any hint of sexual misconduct, take the allegations seriously and step up your investigation into related kinds of misconduct (plagiarism, lying on expense reports, embezzlement) as well as other incidents of sexual misconduct.

Because sexual harassers sometimes go to great lengths to hide their behavior, you almost certainly need to expand your professional network to include more people who are likely to be targets of sexual harassment by your colleagues – and gain their trust. If you aren’t already tapped into this crucial network, here are some things you can do to get more access:

These are all aspects of ally skills – concrete actions that people with more power and privilege can take to support people who have less.

Finally, we’ve seen a bunch of VCs pledging to donate the profits of their investments in funds run by accused sexual harassers to charities supporting women in tech. We will echo many other women entrepreneurs and say: don’t donate that money, invest it in women-led ventures – especially those led by women of color.


Tagged: ally skills, feminism, tech

TEDThe TED2018 Fellows application is open. Apply now!

TED2018_Fellows_application

TED is looking for early-career, visionary thinkers from around the world to join the Fellows program at the upcoming TED2018 conference in Vancouver, British Columbia.

Do you have an original approach to your work that’s worth sharing with the world? Are you working to uplift and empower your local community through innovative science, art or entrepreneurship? Are you ready to take full advantage of the TED platform and the support of a dynamic global community of innovators? If yes, you should apply to be a TED Fellow.

TED Fellows are a multidisciplinary group of remarkable individuals who are chosen through an open and rigorous application process. For each TED conference, we select a class of 20 Fellows based on their exceptional achievement and an innovative approach to tackling the world’s toughest problems, as well as on their character, grit and collaborative spirit.

Apply by September 10 at go.ted.com/tedfellowsapply.

TED2018 — themed “The Age of Amazement” — will take a deep-dive into the key developments driving our future, from jaw-dropping AI to glorious new forms of creativity to courageous advocates of radical social change. If selected, you will attend the TED2018 conference and participate in a Fellows-only pre-conference designed especially to inspire, empower and support your work. Fellows also deliver a TED Talk at the conference, filmed and considered for publication on TED.com.  

The TED Fellows program is designed to catapult your career through transformational support like coaching and mentorship, public relations guidance for sharing your latest projects, hands on speaker training — and, most importantly, access to the vibrant global network of more than 400 Fellows from over 90 countries.

The online application includes general biographical questions, short essays on your work and three references. Only those aged 18 and older can apply. If selected, Fellows must reserve April 10 – April 15, 2018 on their calendars for the TED2018 conference in Vancouver, British Columbia.

Think you have what it takes to be a TED Fellow? Apply now.

More information
Questions?: ted.com/participate/ted-fellows-program
Visit: ted.com/fellows
Follow: @TEDFellow
Like: facebook.com/TEDFellow
Read: fellowsblog.ted.com


CryptogramMany of My E-Books for Cheap

Humble Bundle is selling a bunch of cybersecurity books very cheaply. You can get copies of Applied Cryptography, Secrets and Lies, and Cryptography Engineering -- and also Ross Anderson's Security Engineering, Adam Shostack's Threat Modeling, and many others.

This is the cheapest you'll ever see these books. And they're all DRM-free.

Worse Than FailureThe Little Red Button

Bryan T. had worked for decades to amass the skills, expertise and experience to be a true architect, but never quite made the leap. Finally, he got a huge opportunity in the form of an interview with a Silicon Valley semi-conductor firm project manager who was looking for a consultant to do just that. The discussions revolved around an application that three developers couldn't get functioning correctly in six months, and Bryan was to be the man to reign it in and make it work; he was lured with the promise of having complete control of the software.

The ZF-1 pod weapon system from the Fifth Element

Upon starting and spelunking through the code-base, Bryan discovered the degree of total failure that caused them to yield complete control to him. It was your typical hodgepodge of code slapped together with anti-patterns, snippets of patterns claiming to be the real deal, and the usual Assortment-o-WTF™ we've all come to expect.

Once he recognized the futility of attempting to fix this mess, Bryan scrapped it and rewrote it as a full-blown modular and compositional application, utilizing MVVM, DDD, SOA, pub/sub; the works. Within three weeks, he had it back to the point it was when he started, only his version actually worked.

While he had righted the sinking ship, it was so successful that the project team started managing it, which proved to be its undoing.

Given the sudden success of the project, the department head committed the application to all the divisions company wide within three quarters - without informing Bryan or anyone else on the team. After all, it's not like developers need to plan for code and resource scalability issues beyond the original design requirements or anything.

We've read countless stories about how difficult it is to work with things like dates and even booleans, but buttons are pretty much solidly understood. Some combination of text, text+image or just image, and an onAction callback pretty much covers it. Oh sure, you can set fg/bg colors and the font, but that's usually to just give visual clues. Unfortunately, buttons would be the beginning of a downward spiral so steep, that sheer inertia would derail the project.

The project manager decided that images were incredibly confusing, so all buttons should have text instead of icons. Bryan had created several toolbars (because ribbons were shot down already) which, according to management, made the application unusable. In particular, there was a fairly standard user icon with a pencil bullet that was meant to (as you might have guessed it) edit users...

  Manager:  So I looked at it with Lisa and she had no clue what it was.  
            It was so confusing that no one would ever be able to use our 
            application with it.  Buttons should all be text and not images!

OK, let’s forget that ribbons and toolbars have been an application standard for decades now; let’s focus on how confusing this really is. To that end, Bryan did the nanny test. He asked his nanny what she thought it meant and she thought that the button had something to do with people. Awesome, on the ball! After explaining what it did she agreed it made sense.

  Bryan: How about we explain it to the users and add a tooltip?
  Mgr:   Tooltips take way too long to display and it’s still 
         incredibly confusing – no one would remember it. We
         don't want people pressing the wrong buttons!
         And why are some of the buttons different colors than others?

Bryan wasn't sure if the manager realized how stupidly he was treating his users, if he was just oblivious, or if he was just pushing for his personal preference. In the end, all the toolbars were removed and the icons were replaced with text. This left an application with assorted colored buttons with text. Unfortunately, some of the buttons were so small that the text got displayed as a truncated string. Also, no amount of explanation could convey that color could also convey meaning (think traffic lights).

As his opinion in UI matters dwindled to nothing over the next couple of months, one of the four BA’s on the team of six pinged Bryan for a meeting about scalability. He wanted to make sure that the project was scalable for the next three quarters. Enter the Holy Hell Twilight Zone moment in the land where no ribbons or toolbars exist, as the project manager was also involved.

  Mgr:   I’ve got to make sure we have everything we need to 
         scale for the next three quarters.
  Bryan: I can’t get the project manager to commit to lay out
         three weeks of planning for development. I can’t even 
         begin to guess if we have what we need for the next three 
         quarters.
  Mgr:   Well the vice president has a commitment to deploy this 
         to all divisions in the company within three quarters and 
         I’m tasked to make sure we have what we need.

Now Bryan could make up statistics better than 84.3% of people, but what was asked was impossible to determine. Additionally there was a flat out refusal to even vaguely commit to development more than a week or two in advance, so there was heavy resistance just to get the information needed to try!

At this point in time, Bryan felt the need to bail out, but before he left town he grabbed his prized coffee mug from the office. He wasn’t going to be back in town for at least three weeks and from his prior experiences he knew where this was going.

Of course the guy who originally sunk the ship in the first place had a true killer instinct, apparently knew better than Bryan and was left to steer the ship again. All these problems and issues that Bryan saw coming were either over exaggerations or without merit. The project manager felt so comfortable with the architecture and frameworks that Bryan put in place that he felt confident that there was absolutely nothing that he couldn’t handle. After all, he now had the buttons he wanted and understood. Bryan repeatedly asked if he wanted code walkthroughs and was denied. He didn't need to know what the different colors on the buttons were for. Bryan was even given 40 free consulting hours and even told not to check in his latest bug fixes.

Bryan sent his final farewell with a picture of him drinking from his coffee mug at home and out of state.

A real killer, when handed the ZF-1, would've immediately asked about the little red button on the bottom of the gun.

[Advertisement] Infrastructure as Code built from the start with first-class Windows functionality and an intuitive, visual user interface. Download Otter today!

Don MartiStupid ideas department

Here's a probably stupid idea: give bots the right to accept proposed changes to a software project. Can automation encourage less burnout-provoking behavior?

A set of bots could interact in interesting ways.

  • Regression-test-bot: If a change only adds a test, applies cleanly to both the current version and to a previous version, and the previous version passses the test, accept it, even if the test fails for the current version.

  • Harmless-change-bot: If a change is below a certain size, does not modify existing tests, and all tests (including any new ones) pass, accept it.

  • Revert-bot: If any tests are failing on the current version, and have been failing for more than a certain amount of time, revert back to a version that passes.

Would more people write regression tests for their issues if they knew that a bot would accept them? Or say that someone makes a bad change but gets it past harmless-change-bot because no existing test covers it. No lengthy argument needed. Write a regression test and let regression-test-bot and revert-bot team up to take care of the problem. In general, move contributor energy away from arguing with people and toward test writing, and reduce the size of the maintainer's to-do list.

,

Cory DoctorowSan Diego! Come hear me read from Walkaway tomorrow night at Comickaze Liberty Station!

I’m teaching the Clarion Science Fiction writing workshop at UCSD in La Jolla this week, and tomorrow night at 7PM, I’ll be reading from my novel Walkaway at Comickaze Liberty Station, 2750 Historic Decatur Rd #101, San Diego, CA 92106. Hope to see you!

CryptogramAustralia Considering New Law Weakening Encryption

News from Australia:

Under the law, internet companies would have the same obligations telephone companies do to help law enforcement agencies, Prime Minister Malcolm Turnbull said. Law enforcement agencies would need warrants to access the communications.

"We've got a real problem in that the law enforcement agencies are increasingly unable to find out what terrorists and drug traffickers and pedophile rings are up to because of the very high levels of encryption," Turnbull told reporters.

"Where we can compel it, we will, but we will need the cooperation from the tech companies," he added.

Never mind that the law 1) would not achieve the desired results because all the smart "terrorists and drug traffickers and pedophile rings" will simply use a third-party encryption app, and 2) would make everyone else in Australia less secure. But that's all ground I've covered before.

I found this bit amusing:

Asked whether the laws of mathematics behind encryption would trump any new legislation, Mr Turnbull said: "The laws of Australia prevail in Australia, I can assure you of that.

"The laws of mathematics are very commendable but the only law that applies in Australia is the law of Australia."

Next Turnbull is going to try to legislate that pi = 3.2.

Another article. BoingBoing post.

EDITED TO ADD: More commentary.

Cory DoctorowI’m profiled in the new issue of Locus Magazine

Cory Doctorow: Bugging In:

‘‘Walkaway is an ‘optimistic disaster novel.’ It’s about people who, in a crisis, come together, rather than turning on each other. Its villains aren’t the people next door, who’ve secretly been waiting for civilization’s breakdown as an excuse to come and eat you, but the super-rich who are convinced that without the state and its police, the poors are coming to eat them.

‘‘In Walkaway, the economy has comprehensively broken down, and so has the planet. Climate refugees drift in huge, unstoppable numbers from place to place, seeking refuge. The world has no jobs for most people, because when robots do all the work, the forces of capital require a few foremen to boss the robots, and a few unemployed people mooching around the factory gates to threaten the supervisors with if they demand higher wages. Everyone else is surplus to requirements.

‘‘Awareness of self-deception is a tactic that’s deployed very usefully by a lot of people now. It’s at the core of things like cognitive behavioral therapy – the idea that you must become an empiricist of your emotions because your recollections of emotions are always tainted, so you have to write down your experiences and go back to see what actually happened. Do you remember the term Endless September? It’s from when AOL came on to the net, and suddenly new people were getting online all the time, who didn’t know how things worked. The onboarding process to your utopian project is always difficult. It’s a thing Burning Man is struggling with, and it’s a thing fandom is struggling with right now. We were just talking about what it’s like to go to a big media convention, a San Diego Comic-Con or something, and to what extent that’s a new culture, or it’s continuous with the old culture, or it’s preserving the best things or bringing in the worst things, or it’s overwhelming the old, or whatever. It’s a real problem, and there is a shibboleth, which is, ‘I don’t object to all these newcomers, but they’re coming in such numbers that they’re overwhelming our ability to assimilate them.’ This is what every xenophobe who voted for Brexit said, but you hear that lament in science fiction too, and you hear it even about such things as gender parity in the workplace.”

*

‘‘For me, I live by the aphorism, ‘fail better, fail faster.’ To double your success rate, triple your failure rate. What the walkaways figured out how to do is reduce the cost of failure, to make it cheaper to experiment with new ways of succeeding. One of the great bugaboos of the rationalist movement is loss aversion. There is another name for it, ‘the entitlement effect’: basically, people value something they have more than they would pay for it before they got it. How much is your IKEA furniture worth before and after you assemble it? People grossly overestimate the value of their furniture after they’ve assembled it, because having infused it with their labor and ownership, they feel an attachment to it that is not economically rational. Sunk cost is another great fallacy. You can offer somebody enough money to buy the furniture again, and pay somebody to assemble it, and they’ll turn you down, because now that they have it, they don’t want to lose it. That was the wisdom of Obama with Obamacare. He understood that Obamacare is not sustainable, that basically letting insurance companies set the price without any real limits means that the insurance companies will eventually price it out of the government’s ability to pay, but he also understood that once you give 22 million people healthcare, when the insurance companies blew it up, the people would then demand some other healthcare system be found. The idea of just going without healthcare, which was a thing that people were willing to put up with for decades, is something they’ll never go back to. Any politician who proposes that when Obamacare blows up that we replace it with nothing, as opposed to single payer – where it’s going to end up – that politician is dead in the water. ”


More…

Worse Than FailureCodeSOD: Impersonated Programming

Once upon a time, a long long time ago, I got contracted to show a government office how to build and deliver applications… in Microsoft Access. I’m sorry. I’m so, so sorry. As horrifying and awful as it is, Access is actually built with some mechanisms to actually support that- you can break the UI and behavior off into one file, while keeping the data in another, and you can actually construct linked tables that connect to a real database, if you don’t mind gluing a UI made out of evil and sin to your “real” database.

Which brings us to poor Alex Rao. Alex has an application built in Access. This application uses linked tables, which he wants to convert to local tables. The VBA API exposed by Access doesn’t give him any way to do this, so he came up with this solution…

Public Function convertToLocal()
    Dim dBase As DAO.Database
    Dim tdfTable As DAO.TableDef

    Set dBase = CurrentDb

    For Each tdfTable In dBase.TableDefs
        If tdfTable.Connect <> "" Then
            ' OH. MY. GOSH. I hate myself so much for doing this. For the love of everything holy,
            ' dear reader, if you can come up with a better way to do this, please tell me about it
            ' AS SOON AS POSSIBLE
            ' I have literally been trying to do this for the past week. For reference, here is what I
            ' am trying to do:
            '   Convert a "linked" table to a "local" one
            '   Keep relationships intact.
            ' Now, Access has this handy tool, "Convert to Local Table" - you'll see it if you right click
            ' on a linked table. However, THERE IS NO WAY IN VBA TO DO THIS. I am aware of the following:
            '   DoCmd.SelectObject acTable, "Company", True RunCommand acCmdConvertLinkedTableToLocal
            ' Note that this no longer works as of Access 2016 because the wonderful programmers at Microsoft decided
            ' that "It wasn't used anymore".
            '
            ' So, onto my solution:
            '   First, I select the table object, making sure it's actually selected (i.e., like a user selected it)
            '   Then, I pause for one second (I hope to the man upstairs that's long enough)
            '   Then, I send the "Context Menu" key (SHIFT+F10)
            '   Then, I pause for another second (Again, fingers crossed)
            '   Then, I send the "v" key - to activate the "ConVert to Local Table" command shortcut
            '
            ' I literally send KEYPRESSES to the active application, and hope to God that Access is ready to go.
            ' And if the user selected a different application (or literally anything else) in that time? Well,
            ' then Screw you, user.
            '
            ' God help us.
            DoCmd.SelectObject acTable, tdfTable.Name, True
            Pause 1
            SendKeys "+{F10}", True
            Pause 1
            SendKeys "v", True
        End If
    Next tdfTable
End Function

Don’t feel bad, Alex. I’m certain this isn’t the worst thing ever built in Access.

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Don MartiPlaying for third place

Just tried a Twitter advertising trick that a guy who goes by "weev" posted two years ago.

It still works.

They didn't fix it.

Any low-budget troll who can read that old blog post and come up with a valid credit card number can still do it.

Maybe Twitter is a bad example, but the fast-moving nationalist right wing manages to outclass its opponents on other social marketing platforms, too. Facebook won't even reveal how badly they got played in 2016. They thought they were putting out cat food for cute Internet kittens, but the rats ate it.

This is not new. Right-wing shitlords, at least the best of them, are the masters of database marketing. They absolutely kill it, and they have been ever since Marketing as we know it became a thing. Some good examples:

All the creepy surveillance marketing stuff they're doing today is just another set of tools in an expanding core competency.

Every once in a while you get an exception. The environmental movement became a direct mail operation in response to Interior Secretary James G. Watt, who alarmed environmentalists enough that organizations could reliably fundraise with direct mail copy quoting from Watt's latest speech. And the Democrats tried that "Organizing for America" thing for a little while, but, man, their heart just wasn't in it. They dropped it like a Moodle site during summer vacation. Somehow, the creepier the marketing, the more it skews "red". The more creativity involved, the more it skews "blue" (using the USA meanings of those colors.) When we make decisions about how much user surveillance we're going to allow on a platform, we're making a political decision.

Anyway. News Outlets to Seek Bargaining Rights Against Google and Facebook.

The standings so far.

  1. Shitlords and fraud hackers

  2. Adtech and social media bros

  3. NEWS SITES HERE (?)

News sites want to go to Congress, to get permission to play for third place in their own business? You want permission to bring fewer resources and less experience to a surveillance marketing game that the Internet companies are already losing?

We know the qualities of a medium that you win by being creepier, and we know the qualities of a medium that you can win with reputation and creativity. Why waste time and money asking Congress for the opportunity to lose, when you could change the game instead?

Maybe achieving balance in political views depends on achieving balance in business model. Instead of buying in to the surveillance marketing model 100%, and handing an advantage to one side, maybe news sites should help users control what data they share in order to balance competing political interests.

Planet Linux Australiasthbrx - a POWER technical blog: XDP on Power

This post is a bit of a break from the standard IBM fare of this blog, as I now work for Canonical. But I have a soft spot for Power from my time at IBM - and Canonical officially supports 64-bit, little-endian Power - so when I get a spare moment I try to make sure that cool, officially-supported technologies work on Power before we end up with a customer emergency! So, without further ado, this is the story of XDP on Power.

XDP

eXpress Data Path (XDP) is a cool Linux technology to allow really fast processing of network packets.

Normally in Linux, a packet is received by the network card, an SKB (socket buffer) is allocated, and the packet is passed up through the networking stack.

This introduces an inescapable latency penalty: we have to allocate some memory and copy stuff around. XDP allows some network cards and drivers to process packets early - even before the allocation of the SKB. This is much faster, and so has applications in DDOS mitigation and other high-speed networking use-cases. The IOVisor project has much more information if you want to learn more.

eBPF

XDP processing is done by an eBPF program. eBPF - the extended Berkeley Packet Filter - is an in-kernel virtual machine with a limited set of instructions. The kernel can statically validate eBPF programs to ensure that they terminate and are memory safe. From this it follows that the programs cannot be Turing-complete: they do not have backward branches, so they cannot do fancy things like loops. Nonetheless, they're surprisingly powerful for packet processing and tracing. eBPF programs are translated into efficient machine code using in-kernel JIT compilers on many platforms, and interpreted on platforms that do not have a JIT. (Yes, there are multiple JIT implementations in the kernel. I find this a terrifying thought.)

Rather than requiring people to write raw eBPF programs, you can write them in a somewhat-restricted subset of C, and use Clang's eBPF target to translate them. This is super handy, as it gives you access to the kernel headers - which define a number of useful data structures like headers for various network protocols.

Trying it

There are a few really interesting project that are already up and running that allow you to explore XDP without learning the innards of both eBPF and the kernel networking stack. I explored the samples in the bcc compiler collection and also the samples from the netoptimizer/prototype-kernel repository.

The easiest way to get started with these is with a virtual machine, as recent virtio network drivers support XDP. If you are using Ubuntu, you can use the uvt-kvm tooling to trivially set up a VM running Ubuntu Zesty on your local machine.

Once your VM is installed, you need to shut it down and edit the virsh XML.

You need 2 vCPUs (or more) and a virtio+vhost network card. You also need to edit the 'interface' section and add the following snippet (with thanks to the xdp-newbies list):

<driver name='vhost' queues='4'>
    <host tso4='off' tso6='off' ecn='off' ufo='off'/>
    <guest tso4='off' tso6='off' ecn='off' ufo='off'/>
</driver>

(If you have more than 2 vCPUs, set the queues parameter to 2x the number of vCPUs.)

Then, install a modern clang (we've had issues with 3.8 - I recommend v4+), and the usual build tools.

I recommend testing with the prototype-kernel tools - the DDOS prevention tool is a good demo. Then - on x86 - you just follow their instructions. I'm not going to repeat that here.

POWERful XDP

What happens when you try this on Power? Regular readers of my posts will know to expect some minor hitches.

XDP does not disappoint.

Firstly, the prototype-kernel repository hard codes x86 as the architecture for kernel headers. You need to change it for powerpc.

Then, once you get the stuff compiled, and try to run it on a current-at-time-of-writing Zesty kernel, you'll hit a massive debug splat ending in:

32: (61) r1 = *(u32 *)(r8 +12)
misaligned packet access off 0+18+12 size 4
load_bpf_file: Permission denied

It turns out this is because in Ubuntu's Zesty kernel, CONFIG_HAS_EFFICIENT_UNALIGNED_ACCESS is not set on ppc64el. Because of that, the eBPF verifier will check that all loads are aligned - and this load (part of checking some packet header) is not, and so the verifier rejects the program. Unaligned access is not enabled because the Zesty kernel is being compiled for CPU_POWER7 instead of CPU_POWER8, and we don't have efficient unaligned access on POWER7.

As it turns out, IBM never released any officially supported Power7 LE systems - LE was only ever supported on Power8. So, I filed a bug and sent a patch to build Zesty kernels for POWER8 instead, and that has been accepted and will be part of the next stable update due real soon now.

Sure enough, if you install a kernel with that config change, you can verify the XDP program and load it into the kernel!

If you have real powerpc hardware, that's enough to use XDP on Power! Thanks to Michael Ellerman, maintainer extraordinaire, for verifying this for me.

If - like me - you don't have ready access to Power hardware, you're stuffed. You can't use qemu in TCG mode: to use XDP with a VM, you need multi-queue support, which only exists in the vhost driver, which is only available for KVM guests. Maybe IBM should release a developer workstation. (Hint, hint!)

Overall, I was pleasantly surprised by how easy things were for people with real ppc hardware - it's encouraging to see something not require kernel changes!

eBPF and XDP are definitely growing technologies - as Brendan Gregg notes, now is a good time to learn them! (And those on Power have no excuse either!)

,

Planet Linux AustraliaOpenSTEM: This Week in HASS – term 3, week 2

This week older students start their research projects for the term, whilst younger students are doing the Timeline Activity. Our youngest students are thinking about the places where people live and can join together with older students as buddies to Build A Humpy together.

Foundation/Prep/Kindy to Year 3

Students in stand-alone Foundation/Prep/Kindy classes (Unit F.3), or those in classes integrated with Year 1 (Unit F-1.3) are considering different types of homes this week. They will think about where the people in the stories from last week live and compare that to their own houses. They can consider how homes were different in the past and how our homes help us meet our basic needs. There is an option this week for these students to buddy with older students, especially those in Years 4, 5 and 6, to undertake the Building A Humpy activity together. In this activity students collect materials to build a replica Aboriginal humpy or shelter outside. Many teachers find that both senior primary and the younger students get a lot of benefit from helping each other with activities, enriching the learning experience. The Building a Humpy activity is one where the older students can assist the younger students with the physical requirements of building a humpy, whilst each group considers aspects of the activity relevant to their own studies, and comparing past ways of life to their own.

Students in Years 1 (Unit 1.3), 2 (Unit 2.3) and 3 (Unit 3.3) are undertaking the Timeline Activity this week. This activity is designed to complement the concept of the number line from the Mathematics curriculum, whilst helping students to learn to visualise the abstract concepts of the past and different lengths of time between historical events and the present. In this activity students walk out a timeline, preferably across a large open space such as the school Oval, whilst attaching pieces of paper at intervals to a string. The pieces of paper refer to specific events in history (starting with their own birth years) and cover a wide range of events from the material covered this year. Teachers can choose from events in Australian and world history, covering 100s, 1000s and even millions of years, back to the dinosaurs. Teachers can also add their own events. Thus the details of the activity are able to be altered in different years to maintain student interest. Depending on the class, the issue of scale can be addressed in various ways. By physically moving their bodies, students will start to understand the lengths of time involved in examinations of History. This activity is repeated in increasing detail in higher years, to make sure that the fundamental concepts are absorbed by students over time.

Years 3 to 6

Science ExplosionStudents in Years 3 to 6 are starting their term research projects on Australian history this week. Students in Year 3 (Unit 3.7) concentrate on topics from the history of their capital city or local community. Suggested topics are included for Brisbane, Melbourne, Sydney, Adelaide, Darwin, Hobart, Perth and Canberra. Teachers can substitute their own topics for a local community study. Students will undertake a Scientific Investigation into an aspect of their chosen research project and will produce a Scientific Report. It is recommended that teachers supplement the resources provided with old photographs, books, newspapers etc, many of which can be accessed online, to provide the students with extra material for their investigation.

First Fleet 1788First Fleet

Students in Year 4 (Unit 4.3) will be focusing on Australia in the period up to and including the arrival of the First Fleet and the early colonial period. OpenSTEM’s Understanding Our World® program encompasses the whole Australian curriculum for HASS and thus does not simply rely on “flogging the First Fleet to death”! There are 7 research themes for Year 4 students: “Australia Before 1788”; “The First Fleet”; “Convicts and Settlers”; “Aboriginal People in Colonial Australia”; “Australia and Other Nations in the 17th, 18th and 19th centuries”; “Colonial Children”; “Colonial Animals and their Impact”. These themes are allocated to groups of students and each student chooses an individual research topic within their groups themes. Suggested topics are given in the Teacher Handbook, as well as suggested resources.

19th century china dolls

Year 5 (Unit 5.3) students focus on the colonial period in Australia. There are 9 research themes for Year 5 students. These are: “The First Fleet”; “Convicts and Settlers”; “The 6 Colonies”; “Aboriginal People in Colonial Australia”; “Resistance to Colonial Authorities”; “Sugar in Queensland”; “Colonial Children”; “Colonial Explorers” and “Colonial Animals and their Impact”. As well as themes unique to Year 5, some overlap is provided to facilitate teaching in multi-year classes. The range of themes also allows for the possibility of teachers choosing different themes in different years. Once again individual topics and resources are suggested in the Teacher Handbook.

Year 6 (Unit 6.3) students will examine research themes around Federation and the early 20th century. There are 8 research themes for Year 6 students: “Federation and Sport”; “Women’s Suffrage”; “Aboriginal Rights in Australia”; “Henry Parkes and Federation”; “Edmund Barton and Federation”; “Federation and the Boer War”; “Samuel Griffith and the Constitution”; “Children in Australian History”. Individual research topics and resources are suggested in the Teachers Handbook. It is expected that students in Year 6 will be able to research largely independently, with weekly guidance from their teacher. OpenSTEM’s Understanding Our World® program is aimed at developing research skills in students progressively, especially over the upper primary years. If the program is followed throughout the primary years, students are well prepared for high school by the end of Year 6, having practised individual research skills for several years.

 

Rondam RamblingsThings I wish someone had told me before I started angel investing

Back in 2005 I suddenly found myself sitting on a big pile of money after the Google IPO so I did what any young nouveau-riche high-tech dilettante would do: I started angel investing.  I figured it would be more fun to be the beggee than the beggor for a change, and I was right about that.  But I was wrong about just about everything else, and I got a very expensive education as a result. Now

Krebs on SecurityPorn Spam Botnet Has Evil Twitter Twin

Last month KrebsOnSecurity published research into a large distributed network of apparently compromised systems being used to relay huge blasts of junk email promoting “online dating” programs — affiliate-driven schemes traditionally overrun with automated accounts posing as women. New research suggests that another bot-promoting botnet of more than 80,000 automated female Twitter accounts has been pimping the same dating scheme and prompting millions of clicks from Twitter users in the process.

One of the 80,000+ Twitter bots ZeroFOX found that were enticing male Twitter users into viewing their profile pages.

One of the 80,000+ Twitter bots ZeroFOX found that were enticing male Twitter users into viewing their profile pages.

Not long after I published Inside a Porn-Pimping Spam Botnet, I heard from researchers at ZeroFOX, a security firm that helps companies block attacks coming through social media.

Zack Allen, manager of threat operations at ZeroFOX, said he had a look at some of the spammy, adult-themed domains being promoted by the botnet in my research and found they were all being promoted through a botnet of bogus Twitter accounts.

Those phony Twitter accounts all featured images of attractive or scantily-clad women, and all were being promoted via suggestive tweets, Allen said.

Anyone who replied was ultimately referred to subscription-based online dating sites run by Deniro Marketing, a company based in California. This was the same company that was found to be the beneficiary of spam from the porn botnet I’d written about in June. Deniro did not respond to requests for comment.

“We’ve been tracking this thing since February 2017, and we concluded that the social botnet controllers are probably not part of Deniro Marketing, but most likely are affiliates,” Allen said.

ZeroFOX found more than 86,262 Twitter accounts were responsible for more than 8.6 million posts on Twitter promoting porn-based sites, many of them promoting domains in a swath of Internet address space owned by Deniro Marketing (ASN19884).

Allen said 97.4% of bot display names had the pattern “Firstname Surname” with the first letters of each name capitalized, and each name separated by a single whitespace character that corresponded to common female names.

An analysis of the Twitter bot names used in the scheme. Graphic: ZeroFOX.

An analysis of the Twitter bot names used in the scheme. Graphic: ZeroFOX.

The accounts advertise adult content by routinely injecting links from their twitter profiles to a popular hashtag, or by @-mentioning a popular user or influencer on Twitter. Those profile links are shortened with Google’s goo.gl link shortening service, which then redirects to a free hosting domain in the dot-tk (.tk) domain space (.tk is the country code for Tokelau — a group of atolls in the South Pacific).

From there the system is smart enough to redirect users back to Twitter if they appear to be part of any automated attempt to crawl the links (e.g. by using site download and mirroring tools like cURL), the researchers found. They said this was likely a precaution on the part of the spammers to avoid detection by automated scanners looking for bot activity on Twitter. Requests from visitors who look like real users responding to tweets are redirected to the porn spam sites.

Because the links promoted by those spammy Twitter accounts all abused short link services from Twitter and Google, the researchers were able to see that this entire botnet has generated more than 30 million unique clicks from February to June 2017.

[SIDE NOTE: Anyone seeking more context about what’s being promoted here can check out the Web site datinggold[dot]com [Caution: Not-Safe-for-Work], which suggests it’s an affiliate program that rewards marketers who drive new signups to its array of “online dating” offerings — mostly “cheating,” “hookup” and “affair-themed” sites like “AdsforSex,” “Affair Hookups,” and “LocalCheaters.” Note that this program is only interested in male signups.]

The datinggold affiliate site which pays spammers to bring male signups to "online dating" services.

The datinggold affiliate site which pays spammers to bring male signups to “online dating” services.

Allen said the Twitter botnet relies heavily on accounts that have been “aged” for a period of time as another method to evade anti-spam techniques used by Twitter, which may treat tweets from new accounts with more prejudice than those from established accounts. ZeroFOX said about 20 percent of the Twitter accounts identified as part of the botnet were aged at least one year before sending their first tweet, and that the botnet overall demonstrates that these affiliate programs have remained lucrative by evolving to harness social media.

“The final redirect sites encourage the user to sign up for subscription pornography, webcam sites, or fake dating,” ZeroFOX wrote in a report being issued this week. “These types of sites, although legal, are known to be scams.”

Perhaps the most well-known example of the subscription-based dating/cheating service that turned out to be mostly phony was AshleyMadison. After AshleyMadison’s user databases were plundered and published online, the company admitted that its service used at least 70,000 female chatbots that were programmed to message new users and try to entice them into replying — which required a paid account.

“Many of the sites’ policies claim that the site owners operate most of the profiles,” ZeroFOX charged. “They also have overbearing policies that can use personally information of their customers to send to other affiliate programs, yielding more spam to the victim. Much like the infamous ‘partnerka’ networks from the Russian Business Network, money is paid out via clicks and signups on affiliate programs” [links added].

Although the Twitter botnet discovered by ZeroFOX has since been dismantled, it not hard to see how this same approach could be very effective at spreading malware. Keep your wits about you while using or cruising social media sites, and be wary of any posts or profiles that match the descriptions and behavior of the bot accounts described here.

For more on this research, see ZeroFOX’s blog post Inside a Massive Siren Social Network Spam Botnet.

,

CryptogramBook Review: Twitter and Tear Gas, by Zeynep Tufekci

There are two opposing models of how the Internet has changed protest movements. The first is that the Internet has made protesters mightier than ever. This comes from the successful revolutions in Tunisia (2010-11), Egypt (2011), and Ukraine (2013). The second is that it has made them more ineffectual. Derided as "slacktivism" or "clicktivism," the ease of action without commitment can result in movements like Occupy petering out in the US without any obvious effects. Of course, the reality is more nuanced, and Zeynep Tufekci teases that out in her new book Twitter and Tear Gas.

Tufekci is a rare interdisciplinary figure. As a sociologist, programmer, and ethnographer, she studies how technology shapes society and drives social change. She has a dual appointment in both the School of Information Science and the Department of Sociology at University of North Carolina at Chapel Hill, and is a Faculty Associate at the Berkman Klein Center for Internet and Society at Harvard University. Her regular New York Times column on the social impacts of technology is a must-read.

Modern Internet-fueled protest movements are the subjects of Twitter and Tear Gas. As an observer, writer, and participant, Tufekci examines how modern protest movements have been changed by the Internet­ -- and what that means for protests going forward. Her book combines her own ethnographic research and her usual deft analysis, with the research of others and some big data analysis from social media outlets. The result is a book that is both insightful and entertaining, and whose lessons are much broader than the book's central topic.

"The Power and Fragility of Networked Protest" is the book's subtitle. The power of the Internet as a tool for protest is obvious: it gives people newfound abilities to quickly organize and scale. But, according to Tufekci, it's a mistake to judge modern protests using the same criteria we used to judge pre-Internet protests. The 1963 March on Washington might have culminated in hundreds of thousands of people listening to Martin Luther King Jr. deliver his "I Have a Dream" speech, but it was the culmination of a multi-year protest effort and the result of six months of careful planning made possible by that sustained effort. The 2011 protests in Cairo came together in mere days because they could be loosely coordinated on Facebook and Twitter.

That's the power. Tufekci describes the fragility by analogy. Nepalese Sherpas assist Mt. Everest climbers by carrying supplies, laying out ropes and ladders, and so on. This means that people with limited training and experience can make the ascent, which is no less dangerous -- to sometimes disastrous results. Says Tufekci: "The Internet similarly allows networked movements to grow dramatically and rapidly, but without prior building of formal or informal organizational and other collective capacities that could prepare them for the inevitable challenges they will face and give them the ability to respond to what comes next." That makes them less able to respond to government counters, change their tactics­ -- a phenomenon Tufekci calls "tactical freeze" -- make movement-wide decisions, and survive over the long haul.

Tufekci isn't arguing that modern protests are necessarily less effective, but that they're different. Effective movements need to understand these differences, and leverage these new advantages while minimizing the disadvantages.

To that end, she develops a taxonomy for talking about social movements. Protests are an example of a "signal" that corresponds to one of several underlying "capacities." There's narrative capacity: the ability to change the conversation, as Black Lives Matter did with police violence and Occupy did with wealth inequality. There's disruptive capacity: the ability to stop business as usual. An early Internet example is the 1999 WTO protests in Seattle. And finally, there's electoral or institutional capacity: the ability to vote, lobby, fund raise, and so on. Because of various "affordances" of modern Internet technologies, particularly social media, the same signal -- a protest of a given size -- reflects different underlying capacities.

This taxonomy also informs government reactions to protest movements. Smart responses target attention as a resource. The Chinese government responded to 2015 protesters in Hong Kong by not engaging with them at all, denying them camera-phone videos that would go viral and attract the world's attention. Instead, they pulled their police back and waited for the movement to die from lack of attention.

If this all sounds dry and academic, it's not. Twitter and Tear Gasis infused with a richness of detail stemming from her personal participation in the 2013 Gezi Park protests in Turkey, as well as personal on-the-ground interviews with protesters throughout the Middle East -- particularly Egypt and her native Turkey -- Zapatistas in Mexico, WTO protesters in Seattle, Occupy participants worldwide, and others. Tufekci writes with a warmth and respect for the humans that are part of these powerful social movements, gently intertwining her own story with the stories of others, big data, and theory. She is adept at writing for a general audience, and­despite being published by the intimidating Yale University Press -- her book is more mass-market than academic. What rigor is there is presented in a way that carries readers along rather than distracting.

The synthesist in me wishes Tufekci would take some additional steps, taking the trends she describes outside of the narrow world of political protest and applying them more broadly to social change. Her taxonomy is an important contribution to the more-general discussion of how the Internet affects society. Furthermore, her insights on the networked public sphere has applications for understanding technology-driven social change in general. These are hard conversations for society to have. We largely prefer to allow technology to blindly steer society or -- in some ways worse -- leave it to unfettered for-profit corporations. When you're reading Twitter and Tear Gas, keep current and near-term future technological issues such as ubiquitous surveillance, algorithmic discrimination, and automation and employment in mind. You'll come away with new insights.

Tufekci twice quotes historian Melvin Kranzberg from 1985: "Technology is neither good nor bad; nor is it neutral." This foreshadows her central message. For better or worse, the technologies that power the networked public sphere have changed the nature of political protest as well as government reactions to and suppressions of such protest.

I have long characterized our technological future as a battle between the quick and the strong. The quick -- dissidents, hackers, criminals, marginalized groups -- are the first to make use of a new technology to magnify their power. The strong are slower, but have more raw power to magnify. So while protesters are the first to use Facebook to organize, the governments eventually figure out how to use Facebook to track protesters. It's still an open question who will gain the upper hand in the long term, but Tufekci's book helps us understand the dynamics at work.

This essay originally appeared on Vice Motherboard.

The book on Amazon.com.

CryptogramFriday Squid Blogging: Eyeball Collector Wants a Giant-Squid Eyeball

They're rare:

The one Dubielzig really wants is an eye from a giant squid, which has the biggest eye of any living animal -- it's the size of a dinner plate.

"But there are no intact specimens of giant squid eyes, only rotten specimens that have been beached," he says.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Read my blog posting guidelines here.

CryptogramForged Documents and Microsoft Fonts

A set of documents in Pakistan were detected as forgeries because their fonts were not in circulation at the time the documents were dated.

Worse Than FailureError'd: Unfortunate Timing

"Apparently, I viewed the page during one of those special 31 seconds of the year," wrote Richard W.

 

"Well, it looks like I'll be paying full price for this repair," wrote Ryan W.

 

Marco writes, "So...that's like September 2nd?"

 

"Office 365????? I guess so, but ONLY if you're sure, Microsoft..." writes Leonid T.

 

Brandon writes, "Being someone who lives in 'GMT +1', I have to wonder, is it the 6th, 11th, or 12th of July or the 7th of June, November, or December we're talking about here?"

 

"Translated, it reads, 'Bayrou and Modem in angry mode', assuming of course it really is Mr. Bayrou in the pic," wrote Matt.

 

[Advertisement] Universal Package Manager - ProGet easily integrates with your favorite Continuous Integration and Build Tools, acting as the central hub to all your essential components. Learn more today!

Don MartiSmart futures contracts on software issues talk, and bullshit walks?

Previously: Benkler’s Tripod, transactions from a future software market, more transactions from a future softwware market

Owning "equity" in an outcome

John Robb: Revisiting Open Source Ventures:

Given this, it appears that an open source venture (a company that can scale to millions of worker/owners creating a new economic ecosystem) that builds massive human curated databases and decentralizes the processing load of training these AIs could become extremely competitive.

But what if the economic ecosystem could exist without the venture? Instead of trying to build a virtual company with millions of workers/owners, build a market economy with millions of participants in tens of thousands of projects and tasks? All of this stuff scales technically much better than it scales organizationally—you could still be part of a large organization or movement while only participating directly on a small set of issues at any one time. Instead of holding equity in a large organization with all its political risk, you could hold a portfolio of positions in areas where you have enough knowledge to be comfortable.

Robb's opportunity is in training AIs, not in writing code. The "oracle" for resolving AI-training or dataset-building contracts would have to be different, but the futures market could be the same.

The cheating project problem

Why would you invest in a futures contract on bug outcomes when the project maintainer controls the bug tracker?

And what about employees who are incentivized from both sides: paid to fix a bug but able to buy futures contracts (anonymously) that will let them make more on the market by leaving it open?

In order for the market to function, the total reputation of the project and contributors must be high enough that outside participants believe that developers are more motivated to maintain that reputation than to "take a dive" on a bug.

That implies that there is some kind of relationship between the total "reputation capital" of a project and the maximum market value of all the futures contracts on it.

Open source metrics

To put that another way, there must be some relationship between the market value of futures contracts on a project and the maximum reputation value of the project. So that could be a proxy for a difficult-to-measure concept such as "open source health."

Open source journalism

Hey, tickers to put into stories! Sparklines! All the charts and stuff that finance and sports reporters can build stories around!

,

Krebs on SecurityThieves Used Infrared to Pull Data from ATM ‘Insert Skimmers’

A greater number of ATM skimming incidents now involve so-called “insert skimmers,” wafer-thin fraud devices made to fit snugly and invisibly inside a cash machine’s card acceptance slot. New evidence suggests that at least some of these insert skimmers — which record card data and store it on a tiny embedded flash drive  — are equipped with technology allowing them to transmit stolen card data wirelessly via infrared, the same communications technology that powers a TV remote control.

Last month the Oklahoma City metropolitan area experienced rash of ATM attacks involving insert skimmers. The local KFOR news channel on June 28, 2017 ran a story stating that at least four banks in the area were hit with insert skimmers.

The story quoted a local police detective saying “the skimmer contains an antenna which transmits your card information to a tiny camera hidden somewhere outside the ATM.”

Financial industry sources tell KrebsOnSecurity that preliminary analysis of the insert skimmers used in the attacks suggests they were configured to transmit stolen card data wirelessly to the hidden camera using infrared, a short-range communications technology most commonly found in television remote controls.

Here’s a look at one of the insert skimmers that Oklahoma authorities recently seized from a compromised ATM:

An insert skimmer retrieved from a compromised cash machine in Oklahoma City.

An insert skimmer retrieved from a compromised cash machine in Oklahoma City. Image: KrebsOnSecurity.com.

In such an attack, the hidden camera has a dual function: To record time-stamped videos of ATM users entering their PINs; and to receive card data recorded and transmitted by the insert skimmer. In this scenario, the fraudster could leave the insert skimmer embedded in the ATM’s card acceptance slot, and merely swap out the hidden camera whenever its internal battery is expected to be depleted.

Of course, the insert skimmer also operates on an embedded battery, but according to my sources the skimmer in question was designed to turn on only when someone uses the cash machine, thereby preserving the battery.

Thieves involved in skimming attacks have hidden spy cameras in some pretty ingenious places, such as a brochure rack to the side of the cash machine or a safety mirror affixed above the cash machine (some ATMs legitimately place these mirrors so that customers will be alerted if someone is standing behind them at the machine).

More often than not, however, hidden cameras are placed behind tiny pinholes cut into false fascias that thieves install directly above or beside the PIN pad. Unfortunately, I don’t have a picture of a hidden camera used in the recent Oklahoma City insert skimming attacks.

Here’s a closer look at the insert skimmer found in Oklahoma:

Image: KrebsOnSecurity.com.

Image: KrebsOnSecurity.com.

A source at a financial institution in Oklahoma shared the following images of the individuals who are suspected of installing these insert skimming devices.

Individuals suspected of installing insert skimmers in a rash of skimming attacks last month in Oklahoma City. Image: KrebsOnSecurity.com.

Individuals suspected of installing insert skimmers in a rash of skimming attacks last month in Oklahoma City. Image: KrebsOnSecurity.com.

As this skimming attack illustrates, most skimmers rely on a hidden camera to record the victim’s PIN, so it’s a good idea to cover the pin pad with your hand, purse or wallet while you enter it.

Yes, there are skimming devices that rely on non-video methods to obtain the PIN (such as PIN pad overlays), but these devices are comparatively rare and quite a bit more expensive for fraudsters to build and/or buy.

So cover the PIN pad. It also protects you against some ne’er-do-well behind you at the ATM “shoulder surfing” you to learn your PIN (which would likely be followed by a whack on the head).

It’s an elegant and simple solution to a growing problem. But you’d be amazed at how many people fail to take this basic, hassle-free precaution.

If you’re as fascinated as I am with all these skimming devices, check out my series All About Skimmers.

CryptogramTomato-Plant Security

I have a soft spot for interesting biological security measures, especially by plants. I've used them as examples in several of my books. Here's a new one: when tomato plants are attacked by caterpillars, they release a chemical that turns the caterpillars on each other:

It's common for caterpillars to eat each other when they're stressed out by the lack of food. (We've all been there.) But why would they start eating each other when the plant food is right in front of them? Answer: because of devious behavior control by plants.

When plants are attacked (read: eaten) they make themselves more toxic by activating a chemical called methyl jasmonate. Scientists sprayed tomato plants with methyl jasmonate to kick off these responses, then unleashed caterpillars on them.

Compared to an untreated plant, a high-dose plant had five times as much plant left behind because the caterpillars were turning on each other instead. The caterpillars on a treated tomato plant ate twice as many other caterpillars than the ones on a control plant.