Planet Javasummit

November 18, 2017

Tim Bray Dueling Camera Apps

I got a Pixel 2, largely because it’s said to have a really great camera, with software-driven magic — machine learning at work. Here are two shot comparisons between the Google and Lightroom Android camera apps to see what that means in practice.

Why Lightroom?

Given a choice, I prefer the Lightroom app to Android’s. It has better, more intuitive ergonomics, including a level; makes the phone feel more like a camera. Also, you can edit in the Android version of Lightroom, which has basically the same controls as the desktop version I live in. Also, it shoots and edits DNG “RAW” files. Finally, anything you don’t erase is synced through the Adobe cloud and is auto-magically there in my Mac Lightroom’s “All synced photos” collection. (Yes, even in the Lightroom CC Classic version.) Not only that, but when I edit it on the Mac, the edits are synced back to the phone, so I can show people the improved versions while we’re having lunch. A pretty sweet package, all things considered.

There is a fly in the ointment. The Lightroom app’s pix’s names may end with “.dng” but if you’re used to the massive depth of the files like the ones I get from my Fujifilm XT-1, where you can pull lost beauty out of darkness or dazzle, you’ll be disappointed. Sure, you can pull the “Highlights” slider down or the “Shadows” slider up, and it sort of works, but not like with real camera files.

When it doesn’t matter

These days all cameras are great, given enough light and an appropriate subject. I’ve pretty well totally stopped using the 10-24mm wide-angle with the big camera because my phone is basically Good Enough.

So if we’re going to compare these apps meaningfully, we need to work with hard-to-take pictures that stress out the sensor; the most obvious examples are low light and high contrast.

Well, in November in the Pacific Northwest there really isn’t that much high contrast, but we got plenty of low light. I tried for a different kind of high contrast anyhow like so.

Vancouver night street scene Vancouver night street scene

I’m going to call this one pretty well a wash. The Android-camera version achieved slightly sharper focus, but that’s not really the point in an impressionistic piece like this. What’s significant is that I had to put in a couple minutes photo-editing on the Lightroom DNG to get it to look as good; the sky had a bit of grey luminance noise and the whole scene leaned yellow. Having said that, I like photo-editing.

Oh, I didn’t say, did I? It’s Android above, Lightroom below.

So what the Android camera is doing here is taking whatever comes off the sensor, putting it through a little photo-editing session right there on the camera, giving me a JPEG, and saying, in effect, “don’t bother your pretty little head about how I got this.”

Finally, I’m usually really happy with Lightroom’s photo-export software, but in this case both pix lost some life, in particular in the trolley-wire sparkles and taillight reds. I’m going to have to try some new tricks.

Also worth noting: These are not terribly difficult or challenging for the sensor: The objects in the picture are pretty well self-illuminating.

When it matters

Here’s a hard one, our new calico cat, asleep on the sofa after a hard night of watching Star Trek Discovery with Mom & Dad. This is a softly-lit book-lined room with black furniture and a dark floor. And, in this case, Android pretty well wiped the floor with Lightroom.

Cat, by Lightroom Cat, by Android Camera

Once again, Lightroom above/Android below. That Lightroom version has been heavily edited, and it’s still not close. The Android version has truer colors, better focus, and less noise. I’m seriously impressed with whatever is going on inside that app.

You know, when you look at the two of these side by side in Lightroom on my 15" Retina Mac screen, it’s like night and day. But as I look at the 720-wide presentation here in the blog draft, I wonder if the differences really matter.

More on the Android app

It’s nice, but trying too hard. No, I don’t want a little slab of video prepended to my photos so they shimmer into place (and can’t be edited). No, I don’t want color-balance modes, since the ML seems to get that right.

Also, since everything is apparently auto-magically cloudified, there ought to be an easy/automatic way to get the full-rez versions of the pix out of the cloud and into Lightroom, but I haven’t found it yet. For the moment, I share from the phone to Dropbox, and Lightroom is happy pulling from there.

Also it’s dumb that I have to switch apps to edit the photo I just took, and then the editing controls are all presets and oversimplification. Having said that, the app has a Chromecast button, which is super nice.

What next?

For shots that don’t challenge the camera, I’ll go on using Lightroom; it’s a better shooting experience and better integrated with my workflow. When it gets tricky, I’ll bring that Android ML to bear.

November 17, 2017

Worse Than FailureError'd: Never ASSume that You're Free from Errors

"This was in an email from Nest. I'm sure in some other font this shows a heartwarming image of fluffy bunnies frolicking in an energy saving Utopia, but instead, we get this," wrote Matthew W.


"Um...yeah, sure I guess?" writes Chris U.


Stuart L. wrote, "Looks like the weather has made an 8-bit turn for the worse."


"I had no idea that the success of entering my enrollment depended on whether or not my donkey was nearby," writes Ernie D.


Jamie S. writes, "What exactly are you trying to smuggle in, Fujitsu updater?"


"I'm the fastest man alive. Don't believe me? Check this out," writes John W.


[Advertisement] Atalasoft’s imaging SDKs come with APIs & pre-built controls for web viewing, browser scanning, annotating, & OCR/barcode capture. Try it for 30 days with included support.

XKCDHow to Make Friends

November 16, 2017

Worse Than FailureCodeSOD: Delebation

When faced with an API or programming paradigm that requires repetitive, boilerplate code, a developer is left with two options. They may refine or adapt the API/paradigm, using the idioms of their language to make something tedious and verbose into something elegant and clear.

Or they just automate it. If you have a mile of boilerplate that’s mostly the same across the application, just generate that. It’s like copy/paste, but, y’know… automatic.

Which is why Derf Skren found this pile in their codebase:

  public abstract class ExchangeSingleData : IExchangeData
    private readonly string mName;
    private readonly int mLength;

    private Dictionary<string, string> mMapValidData;
    private byte[] mBuffer;

    void AddValidValue(string name, string value) {
        mMapValidData[name] = value;

  public class NetChangeSign : ExchangeSingleData
        public const string Plus = "+";
        public const string Minus = "-";

    public NetChangeSign()
      : base("NetChangeSign", 1)
            AddValidValue("Plus", Plus);
            AddValidValue("Minus", Minus);

  public class BidPriceSign : ExchangeSingleData
        public const string Plus = "+";
        public const string Minus = "-";

    public BidPriceSign()
      : base("BidPriceSign", 1)
            AddValidValue("Plus", Plus);
            AddValidValue("Minus", Minus);

  public class AskPriceSign : ExchangeSingleData
        public const string Plus = "+";
        public const string Minus = "-";

    public AskPriceSign()
      : base("AskPriceSign", 1)
            AddValidValue("Plus", Plus);
            AddValidValue("Minus", Minus);

  // ... and 7 more versions of the same class

The goal of this code is so that they can prepend a “+” or a “-” to a transaction’s value. Note the mBuffer in the base class- they don’t use strings (or, y’know… numbers) to represent the transaction value, but a byte array instead. The “value” is that it lets them write a line like this:


Which allows the instance stored in NetChangeSign to flip that +/- based on the return value of GeneratePriceSign. Obviously, this lets the NetChangeSign instance have full control of the logic of how the sign gets set, right? I mean, each instance has its own map that contains all the allowed values, right? Well… sure, but how do they decide? Based on GeneratePriceSign… which looks like this:

  private static string GeneratePriceSign(Side aSide)
    if (aSide.Equals(Side.Buy))
      return "+";
      return "-";

In design patterns terms, we call this “delebation”. It’s like delegation, but only the person doing it to themselves enjoys it.

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

November 15, 2017

Worse Than FailureThe For While Loop

Alex R. was the architect of a brand spanking new system that was to read inputs from numerous other internal systems, crunch a whole bunch of numbers, record everything in a database and spew forth a massive report file. He spent months designing the major details of the system, and more months designing the various sub-components. From all this came a variety of business-level data structures which spawned POJOs and the underlying DB tables to store assorted inputs, flags and outputs. He did a fairly thorough job of documenting all the interfaces, and provided detailed specifications for all of the next-level methods that were left as TBDs in the design.

Java Programming Cover

The project manager then assigned units of work to numerous offshored junior developers who managed to get virtually everything wrong. If they couldn't understand what a spec required, they changed the spec to reflect what they actually wrote. This caused Alex to start versioning the requirements document in order to catch the changes by the junior developers so that they could be rolled back.

After a while, the number of junior-developer-caused issues was piling up and Alex suggested some training sessions on certain ways of doing things to reduce the chaff he had to deal with. Management turned him down because they couldn't afford to take developers off of coding tasks for purposes of training; there was a schedule to keep! The fact that oodles of time were being wasted on them building the wrong stuff only to have to have why it was wrong explained and then have them go back and re-do it - sometimes 6 or 7 times - was irrelevant.

So how does one deal with idiotic management like this?

Alex thought that he had found a way to expose the problem and (hopefully) force something to be done. He would put in something (that any experienced developer should be able to spot as a simple code formatting issue) that the junior developers would never spot. The code would work correctly, but it would stymie them so that they had to first understand it before they could change it. He used the following coding style in a variety of locations throughout the codebase and waited:

  List<Widget> widgets;
  for (int i=0; i<limit; i++) {
      // Do stuff
  } while ((widgets = getWidgets()) == null);

For those not familiar with Java, the closing brace of a for-loop is followed by an implicit semicolon, so the while (expression); statement is unrelated to the for-loop. However, the junior developers didn't know this, and couldn't find any documentation on a for-while statement. Although they were able to create little test programs, they didn't understand how the while-expression controlled the for-loop (it doesn't). In this case, the underlying DAO either returned a populated list or threw an exception, so it was effectively while-false (the function call and assignment occurred once) and was just syntactic nonsense that confused the junior developers.

They couldn't recognize a Java 101 code format issue and they were sufficiently stubborn that they refused to simply ask Alex what the code was doing. They were even foolish enough to openly discuss it amongst themselves on a conference line - agreeing not to ask for help until they figured it out - before a meeting with Alex and his boss began.

After 6 weeks of them floundering around on it, the offshore manager finally brought the issue up with Alex and his boss, at which time Alex explained what running the code formatter would show. He then pointed out that since they didn't know the basics of reading Java code and preferred to waste massive amounts of time rather than just asking about something they didn't understand, it was clear that they didn't have the wherewithall to make technical decisions on a larger scale, or change the design documents as they saw fit.

He continued to point out that until the junior developers showed marked improvements in their understanding of simple code, they should concentrate on learning to do basic programming instead of trying to be architects. To this end, he again offered to have ongoing training sessions where he would attempt to raise their skill level.

Of course management backed the cheap offshore labor. It was at this point that Alex realized it was a lost cause, so he fixed all the for-while snippets and updated the latest version of the detailed design document with a new opening paragraph:

  To Whomever Inherits This System:

  Detailed design documents were created by experienced people. Management decreed
  that junior developers could ignore them, at will and without penalty. The state
  of the code reflects this.
  Fair Warning!

Then he committed it, secure in the knowledge that the junior developers would never bother to look at it again once he was gone. Then he gave two weeks notice.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

XKCDTemperature Preferences

November 14, 2017

Worse Than FailureCodeSOD: One's Company

The more you learn about something, the less confident you often become in making statements about it, because you understand the complexities of the matter. If, for example, I asked you to help me refine my definition of how dates and times work, you know that many assumptions are wrong. Or if we tried to define what makes a string a person’s name, we’ll run into similar problems. This is even true for a value we’ve all probably seen implemented as a boolean value: gender. The more you learn about these subjects, the more complex and nuanced your understanding of them becomes. More and more, your answers start with, “It’s complicated…”.

Eugene was going through some code at a customer’s site, and he found that their business logic depended heavily on a flag ISCOMAPNY (sic), but there was no ISCOMPANY field anywhere in the database. There was, however, a SEX field on the customer records, implemented as an integer.

Digging through the queries, Eugene found a new approach to defining a company:

    WHEN '6'
THEN '-1'
    WHEN '9'
THEN '-1'
    ELSE '0'
FROM customers

Like I said, it's complicated.

[Advertisement] Easily create complex server configurations and orchestrations using both the intuitive, drag-and-drop editor and the text/script editor.  Find out more and download today!

November 13, 2017

XKCDNightmare Email Feature

November 10, 2017

XKCDTwitter Verification

November 05, 2017

Tim Bray Vintage Nauticals

We keep our boat at Horseshoe Bay, a pretty little place when approached by land. Boaters inbound by ocean come face-to-face with a huge antique float/pier/breakwater kind of thing, whose ugliness I’ve long found magnificent and which today I took the time to tour and photograph.

Vintage Nautical

Turns out this huge steel thing is afloat, actually; here’s how it’s fastened to the land. Those are big chains and huge truck tires.

What happened was, I was out winterizing the boat. Vancouver missed autumn this year, snapped over from 18°C Indian-summer to basically zero more or less overnight. Today a few snowflakes drifted down and I was wearing a nontraditional Canadian boating toque. You can see snowflakes in all but one of today’s photos, if you look closely.

I’d neglected my camera and was thus shooting with the Pixel, really not much of a handicap for daylight photos when they’re close-ups and big landscapes. Here are one of each.

Vintage Nautical

Wow, this thing is, well, “going back to nature” doesn’t feel like the right phrase. But there’s definitely an entropy gradient.

Boat approaching Horseshoe Bay

I’ve been using the Lightroom camera app, which is a little on the slow side but has nice controls and pulls the raw bits off the sensor, and I like the app’s editing primitives. But in recent weeks it’s become unusably crashy — I’ve already cleared its data once to no effect — so my fallback is the well-regarded native Android Camera app. It’s quick and the pix look good, I have to say. And while Lightroom claims to be shooting RAW and indeed emits DNG’s, they don’t contain anything like the amount of hidden potential a good RAW file from a real camera exhibits. So the native Android JPEGs are not making me unhappy.

I have to shuffle them through DropBox to get them on the Mac so I can prettify them with Lightroom; I miss Adobe’s nice smooth mobile/PC sync. Anyhow, I thought Lightroom did a nice job on this monochrome.

Vintage Nautical

This float is quite a structure. It’s not solid, it’s sort of a frame so you can look down at the ocean in the middle. Well, you could, except for that piece of water is stacked up with old styrofoam, the kind that’s used to make floating docks float.

Vintage Nautical

This picture isn’t beautiful or anything, but
there’s a lot happening, top to bottom, left to right.

As for the boat, I installed its low-level winter heating apparatus, ran the engine for a bit, set up support poles to protect the canvas cover from heavy snow, and adjusted the lines against winter storms; then went for a burger and beer at the waterfront alehouse.

That’s a satisfying early-winter half-day.

November 03, 2017

etbeWork Stuff

Does anyone know of a Linux support company that provides 24*7 support to Ruby and PHP applications? I have a client that is looking for such a company.

Also I’m looking for more consulting work. If anyone knows of an organisation that needs some SE Linux consulting, or support for any of the FOSS software I’ve written then let me know. I take payment by Paypal and Bitcoin as well as all the usual ways. I can make a private build of any of my FOSS software to suit your requirements or if you want features that could be used by other people (and don’t conflict with the general use cases) I can add them on request. Small changes start at $100.

October 30, 2017

etbeLogic of Zombies

Most zombie movies feature shuffling hordes which prefer to eat brains but also generally eat any human flesh available. Because in most movies (pretty much everything but the 28 Days Later series [1]) zombies move slowly they rely on flocking to be dangerous.

Generally the main way of killing zombies is severe head injury, so any time zombies succeed in their aim of eating brains they won’t get a new recruit for their horde. The TV series iZombie [2] has zombies that are mostly like normal humans as long as they get enough brains and are smart enough to plan to increase their horde. But most zombies don’t have much intelligence and show no signs of restraint so can’t plan to recruit new zombies. In 28 Days Later the zombies aren’t smart enough to avoid starving to death, in contrast to most zombie movies where the zombies aren’t smart enough to find food other than brains but seem to survive on magic.

For a human to become a member of a shuffling horde of zombies they need to be bitten but not killed. They then need to either decide to refrain from a method of suicide that precludes becoming a zombie (gunshot to the head or jumping off a building) or unable to go through with it. Most zombie movies (I think everything other than 28 Days Later) has the transition process taking some hours so there’s plenty of time for an infected person to kill themself or be killed by others. Then they need to avoid having other humans notice that they are infected and kill them before they turn into a zombie. This doesn’t seem likely to be a common occurrence. It doesn’t seem likely that shuffling zombies (as opposed to the zombies in 28 Days Later or iZombie) would be able to form a horde.

In the unlikely event that shuffling zombies managed to form a horde that police couldn’t deal with I expect that earth-moving machinery could deal with them quickly. The fact that people don’t improvise armoured vehicles capable of squashing zombies is almost as ridiculous as all the sci-fi movies that feature infantry.

It’s obvious that logic isn’t involved in the choice of shuffling zombies. It’s more of a choice of whether to have the jump-scare aspect of 18 Days Later, the human-drama aspect of zombies that pass for human in iZombie, or the terror of a slowly approaching horrible fate that you can’t escape in most zombie movies.

I wonder if any of the music streaming services have a horror-movie playlist that has screechy music to set your nerves on edge without the poor plot of a horror movie. Could listening to scary music in the dark become a thing?

October 27, 2017

Tim Bray Working at Amazon

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?


That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.


It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.


If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

October 26, 2017

etbeAnarchy in the Office

Some of the best examples I’ve seen of anarchy working have been in corporate environments. This doesn’t mean that they were perfect or even as good as a theoretical system in which a competent manager controlled everything, but they often worked reasonably well.

In a well functioning team members will encourage others to do their share of the work in the absence of management. So when the manager disappears (doesn’t visit the team more than once a week and doesn’t ask for any meaningful feedback on how things are going) things can still work out. When someone who is capable of doing work isn’t working then other people will suggest that they do their share. If resources for work (such as a sufficiently configured PC for IT work) aren’t available then they can be found (abandoned PCs get stripped and the parts used to upgrade the PCs that need it most).

There was one time where a helpdesk worker who was about to be laid off was assigned to the same office as me (apparently making all the people in his group redundant took some time). So I started teaching him sysadmin skills, assigned work to him, and then recommended that my manager get him transferred to my group. That worked well for everyone.

One difficult case is employees who get in the way of work being done, those who are so incompetent that they break enough things to give negative productivity. One time when I was working in Amsterdam I had two colleagues like that, it turned out that the company had no problem with employees viewing porn at work so no-one asked them to stop looking at porn. Having them paid to look at porn 40 hours a week was much better than having them try to do work. With anarchy there’s little option to get rid of bad people, so just having them hang out and do no work was the only option. I’m not advocating porn at work (it makes for a hostile work environment), but managers at that company did worse things.

One company I worked for appeared (from the non-management perspective) to have a management culture of doing no work. During my time there I did two “annual reviews” in two weeks, and the second was delayed by over 6 months. The manager in question only did the reviews at that time because he was told he couldn’t be promoted until he got the backlog of reviews done, so apparently being more than a year behind in annual reviews was no obstacle to being selected for promotion. On one occasion I raised the issue of a colleague who had done no work for over a year (and didn’t even have a PC to do work) with that manager, his response was “what do you expect me to do”! I expected him to do anything other than blow me off when I reported such a serious problem! But in spite of that strictly work-optional culture enough work was done and the company was a leader in it’s field.

There has been a lot of research into the supposed benefits of bonuses etc which usually turn out to reduce productivity. Such research is generally ignored presumably because the people who are paid the most are the ones who get to decide whether financial incentives should be offered so they choose the compensation model for the company that benefits themselves. But the fact that teams can be reasonably productive when some people are paid to do nothing and most people have their work allocated by group consensus rather than management plan seems to be a better argument against the typical corporate management.

I think it would be interesting to try to run a company with an explicit anarchic management and see how it compares to the accidental anarchy that so many companies have. The idea would be to have minimal management that just does the basic HR tasks (preventing situations of bullying etc), a flat pay rate for everyone (no bonuses, pay rises, etc) and have workers decide how to spend money for training, facilities, etc. Instead of having middle managers you would have representatives elected from each team to represent their group to senior management.

PS Australia has some of the strictest libel laws in the world. Comments that identify companies or people are likely to be edited or deleted.

October 14, 2017

Tim Bray Live Metal is Better

Recently I went out for a live metal triple-bill, with Endon and SUMAC opening for Boris, whom I’ve covered here before (with groovy pix). More pix today, with words on the history and meaning of metal, and how to photograph it.

Endon, the openers, are described as catastrophic noise metal and yeah, they were pretty catastrophic. A little too abstract for me, but sincere and really trying to bring it. Didn’t get any pix.

Here’s my problem: I wholeheartedly love this music, in particular live, and yet I have to acknowledge that it’s sort of, well, ridiculous. The volume is much louder than can be sanely necessary. The choreography is all moves invented around 1970 by Robert Plant or Jon Anderson or someone like that. The musical content is built monomaniacally around a single sound, that of an overdriven guitar amp.

SUMAC (on Bandcamp) are very traditional metallistas; the poses, the riffs, the look, the barked vocals.

SUMAC guitarist

Metal is harder to photograph than many other genres, because they like to keep the stage dark, and bathed in subdued reds and blues that stress out the poor sensor. Sometimes you get an incredibly dramatic color treatment, but often the best bet is just to switch to high-contrast B&W.

Wikipedia says the guitarist is Aaron Turner and the bassist Brian Cook, but their picture doesn’t look like this guy.

SUMAC bass player


Metal’s not just one thing, there are many linked strands in its fabric. Check out Heavy metal genres over at Wikipedia (part of WikiProject Metal). For more fun, consider the heavy-metal documentaries by Canadian sociologist Sam Dunn. I haven’t seen them all, but I can heartily recommend Metal — A Headbanger’s Journey, his first, which dives deep, deep, deep, on the subject and is also full of heavy guitar riffs. Best played loud.

Dunn points out that while you can argue all you want about the birth of metal, Black Sabbath usually gets the most votes. I’d go further: It helps that I’m old and was thus there at the time. It was in 1970; I was fifteen and we were visiting my cousin who was a couple years older and infinitely cooler. “Come listen to this record” he said, and put on Sabbath’s eponymous Black Sabbath. Go listen (not to the flavorless sample on the Wikipedia page, the real thing’s all over the Internet). It’s all there, and I mean all there in the first 1:08. Portentous thunderstorm noise, church bell, then four repetitions of a beautifully-heavy three-note riff. No vocals, no nothing, just that awesome guitar tone. I hadn’t imagined that such a sound could exist in this world and my life was never the same after.

There’s more to the song, some pretty good singing from Ozzy and of course the explicitly Satanic lyrics. But Heavy Metal is basically about taking those sixty-eight seconds and building a hundred subgenres and a rabid following around them. I don’t think the occult angle was initially necessary; but it was a clever move by Ozzy and Geezer, and got dragged along with that guitar sound in later years.

Enough history and sociology.


Like I said, the good pictures of Boris are here. On this outing, the lights were darker and the fake-fog flow almost continuous. This time around, they obviously and by design wanted to play behind a curtain of bright red/blue/violet smoke; which as an artistic choice, is nicely harmonious with their music, which itself is nicely harmonious albeit crushingly loud.

For this outing, I rented the Fujinon 23mm f/1.4 from Beau Photo, and the wide-open/wide-angle combination was helpful. But it was still tough; here’s what I got.

Boris in 2017 Boris in 2017 Boris in 2017 Boris in 2017

Pro tip: When the last opener quits playing, you can almost always snag a great photographers’ spot up near the front of the stage. And then, after the first few songs, it’s polite to turn it over to a fellow fan. On this occasion I was dead center and had the rare experience of live metal in stereo, with Wata’s amps to my left and Atsuo’s to my right. Cool!

Yes, Wata picked up an accordion for a couple tunes. The general sound is remarkably like her guitar, but doesn’t allow for her rare-but-wonderful excursions up the neck into screaming-treble territory.


Damn, I love their music. Wata’s guitar tone is up there with the greats, and the beauty of their crushing melodious drone is not equalled by anyone except maybe Sunn O))); and it’s worth mentioning that the Boris/Sunn O))) collaboration Altar is a very beautiful piece of music.

Other Boris outings that I strongly recommend are Pink (from 2005) and the much more recent Attention Please.

Why it’s OK to love Metal

I mean, I’m old and completely out of fucks to give about what others think of my tastes. But still…

First of all, Metal is serious music. Given the size of the crowds at this point in the 21st century (small), it’s no road to riches. And the players are completely oblivious to issues of packaging and pandering. And if you actually listen closely past the surface of the roar, the music is constructed with care, and then obviously the result of endless practice and dedication to the craft. It may lack a backbeat and a catchy chorus, but there’s beauty aplenty in there.

I’ve quoted Sasha Frere-Jones, erstwhile New Yorker rock critic, before on this subject, but he said two things that are worth repeating. First, “Get past the novelty, though, and you find a level of passion and an attention to detail that make a number of mere rock bands look lazy.” And “You may eventually find a TV that is sufficiently large that it makes going to a movie theatre pointless, but you are never going to replicate anything like a black-metal show at home, no matter how fancy your stereo is.”.

Metal people

There are very few gatherings in the world at which I feel totally at home. Examples are O’Reilly’s OSCON and an Amazon principal-engineers’ gathering. But a metal concert is another. Here’s the cute end of metal style.

Metal fans

Aren’t they adorable?

When I was young, I aspired to hair like that, but mine was always too thin and stringy, and now I don’t have much. But the crowd included grizzled hardasses with facial tattoos, a couple of subgroups of Japanese extreme-music culture that I don’t begin to understand, and then a lot of ordinary people off the street who just share the love of this extreme, crazy, deadly-serious art form.

It’s OK to crowd up to the front and stand there like a post. It’s OK to head-bang when the beat picks up. It’s OK to sit motionless in the back row, eyes closed. It’s a perfectly fine thing, in 2017, to be a greybearded metal-head.

September 28, 2017

etbeProcess Monitoring

Since forking the Mon project to etbemon [1] I’ve been spending a lot of time working on the monitor scripts. Actually monitoring something is usually quite easy, deciding what to monitor tends to be the hard part. The process monitoring script ps.monitor is the one I’m about to redesign.

Here are some of my ideas for monitoring processes. Please comment if you have any suggestions for how do do things better.

For people who don’t use mon, the monitor scripts return 0 if everything is OK and 1 if there’s a problem along with using stdout to display an error message. While I’m not aware of anyone hooking mon scripts into a different monitoring system that’s going to be easy to do. One thing I plan to work on in the future is interoperability between mon and other systems such as Nagios.

Basic Monitoring

ps.monitor tor:1-1 master:1-2 auditd:1-1 cron:1-5 rsyslogd:1-1 dbus-daemon:1- sshd:1- watchdog:1-2

I’m currently planning some sort of rewrite of the process monitoring script. The current functionality is to have a list of process names on the command line with minimum and maximum numbers for the instances of the process in question. The above is a sample of the configuration of the monitor. There are some limitations to this, the “master” process in this instance refers to the main process of Postfix, but other daemons use the same process name (it’s one of those names that’s wrong because it’s so obvious). One obvious solution to this is to give the option of specifying the full path so that /usr/lib/postfix/sbin/master can be differentiated from all the other programs named master.

The next issue is processes that may run on behalf of multiple users. With sshd there is a single process to accept new connections running as root and a process running under the UID of each logged in user. So the number of sshd processes running as root will be one greater than the number of root login sessions. This means that if a sysadmin logs in directly as root via ssh (which is controversial and not the topic of this post – merely something that people do which I have to support) and the master process then crashes (or the sysadmin stops it either accidentally or deliberately) there won’t be an alert about the missing process. Of course the correct thing to do is to have a monitor talk to port 22 and look for the string “SSH-2.0-OpenSSH_”. Sometimes there are multiple instances of a daemon running under different UIDs that need to be monitored separately. So obviously we need the ability to monitor processes by UID.

In many cases process monitoring can be replaced by monitoring of service ports. So if something is listening on port 25 then it probably means that the Postfix “master” process is running regardless of what other “master” processes there are. But for my use I find it handy to have multiple monitors, if I get a Jabber message about being unable to send mail to a server immediately followed by a Jabber message from that server saying that “master” isn’t running I don’t need to fully wake up to know where the problem is.

SE Linux

One feature that I want is monitoring SE Linux contexts of processes in the same way as monitoring UIDs. While I’m not interested in writing tests for other security systems I would be happy to include code that other people write. So whatever I do I want to make it flexible enough to work with multiple security systems.

Transient Processes

Most daemons have a second process of the same name running during the startup process. This means if you monitor for exactly 1 instance of a process you may get an alert about 2 processes running when “logrotate” or something similar restarts the daemon. Also you may get an alert about 0 instances if the check happens to run at exactly the wrong time during the restart. My current way of dealing with this on my servers is to not alert until the second failure event with the “alertafter 2” directive. The “failure_interval” directive allows specifying the time between checks when the monitor is in a failed state, setting that to a low value means that waiting for a second failure result doesn’t delay the notification much.

To deal with this I’ve been thinking of making the ps.monitor script automatically check again after a specified delay. I think that solving the problem with a single parameter to the monitor script is better than using 2 configuration directives to mon to work around it.


Mon currently has a loadavg.monitor script that to check the load average. But that won’t catch the case of a single process using too much CPU time but not enough to raise the system load average. Also it won’t catch the case of a CPU hungry process going quiet (EG when the SETI at Home server goes down) while another process goes into an infinite loop. One way of addressing this would be to have the ps.monitor script have yet another configuration option to monitor CPU use, but this might get confusing. Another option would be to have a separate script that alerts on any process that uses more than a specified percentage of CPU time over it’s lifetime or over the last few seconds unless it’s in a whitelist of processes and users who are exempt from such checks. Probably every regular user would be exempt from such checks because you never know when they will run a file compression program. Also there is a short list of daemons that are excluded (like BOINC) and system processes (like gzip which is run from several cron jobs).

Monitoring for Exclusion

A common programming mistake is to call setuid() before setgid() which means that the program doesn’t have permission to call setgid(). If return codes aren’t checked (and people who make such rookie mistakes tend not to check return codes) then the process keeps elevated permissions. Checking for processes running as GID 0 but not UID 0 would be handy. As an aside a quick examination of a Debian/Testing workstation didn’t show any obvious way that a process with GID 0 could gain elevated privileges, but that could change with one chmod 770 command.

On a SE Linux system there should be only one process running with the domain init_t. Currently that doesn’t happen in Stretch systems running daemons such as mysqld and tor due to policy not matching the recent functionality of systemd as requested by daemon service files. Such issues will keep occurring so we need automated tests for them.

Automated tests for configuration errors that might impact system security is a bigger issue, I’ll probably write a separate blog post about it.

November 18, 2014

Kelvin Lawrence - personal25 Years of the World Wide Web

I have been so busy that I am a few days late putting this post together but hopefully better late than never!

A few days ago, hard though it is to believe, the Worldwide Web, that so many of us take for granted these days, celebrated it's 25th anniversary. Created in 1989 by Sir Tim Berners-Lee , for many of us, "Web" has become as essential in our daily lives as electricity or natural gas. Built from its earliest days upon the notion of open standards the Web has become the information backbone of our current society. My first exposure, that I can remember, to the concept of the Web was in the early 1990s when I was part of the OS/2 team at IBM and we put one of the earliest browsers, Web Explorer, into the operating system and shipped it. Back then, an HTML web page was little more than text, images, animated GIFs and most importantly of all hyperlinks. I was also involved with the team that did some of the early ports of Netscape Navigator to OS/2 and I still recall being blown away by some of what I saw that team doing upon some of my many visits to Netscape in California what seems like a lifetime ago now!

 From those modest but still highly effective beginnings, the Web and most importantly perhaps, the Web browser, has evolved into the complete business and entertainment platform that it is today.

The Web, and open standards, have been part of my personal and work life ever since. I am honored to have been a small part of the evolution of the web myself. I have worked on a number of different projects with great people from all over the World under the auspices of the W3C for longer than I care to remember! I have done a lot of fun things in my career, but one of the highlights was definitely working with so many talented people on the original Scalable Vector Graphics (SVG) specification which is now supported by most of the major browsers and of course you can find my library of SVGsamples here on my site.

It is also so fitting that the latest evolution of Web technology, the finished HTML 5 specification was announced to coincide with the 25th anniversary of the Web.

I could write so much more about what the Web has meant to me but most of all I think my fondest memory will always be all of the great friends I have met and the large number of very talented people that I have had the good fortune to work with through our joint passion to make the Web a better and even more open, place.

Happy (slightly belated) Birthday Worldwide Web and here's to the next 25!

November 13, 2014

Kelvin Lawrence - personalAsian Tiger Mosquitoes

The weather has been unusually cold for the time of year the last day or so. I was actually hoping that if we get a hard freeze it will kill off for now the Asian Tiger mosquitoes that we have been overrun with this year. However I have my doubts as apparently, unlike other mosquitoes, their eggs, which they lay in vegetation and standing water, can survive a harsh winter. They apparently got into the USA in a shipment of waterlogged tires (tyres for my UK friends) some time ago and they are now spreading more broadly. They are covered in black and white stripes and look quite different than the regular "brown" colored mosquitoes we are used to seeing here. They are also a lot more aggressive. They bite all day long (not just at dusk) and even bite animals but definitely prefer humans. It has got so bad that we have had to pay to have our yard sprayed regularly almost all year so that we even have a chance to sit outside and enjoy our yard. These nasty little guys also transmit the chikungunya virus for which I currently believe there is no vaccine. It's not usually fatal but does have some nasty symptoms if you are unlucky enough to catch it. Here's a link to a WebMD write up on these little nasties.

November 12, 2014

Kelvin Lawrence - personalPink Floyd's Endless River - The End of an Era

I just purchased the new Pink Floyd CD from Amazon which includes a free digital download as well. I have been listening to it while I work today. Given the way the album was put together (using material the late Richard Wright recorded almost 20 years ago during the making of The Division Bell) much of the music is immediately familiar. I definitely also hear flashbacks to Wish You Were Here, Dark Side of the Moon and many other albums as well. It's mostly instrumental and there is a lot of it - four sides if you buy the vinyl version!! A lot of the music has an almost eerie tone to it - definitely a good one for the headphones with the lights off. It's a really good listen but left me feeling sad in a way, in a good way I guess, as much of their music has been the backdrop to the last 40 years or so of my life and this is definitely the end of a musical era as supposedly this is the last album the band plan to release. It has a bit of everything for Pink Floyd fans, especially those who like some of the "more recent" albums. Don't expect a bunch of rocking songs that you will be humming along to all day but as a complete work, listened to end to end, I found it very moving. Very much not your modern day pop tune and thank goodness for that!

October 26, 2014

Kelvin Lawrence - personalSeven years post cancer surgery

Today marks another big milestone for me. It has now been seven years since my cancer surgery. As always, I am grateful for all of my family, friends and doctors and every minute that I get to spend with them.

June 03, 2009

Software Summit June 3, 2009: The Finale of Colorado Software Summit

To Our Friends and Supporters,

In these challenging economic times, business has slowed, many companies have had to resort to layoffs and/or closures, and everyone has been tightening their belts. Unfortunately, Colorado Software Summit has not been immune to this downturn. As have so many companies and individuals, we too have experienced a severe decline in our business, and as a result we are not able to continue producing this annual conference.

This year would have been our 18th conference, and we had planned to continue through our 20th in 2011, but instead we must end it now.

Producing this conference has been a wonderful experience for us, truly a labor of love, and we have been extremely privileged to have been able to do well by doing good.  We are very proud of the many people whose careers flourished through what they learned here, of the extensive community we built via the conference, and of the several businesses that were begun through friendships made here. We treasure the friends we made, and we consider them to be part of our extended family. Just as in any family, we celebrated with them through joyous life events and grieved with them through tragic ones.

This is a sad time for us, of course, but not overwhelmingly so. It's sort of the feeling you have when your son leaves for college, or your daughter gets married. You knew it was coming someday, but it is here much sooner than you imagined, and the sadness is sweetened with the joy you had in all that has come before.

We have been privileged to have created a thriving community of friends who met for the first time at the conference, and we want that community to continue. We hope that all of you will stay in touch with us and with each other, and that the Colorado Software Summit community will continue as a source of wisdom and friendship to all of you. If you have ever attended one of our conference, we hope you will consider joining the Colorado Software Summit LinkedIn group as one means to keep in touch.

With our very best wishes for your future, and with unbounded gratitude for your support,

- Wayne and Peggy Kovsky -

All presentations from Colorado Software Summit 2008 have been posted.

May 18, 2009

Software Summit May 17, 2009: Additions to Preliminary Agenda for Colorado Software Summit 2009

We have posted additions to the preliminary agenda for Colorado Software Summit 2009, in two formats:

We will continue to post additions to this agenda during the coming weeks. Please check back here from time to time for additions and/or changes to the agenda, or subscribe to our RSS feed to receive notifications of updates automatically.

Presentations from the 2008 Conference

We have posted presentations for these speakers from Colorado Software Summit 2008:

Presentations from Colorado Software Summit 2008 will be posted periodically throughout the year.

May 03, 2009

Software Summit May 3, 2009: Additions to Preliminary Agenda for Colorado Software Summit 2009

We have posted additions to the preliminary agenda for Colorado Software Summit 2009, in two formats:

We will continue to post additions to this agenda during the coming weeks. Please check back here from time to time for additions and/or changes to the agenda, or subscribe to our RSS feed to receive notifications of updates automatically.

Presentations from the 2008 Conference

We have posted presentations for these speakers from Colorado Software Summit 2008:

Presentations from Colorado Software Summit 2008 will be posted periodically throughout the year.

April 26, 2009

Software Summit April 25, 2009: Preliminary Agenda for Colorado Software Summit 2009

We have posted the preliminary agenda for Colorado Software Summit 2009, in two formats:

We will continue to post additions to this agenda during the coming weeks. Please check back here from time to time for additions and/or changes to the agenda, or subscribe to our RSS feed to receive notifications of updates automatically.

Presentations from the 2008 Conference

We have posted presentations for these speakers from Colorado Software Summit 2008:

Presentations from Colorado Software Summit 2008 will be posted periodically throughout the year.