Planet Russell

,

CryptogramOwn a Pair of Clipper Chips

The AT&T TSD was an early 1990s telephone encryption device. It was digital. Voice quality was okay. And it was the device that contained the infamous Clipper Chip, the U.S. government's first attempt to put a back door into everyone's communications.

Marcus Ranum is selling a pair on eBay. He has the decryption wrong, though. The TSD-3600-E is the model with the Clipper Chip in it. The TSD-3600-F is the version with the insecure exportable algorithm.

Worse Than FailureCodeSOD: What A Load

JCB 3CX Backhoe loader

In the mid-2000s, Amani was contracted to refactor a legacy codebase. He enjoyed breathing new life into old garbage, until the fateful day he came upon something completely unexpected.

One of the webpages he tended to was making calls to a database. Amani couldn't figure out why at first, especially since there were no Ajax scripts on the page. But then he paid careful attention to the CSS details in the webpage's header:

<link rel="stylesheet" href="/style/pictures.css.php"/>

The stylesheet was a PHP file?

Agape, Amani hunted down the file in question and unearthed the horror below:

/* snip */

.popup-yellow .popup-innerHtml {
    background-color: #FFCC00;
}

.top-down {
    font-size: 12px;
    cursor: pointer;
    text-align: left;
}

.grid-a {
    cursor: pointer;
    text-decoration: none;
    color: #00C;
}

query($sql);
while($row = $db->fetchNextObject($result)){
    $id = $row->id;
     $name = "shape".$id;
     $image = "../images/layouts/".$row->image;
    $img = createfrompng($image);
    $width = imageX($img);
    $height = imageY($img);
    echo "#".$name."{ position: relative; width: ".$width."px;  height: ".$height."px; background-color: #ccc; border: 2px solid #FFFFFF; visibility: visible; background-image: url('/images/layouts/".$row->image."'); cursor: pointer; margin: 0; padding: 0; } ";}
?>

The progenitor of this hell-born abomination had written PHP to query a database, load the retrieved images in memory, assign them unique IDs, generate unique CSS style classes for each one, then throw them out—every single time the webpage was requested.

Amani replaced this with a single class of style attributes for all images to share, setting widths and heights to auto. He also removed the unnecessary position: relative; attribute, as nothing on the page was being anchored on the elements.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

,

Cryptogram$7 Million Social Media Privacy Mistake

Forbes estimates that football player Laremy Tunsil lost $7 million in salary because of an ill-advised personal video made public.

Sociological ImagesWomen are less happy than men in marriage, so why does the media insist otherwise?

To Post Secret, a project that collects personal secrets written artistically onto postcards, someone recently sent in the following bombshell: “Ever since we started getting married and buying houses,” she writes, “my girlfriends and I don’t laugh much anymore.”

4

Her personal secret is, in fact, a national one.  It’s part of what has been called the “paradox of declining female happiness.” Women have more rights and opportunities than they have had in decades and yet they are less happy than ever in both absolute terms and relative to men.

Marriage is part of why. Heterosexual marriage is an unequal institution. Women on average do more of the unpaid and undervalued work of households, they work more each day, and they are more aware of this inequality than their husbands. They are more likely to sacrifice their individual leisure and career goals for marriage. Marriage is a moment of subordination and women, more so than men, subordinate themselves and their careers to their relationship, their children, and the careers of their husbands.

Compared to being single, marriage is a bum deal for many woman. Accordingly, married women are less happy than single women and less happy than their husbands, they are less eager than men to marry, they’re more likely to file for divorce and, when they do, they are happier as divorcees than they were when married (the opposite is true for men) and they are more likely than men to prefer never to remarry.

The only reason this is surprising is because of the torrent of propaganda we get that tells us otherwise. We are told by books, sitcoms, reality shows, and romantic comedies that single women are wetting their pants to get hitched. Men are metaphorically or literally drug to the altar in television commercials and wedding comedies, an idea invented by Hugh Hefner in the 1950s (before the “playboy,” men who resisted marriage were suspected of being gay). Not to mention the wedding-themed toys aimed at girls and the ubiquitous wedding magazines aimed solely at women. Why, it’s almost as if they were trying very hard to convince us of something that isn’t true.

But if women didn’t get married to men, what would happen? Marriage reduces men’s violence and conflict in a society by giving men something to lose. It increases men’s efforts at work, which is good for capitalists and the economy. It often leads to children, which exacerbate cycles of earning and spending, makes workers more reliable and dependent on employers, reduces mobility, and creates a next generation of workers and social security investors. Marriage inserts us into the machine. And if it benefits women substantially less than men, then it’s no surprise that so many of our marriage promotion messages are aimed squarely at them.

Lisa Wade is a professor at Occidental College and the co-author of Gender: Ideas, Interactions, Institutions. Find her on TwitterFacebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

CryptogramCredential Stealing as an Attack Vector

Traditional computer security concerns itself with vulnerabilities. We employ antivirus software to detect malware that exploits vulnerabilities. We have automatic patching systems to fix vulnerabilities. We debate whether the FBI should be permitted to introduce vulnerabilities in our software so it can get access to systems with a warrant. This is all important, but what's missing is a recognition that software vulnerabilities aren't the most common attack vector: credential stealing is.

The most common way hackers of all stripes, from criminals to hacktivists to foreign governments, break into networks is by stealing and using a valid credential. Basically, they steal passwords, set up man-in-the-middle attacks to piggy-back on legitimate logins, or engage in cleverer attacks to masquerade as authorized users. It's a more effective avenue of attack in many ways: it doesn't involve finding a zero-day or unpatched vulnerability, there's less chance of discovery, and it gives the attacker more flexibility in technique.

Rob Joyce, the head of the NSA's Tailored Access Operations (TAO) group -- basically the country's chief hacker -- gave a rare public talk at a conference in January. In essence, he said that zero-day vulnerabilities are overrated, and credential stealing is how he gets into networks: "A lot of people think that nation states are running their operations on zero days, but it's not that common. For big corporate networks, persistence and focus will get you in without a zero day; there are so many more vectors that are easier, less risky, and more productive."

This is true for us, and it's also true for those attacking us. It's how the Chinese hackers breached the Office of Personnel Management in 2015. The 2014 criminal attack against Target Corporation started when hackers stole the login credentials of the company's HVAC vendor. Iranian hackers stole US login credentials. And the hacktivist that broke into the cyber-arms manufacturer Hacking Team and published pretty much every proprietary document from that company used stolen credentials.

As Joyce said, stealing a valid credential and using it to access a network is easier, less risky, and ultimately more productive than using an existing vulnerability, even a zero-day.

Our notions of defense need to adapt to this change. First, organizations need to beef up their authentication systems. There are lots of tricks that help here: two-factor authentication, one-time passwords, physical tokens, smartphone-based authentication, and so on. None of these is foolproof, but they all make credential stealing harder.

Second, organizations need to invest in breach detection and -- most importantly -- incident response. Credential-stealing attacks tend to bypass traditional IT security software. But attacks are complex and multi-step. Being able to detect them in process, and to respond quickly and effectively enough to kick attackers out and restore security, is essential to resilient network security today.

Vulnerabilities are still critical. Fixing vulnerabilities is still vital for security, and introducing new vulnerabilities into existing systems is still a disaster. But strong authentication and robust incident response are also critical. And an organization that skimps on these will find itself unable to keep its networks secure.

This essay originally appeared on Xconomy.

Worse Than FailureCongraubullations

“Java and C# are kind of the same thing, right?”

Josh was a Java developer, but his company was doing a big upgrade of some .NET applications written years earlier by a developer named Ray. Ray had left the company the previous year, and somebody needed to help port his .NET 2.5 code to .NET 4.0 and a new version of IIS. Josh was a team player, and also knew that newer versions of .NET were almost always backwards compatible, so he didn’t expect it to create that much work for him.

Most of the applications transferred to the new servers without any issues. Most. Of course, the one that didn’t transfer was the “mission critical” package. The ERP system didn’t have all the tools the users wanted for manipulating data, so they wanted a tool that could export some records to Excel, where they could edit them, and then reimport the data. The import process was reporting that it was complete, but actually crashed and didn’t finish the job (and for bonus points, left the database in an inconsistent state).

Ray hadn’t implemented any logging, so Josh fired it up on his dev box and fed it an Excel file. The program cheerfully reported what it was up to at each step of the bulk loading process, displaying a message with a throbber image that turned into a check-mark when that step completed. Despite the crash, every message reported success.

  • You’re file is on it’s way!
  • We have recieved you’re file and are procesing it right now!
  • Almoast done!
  • Table’s are bieng added to teh system.
  • The system is reading teh table!
  • You’re procesing is almoast compleeted.
  • Congraubullations! You’re file has been procesed!

Ugh. Josh had minored in English, and just seeing those messages gave him a headache. He pushed his annoyance aside- this was cosmetic and not directly relevant to the problem at hand.

With a few breakpoints, it only took Josh a few minutes to identify the problem. Ray’s code was spawning a background thread using the new Thread() syntax, which was one of the few places where .NET 4.0 made some breaking changes with older versions. Josh whipped up a quick workaround and ran it again- everything worked.

With the real problem solved, Josh tracked down the messages. They were hardcoded strings. He touched up the spelling, checked in his changes, and then let QA know there was a fix they could test.

They replied back minutes later: “The file gets uploaded now, but the screen hangs on the last step. The spinner just sits there.”

  • Your file is on its way!
  • We have received your file and are processing it right now!
  • Almost done!
  • Tables are being added to the system.
  • The system is reading the table!
  • Your processing is almost completed.
  • Congratulations! Your file has been processed!

That was strange, since he’d just seen the code run successfully. Well… before he’d fixed the spelling. Could it be? Josh dug through the code that controlled the status display, and found the problem in a file called StatisUpdator.cs.

if (StatisMesages[i].Contains("Congraubullations"))
{
        finished = true; //causes the throbber to turn into a check
}

At least Ray is consistent.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

Planet DebianPetter Reinholdtsen: The Pyra - handheld computer with Debian preinstalled

A friend of mine made me aware of The Pyra, a handheld computer which will be delivered with Debian preinstalled. I would love to get one of those for my birthday. :)

The machine is a complete ARM-based PC with micro HDMI, SATA, USB plugs and many others connectors, and include a full keyboard and a 5" LCD touch screen. The 6000mAh battery is claimed to provide a whole day of battery life time, but I have not seen any independent tests confirming this. The vendor is still collecting preorders, and the last I heard last night was that 22 more orders were needed before production started.

As far as I know, this is the first handheld preinstalled with Debian. Please let me know if you know of any others. Is it the first computer being sold with Debian preinstalled?

Planet DebianDebian Java Packaging Team: What's new since Jessie?

Jessie was released one year ago now and the Java Team has been busy preparing the next release. Here is a quick summary of the current state of the Java packages:

  • A total of 136 packages have been added, 63 removed, 213 upgraded to a new upstream release, and 145 updated. We are now maintaining 892 packages (+12.34%).
  • OpenJDK 8 is now the default Java runtime in testing/unstable. OpenJDK 7 has been removed, as well as several packages that couldn't be upgraded to work with OpenJDK 8 (avian, eclipse).
  • OpenJDK 9 is available in experimental. As a reminder, it won't be part of the next release; OpenJDK 8 will be the only Java runtime supported for Stretch.
  • Netbeans didn't make it into Jessie, but it is now back and up to date.
  • The main build tools are close to their latest upstream releases, especially Maven and Gradle which were seriously lagging behind.
  • Scala has been upgraded to the version 2.11. We are looking for Scala experts to maintain the package and its dependencies.
  • Freemind has been removed due to lack of maintenance, Freeplane is recommended instead.
  • The reproducibility rate has greatly improved, climbing from 50% to 75% in the past year.
  • Backports are continuously provided for the key packages and applications: OpenJDK 8, OpenJFX, Ant, Maven, Gradle, Tomcat 7 & 8, Jetty 8 & 9, jEdit.
  • The transition to Maven 3 has been completed, and packages are no longer built with Maven 2.
  • We replaced several obsolete libraries and transitioned them to their latest versions - for example, asm2, commons-net1 and commons-net2. Groovy 1.x was replaced with Groovy 2, and we upgraded BND, an important tool to develop with OSGi, and more than thirty of its reverse-dependencies from the 1.x series to version 2.4.1.
  • New packaging tools have been created to work with Gradle (gradle-debian-helper) and Ivy (ivy-debian-helper).

Outlook, goals and request for help

  • We have several difficult transitions ahead: BND 3, Tomcat 7 to 8, Jetty 8 to 9, ASM 5, and of course Java 9. Any help would be welcome.
  • Eclipse is severely outdated and currently not part of testing. We would like to update this important piece of software and its corresponding modules to the latest upstream release, but we need more active people who want to maintain them. If you care about the Eclipse ecosystem, please get in touch with us.
  • We still are in the midst of removing old libraries like asm3, commons-httpclient and the servlet 2.5 API, which is part of the Tomcat 6 source package.
  • Want to see Azureus/Vuze in Stretch again? Packaging is almost complete but we are looking for someone who can clarify remaining licensing issues with upstream and wants to maintain the software for the foreseeable future.
  • Do you have more ideas and want to get involved with the Java Team? Just send your suggestions to debian-java@lists.debian.org or chat with us on IRC at irc.debian.org, #debian-java.

Java and Friends

  • The Java Team is not the only team that maintains Java software in Debian. DebianMed, DebianScience and the Android Tools Maintainers rely heavily on Java. By helping the Java Team and working together, you can improve the Java ecosystem and further the efforts of multiple other fields of endeavor all at once.

Package updates

The packages listed below detail the changes in jessie-backports and testing. Libraries and Debian specific tools have been excluded.

Packages added to jessie-backports:

Packages removed from testing:

Packages added to testing:

Packages upgraded in testing:

Planet DebianJunichi Uekawa: Fighting with my emacs configuration.

Fighting with my emacs configuration. I'm trying to get a nice emacs terminal, and trying to set up 256 color mode in screen. This is so hard.

Rondam RamblingsWell, I sure didn't see that coming

When I wrote about Donald Trump's prospects to win the Republican nomination it was just before the New York primary, and he had just lost three states in a row.  I did the math and concluded that he had a shot at winning the nomination outright, but that it would come down to the wire on June 7. Since then there has been a very interesting new development: The Donald won New York, and every

,

Planet DebianMartín Ferrari: New sources for contributors.debian.org

Many people might not be aware of it, but since a couple of years ago, we have an excellent tool for tracking and recognising contributors to the Debian Project: Debian Contributors

Debian is a big project, and there are many people working that do not have great visibility, specially if they are not DDs or DMs. We are all volunteers, so it is very important that everybody gets credited for their work. No matter how small or unimportant they might think their work is, we need to recognise it!

One great feature of the system is that anybody can sign up to provide a new data source. If you have a way to create a list of people that is helping in your project, you can give them credit!

If you open the Contributors main page, you will get a list of all the groups with recent activity, and the people credited for their work. The data sources page gives information about each data source and who administers it.

For example, my Contributors page shows the many ways in which the system recognises me, all the way back to 2004! That includes commits to different projects, bug reports, and package uploads.

I have been maintaining a few of the data sources that track commits to Git and Subversion repositories:

The last two are a bit problematic, as they group together all commits to the respective VCS repositories without distinguishing to which sub-projects the contributions were made.

The Go and Perl groups' contributions are already extracted from that big pile of data, but it would be much nicer if each substantial packaging team had their own data source. Sadly, my time is limited, so this is were you come into the picture!

If you are a member of a team, and want to help with this effort, adopt a new data source. You can be providing commit logs, but it is not limited to that; think of translators, event volunteers, BSP attendants, etc.

The initial work is very small, and there is almost no maintenance. There is information on how to contribute here and here, but I would be more than happy to guide you if you contact me.

Comment

Planet DebianNeil Williams: Moving to Pelican

Prompted by Tollef, moving to Hugo, I investigated a replacement blog engine. The former site used Wordpress which is just overhead - my blog doesn't need to be generated on every view, it doesn't need the security implications of yet another website login and admin interface either.

The blog is static, so I've been looking at static generators. I didn't like the look of Hugo and wanted something where the syntax was familiar - so either Jinja2 or ReST.

So, I've chosen Pelican with the code living in a private git repo, naturally. I wanted a generator that was supported in Jessie. I first tried nikola but it turns out that nikola in jessie has syntax changes. I looked at creating backports but then there is a new upstream release which adds a python module not yet in Debian, so that would be an extra amount of work.

Hopefully, this won't flood planet - I've gone through the RSS content to update timestamps but the URLs have changed.

CryptogramJulian Sanchez on the Feinstein-Burr Bill

Two excellent posts.

It's such a badly written bill that I wonder if it's just there to anchor us to an extreme, so we're relieved when the actual bill comes along. Me:

"This is the most braindead piece of legislation I've ever seen," Schneier -- who has just been appointed a Fellow of the Kennedy School of Government at Harvard -- told The Reg. "The person who wrote this either has no idea how technology works or just doesn't care."

Cory DoctorowThe open web’s guardians are acting like it’s already dead

The World Wide Web Consortium — an influential standards body devoted to the open web — used to make standards that would let anyone make a browser that could view the whole Web; now they’re making standards that let the giant browser companies and giant entertainment companies decide which browsers will and won’t work on the Web of the future.

When you ask them why they’re doing this, they say that the companies are going to build technology that locks out new entrants no matter what they do, and by capitulating to them, at least there’s a chance of softening the control the giants will inevitably get.

In my latest Guardian column, Why the future of web browsers belongs to the biggest tech firms, I explain how the decision of the W3C to let giant corporations lock up the Web betrays a belief that the open Web is already dead, and all that’s left to argue about are the terms on which our new overlords will present to us.

Today is the International Day Against DRM. EME, the W3C project that hands control over the Web to giant corporations, uses DRM to assert this control.

We will get the open Web we deserve. If you and I and everyone we know stand up to the bullies who want to use entertainment technology to seize control over the future, we can win.

Otherwise, we’ll be Huxleyed into the full Orwell.

Make it easy for today’s crop of web giants to sue any new entrants into oblivion and you can be pretty certain there won’t be any new entrants.

It marks a turning point in the history of those companies. Where once web giants were incubators for the next generation of entrepreneurs who struck out and started competitors that eclipsed their former employers, now those employees are setting the stage for a future where they can stay where they are, or slide sideways to another giant. Forget overturning the current order, though. Maybe they, too, think the web is cooked.

In case there was any doubt of where the W3C stood on whether the future web needed protection from the giants of today, that doubt was dispelled last month. Working with the Electronic Frontier Foundation, I proposed that the W3C adapt its existing policies – which prohibit members from using their patents to block new web companies – to cover EME, a moved that was supported by many W3C members.

Rather than adopt this proposal or a version of it, last month, the W3C executive threw it out, giving the EME group a green light to go forward with no safeguards whatsoever.

Why the future of web browsers belongs to the biggest tech firms
[The Guardian]

Krebs on SecurityFraudsters Steal Tax, Salary Data From ADP

Identity thieves stole tax and salary data from payroll giant ADP by registering accounts in the names of employees at more than a dozen customer firms, KrebsOnSecurity has learned. ADP says the incidents occurred because the victim companies all mistakenly published sensitive ADP account information online that made those firms easy targets for tax fraudsters.

adpPatterson, N.J.-based ADP provides payroll, tax and benefits administration for more than 640,000 companies. Last week, U.S. Bancorp (U.S. Bank) — the nation’s fifth-largest commercial bank — warned some of its employees that their W-2 data had been stolen thanks to a weakness in ADP’s customer portal.

ID thieves are interested in W-2 data because it contains much of the information needed to fraudulently request a large tax refund from the U.S. Internal Revenue Service (IRS) in someone else’s name. A reader who works at U.S. Bank shared a letter received from Jennie Carlson, the financial institution’s executive vice president of human resources.

“Since April 19, 2016, we have been actively investigating a security incident with our W-2 provider, ADP,” Carlson wrote. “During the course of that investigation we have learned that an external W-2 portal, maintained by ADP, may have been utilized by unauthorized individuals to access your W-2, which they may have used to file a fraudulent income tax return under your name.”

The letter continued:

“The incident originated because ADP offered an external online portal that has been exploited. For individuals who had never used the external portal, a registration had never been established. Criminals were able to take advantage of that situation to use confidential personal information from other sources to establish a registration in your name at ADP. Once the fraudulent registration was established, they were able to view or download your W-2.”

U.S. Bank spokesman Dana Ripley said the letter was sent to a “small population” of the bank’s more than 64,000 employees. Asked to comment on the letter from U.S. Bank, ADP confirmed that the fraud visited upon U.S. Bank also hit “a very small subset” of the ADP’s total customers this year.

ADP emphasized that the fraudsters needed to have the victim’s personal data — including name, date of birth and Social Security number — to successfully create an account in someone’s name. ADP also stressed that this personal data did not come from its systems, and that thieves appeared to already possess that data when they created the unauthorized accounts at ADP’s portal.

ADP Chief Security Officer Roland Cloutier said customers can choose to create an account at the ADP portal for each employee, or they can defer that process to a later date (but employers do have to chose one or the other, Cloutier said).

According to ADP, new users need to be in possession of two other things (in addition to the victim’s personal data) at a minimum in order to create an account: A custom, company-specific link provided by ADP, and a static code assigned to the customer by ADP.

The problem, Cloutier said, seems to stem from ADP customers that both deferred that signup process for some or all of their employees and at the same time inadvertently published online the link and the company code. As a result, for users who never registered, criminals were able to register as them with fairly basic personal info, and access W-2 data on those individuals.

U.S. Bank’s Ripley acknowledged that the bank published the link and company code to an employee resource online, but said the institution never considered that the data itself was privileged.

“We viewed the code as an identification code, not as an authentication code, and we posted it to a Web site for the convenience of our employees so they could access their W-2 information,” Ripley said. “We have discontinued that practice.”

In the meantime, ADP says it has developed systems to monitor the Web for any other customers that may inadvertently publish their signup link and code.

“We’ve now aggressively put in some security intelligence by trying to look for that code and turn off self-service registration access if we find that code” published online, Cloutier said.

ANALYSIS

ADP’s portal, like so many other authentication systems, relies entirely on static data that is available on just about every American for less than $4 in the cybercrime underground (SSN/DOB, address, etc). It’s true that companies should know better than to publish such a crucial link online along with the company’s ADP code, but then again these are pretty weak authenticators.

Cloutier said ADP does offer an additional layer of authentication — a personal identification code (PIC) — basically another static code that can be assigned to each employee. He added that ADP is trialing a service that will ask anyone requesting a new account to successfully answer a series of questions based on information that only the real account holder is supposed to know.

Cloutier declined to say who was providing the verification service, but these so-called knowledge-based authentication (KBA) or “out-of-wallet” questions generally focus on things such as previous address, loan amounts and dates and can be successfully enumerated with random guessing. In many cases, the answers can be found by consulting free online services, such as Zillow and Facebook.

The IRS found this out the hard way, and over the past year has removed two separate authentication systems that placed too much reliance on KBA and static data to authenticate taxpayers. In May 2015, the IRS took down its “Get Transcript” service after tax refund fraudsters began using it to pull W-2 data on more than 724,000 taxpayers. In those cases, the fraudsters also already had the victim’s SSN, DoB and other personal data. In March 2016, the IRS suspended its “Get IP PIN” feature for the same reason.

But somehow, KBA questions are an innovation that’s worth looking forward to at ADP.

“The IRS didn’t have a PIC code or client code,” Cloutier said when I brought up the IRS’s experience. “They didn’t have as many levels and individual authentication components that we provide our clients.”

Cloutier’s words recalled to mind a scene from the movie Office Space, in which Jennifer Aniston’s character is upbraided by her manager for wearing too few “pieces of flair” on her ‘Chotchkie’s’ uniform. His comment also made me think about one of the best scenes from the cult hit “This is Spinal Tap,” in which the character Nigel Tufnel shows off how all the knobs on his amplifier go to “level 11,” while other amps only go to the more boring and standard level 10.

It’s truly a measure of the challenges ahead in improving online authentication that so many organizations are still looking backwards to obsolete and insecure approaches. ADP’s logo includes the clever slogan, “A more human resource.” It’s hard to think of a more apt mission statement for the company. After all, it’s high time we started moving away from asking people to robotically regurgitate the same static identifiers over and over, and shift to a more human approach that focuses on dynamic elements for authentication. But alas, that’s fodder for a future post.

Update 1:59 p.m. ET: Clarified Spinal Tap reference.

Update, 10:07 p.m. ET: It looks like ADP’s stock took a pretty big hit immediately after this story ran today.

adp-twitter

The stock later rebounded:

adp-stocknews

Planet DebianCarl Chenet: Feed2tweet, your RSS feed to Twitter Python self-hosted app

Feed2tweet is a self-hosted Python app to send you RSS feed to Twitter.

Feed2tweet is in production for Le Journal du hacker, a French Hacker News-style FOSS website and LinuxJobs.fr, the job board of the French-speaking FOSS community.

linuxjobs-horizontale

Feed2tweet 0.3 now only runs with Python 3.  It also fixes a nasty bug with RSS feeds modifying the RSS entry orders. Have a look at the Feed2tweet 0.3 changelog:

Using Feed2tweet? Send us bug reports/feature requests/push requests/comments about it!


Sociological ImagesHow Much Do Racial Wealth Gaps Affect the Next Generation?

The Wall Street Journal’s Real Time Economics recently looked at wealth inequality.  The first chart taken from the post shows wealth differences by race and age of head of family.

wealth gap

Racial differences (white versus black and Hispanic) dominate whether looking at average or median net worth, and the gap grows as the head of the family ages.  Median figures are especially sobering, showing the limited wealth generation of representative black and Hispanic heads of families regardless of age.

So, do these advantages and disadvantages transfer to the next generation? Yes, and not just laterally. This second chart looks at the relationship between inheritance and wealth generation.

Inheritance

Inheritance was divided into ten groups.  WARNING: THE TENTH GROUP, WHICH RECEIVED THE LARGEST INHERITANCE, IS NOT SHOWN.

As Josh Zumrun, the author of the blog, explains:

The bottom 10% of inheritors received an inheritance averaging only about $2,000. Families receiving this much inheritance aren’t that wealthy.

But among families that received a $35,000 inheritance, their net worth is over half a million. Families that received a $125,000 inheritance are worth $780,000 on average and those that receive a $200,000 inheritance are, on average, millionaires. (The top 10% of inheritors, not pictured in this chart, inherit $1.6 million on average and have a net worth of $4.2 million.)

The take-away is pretty simple: Wealth inequality is real, with strong racial determinants, and is also, to a significant degree, self-reinforcing.

Originally posted at Reports from the Economic Front.

———————————

Martin Hart-Landsberg is a professor of economics at Lewis and Clark College. You can follow him at Reports from the Economic Front.

(View original at https://thesocietypages.org/socimages)

Planet DebianJamie McClelland: Monitoring Deflect

May First/People Link has several members that are targets of politically motivated denial of service attacks (mostly groups that support reproductive justice for women and palestinian rights). To fight off the attacks, we work closely with Deflect - a non-governmental organization based in Canada that fights against this kind of censorship.

When a site is down, it's not always easy to understand why. Deflect runs as many as 5 edge servers, any of them could be down. And, of course, the origin server could also be down.

I tried using a commericial/free as in beer service for monitoring up time, but when it reported the site being down, I had no idea which part was down.

So... httping to the rescue. Unfortunately, it depends on --divert-connect which is only available in Debian Stretch. I run the script via a cron job and output the results to a log file.

    #!/bin/bash

    # Test all given edges 

    domain="$1"
    origin="$2"
    proto=http
    if [ -n "$3" ]; then
        proto="$3"
    fi

    if [ -z "$domain" ]; then
        printf "Please pass the domain as first argument.\n"
        exit 1
    fi

    if ! ping -c 1 4.2.2.1 >/dev/null; then
        # printf "We are offline. Not running.\n"
        exit 1
    fi

    ips=$(dig +short "$domain")
    if [ "$?" -ne "0" ]; then
        # printf "DNS lookup failure. Not running.\n"
        exit 1
    fi
    if [ -n "$origin" ]; then
        ips="$ips $origin"
    fi

    l=
    if [ "$proto" = "https" ]; then
        l=-l
    fi

    for ip in $ips; do
        date=$(date +%Y.%m.%d-%H:%M)
        for i in 1 2 3; do
            out=$(httping $l -m -t 5 -c 1 --divert-connect "$ip" "$proto://$domain")
            [ -z "$out" ] && out=1
            printf "%s %s %s\n" "$date" "$ip" "$out"
        done
    done

Worse Than FailureCodeSOD: An Ant Pushes a Perl

It’s an old joke that Perl is a “write only language”. Despite some of its issues, back in the early 2000s, Perl was one of the best options out there for scripting languages and rapid-development automation.

Speaking of automation, build automation is really important. Back in the early 2000s, before Maven really caught on, your build automation tool for Java was Ant. Ant, like everything invented in the early 2000s, was driven by an XML scripting tool. Since it was tuned specifically for Java, it had some high-level operations to streamline tasks like generating proxy classes for calling web services based on a supplied WSDL file.

Actually, speaking of code generation, Carrie sends us this code block. It’s a Perl script that’s called from an Ant build. It runs after generating a class based off a WSDL. It parses Java code using Regular Expressions and injects a ListWrapper class which doesn’t adhere to the List contract. But hey, it does have a use strict declaration, guaranteeing you’ll get errors if you access uninitialized variables.

use strict;
use warnings;

my $dir = $ARGV[0];
readDir($dir);

sub readDir {
        my ($dir) = @_;
        opendir my $dirHandle, $dir or die "Cannot open $dir.\n$!\n";
        foreach my $file(readdir($dirHandle)) {
                next if $file =~ m/^\./;

                if(-d "$dir/$file") {
                        readDir("$dir/$file");
                        next;
                }

                my %seenFields;
        my %multiples;

                my $fileName = "$dir/$file";

                open IN, "<$fileName" or die "Cannot open $fileName.\n$!\n";

                my @methods;

                my $file = "";
                my $currentLine;
                my $containsContent = 0;
                while(<IN>) {
                        if($_ =~ m/\@XmlElementRef\(name\s*=\s*"(.*?)".+namespace\s*=\s*"(.*?)".+type\s*=\s*(.*?.class)/) {
                                my $field = ucfirst($1);
                my $namespace = $2;
                                my $class = $3;

                $multiples{$1}++;
                if($multiples{$field} > 1) {
                    $_ =~ s/name\s*=\s*"$1"/name = "$1$multiples{$1}"/gis;
                    $field = $1 . $multiples{$1};
                }

                                my $fieldlc = lc($field);
                                my $retObject = substr($class, 0, length($class) - 6);
                                die if not defined $retObject;
                                my $methodName = $field;
                                unless(defined $seenFields{$methodName}) {
                                        $seenFields{$methodName} = 1;
                                        my $method = <<EOF;

        public List get$field() {
                List all = getContent();
                ListWrapper retVal = new ListWrapper(all);
                for(java.lang.Object current : all) {
                        java.lang.String className = null;
                        if(current instanceof javax.xml.bind.JAXBElement) {
                                className = ((javax.xml.bind.JAXBElement)current).getName().getLocalPart().toLowerCase();
                        } else {
                                className = current.getClass().getSimpleName().toLowerCase();
                        }
                        boolean good = false;
                  if(className.equalsIgnoreCase("$fieldlc")) {
                          good = true;
                  } else {
                          if(className.length() > 4) {
                                  className = className.substring(0, className.length() - 4);
                                  if(className.equalsIgnoreCase("$fieldlc")) {
                                          good = true;
                                  }
                          }
                  }
                  if(good) {
                                retVal.addWrapped(current);
                        }
                }
                return retVal;
        }

EOF
                                        push(@methods, $method);
                                }
                        } elsif ($_ =~ m/getContent\(\)/) {
                                $containsContent = 1;
            }
                        $currentLine = $_;
                        $file .= $currentLine if defined $currentLine;
                }
                close IN;

                if($containsContent) {
                        print "$fileName\n";

                        for my $method(@methods) {
                                $file .= $method;
                        }

                        $file .= <<EOF;

        private class ListWrapper<T> extends ArrayList<T> {
            private List contentsList;

            public ListWrapper(List contents) {
                super();
                contentsList = contents;
            }

            public boolean addWrapped(T toAdd) {
                return super.add(toAdd);
            }

            \@Override
            public boolean add(T toAdd) {
                return contentsList.add(toAdd);
            }

            \@Override
            public boolean addAll(java.util.Collection<? extends T> toAdd) {
                return contentsList.addAll(toAdd);
            }

            \@Override
            public T set(int index, T element) {
                int realIndex = contentsList.indexOf(this.get(index));
                return (T)contentsList.set(realIndex, element);
            }
        }
}
EOF
                        open OUT, ">$fileName" or die "Cannot open $fileName.\n$!\n";
                        print OUT $file;
                        close OUT;
                }
        }
        closedir $dirHandle;
}
[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Worse Than FailureCodeSOD: An Ant Pushes a Perl

It’s an old joke that Perl is a “write only language”. Despite some of its issues, back in the early 2000s, Perl was one of the best options out there for scripting languages and rapid-development automation.

Speaking of automation, build automation is really important. Back in the early 2000s, before Maven really caught on, your build automation tool for Java was Ant. Ant, like everything invented in the early 2000s, was driven by an XML scripting tool. Since it was tuned specifically for Java, it had some high-level operations to streamline tasks like generating proxy classes for calling web services based on a supplied WSDL file.

Actually, speaking of code generation, Carrie sends us this code block. It’s a Perl script that’s called from an Ant build. It runs after generating a class based off a WSDL. It parses Java code using Regular Expressions and injects a ListWrapper class which doesn’t adhere to the List contract. But hey, it does have a use strict declaration, guaranteeing you’ll get errors if you access uninitialized variables.

use strict;
use warnings;

my $dir = $ARGV[0];
readDir($dir);

sub readDir {
        my ($dir) = @_;
        opendir my $dirHandle, $dir or die "Cannot open $dir.\n$!\n";
        foreach my $file(readdir($dirHandle)) {
                next if $file =~ m/^\./;

                if(-d "$dir/$file") {
                        readDir("$dir/$file");
                        next;
                }

                my %seenFields;
        my %multiples;

                my $fileName = "$dir/$file";

                open IN, "<$fileName" or die "Cannot open $fileName.\n$!\n";

                my @methods;

                my $file = "";
                my $currentLine;
                my $containsContent = 0;
                while(<IN>) {
                        if($_ =~ m/\@XmlElementRef\(name\s*=\s*"(.*?)".+namespace\s*=\s*"(.*?)".+type\s*=\s*(.*?.class)/) {
                                my $field = ucfirst($1);
                my $namespace = $2;
                                my $class = $3;

                $multiples{$1}++;
                if($multiples{$field} > 1) {
                    $_ =~ s/name\s*=\s*"$1"/name = "$1$multiples{$1}"/gis;
                    $field = $1 . $multiples{$1};
                }

                                my $fieldlc = lc($field);
                                my $retObject = substr($class, 0, length($class) - 6);
                                die if not defined $retObject;
                                my $methodName = $field;
                                unless(defined $seenFields{$methodName}) {
                                        $seenFields{$methodName} = 1;
                                        my $method = <<EOF;

        public List get$field() {
                List all = getContent();
                ListWrapper retVal = new ListWrapper(all);
                for(java.lang.Object current : all) {
                        java.lang.String className = null;
                        if(current instanceof javax.xml.bind.JAXBElement) {
                                className = ((javax.xml.bind.JAXBElement)current).getName().getLocalPart().toLowerCase();
                        } else {
                                className = current.getClass().getSimpleName().toLowerCase();
                        }
                        boolean good = false;
                  if(className.equalsIgnoreCase("$fieldlc")) {
                          good = true;
                  } else {
                          if(className.length() > 4) {
                                  className = className.substring(0, className.length() - 4);
                                  if(className.equalsIgnoreCase("$fieldlc")) {
                                          good = true;
                                  }
                          }
                  }
                  if(good) {
                                retVal.addWrapped(current);
                        }
                }
                return retVal;
        }

EOF
                                        push(@methods, $method);
                                }
                        } elsif ($_ =~ m/getContent\(\)/) {
                                $containsContent = 1;
            }
                        $currentLine = $_;
                        $file .= $currentLine if defined $currentLine;
                }
                close IN;

                if($containsContent) {
                        print "$fileName\n";

                        for my $method(@methods) {
                                $file .= $method;
                        }

                        $file .= <<EOF;

        private class ListWrapper<T> extends ArrayList<T> {
            private List contentsList;

            public ListWrapper(List contents) {
                super();
                contentsList = contents;
            }

            public boolean addWrapped(T toAdd) {
                return super.add(toAdd);
            }

            \@Override
            public boolean add(T toAdd) {
                return contentsList.add(toAdd);
            }

            \@Override
            public boolean addAll(java.util.Collection<? extends T> toAdd) {
                return contentsList.addAll(toAdd);
            }

            \@Override
            public T set(int index, T element) {
                int realIndex = contentsList.indexOf(this.get(index));
                return (T)contentsList.set(realIndex, element);
            }
        }
}
EOF
                        open OUT, ">$fileName" or die "Cannot open $fileName.\n$!\n";
                        print OUT $file;
                        close OUT;
                }
        }
        closedir $dirHandle;
}
[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Planet DebianRaphaël Hertzog: My Free Software Activities in April 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it’s one of the best ways to find volunteers to work with me on projects that matter to me.

Debian LTS

I handled a new LTS sponsor that wanted to see wheezy keep supporting armel and armhf. This was not part of our initial plans (set during last Debconf) and I thus mailed all teams that were impacted if we were to collectively decide that it was OK to support those architectures. While I was hoping to get a clear answer rather quickly, it turns out that we never managed to get an answer to the question from all parties. Instead the discussion drifted on the more general topic of how we handle sponsorship/funding in the LTS project.

Fortunately, the buildd maintainers said they were OK with this and the ftpmasters had no objections, and they both implicitly enacted the decision: Ansgar Burchardt kept the armel/armhf architectures in the wheezy/updates suite when he handled the switch to the LTS team, and Aurélien Jarno also configured wanna-build to keep building armel/armhf for the suite. The DSA team did not confirm that this change was not interfering with one of their plans to decommission some hardware. Build daemons are a shared resource anyway and a single server is likely to handle builds for multiple releases.

DebConf 16

This month I registered for DebConf 16 and submitted multiple talk/BoF proposals:

  • Kali Linux’s Experience of a Debian Derivative Based on Testing (Talk)
  • 2 Years of Work of Paid Contributors in the Debian LTS Project (Talk)
  • Using Debian Money to Fund Debian Projects (BoF)

I want to share the setup we use in Kali as it can be useful for other derivatives and also for Debian itself to help smooth the relationship with derivatives.

I also want to open again the debate on the usage of money within Debian. It’s a hard topic but we should really strive to take some official position on what’s possible and what’s not possible. With Debian LTS and its sponsorship we have seen that we can use money to some extent without hurting the Debian project as a whole. Can this be transposed to other teams or projects? What are the limits? Can we define a framework and clear rules? I expect the discussion to be very interesting in the BoF. Mehdi Dogguy has agreed to handle this BoF with me.

Packaging

Django. I uploaded 1.8.12 to jessie-backports and 1.9.5 to unstable. I filed two upstream bugs (26473 and 26474) for two problems spotted by lintian.

Unfortunately, when I wanted to upload it to unstable, the test suite did not ran. I pinned this down to a sqlite regression. Chris Lamb filed #820225 and I contacted the SQLite and Django upstream developers by email to point them to this issue. I helped the SQLite upstream author (Richard Hipp) to reproduce the issue and he was quick to provide a patch which landed in 3.12.1.

Later in the month I made another upload to fix an upgrade bug (#821789).

GNOME 3.20. As for each new version, I updated gnome-shell-timer to ensure it works with the new GNOME. This time I spent a bit more time to fix a regression (805347) that dates back to a while and that would never be fixed otherwise since the upstream author orphaned this extension (as he no longer uses GNOME).

I have also been bitten by display problems where accented characters would be displayed below the character that follows. With the help of members of the GNOME team, we found out that this was a problem specific to the cantarell font and was only triggered with Harfbuzz 1.2. This is tracked in Debian with #822682 on harfbuzz and #822762 in fonts-cantarell. There’s a new upstream release (with the fix) ready to be packaged but unfortunately it is blocked by the lack of a recent fontforge in Debian. I thus mailed debian-mentors in the hope to find volunteers to help the pkg-fonts team to package a newer version…

Misc Debian/Kali work

Distro Tracker. I started to mentor Vladimir Likic who contacted me because he wants to contribute to Distro Tracker. I helped him to setup his development environment and we fixed a few issues in the process.

Bug reports. I filed many bug reports, most of them due to my work on Kali:

  • #820288: a request to keep the wordpress package installable in older releases (due to renaming of many php packages)
  • #820660: request support of by-hash indices in reprepro
  • #820867: possibility to apply overrides on already installed packages in reprepro
  • #821070: jessie to stretch upgrade problem with samba-vfs-modules
  • #822157: python-future hides and breaks python-configparser
  • #822669: dh_installinit inserts useless autoscript for System V init script when package doesn’t contain any
  • #822670: dh-systemd should be merged into debhelper, we have systemd by default and debhelper should have proper support for it by default

I also investigated #819958 that was affecting testing since it has been reported to Kali as well. And I made an NMU of dh-make-golang to fix #819472 that I reported earlier.

Thanks

See you next month for a new summary of my activities.

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

Planet DebianRuss Allbery: Review: The Effective Engineer

Review: The Effective Engineer, by Edmond Lau

Publisher: Effective Bookshelf
Copyright: 2015
ISBN: 0-9961281-0-7
Format: Trade paperback
Pages: 222

Silicon Valley start-up tech companies have a standard way of thinking about work. Large chunks of this come from Google, which pioneered a wide variety of new, or at least not-yet-mainstream, ways of organizing and thinking about work. The rest accreted through experience with fast-paced start-ups, engineer-focused companies, web delivery of products, and rabid turnover and high job mobility within a hothouse of fairly similar companies. A key part of this mindset is the firm belief that this atmosphere has created a better way to work, at least for software engineers (and systems administrators, although heaven forbid that one call them that any more): more effective, more efficient, more focused on what really matters.

I think this is at least partly true, at least from the perspective of a software engineer. This Silicon Valley work structure focuses on data gathering, data-based decision-making, introspection, analysis, and continuous improvement, all of which I think are defensibly pointed in the right direction (if rarely as rigorous as one might want to believe). It absorbs bits and pieces of work organization techniques that are almost certainly improvements for the type of work software engineers do: Agile, Lean, continuous deployment, and fast iteration times.

In other cases, though, I'm less convinced that this Silicon Valley consensus is objectively better as opposed to simply different; interviewing, for instance, is a puzzle that I don't think anyone has figured out, and the remarkable consensus in Silicon Valley on how to interview (basically, "like Google except for the bits we thought were obnoxious") feels more like a social fad than a sign of getting it right. But every industry has its culture of good ideas, bad ideas, fads, and fashion, and it's quite valuable to know that culture if you want to work in that industry.

The Effective Engineer is a self-published book by Edmund Lau, a Silicon Valley software engineer who also drifted (as is so common in Silicon Valley) into mentoring, organizing, and speaking to other software engineers. Its purpose, per the subtitle, is to tell you "how to leverage your efforts in software engineering to make a disproportionate and meaningful impact." While that's not exactly wrong, and the book contains some useful and valuable tips, I'd tend to give it a slightly different subtitle: "a primer on how a Silicon Valley software engineer is expected to think about their work." This is a bit more practical, a bit less confident, and a bit less convinced of its own correctness than Lau might want to present his work, but it's just as valuable of a purpose if you want to work in the industry. (And is a bit more honest about its applicability outside of that industry.)

What this book does extremely well is present, in a condensed, straightforward, and fast-moving form, most of the highlights of how start-ups and web-scale companies approach software engineering and the SWE role in companies (SWE, meaning software engineer, is another bit of Google terminology that's now nearly universal). If you've already worked in or around this industry for a while, you've probably picked up a lot of this via osmosis: prioritize based on impact and be unapologetic about letting other things drop, have a growth mindset, reprioritize regularly, increase your iteration speed, measure everything constantly, check your assumptions against data, derisk your estimates, use code review and automated testing (but not too much), automate operations, and invest heavily in hiring and onboarding. (The preceding list is a chapter list for this book.) If you're working at one of these sorts of companies, you're probably currently somewhere between nodding and rolling your eyes because no one at work will shut up about these topics. But if you've not worked inside one of these companies, even if you've done software engineering elsewhere, this is a great book to read to prepare yourself. You're going to hear about these ideas constantly, and, if it achieves nothing else at all, The Effective Engineer will give you a firm enough grounding in the lingo and mindset that you can have intelligent conversations with people who assume this is the only way to think about software engineering.

By this point, you might be detecting a certain cynicism in this review. It's not entirely fair: a lot of these ideas are clearly good ones, and Lau does a good job of describing them quickly and coherently. It's a good job for what it is. But there are a couple of things that limited its appeal for me.

First, it's definitely a primer. I read it after having worked at a web-scale start-up for a year and a half. There wasn't much in it that seemed particularly new, and it's somewhat superficial. The whole middle section in particular (build tools for yourself, measure everything, be data-driven) are topics for which the devil is often in the details. Lau gives you the terminology and the expected benefits, but putting any one of these techniques into practice could be a book (or several) by itself. Don't expect to come away from The Effective Engineer with much of a concrete plan for how to do these things in your day-to-day software development projects. But it's a good reminder to be thinking about, say, how to embed metrics and data-gathering hooks into the software you write. This is the nature of a primer; no 222-page book can get into much depth about the fractal complexity of doing good, fast, scalable software development.

Second, there's a fundamental question raised by a book like this: effective at what? Lau tackles that in the first chapter with his focus on impact and leverage, and it's good advice as far as it goes. (Regular readers of my book reviews know that I love this sort of time management and prioritization discussion.) But measuring impact is a hard problem that requires a prioritization framework, and this is not really the book for this. The Effective Engineer is written primarily for software developers at start-ups, leaves the whole venture-capital start-up process as unquestioned background material, and accepts without comment the standard measures of value in that world: fast-deployed products, hypergrowth, racing competitors for perceived innovation, and finding ways to extract money. That's as deep into the question of impact as Lau gets: increases in company revenue.

There's nothing wrong with this for the kind of book Lau intended to write, and it's not his fault that I find it unsatisfying. But don't expect The Effective Engineer to ask any hard questions about whether that's a meaningful definition of impact, or to talk much about less objective goals: quality of implementation, craftsmanship, giving back to a broader community via free software contributions, impact on the world in ways that can't be measured in market share, or anything else that is unlikely to lead to objective impact for company profits. At best he leaves a bit of wiggle room around using the concept of impact with different goals.

If you're a new graduate who wants to work at Silicon-Valley-style start-ups, this is a great orientation, and likewise if you're coming from a different area of software development into that world. If you're not working in that industry, The Effective Engineer may still be moderately interesting, but it's not written for that audience and has little or nothing to say of the challenges of other types of businesses. But if you've already worked in the industry for a while, or if you're more interested in deeper discussions of goals and subjective values, you may not get much out of this.

Rating: 7 out of 10

,

CryptogramFake Security Conferences

Turns out there are two different conferences with the title International Conference on Cyber Security (ICCS 2016), one real and one fake. Richard Clayton has the story.

TEDHow drones could deliver better health

In Katete, a community in Malawi’s capital, Lilongwe, a group of residents watch a test flight of a new drone from Matternet that could, someday soon, deliver test results for newborn babies in their town. Photo: Aris Messinis for Matternet

Drones are the sexy, mysterious tech craze that are often a butt of a joke, a vague solution or an ominous threat. But no matter how flashy, good technology should still solve concrete problems. Amid the hype of getting packages and pizzas delivered in half the time, what if drones could revolutionize transportation networks, connecting people to what they actually need the most?

The health of a community relies on efficient networks. Medicine and medical supplies need to be transported reliably from Point A to Point B. This, in turn, relies on roads to travel on. Sturdy roads, able to endure heavy rains and bad weather, are something many in developed countries take for granted. But worldwide, a staggering one billion people lack access to all-season roads. What that means for people who live in remote villages: a simple rainstorm can turn a dirt road into a muddy and impassable mess, cutting them off from vital goods, services and medicines.

A surprising new technology might make a difference. Andreas Raptopoulos (watch his TED Talk) and Paola Santana co-founded Matternet, a company that adds smart software to nimble drones. Their signature product, Matternet ONE, can carry 1 kilogram of cargo over 15-20 kilometers on a single battery charge.

Matternet has worked with WHO in Bhutan to deliver medical supplies to remote hospitals, and with MSF (Doctors Without Borders) in Papua New Guinea to transport medical tests for tuberculosis.

In March 2016, Matternet started tests, along with UNICEF, in Malawi to see how well drones could deliver medical tests and blood samples from remote villages to labs. They’re working with the Malawian government to see if the drones could reduce the wait times for HIV test results, specifically for infants.

Local children watch the test flights in Katete.

Local children watch the test flights in Katete. Photo: Aris Messinis for Matternet

An estimated one million people in Malawi live with the disease, as of 2014. Testing — and regular medication — is crucial to survival, and yet it takes an average of 11 days to deliver supplies from health centers to labs and eight weeks to return test results. Also, according to Raptopoulos, “More than three-quarters of them are within 20 kilometers of each other or a diagnostic lab.” Using drones could dramatically speed up this process and solve the problem of distance.

The first step to solving a problem is assessing what’s already happening. Paola Santana, who is also the Head of Network Operations and Lead Coordinator for this project, studied Malawi’s current system for transporting HIV tests. “There is someone on a motorcycle or in a car that does a circuit every day. They need to go and select the number of labs where they start in the morning, and by the afternoon, they have picked up all of these samples from these clinics. Then they go to the hospital where these samples are going to be analyzed,” she says. The problem is, though, that these routes are outdated. “These circuits have been created based on data that has been there for a while. So imagine that as populations shift, more people now live in other areas that are not being served every day. Still, the logistics circuit stays the same.”

Young girls watch test flights at the Area 18 Health Center.

Young girls watch test flights at the Area 18 Health Center. Photo: Aris Messinis for Matternet

Logistics, she says, need to account for how people are moving, how outbreaks are spreading and how many samples are transported in real time. That way, a regular and efficient system is created, one that can transport tests once a day, as opposed to once a week or even twice a month, which is the current norm.

With this data, Santana and her team clearly understood the task at hand. “The challenge is trying to envision how the system should work that has nothing to do with how the system works today. How do we adapt the flow of logistics to a new technology?”

Working with Malawi’s Ministry of Health, Department of Civil Aviation and Ministry of Transportation, among others, Matternet first mapped the geography of the approved flying area. Then, they entered the data into the software so that any obstacles could be accounted for in determining the safest and most effective routes.

Through the app, a user can select the landing location and command the drone to take off. Because it flies at around 400 feet, a user can also track its course on the app and get a notification when it lands.

This means that in the future, lab technicians could autonomously use the app to send HIV tests from their remote villages to central hospitals, and get the results back in a timely manner.

A lab technician from Kamuzu Central Hospital holds the UNICEF drone, flanked by Jim O’Sullivan, left, Matternet’s electronics lead, and Paola Santana, right, Matternet’s head of network operations.

A lab technician from Kamuzu Central Hospital holds the UNICEF drone, flanked by Jim O’Sullivan, left, Matternet’s electronics lead, and Paola Santana, right, Matternet’s head of network operations. Photo: Aris Messinis for Matternet

Santana and her team were in Lilongwe, Malawi, for a total of two weeks. During the first half of the trip, they worked from 8am to 5pm every day to acclimate the drones to the new geography and make sure they could fly safely over densely populated areas. “The goal was to collect diagnostics from Area 25 Health Center and Area 18 Health Center to the Kamuzu Central Hospital to be analyzed,” Santana says.

As they prepped, Matternet, UNICEF and Malawi’s Ministry of Information hosted hands-on sessions and a Community Demo Day so that the locals could see how the drones work and learn what they’d be carrying. These demos also stressed the importance of HIV awareness, getting tested and taking antiretrovirals.

At Kamuzu Central Hospital, UAV technician Brandon Landry speaks to a group of moms-to-be, showing how the drone works and what it will be carrying: HIV tests for infants.

At Kamuzu Central Hospital, UAV technician Brandon Landry speaks to a group of moms-to-be, showing how the drone works and what it will be carrying: HIV tests for infants. Photo: Aris Messinis for Matternet

 

Plenty of locals turned out for Community Demo Day at Area 25 Primary School.

Plenty of locals turned out for Community Demo Day at Area 25 Primary School. Photo: Aris Messinis for Matternet

The next week, they conducted the first official test launch. The drone completed a 10-kilometer journey from Area 25 Health Center to Kamuzu Central Hospital, a journey UNICEF calls the “first known use of UAVs [unmanned aerial vehicles] on the continent for improvement of HIV services.”

On the day of the inaugural test launch at Kamuzu Central Hospital, Dr. Peter Kumpalume, Malawi’s Minister of Health, works with drone technician Brandon Landry to get ready for launch.

On the day of the inaugural test launch at Kamuzu Central Hospital, Dr. Peter Kumpalume, Malawi’s Minister of Health, works with drone technician Brandon Landry to get ready for launch. Photo: Aris Messinis for Matternet

Malawi’s Minister of Health, Peter Kumpalume, praised the launch as a huge step forward in the country’s fight against HIV/AIDS. “We are at the forefront of technology. The countdown to ending HIV/Aids has begun and today is another step forward in meeting that goal.”

UNICEF’s Malawi representative, Mahimbo Mdoe, also remarked on the implications for transportation in the country. “HIV is still a barrier to development in Malawi. This innovation could be the breakthrough in overcoming transport challenges and associated delays experienced by health workers in remote areas of Malawi.”

What’s next? UNICEF, the Malawian government and other partners are assessing the results of the feasibility study. This way, they can look at impact before moving on to any subsequent phases.

 

 


Planet DebianReproducible builds folks: Reproducible builds: week 53 in Stretch cycle

What happened in the Reproducible Builds effort between April 24th and 30th 2016.

Media coverage

Reproducible builds were mentioned explicitly in two talks at the Mini-DebConf in Vienna:

  • Martin Michlmayr had a talk in which he presented an overview about innovations and changes in Debian in the last years. Martin expressed his disappointment that there was no talk from us in Vienna (we'll fix this at DebConf16 in Cape Town) and described the reproducible builds work as "a real innovation". His talk is very much worth seeing, whatever your current perspective, it might change your view on Debian.
  • Ben Hutchings explains how Secure Boot will use signed kernels via separate signature packages and how this was designed with reproducible builds in mind.

Aspiration together with the OTF CommunityLab released their report about the Reproducible Builds summit in December 2015 in Athens.

Toolchain fixes

Now that the GCC development window has been opened again, the SOURCE_DATE_EPOCH patch by Dhole and Matthias Klose to address the issue timestamps_from_cpp_macros (__DATE__ / __TIME__) has been applied upstream and will be released with GCC 7.

Following that Matthias Klose also has uploaded gcc-5/5.3.1-17 and gcc-6/6.1.1-1 to unstable with a backport of that SOURCE_DATE_EPOCH patch.

Emmanuel Bourg uploaded maven/3.3.9-4, which uses SOURCE_DATE_EPOCH for the maven.build.timestamp.

(SOURCE_DATE_EPOCH specification)

Other upstream changes

Alexis Bienvenüe submitted a patch to Sphinx which extends SOURCE_DATE_EPOCH support for copyright years in generated documentation.

Packages fixed

The following 12 packages have become reproducible due to changes in their build dependencies: hhvm jcsp libfann libflexdock-java libjcommon-java libswingx1-java mobile-atlas-creator not-yet-commons-ssl plexus-utils squareness svnclientadapter

The following packages have became reproducible after being fixed:

Some uploads have fixed some reproducibility issues, but not all of them:

Patches submitted that have not made their way to the archive yet:

  • #822566 against stk by Alexis Bienvenüe: sort lists of object files for reproducible linking order.
  • #822948 against shotwell by Alexis Bienvenüe: normalize tarball permissions and use locale/timezone-independent modification time.
  • #822963 against htop by Alexis Bienvenüe: use SOURCE_DATE_EPOCH for embedded copyright year, which has before already been applied in git and upstream.

Package reviews

95 reviews have been added, 15 have been updated and 129 have been removed in this week.

22 FTBFS bugs have been reported by Chris Lamb and Martin Michlmayr.

diffoscope development

  • diffoscope 52~bpo8+1 has been uploaded to jessie-backports by Mattia Rizzolo, where it is currently waiting for NEW-approval.
  • Support for the deb(5) format (uncompressed data.tar/control.tar, control.tar.xz) (Closes: #818414) has been completed by Reiner Herrmann in git.

strip-nondeterminism development

  • Support for EPUB documents has been added (to the development version in git) by Holger Levsen, to address the timestamps_in_epub issue.

tests.reproducible-builds.org

Misc.

Amongst the 29 interns who will work on Debian through GSoC and Outreachy there are four who will be contributing to Reproducible Builds for Debian and Free Software. We are very glad to welcome ceridwen, Satyam Zode, Scarlett Clark and Valerie Young and look forward to working together with them the coming months (and maybe beyond)!

This week's edition was written by Reiner Herrmann and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

Planet DebianVincent Bernat: Pragmatic Debian packaging

While the creation of Debian packages is abundantly documented, most tutorials are targeted to packages implementing the Debian policy. Moreover, Debian packaging has a reputation of being unnecessarily difficult1 and many people prefer to use less constrained tools2 like fpm or CheckInstall.

However, I would like to show how building Debian packages with the official tools can become straightforward if you bend some rules:

  1. No source package will be generated. Packages will be built directly from a checkout of a VCS repository.

  2. Additional dependencies can be downloaded during build. Packaging individually each dependency is a painstaking work, notably when you have to deal with some fast-paced ecosystems like Java, Javascript and Go.

  3. The produced packages may bundle dependencies. This is likely to raise some concerns about security and long-term maintenance, but this is a common trade-off in many ecosystems, notably Java, Javascript and Go.

Pragmatic packages 101§

In the Debian archive, you have two kinds of packages: the source packages and the binary packages. Each binary package is built from a source package. You need a name for each package.

As stated in the introduction, we won’t generate a source package but we will work with its unpacked form which is any source tree containing a debian/ directory. In our examples, we will start with a source tree containing only a debian/ directory but you are free to include this debian/ directory into an existing project.

As an example, we will package memcached, a distributed memory cache. There are four files to create:

  • debian/compat,
  • debian/changelog,
  • debian/control, and
  • debian/rules.

The first one is easy. Just put 9 in it:

echo 9 > debian/compat

The second one has the following content:

memcached (0-0) UNRELEASED; urgency=medium

  * Fake entry

 -- Happy Packager <happy@example.com>  Tue, 19 Apr 2016 22:27:05 +0200

The only important information is the name of the source package, memcached, on the first line. Everything else can be left as is as it won’t influence the generated binary packages.

The control file§

debian/control describes the metadata of both the source package and the generated binary packages. We have to write a block for each of them.

Source: memcached
Maintainer: Vincent Bernat <bernat@debian.org>

Package: memcached
Architecture: any
Description: high-performance memory object caching system

The source package is called memcached. We have to use the same name as in debian/changelog.

We generate only one binary package: memcached. In the remaining of the example, when you see memcached, this is the name of a binary package. The Architecture field should be set to either any or all. Use all exclusively if the package contains only arch-independent files. In doubt, just stick to any.

The Description field contains a short description of the binary package.

The build recipe§

The last mandatory file is debian/rules. It’s the recipe of the package. We need to retrieve memcached, build it and install its file tree in debian/memcached/. It looks like this:

#!/usr/bin/make -f

DISTRIBUTION = $(shell lsb_release -sr)
VERSION = 1.4.25
PACKAGEVERSION = $(VERSION)-0~$(DISTRIBUTION)0
TARBALL = memcached-$(VERSION).tar.gz
URL = http://www.memcached.org/files/$(TARBALL)

%:
    dh $@

override_dh_auto_clean:
override_dh_auto_test:
override_dh_auto_build:
override_dh_auto_install:
    wget -N --progress=dot:mega $(URL)
    tar --strip-components=1 -xf $(TARBALL)
    ./configure --prefix=/usr
    make
    make install DESTDIR=debian/memcached

override_dh_gencontrol:
    dh_gencontrol -- -v$(PACKAGEVERSION)

The empty targets override_dh_auto_clean, override_dh_auto_test and override_dh_auto_build keep debhelper from being too smart. The override_dh_gencontrol target sets the package version3 without updating debian/changelog. If you ignore the slight boilerplate, the recipe is quite similar to what you would have done with fpm:

DISTRIBUTION=$(lsb_release -sr)
VERSION=1.4.25
PACKAGEVERSION=${VERSION}-0~${DISTRIBUTION}0
TARBALL=memcached-${VERSION}.tar.gz
URL=http://www.memcached.org/files/${TARBALL}

wget -N --progress=dot:mega ${URL}
tar --strip-components=1 -xf ${TARBALL}
./configure --prefix=/usr
make
make install DESTDIR=/tmp/installdir

# Build the final package
fpm -s dir -t deb \
    -n memcached \
    -v ${PACKAGEVERSION} \
    -C /tmp/installdir \
    --description "high-performance memory object caching system"

You can review the whole package tree on GitHub and build it with dpkg-buildpackage -us -uc -b.

Pragmatic packages 102§

At this point, we can iterate and add several improvements to our memcached package. None of those are mandatory but they are usually worth the additional effort.

Build dependencies§

Our initial build recipe only work when several packages are installed, like wget and libevent-dev. They are not present on all Debian systems. You can easily express that you need them by adding a Build-Depends section for the source package in debian/control:

Source: memcached
Build-Depends: debhelper (>= 9),
               wget, ca-certificates, lsb-release,
               libevent-dev

Always specify the debhelper (>= 9) dependency as we heavily rely on it. We don’t require make or a C compiler because it is assumed that the build-essential meta-package is installed and it pulls those. dpkg-buildpackage will complain if the dependencies are not met. If you want to install those packages from your CI system, you can use the following command4:

mk-build-deps \
    -t 'apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -qqy' \
    -i -r debian/control

You may also want to investigate pbuilder or sbuild, two tools to build Debian packages in a clean isolated environment.

Runtime dependencies§

If the resulting package is installed on a freshly installed machine, it won’t work because it will be missing libevent, a required library for memcached. You can express the dependencies needed by each binary package by adding a Depends field. Moreover, for dynamic libraries, you can automatically get the right dependencies by using some substitution variables:

Package: memcached
Depends: ${misc:Depends}, ${shlibs:Depends}

The resulting package will contain the following information:

$ dpkg -I ../memcached_1.4.25-0\~unstable0_amd64.deb | grep Depends
 Depends: libc6 (>= 2.17), libevent-2.0-5 (>= 2.0.10-stable)

Integration with init system§

Most packaged daemons come with some integration with the init system. This integration ensures the daemon will be started on boot and restarted on upgrade. For Debian-based distributions, there are several init systems available. The most prominent ones are:

  • System-V init is the historical init system. More modern inits are able to reuse scripts written for this init, so this is a safe common denominator for packaged daemons.
  • Upstart is the less-historical init system for Ubuntu (used in Ubuntu 14.10 and previous releases).
  • systemd is the default init system for Debian since Jessie and for Ubuntu since 15.04.

Writing a correct script for the System-V init is error-prone. Therefore, I usually prefer to provide a native configuration file for the default init system of the targeted distribution (Upstart and systemd).

System-V§

If you want to provide a System-V init script, have a look at /etc/init.d/skeleton on the most ancient distribution you want to target and adapt it5. Put the result in debian/memcached.init. It will be installed at the right place, invoked on install, upgrade and removal. On Debian-based systems, many init scripts allow user customizations by providing a /etc/default/memcached file. You can ship one by putting its content in debian/memcached.default.

Upstart§

Providing an Upstart job is similar: put it in debian/memcached.upstart. For example:

description "memcached daemon"

start on runlevel [2345]
stop on runlevel [!2345]
respawn
respawn limit 5 60
expect daemon

script
  . /etc/default/memcached
  exec memcached -d -u $USER -p $PORT -m $CACHESIZE -c $MAXCONN $OPTIONS
end script

When writing an Upstart job, the most important directive is expect. Be sure to get it right. Here, we use expect daemon and memcached is started with the -d flag.

systemd§

Providing a systemd unit is a bit more complex. The content of the file should go in debian/memcached.service. For example:

[Unit]
Description=memcached daemon
After=network.target

[Service]
Type=forking
EnvironmentFile=/etc/default/memcached
ExecStart=/usr/bin/memcached -d -u $USER -p $PORT -m $CACHESIZE -c $MAXCONN $OPTIONS
Restart=on-failure

[Install]
WantedBy=multi-user.target

We reuse /etc/default/memcached even if it is not considered a good practice with systemd6. Like for Upstart, the directive Type is quite important. We used forking as memcached is started with the -d flag.

You also need to add a build-dependency to dh-systemd in debian/control:

Source: memcached
Build-Depends: debhelper (>= 9),
               wget, ca-certificates, lsb-release,
               libevent-dev,
               dh-systemd

And you need to modify the default rule in debian/rules:

%:
    dh $@ --with systemd

The extra complexity is a bit unfortunate but systemd integration is not part of debhelper7. Without those additional modifications, the unit will get installed but you won’t get a proper integration and the service won’t be enabled on install or boot.

Dedicated user§

Many daemons don’t need to run as root and it is a good practice to ship a dedicated user. In the case of memcached, we can provide a _memcached user8.

Add a debian/memcached.postinst file with the following content:

#!/bin/sh

set -e

case "$1" in
    configure)
        adduser --system --disabled-password --disabled-login --home /var/empty \
                --no-create-home --quiet --force-badname --group _memcached
        ;;
esac

#DEBHELPER#

exit 0

There is no cleanup of the user when the package is removed for two reasons:

  1. Less stuff to write.
  2. The user could still own some files.

The utility adduser will do the right thing whatever the requested user already exists or not. You need to add it as a dependency in debian/control:

Package: memcached
Depends: ${misc:Depends}, ${shlibs:Depends}, adduser

The #DEBHELPER# marker is important as it will be replaced by some code to handle the service configuration files (or some other stuff).

You can review the whole package tree on GitHub and build it with dpkg-buildpackage -us -uc -b.

Pragmatic packages 103§

It is possible to leverage debhelper to reduce the recipe size and to make it more declarative. This section is quite optional and it requires understanding a bit more how a Debian package is built. Feel free to skip it.

The big picture§

There are four steps to build a regular Debian package:

  1. debian/rules clean should clean the source tree to make it pristine.

  2. debian/rules build should trigger the build. For an autoconf-based software, like memcached, this step should execute something like ./configure && make.

  3. debian/rules install should install the file tree of each binary package. For an autoconf-based software, this step should execute make install DESTDIR=debian/memcached.

  4. debian/rules binary will pack the different file trees into binary packages.

You don’t directly write each of those targets. Instead, you let dh, a component of debhelper, do most of the work. The following debian/rules file should do almost everything correctly with many source packages:

#!/usr/bin/make -f
%:
    dh $@

For each of the four targets described above, you can run dh with --no-act to see what it would do. For example:

$ dh build --no-act
   dh_testdir
   dh_update_autotools_config
   dh_auto_configure
   dh_auto_build
   dh_auto_test

Each of those helpers has a manual page. Helpers starting with dh_auto_ are a bit “magic”. For example, dh_auto_configure will try to automatically configure a package prior to building: it will detect the build system and invoke ./configure, cmake or Makefile.PL.

If one of the helpers do not do the “right” thing, you can replace it by using an override target:

override_dh_auto_configure:
    ./configure --with-some-grog

Those helpers are also configurable, so you can just alter a bit their behaviour by invoking them with additional options:

override_dh_auto_configure:
    dh_auto_configure -- --with-some-grog

This way, ./configure will be called with your custom flag but also with a lot of default flags like --prefix=/usr for better integration.

In the initial memcached example, we overrode all those “magic” targets. dh_auto_clean, dh_auto_configure and dh_auto_build are converted to no-ops to avoid any unexpected behaviour. dh_auto_install is hijacked to do all the build process. Additionally, we modified the behavior of the dh_gencontrol helper by forcing the version number instead of using the one from debian/changelog.

Automatic builds§

As memcached is an autoconf-enabled package, dh knows how to build it: ./configure && make && make install. Therefore, we can let it handle most of the work with this debian/rules file:

#!/usr/bin/make -f

DISTRIBUTION = $(shell lsb_release -sr)
VERSION = 1.4.25
PACKAGEVERSION = $(VERSION)-0~$(DISTRIBUTION)0
TARBALL = memcached-$(VERSION).tar.gz
URL = http://www.memcached.org/files/$(TARBALL)

%:
    dh $@ --with systemd

override_dh_auto_clean:
    wget -N --progress=dot:mega $(URL)
    tar --strip-components=1 -xf $(TARBALL)

override_dh_auto_test:
    # Don't run the whitespace test
    rm t/whitespace.t
    dh_auto_test

override_dh_gencontrol:
    dh_gencontrol -- -v$(PACKAGEVERSION)

The dh_auto_clean target is hijacked to download and setup the source tree9. We don’t override the dh_auto_configure step, so dh will execute the ./configure script with the appropriate options. We don’t override the dh_auto_build step either: dh will execute make. dh_auto_test is invoked after the build and it will run the memcached test suite. We need to override it because one of the test is complaining about odd whitespaces in the debian/ directory. We suppress this rogue test and let dh_auto_test executes the test suite. dh_auto_install is not overriden either, so dh will execute some variant of make install.

To get a better sense of the difference, here is a diff:

--- memcached-intermediate/debian/rules 2016-04-30 14:02:37.425593362 +0200
+++ memcached/debian/rules  2016-05-01 14:55:15.815063835 +0200
@@ -12,10 +12,9 @@
 override_dh_auto_clean:
-override_dh_auto_test:
-override_dh_auto_build:
-override_dh_auto_install:
    wget -N --progress=dot:mega $(URL)
    tar --strip-components=1 -xf $(TARBALL)
-   ./configure --prefix=/usr
-   make
-   make install DESTDIR=debian/memcached
+
+override_dh_auto_test:
+   # Don't run the whitespace test
+   rm t/whitespace.t
+   dh_auto_test

It is up to you to decide if dh can do some work for you, but you could try to start from a minimal debian/rules and only override some targets.

Install additional files§

While make install installed the essential files for memcached, you may want to put additional files in the binary package. You could use cp in your build recipe, but you can also declare them:

  • files listed in debian/memcached.docs will be copied to /usr/share/doc/memcached by dh_installdocs,
  • files listed in debian/memcached.examples will be copied to /usr/share/doc/memcached/examples by dh_installexamples,
  • files listed in debian/memcached.manpages will be copied to the appropriate subdirectory of /usr/share/man by dh_installman,

Here is an example using wildcards for debian/memcached.docs:

doc/*.txt

If you need to copy some files to an arbitrary location, you can list them along with their destination directories in debian/memcached.install and dh_install will take care of the copy. Here is an example:

scripts/memcached-tool usr/bin

Using those files make the build process more declarative. It is a matter of taste and you are free to use cp in debian/rules instead. You can review the whole package tree on GitHub.

Other examples§

The GitHub repository contains some additional examples. They all follow the same scheme:

  • dh_auto_clean is hijacked to download and setup the source tree
  • dh_gencontrol is modified to use a computed version

Notably, you’ll find daemons in Java, Go, Python and Node.js. The goal of those examples is to demonstrate that using Debian tools to build Debian packages can be straightforward. Hope this helps.


  1. People may remember the time before debhelper 7.0.50 (circa 2009) where debian/rules was a daunting beast. However, nowaday, the boilerplate is quite reduced. 

  2. The complexity is not the only reason. Those alternative tools enable the creation of RPM packages, something that Debian tools obviously don’t. 

  3. There are many ways to version a package. Again, if you want to be pragmatic, the proposed solution should be good enough for Ubuntu. On Debian, it doesn’t cover upgrade from one distribution version to another, but we assume that nowadays, systems get reinstalled instead of being upgraded. 

  4. You also need to install devscripts and equivs package. 

  5. It’s also possible to use a script provided by upstream. However, there is no such thing as an init script that works on all distributions. Compare the proposed with the skeleton, check if it is using start-stop-daemon and if it sources /lib/lsb/init-functions before considering it. If it seems to fit, you can install it yourself in debian/memcached/etc/init.d/. debhelper will ensure its proper integration. 

  6. Instead, a user wanting to customize the options is expected to edit the unit with systemctl edit

  7. See #822670 

  8. The Debian Policy doesn’t provide any hint for the naming convention of those system users. A common usage is to prefix the daemon name with an underscore (like _memcached). Another common usage is to use Debian- as a prefix. The main drawback of the latest solution is that the name is likely to be replaced by the UID in ps and top because of its length. 

  9. We could call dh_auto_clean at the end of the target to let it invoke make clean. However, it is assumed that a fresh checkout is used before each build. 

CryptogramVulnerabilities in Samsung's SmartThings

Interesting research: Earlence Fernandes, Jaeyeon Jung, and Atul Prakash, "Security Analysis of Emerging Smart Home Applications":

Abstract: Recently, several competing smart home programming frameworks that support third party app development have emerged. These frameworks provide tangible benefits to users, but can also expose users to significant security risks. This paper presents the first in-depth empirical security analysis of one such emerging smart home programming platform. We analyzed Samsung-owned SmartThings, which has the largest number of apps among currently available smart home platforms, and supports a broad range of devices including motion sensors, fire alarms, and door locks. SmartThings hosts the application runtime on a proprietary, closed-source cloud backend, making scrutiny challenging. We overcame the challenge with a static source code analysis of 499 SmartThings apps (called SmartApps) and 132 device handlers, and carefully crafted test cases that revealed many undocumented features of the platform. Our key findings are twofold. First, although SmartThings implements a privilege separation model, we discovered two intrinsic design flaws that lead to significant overprivilege in SmartApps. Our analysis reveals that over 55% of SmartApps in the store are overprivileged due to the capabilities being too coarse-grained. Moreover, once installed, a SmartApp is granted full access to a device even if it specifies needing only limited access to the device. Second, the SmartThings event subsystem, which devices use to communicate asynchronously with SmartApps via events, does not sufficiently protect events that carry sensitive information such as lock codes. We exploited framework design flaws to construct four proof-of-concept attacks that: (1) secretly planted door lock codes; (2) stole existing door lock codes; (3) disabled vacation mode of the home; and (4) induced a fake fire alarm. We conclude the paper with security lessons for the design of emerging smart home programming frameworks.

Research website. News article -- copy and paste into a text editor to avoid the ad blocker blocker.

EDITED TO ADD: Another article.

TEDRemembering Jok Church, 1949-2016

Jok Church speaks at TED2007

In March 2007, a handsome iron-haired man stepped onto the low stage at Monterey dressed in full leathers, and gave what might be the most beautiful 3-minute TED Talk of all time. Jok Church told a story from his childhood:

I was the class queer. I was the guy beaten up bloody every week in the boys’ room, until one teacher saved my life. She saved my life by letting me go to the bathroom in the teachers’ lounge. She did it in secret. She did it for three years.

Jok got a little older, escaped to San Francisco and became active in AIDS organizations, learning, as many did in the 1980s and ’90s, how to lovingly care for dying friends. And one day, that teacher, the one who’d saved his life, called him. Watch the talk, would you?

A longtime techie, Jok was the creator of You Can with Beakman and Jax,  a science comic strip for kids that ran in Sunday papers and inspired the long-running TV show Beakman’s World. As his friend Richard Bolingbroke writes:

Kids would write in with questions like “why is the sky blue” and he would create this amazing at-home science experiment to demonstrate how to prove this, along with an upside-down line of commentary, which often offered unsolicited personal advice on being a free thinker.

Which sounds exactly perfect. Jok died of a heart attack on April 29, 2016. He will be much missed.


Krebs on SecurityHow the Pwnedlist Got Pwned

Last week, I learned about a vulnerability that exposed all 866 million account credentials harvested by pwnedlist.com, a service designed to help companies track public password breaches that may create security problems for their users. The vulnerability has since been fixed, but this simple security flaw may have inadvertently exacerbated countless breaches by preserving the data lost in them and then providing free access to one of the Internet’s largest collections of compromised credentials.

PwndlistPwnedlist is run by Scottsdale, Ariz. based InfoArmor, and is marketed as a repository of usernames and passwords that have been publicly leaked online for any period of time at Pastebin, online chat channels and other free data dump sites.

The service until quite recently was free to all comers, but it makes money by allowing companies to get a live feed of usernames and passwords exposed in third-party breaches which might create security problems going forward for the subscriber organization and its employees.

This 2014 article from the Phoenix Business Journal describes one way InfoArmor markets the Pwnedlist to companies: “InfoArmor’s new Vendor Security Monitoring tool allows businesses to do due diligence and monitor its third-party vendors through real-time safety reports.”

The trouble is, the way Pwnedlist should work is very different from how it does. This became evident after I was contacted by Bob Hodges, a longtime reader and security researcher in Detroit who discovered something peculiar while he was using Pwnedlist: Hodges wanted to add to his watchlist the .edu and .com domains for which he is the administrator, but that feature wasn’t available.

In the first sign that something wasn’t quite right authentication-wise at Pwnedlist, the system didn’t even allow him to validate that he had control of an email address or domain by sending him a verification to said email or domain.

On the other hand, he found he could monitor any email address he wanted. Hodges said this gave him an idea about how to add his domains: Turns out that when any Pwnedlist user requests that a new Web site name be added to his “Watchlist,” the process for approving that request was fundamentally flawed.

That’s because the process of adding a new thing for Pwnedlist to look for — be it a domain, email address, or password hash — was a two-step procedure involving a submit button and confirmation page, and the confirmation page didn’t bother to check whether the thing being added in the first step was the same as the thing approved in the confirmation page. [For the Geek Factor 5 crowd here, this vulnerability type is known as “parameter tampering,” and it involves  the ability to modify hidden parameters in POST requests].

“Their system is supposed to compare the data that gets submitted in the second step with what you initially submitted in the first window, but there’s nothing to prevent you from changing that,” Hodges said. “They’re not even checking normal email addresses. For example, when you add an email to your watchlist, that email [account] doesn’t get a message saying they’ve been added. After you add an email you don’t own or control, it gives you the verified check box, but in reality it does no verification. You just typed it in. It’s almost like at some point they just disabled any verification systems they may have had at Pwnedlist.”

Hodges explained that one could easily circumvent Pwnedlist’s account controls by downloading and running a copy of Kali Linux — a free suite of tools made for finding and exploiting software and network vulnerabilities.

Always the student, I wanted to see this first-hand. I had a Pwnedlist account from way back when it first launched in 2011, so I fired up a downloadable virtual version of Kali on top of the free VirtualBox platform on my Mac.  Kali comes with a pretty handy vulnerability scanner called Burpsuite, which makes sniffing, snarfing and otherwise tampering with traffic to and from Web sites a fairly straightforward point-and-click exercise.

Indeed, after about a minute of instruction, I was able to replicate Hodges’ findings, successfully adding Apple.com to my watchlist. I also found I could add basically any resource I wanted. Although I verified that I could add top-level domains like “.com” and “.net,” I did not run these queries because I suspected that doing so would crash the database, and in any case might call unwanted attention to my account. (I also resisted the strong temptation to simply shut up about this bug and use it as my own private breach alerting service for the Fortune 500 firms).

Hodges told me that any newly-added domains would take about 24 hours to populate with results. But for some reason my account was taking far longer. Then I noticed that the email address I’d used to sign up for the free account back in 2011 didn’t have any hits in the Pwnedlist, and that was simply not possible if Pwnedlist was doing a halfway decent job tracking breaches. So I pinged InfoArmor and asked them to check my account. Sure enough, they said, it had never been used and was long ago deactivated.

Less than 12 hours after InfoArmor revived my dormant account, I received an automated email alert from the Pwnedlist telling me I had new results for Apple.com. In fact, the report I was then able to download included more than 100,000 usernames and passwords for accounts ending in apple.com. The data was available in plain text, and downloadable as a spreadsheet.

Some of the more than 100,000 credentials that Pwnedlist returned for me in a report on all passwords tied to email addresses that include "apple.com".

Some of the more than 100,000 credentials that Pwnedlist returned for me in a report on all passwords tied to email addresses that include “apple.com”.

It took a while for the enormity of what had just happened to sink in. I could now effectively request a report including all 866 million account credentials recorded by the Pwnsedlist. In short, the Pwnedlist had been pwned.

At this point, I got back in touch with InfoArmor and told them what Hodges had found and shown me. Their first response was that somehow I been given a privileged account on Pwnedlist, and that this is what allowed me to add any domain I chose. After all, I’d added the top 20 companies in the Fortune 500. How had I been able to do that?

“The account type you had had more privileges than an ordinary user would,” insisted Pwnedlist founder Alen Puzic.

After validating the bug, I added some other domains just for giggles. I deleted them all (except the Apple one) before they could generate reports.

After validating the bug, I added some other domains just for giggles. I deleted them all (except the Apple one) before they could generate reports.

I doubted that was true, and I suspected the vulnerability was present across their system regardless of which account type was used. Puzic said the company stopped allowing free account signups about six months ago, but since I had him on the phone I suggested he create a new, free account just for our testing purposes.

He rather gamely agreed. Within 30 seconds after the account was activated, I was able to add “gmail.com” to my Pwnedlist watchlist. Had we given it enough time, that query almost certainly would have caused Pwnedlist to produce a report with tens of millions of compromised credentials involving Gmail accounts.

“Wow, so you really can add whatever domain you want,” Puzic said in amazement as he loaded and viewed my account on his end.

Pwnedlist.com went offline shortly after my phonecall with InfoArmor.

Pwnedlist.com went offline shortly after my phonecall with InfoArmor.

It’s a shame that InfoArmor couldn’t design better authorization and authentication systems for Pwnedlist, given that the service itself is a monument to object failures in that regard. I’m a big believer in companies getting better intelligence about how large-scale everyday password breaches may impact their security, but it helps no one when a service that catalogs breaches has a lame security weakness that potentially prolongs and exacerbates them.

Update, 12:30 p.m. ET: InfoArmor downplayed the problem on Twitter, noting that “The data that was “exposed” has already been “compromised”- there was no loss of PII or subscriber data.” Also, a new notice is up on Pwnedlist.com, stating that the site is being shut down in a few weeks. The pop-up message reads:

“Thank you for being a subscriber and letting us help alert you of any risks related to your personal credentials. PwnedList launched in 2012 and quickly become the leader in open-source compromised data aggregation. In 2013 PwnedList was acquired by InfoArmor, Inc. a provider of enterprise based services. As part of the transition, the PwnedList Website has been scheduled for decommission on May 16, 2016. If you are interested in obtaining our commercial identity protection, please go to infoarmor.com for more information. It has been our pleasure to help you reduce your risk from compromised credentials.”

Worse Than FailureMercy the Mercenary in… a Heated Argument

Last time, Mercy found out the political campaign she was working for didn't have a candidate that was in his best health- but they were pushing him into the governor's mansion anyway. In today's finale, she confronts a hacker and a harsh reality…

Mercy cringed as Ellis waved her over to his laptop. She left her usual workspace next to the hamilton server and headed to where Ellis had holed up. On his laptop she saw a YouTube video, playing one of Rockwood’s stump speeches. “We can’t have the YouTube logo on here anymore,” he said, indicating the “Righteous Rants” design. The red and white logo clashed loudly with the Thomas Kinkade-inspired backgrounds Ellis had picked at random from a stock photo site. “Can you make it go away? We can’t be seen to endorse a company like that.”

Mercy could see the bags under his eyes.

A bonfire on a lake

“We can’t,” she said. “It’s against the terms of service to hide the YouTube logo from the embedded player. There is a third-party video player that streams YouTube videos, which you can customize all you want, but it only works on browsers with Flash installed.”

“Most people have Flash, don’t they?”

“Not on iOS or Android,” she said. “Just about all of our volunteers use smartphones for voter canvassing and campaign communications. If you switch to Flash only, they won’t be able to watch videos on their phones.”

“So they can just watch TV, then.”

“What about 18–24 year olds, all those young people that Rockwood wants to cover their college tuition? They rarely use desktop computers now. If they can’t watch Rockwood’s speeches, it’ll look like we’re purposefully ignoring them.”

At the bare mention of a demographic, Sullivan’s ears perked up and she came over. Mercy again explained the Flash situation.

“Come on, Ellis,” Sullivan said, “the logo doesn’t look that bad. And it’s only for a few more days.”

Ellis’s eyes narrowed, but he soon rubbed them and turned back to his laptop. “I guess you have much more important things to do.”

Mercy felt Sullivan’s fingernails in her shoulder as the campaign manager pulled her away from Ellis. “Just let him be, Mercy. I told you what he’s going through with the boss.”

“He gets away with a lot because of that,” she said. “Fine. I’ll just keep our servers up and running until after the polls close.”

“I hope you’ll stay on after that,” Sullivan said. “Imagine if you were part of our staff in Tallahassee. You’d be invaluable.”

“I need to think about it,” Mercy said, meaning not just the job offer but Rockwood’s diagnosis. Did she really want to work for a governor who was suffering from dementia?

She found her way back to hamilton. On the screen was a black-cloaked, hooded figure, with skeletal arms, floating above a lakeside castle.

“Okay, who put up the Dementor?” Mercy shouted to the volunteers. No one was brave enough to come forward. She closed the image, then checked the logs on the load balancer. hamilton was unresponsive, likely a memory leak, so she triggered a reboot.

But hamilton didn’t boot back up. And Mercy smelled burning plastic.

My Own Private Chernobyl

hamilton and jay are both out of commission,” Mercy said. Sullivan sat beside her, trying her best to keep up with Mercy’s explanation. “hamilton was the first. I noticed something was off when the load balancer reported that it wasn’t responding to requests. I figured a reboot would fix that. But then it didn’t come back. Just after that, jay did the same thing.”

It was well after hours. The phone operators had left, as no one wanted to hear a campaign call at 10PM. Ellis had escorted Rockwood home. It wasn’t a good day for their candidate, as he had stared listlessly at a wall for several hours, out of sight of their volunteer corps. Although Rockwood was out of it, the campaign was doing gangbusters, inching ahead of Packard in the polls.

But that all could change if their hub went down just a few days before election day.

“I decided to swap out hamilton’s hard drive, as that’s a common point of failure. I opened him up to replace the drive and I saw this.” Mercy spun hamilton around, showing Sullivan the server’s internals. She pointed to a gummy piece of plastic underneath a heat sink. “I smelled burning plastic earlier. That’s the CPU. We don’t overclock it, so that heat sink and the fans should have kept it cool.”

Mercy closed hamilton’s case. “I checked jay, too. Same thing. I thought there might be a recall on those boxes we got. We didn’t exactly buy top-of-the-line rack servers. So I called the manufacturer. There wasn’t any recall, but these CPUs do have a flaw. It’s called a ‘halt-and-catch-fire’ sequence. It’s a manufacturing flaw, a set of instructions that can cause the CPU to overheat.”

“We didn’t accidentally make the CPU do that, did we?”

“No. It’s all instructions that happen in kernel space — the operating system, I mean. Normal software shouldn’t even be able to execute it. Then I remembered something. Earlier today I was working on hamilton when Ellis asked about the YouTube logo. I stepped away for a second. When I came back, there was a picture of a Dementor open in the browser.”

“What’s a Dementor?”

“It’s a Harry Potter thing. I thought maybe one of the volunteers was playing a prank, since I forgot to lock my screen. But I don’t think I did. I think someone used hamilton to plant an exploit on our servers.”

“Well, why did it affect one of the other servers if it was just on that one?”

“I think the exploit copied itself onto the other servers. It specifically targeted our buildout. It used a defect in how Chromium opens images to inject malicious code into kernel space, which copied itself onto the other web servers, that affect only the kinds of computers we bought. So they all have it.” Mercy sighed. “We need to hire some people. I’m really stretched thin on this.”

“We need to keep this quiet, Mercy.” Sullivan frowned.

“Barbie, I can’t fix this on my own.” Mercy hadn’t used Sullivan’s first name since she had begun working on the campaign. She sighed. “I can’t fix this.”

Sullivan took her hand. “You know, with all the staff we have coming and going, I thought it might be prudent to place some security cameras in headquarters.” She pointed to a poster of Rockwood in a corner; one of his eyes had a pencil-sized hole in the pupil, where a small camera could be placed. “If you remember when you saw this demented or whatnot, we could get the footage.”

Enlarge and Enhance

It didn’t take long to find the culprit on the security footage. A man that neither Mercy nor Sullivan recognized had walked into headquarters, with black hair and black eyeliner. He wore a “Rockwood for Governor” shirt bought from the campaign store, not one issued to volunteers. He wandered through the crowd, grabbed a soda, then sat at the computer for about two minutes while Mercy was talking to Ellis. The footage was low-res, but to Mercy it looked like his fingernails were painted in alternating black and green. He inserted a USB drive into the front of the computer, opened the image of the Dementor, and left just before she got back to the computer.

“God, that’s quick,” Mercy said. “He had to know how we operate.”

“It’s too low quality,” Sullivan said. “We can’t make out his face.”

Mercy had an idea. “Hey, I wonder if he parked in the shopping center.”

“I have a camera pointing outside.” Sullivan switched feeds. This one was pointed through the glass storefront. The glare from the Orlando sun was intense, but they could make out the man walking to a black and green domestic car. She could make out three letters from the car’s license plate.

“I think that’s enough to find him,” Mercy said.

“I’ll call the county sheriff.”

“No, wait. You said we should keep this quiet. If the cops show up, Ellis will know what’s going on, and the media could pick up on it. I don’t know how long it could take for the other servers to, well, melt down. You need to get Ellis to put our code up on Seashell Hosting, like we discussed a while ago. We can’t keep managing it here, and this is proof. But I’m going to track him down myself.”

Sullivan guffawed. “What, are you a bounty hunter or something?”

“I have some good friends,” Mercy replied.

The Half-Blood Hacker

Mercy called an ex-girlfriend who worked at the Orange County Sheriff’s Office, who was more than willing to help. She gave Mercy the culprit’s address. He lived in a white stucco apartment complex in Kissimmee. The walls of his apartment building were stained brown from rusty sprinkler water. She found his apartment number, banged on the corrugated metal door.

He answered. His hair was nearly black, he was wearing black eyeliner, and he wore a Slytherin t-shirt and boxers.

“I know you did it,” she said.

A flash of recognition showed on the man’s face. He hid it with a fake smile. “I have no idea what you’re talking about,” he responded, in an affected British accent.

“Ever heard of surveillance? Stupid Slytherin, 10 points to Ravenclaw. Let me in and I won’t call the cops.”

The false bravado left his face, and he opened the door for her. She stepped inside, noting that he was more than just a casual fan of the Harry Potter books. From the movie posters on the walls, to the many collector’s copies of books on his shelves, to a framed photo of Helena Bonham Carter by his bed.

“A Dementor?” Mercy asked. “Seems pretty obvious.”

“It’s a calling card.” The man sat at his computer desk, squirming.

“Are you working alone?”

“I’ve been paid for my discretion.”

Mercy pulled out her phone. “Is it enough to cover bail?”

The man held up his hands. “Fine, it’s Packard.”

Mercy pocketed her phone. “Okay. What’s the fix?”

He giggled. “There isn’t one. I mean, maybe if it weren’t some web developer like you. Yeah, I looked you up on LinkedIn before I stopped by your headquarters. You put together that little cheap server farm yourself, I’ll bet. No, there’s no fix. Shutting down the hardware triggers the HACF sequence. Packard didn’t want everything down at once, because you people would know something was wrong and start digging. But I figure you’d apply rolling updates to each server, rebooting each time. Each one would fail, and you wouldn’t be able to guess why.”

“No fix? I don’t have time to talk to you, then.”

“Hey, could I get your number—“ the man said, the clanging of his front door drowning out his voice.

Executive Decision

“We’re done for,” Mercy said, getting back to headquarters an hour later. She explained the situation to Sullivan, who was at headquarters, and Ellis, who was out with Rockwood on speakerphone. The servers would need to run flawlessly until election day. If they didn’t — if they required a software update or a memory leak forced Mercy to turn any of them off — they’d melt. “We need hosting. I mean it.”

“Absolutely not,” Ellis said. “The boss is adamant.”

“You have no idea what’s going to happen if–”

“I have work to do.” Ellis hung up.

“Mercy, please.” Sullivan rubbed her temples. “Just … do what you can to keep the servers going. It’s only for a few days.”

Mercy considered her options. One: she could do her best to keep the servers up, and watch as they failed one by one, until election day came and there was nothing else that could be done. Two: she could buy more servers as others failed, but that could get expensive, not to mention tedious. Three: she could migrate everything onto Seashell’s safe hardware tonight, and risk Ellis’s wrath.

In the end, Rockwood, a man who couldn’t remember a conversation he had five minutes ago, wasn’t worth it.

“I’m moving the code to Seashell,” Mercy said. Her stomach felt heavy. “I’m leaving the campaign as soon as it’s done.”

Sullivan sunk into her chair. “I won’t stop you. I won’t stop you moving the code, and I won’t stop you leaving. I know it’s your choice.” She smiled wanly. “And Ellis won’t have any idea how to get the code back onto those servers, so it’ll stay running until after the election’s over.”

Mercy made the call to Seashell Hosting. Despite the late hour, they were excited about hosting Rockwood’s campaign and offered affordable rates. In a few hours she had transferred everything to their servers: the database, the code, their internal files. After that, she asked a technician to restart their virtual server, in case some rack-mounted unit suddenly melted. Nothing happened.

After she grabbed her things, she shook Sullivan’s hand. “I’ll keep in touch,” the older woman said.

Mercy drove home.

At her apartment, she turned on her laptop. She played the stump speech Rockwood gave in Stoneford, the one responsible for his meteoric rise and her own time with the campaign, one that would soon see him in the governor’s mansion in Tallahassee. She didn’t pay attention to the words this time. All she could see was a scared old man, desperately improvising to hide the fact that he couldn’t remember what he was doing. In the beginning, Mercy thought he was brilliant. But in the end, he simply had been making it up the whole time.

[Advertisement] Application Release Automation for DevOps – integrating with best of breed development tools. Free for teams with up to 5 users. Download and learn more today!

Worse Than FailureMercy the Mercenary in… a Heated Argument

Last time, Mercy found out the political campaign she was working for didn't have a candidate that was in his best health- but they were pushing him into the governor's mansion anyway. In today's finale, she confronts a hacker and a harsh reality…

Mercy cringed as Ellis waved her over to his laptop. She left her usual workspace next to the hamilton server and headed to where Ellis had holed up. On his laptop she saw a YouTube video, playing one of Rockwood’s stump speeches. “We can’t have the YouTube logo on here anymore,” he said, indicating the “Righteous Rants” design. The red and white logo clashed loudly with the Thomas Kinkade-inspired backgrounds Ellis had picked at random from a stock photo site. “Can you make it go away? We can’t be seen to endorse a company like that.”

Mercy could see the bags under his eyes.

A bonfire on a lake

“We can’t,” she said. “It’s against the terms of service to hide the YouTube logo from the embedded player. There is a third-party video player that streams YouTube videos, which you can customize all you want, but it only works on browsers with Flash installed.”

“Most people have Flash, don’t they?”

“Not on iOS or Android,” she said. “Just about all of our volunteers use smartphones for voter canvassing and campaign communications. If you switch to Flash only, they won’t be able to watch videos on their phones.”

“So they can just watch TV, then.”

“What about 18–24 year olds, all those young people that Rockwood wants to cover their college tuition? They rarely use desktop computers now. If they can’t watch Rockwood’s speeches, it’ll look like we’re purposefully ignoring them.”

At the bare mention of a demographic, Sullivan’s ears perked up and she came over. Mercy again explained the Flash situation.

“Come on, Ellis,” Sullivan said, “the logo doesn’t look that bad. And it’s only for a few more days.”

Ellis’s eyes narrowed, but he soon rubbed them and turned back to his laptop. “I guess you have much more important things to do.”

Mercy felt Sullivan’s fingernails in her shoulder as the campaign manager pulled her away from Ellis. “Just let him be, Mercy. I told you what he’s going through with the boss.”

“He gets away with a lot because of that,” she said. “Fine. I’ll just keep our servers up and running until after the polls close.”

“I hope you’ll stay on after that,” Sullivan said. “Imagine if you were part of our staff in Tallahassee. You’d be invaluable.”

“I need to think about it,” Mercy said, meaning not just the job offer but Rockwood’s diagnosis. Did she really want to work for a governor who was suffering from dementia?

She found her way back to hamilton. On the screen was a black-cloaked, hooded figure, with skeletal arms, floating above a lakeside castle.

“Okay, who put up the Dementor?” Mercy shouted to the volunteers. No one was brave enough to come forward. She closed the image, then checked the logs on the load balancer. hamilton was unresponsive, likely a memory leak, so she triggered a reboot.

But hamilton didn’t boot back up. And Mercy smelled burning plastic.

My Own Private Chernobyl

hamilton and jay are both out of commission,” Mercy said. Sullivan sat beside her, trying her best to keep up with Mercy’s explanation. “hamilton was the first. I noticed something was off when the load balancer reported that it wasn’t responding to requests. I figured a reboot would fix that. But then it didn’t come back. Just after that, jay did the same thing.”

It was well after hours. The phone operators had left, as no one wanted to hear a campaign call at 10PM. Ellis had escorted Rockwood home. It wasn’t a good day for their candidate, as he had stared listlessly at a wall for several hours, out of sight of their volunteer corps. Although Rockwood was out of it, the campaign was doing gangbusters, inching ahead of Packard in the polls.

But that all could change if their hub went down just a few days before election day.

“I decided to swap out hamilton’s hard drive, as that’s a common point of failure. I opened him up to replace the drive and I saw this.” Mercy spun hamilton around, showing Sullivan the server’s internals. She pointed to a gummy piece of plastic underneath a heat sink. “I smelled burning plastic earlier. That’s the CPU. We don’t overclock it, so that heat sink and the fans should have kept it cool.”

Mercy closed hamilton’s case. “I checked jay, too. Same thing. I thought there might be a recall on those boxes we got. We didn’t exactly buy top-of-the-line rack servers. So I called the manufacturer. There wasn’t any recall, but these CPUs do have a flaw. It’s called a ‘halt-and-catch-fire’ sequence. It’s a manufacturing flaw, a set of instructions that can cause the CPU to overheat.”

“We didn’t accidentally make the CPU do that, did we?”

“No. It’s all instructions that happen in kernel space — the operating system, I mean. Normal software shouldn’t even be able to execute it. Then I remembered something. Earlier today I was working on hamilton when Ellis asked about the YouTube logo. I stepped away for a second. When I came back, there was a picture of a Dementor open in the browser.”

“What’s a Dementor?”

“It’s a Harry Potter thing. I thought maybe one of the volunteers was playing a prank, since I forgot to lock my screen. But I don’t think I did. I think someone used hamilton to plant an exploit on our servers.”

“Well, why did it affect one of the other servers if it was just on that one?”

“I think the exploit copied itself onto the other servers. It specifically targeted our buildout. It used a defect in how Chromium opens images to inject malicious code into kernel space, which copied itself onto the other web servers, that affect only the kinds of computers we bought. So they all have it.” Mercy sighed. “We need to hire some people. I’m really stretched thin on this.”

“We need to keep this quiet, Mercy.” Sullivan frowned.

“Barbie, I can’t fix this on my own.” Mercy hadn’t used Sullivan’s first name since she had begun working on the campaign. She sighed. “I can’t fix this.”

Sullivan took her hand. “You know, with all the staff we have coming and going, I thought it might be prudent to place some security cameras in headquarters.” She pointed to a poster of Rockwood in a corner; one of his eyes had a pencil-sized hole in the pupil, where a small camera could be placed. “If you remember when you saw this demented or whatnot, we could get the footage.”

Enlarge and Enhance

It didn’t take long to find the culprit on the security footage. A man that neither Mercy nor Sullivan recognized had walked into headquarters, with black hair and black eyeliner. He wore a “Rockwood for Governor” shirt bought from the campaign store, not one issued to volunteers. He wandered through the crowd, grabbed a soda, then sat at the computer for about two minutes while Mercy was talking to Ellis. The footage was low-res, but to Mercy it looked like his fingernails were painted in alternating black and green. He inserted a USB drive into the front of the computer, opened the image of the Dementor, and left just before she got back to the computer.

“God, that’s quick,” Mercy said. “He had to know how we operate.”

“It’s too low quality,” Sullivan said. “We can’t make out his face.”

Mercy had an idea. “Hey, I wonder if he parked in the shopping center.”

“I have a camera pointing outside.” Sullivan switched feeds. This one was pointed through the glass storefront. The glare from the Orlando sun was intense, but they could make out the man walking to a black and green domestic car. She could make out three letters from the car’s license plate.

“I think that’s enough to find him,” Mercy said.

“I’ll call the county sheriff.”

“No, wait. You said we should keep this quiet. If the cops show up, Ellis will know what’s going on, and the media could pick up on it. I don’t know how long it could take for the other servers to, well, melt down. You need to get Ellis to put our code up on Seashell Hosting, like we discussed a while ago. We can’t keep managing it here, and this is proof. But I’m going to track him down myself.”

Sullivan guffawed. “What, are you a bounty hunter or something?”

“I have some good friends,” Mercy replied.

The Half-Blood Hacker

Mercy called an ex-girlfriend who worked at the Orange County Sheriff’s Office, who was more than willing to help. She gave Mercy the culprit’s address. He lived in a white stucco apartment complex in Kissimmee. The walls of his apartment building were stained brown from rusty sprinkler water. She found his apartment number, banged on the corrugated metal door.

He answered. His hair was nearly black, he was wearing black eyeliner, and he wore a Slytherin t-shirt and boxers.

“I know you did it,” she said.

A flash of recognition showed on the man’s face. He hid it with a fake smile. “I have no idea what you’re talking about,” he responded, in an affected British accent.

“Ever heard of surveillance? Stupid Slytherin, 10 points to Ravenclaw. Let me in and I won’t call the cops.”

The false bravado left his face, and he opened the door for her. She stepped inside, noting that he was more than just a casual fan of the Harry Potter books. From the movie posters on the walls, to the many collector’s copies of books on his shelves, to a framed photo of Helena Bonham Carter by his bed.

“A Dementor?” Mercy asked. “Seems pretty obvious.”

“It’s a calling card.” The man sat at his computer desk, squirming.

“Are you working alone?”

“I’ve been paid for my discretion.”

Mercy pulled out her phone. “Is it enough to cover bail?”

The man held up his hands. “Fine, it’s Packard.”

Mercy pocketed her phone. “Okay. What’s the fix?”

He giggled. “There isn’t one. I mean, maybe if it weren’t some web developer like you. Yeah, I looked you up on LinkedIn before I stopped by your headquarters. You put together that little cheap server farm yourself, I’ll bet. No, there’s no fix. Shutting down the hardware triggers the HACF sequence. Packard didn’t want everything down at once, because you people would know something was wrong and start digging. But I figure you’d apply rolling updates to each server, rebooting each time. Each one would fail, and you wouldn’t be able to guess why.”

“No fix? I don’t have time to talk to you, then.”

“Hey, could I get your number—“ the man said, the clanging of his front door drowning out his voice.

Executive Decision

“We’re done for,” Mercy said, getting back to headquarters an hour later. She explained the situation to Sullivan, who was at headquarters, and Ellis, who was out with Rockwood on speakerphone. The servers would need to run flawlessly until election day. If they didn’t — if they required a software update or a memory leak forced Mercy to turn any of them off — they’d melt. “We need hosting. I mean it.”

“Absolutely not,” Ellis said. “The boss is adamant.”

“You have no idea what’s going to happen if–”

“I have work to do.” Ellis hung up.

“Mercy, please.” Sullivan rubbed her temples. “Just … do what you can to keep the servers going. It’s only for a few days.”

Mercy considered her options. One: she could do her best to keep the servers up, and watch as they failed one by one, until election day came and there was nothing else that could be done. Two: she could buy more servers as others failed, but that could get expensive, not to mention tedious. Three: she could migrate everything onto Seashell’s safe hardware tonight, and risk Ellis’s wrath.

In the end, Rockwood, a man who couldn’t remember a conversation he had five minutes ago, wasn’t worth it.

“I’m moving the code to Seashell,” Mercy said. Her stomach felt heavy. “I’m leaving the campaign as soon as it’s done.”

Sullivan sunk into her chair. “I won’t stop you. I won’t stop you moving the code, and I won’t stop you leaving. I know it’s your choice.” She smiled wanly. “And Ellis won’t have any idea how to get the code back onto those servers, so it’ll stay running until after the election’s over.”

Mercy made the call to Seashell Hosting. Despite the late hour, they were excited about hosting Rockwood’s campaign and offered affordable rates. In a few hours she had transferred everything to their servers: the database, the code, their internal files. After that, she asked a technician to restart their virtual server, in case some rack-mounted unit suddenly melted. Nothing happened.

After she grabbed her things, she shook Sullivan’s hand. “I’ll keep in touch,” the older woman said.

Mercy drove home.

At her apartment, she turned on her laptop. She played the stump speech Rockwood gave in Stoneford, the one responsible for his meteoric rise and her own time with the campaign, one that would soon see him in the governor’s mansion in Tallahassee. She didn’t pay attention to the words this time. All she could see was a scared old man, desperately improvising to hide the fact that he couldn’t remember what he was doing. In the beginning, Mercy thought he was brilliant. But in the end, he simply had been making it up the whole time.

[Advertisement] Application Release Automation for DevOps – integrating with best of breed development tools. Free for teams with up to 5 users. Download and learn more today!

Planet DebianMichal Čihař: Weekly phpMyAdmin contributions 2016-W17

Last week was quite split into many smaller tasks - working on our libraries (both SQL parser and motranslator got new releases with bug fixes), fixing bugs for upcoming 4.6.1 and working on documentation.

From the libraries side, probably most visible is release of motranslator 1.0, just to claim it's now stable enough. Let's see if somebody else will pick it up as well or it will stay only for our use.

Most time was however spent on our documentation. We've agreed to move wiki from our server to GitHub wiki and reduce content available on the wiki. So far it's really mixture of user documentation, notes and developer documentation. The final shape should be that wiki will contain only developer documentation and all end user documentation will go to our documentation. So far I've gone through about half of user docs pages, deleted duplicated ones and moved content to our documentation. It is most visible on the user guide which now contains way more information and hopefully it will get more complete in near future.

Handled issues:

Filed under: English phpMyAdmin | 0 comments

Planet DebianRuss Allbery: Review: The Girl with the Dragon Tattoo

Review: The Girl with the Dragon Tattoo, by Stieg Larsson

Translator: Reg Keeland
Series: Millennium #1
Publisher: Vintage Crime
Copyright: 2005, 2008
Printing: June 2009
ISBN: 0-307-47347-3
Format: Mass market
Pages: 644

As The Girl with the Dragon Tattoo opens, Mikael Blomkvist is losing a criminal libel suit in Swedish court. His magazine, Millennium, published his hard-hitting piece of investigative journalism that purported to reveal sketchy arms deals and financial crimes by Hans-Erik Wennerström, a major Swedish businessman. But the underlying evidence didn't hold up, and Blomkvist could offer no real defense at trial. The result is a short prison stint for him (postponed several months into this book) and serious economic danger for Millennium.

Lisbeth Salander is a (very) freelance investigator for Milton Security. Her specialty is research and background checks: remarkably thorough, dispassionate, and comprehensive. She's almost impossible to talk to, tending to meet nearly all questions with stony silence, but Dragan Armansky, the CEO of Milton Security, has taken her partly under his wing. She, and Milton Security, were hired by a lawyer named Dirch Frode to do a comprehensive background check on Mikael Blomkvist, which she and Dragan present near the start of the book. The reason, as the reader discovers in a few more chapters, is that Frode's employer wants to offer Blomkvist a very strange job.

Over forty years ago, Harriet Vanger, scion of one of Sweden's richest industrial families, disappeared. Her uncle, Henrik Vanger, has been obsessed with her disappearance ever since, but in forty years of investigation has never been able to discover what happened to her. There are some possibilities for how her body could have been transported off the island the Vangers (mostly) lived, and live, on, but motive and suspects are still complete unknowns. Vanger wants Blomkvist to try his hand under the cover of writing a book about the Vanger family. Payment is generous, but even more compelling is Henrik Vanger's offer to give Blomkvist documented, defensible evidence against Wennerström at the end of the year.

The Girl with the Dragon Tattoo (the original Swedish title is Män som hatar kvinnor, "Men who hate women") is the first of three mystery novels written at the very end of Stieg Larsson's life, all published posthumously. They made quite a splash when they were published: won multiple awards, sold millions of copies, and have resulted in four movies to date. I've had a copy of the book sitting around for a while and finally picked it up when in the mood for something a bit different.

A major disclaimer up front: I read very little crime and mystery fiction. Every genre has its own conventions and patterns, and regular genre readers often look for different things than people new to that genre. My review is from a somewhat outside and inexperienced perspective, which may not be useful for regular genre readers.

I'm also a US reader, reading the book in translation. It appears to be a very good translation, but it was also quite obvious to me that The Girl with the Dragon Tattoo was written from a slightly different set of cultural assumptions than I brought to the book. This is one of the merits of reading books from other cultures in translation. It can be eye-opening, and can carry some of the same thrill as science fiction or fantasy, to hit the parts of the book that question your assumptions. But it can also be hard to tell whether some striking aspect of a book is due to a genre convention I wasn't familiar with, a Swedish cultural assumption that I don't share, or just the personal style of the author.

A few things do leap out as cultural differences. Blomkvist has to spend a few months in prison in the middle of this book, and that entire experience is completely foreign to an American understanding of what prison is like. The degradation, violence, and awfulness that are synonymous with prison for an American are almost entirely absent. He even enjoys the experience as quiet time to focus on writing a history of the Vangers (Blomkvist early on decides to take his cover story seriously, since he doubts he'll make any inroads into the mystery of Harriet's disappearance but can at least get a book out of it). It's a minor element in the book, glossed over in a few pages, but it's certainly eye-opening for how minimum security prison could be structured in a civilized country.

Similarly, as an American reader, I was struck by how hard Larsson has to work to ruin Salander's life. Although much of the book is written from Blomkvist's perspective (in tight third person), Lisbeth Salander is the titular girl with the dragon tattoo and becomes more and more involved in the story as it develops. The story Larsson wanted to tell requires that she be in a very precarious position legally and socially. In the US, this would be relatively easy, particularly for someone who acts like Salander does. In Sweden, Larsson has to go to monumental efforts to find ways for Salander to credibly fall through Sweden's comprehensive social safety net, and still mostly relies on Salander's complete refusal to assist or comply with any form of authority or support. I've read a lot about differences in policies around social support between the US and Scandinavian countries, but I've rarely read something that drove the point home more clearly than the amount of work a novelist has to go to in order to mess up their protagonist's life in Sweden.

The actual plot is slow-moving and as much about the psychology of the characters as it is about the mystery. The reader gets inside the thoughts of the characters occasionally, but Larsson shows far more than tells and leaves it to the reader to draw more general conclusions. Blomkvist's relationship with his long-time partner and Millennium co-founder is an excellent example: so much is left unstated that I would have expected other books to lay down in black and white, and the characters seem surprisingly comfortable with ambiguity. (Some of this may be my genre unfamiliarity; SFF tends to be straightforward to a fault, and more literary fiction is more willing to embrace ambiguous relationships.) While the mystery of Harriet's disappearance forms the backbone of the story, rather more pages are spent on Blomkvist navigating the emotional waters of the near-collapse of his career and business, his principles around investigation and journalism, and the murky waters of the Vanger's deeply dysfunctional family.

Harriet's disappearance is something of a locked room mystery. The day she disappeared, a huge crash closed the only bridge from the island to the mainland, both limiting suspects and raising significant questions about why her body was never found on the island. It's also forty years into the past, so Blomkvist has to rely on Henrik Vanger's obsessive archives, old photographs, and old police reports. I found the way it unfolded to be quite satisfying: there are just enough clues to let Blomkvist credibly untangle things with some hard work and research, but they're obscure enough to make it plausible that previous investigators missed them.

Through most of this novel, I wasn't sure what I thought of it. I have a personal interest in Blomkvist's journalistic focus — wrongdoing by rich financiers — but I had trouble warming to Blomkvist himself. He's a very passive, inward character, who spends a lot of the early book reacting to things that are happening to him. Salander is more dynamic and honestly more likable, but she's also deeply messed up, self-destructive, and does some viciously awful things in this book. And the first half of the book is very slow: lots of long conversations, lots of character introduction, and lots of Blomkvist wandering somewhat aimlessly. It's only when Larsson gets the two protagonists together that I thought the book started to click. Salander sees Blomkvist's merits more clearly than the reader can, I think.

I also need to give a substantial warning: The Girl with the Dragon Tattoo is a very violent novel, and a lot of that violence is sexual. By mid-book, Blomkvist realizes that Harriet's disappearance is somehow linked with a serial killer whose trademark is horrific, sexualized symbolism drawn from Leviticus. There is a lot of rape here, including revenge rape by a protagonist. If that sort of thing is likely to bother you, you may want to steer way clear.

That said, despite the slow pace, the nauseating subject matter, the occasionally very questionable ethics of protagonists, and a twist of the knife at the very end of the novel that I thought was gratuitously nasty on Larsson's part and wasn't the conclusion I wanted, I found myself enjoying this. It has a different pace and a different flavor than what I normally read, the characters are deep and complex enough to play off each other in satisfying ways, and Salander is oddly compelling to read about. Given the length, it's a substantial investment of time, but I don't regret reading it, and I'm quite tempted to read the sequel. I'm not sure this is the sort of book I can recommend (or not recommend) given my lack of familiarity with the genre, but I think US readers might get an additional layer of enjoyment out of seeing how different of a slant the Swedish setting puts on some of the stock elements of a crime novel.

Followed by The Girl Who Played with Fire.

Rating: 7 out of 10

,

Planet DebianLior Kaplan: Backporting of PHP security fixes

4 months ago I wrote my thoughts about PHP support during the “PHP 5 support timeline” vote:

I think we should limit what we guarantee (meaning keeping only one year of security support till end of 2017), and encourage project members and the eco-system (e.g. Linux distributions) to maintain further security based on best effort.

This is already the case for out of official support releases like the 5.3 and 5.4 branches (examples for backports done by Debian: 5.3 and 5.4). And of course, we also have companies that make their money out of long term support (e.g. RedHat).

On the other hand, we should help the eco system in doing such extended support, and hosting backported fixes in the project’s git repo instead of having each Linux disto do the same patch work on its own.

But suggesting to others what they should do is easy, so I decided to finally find the time to also implement this myself. I’ve started with back porting PHP 5.5 fixes to PHP 5.4, resulting in a GitHub repository with all the fixes, including CVE info NEWS file entries and references to the original commits. See https://github.com/kaplanlior/php-src/commits/PHP-5.4-security-backports . I hope this would later on find it’s way into PHP LTS packages for Debian Wheezy.

Next step would be to start doing the same for PHP 5.3 (back porting from PHP 5.4, and later on also from PHP 5.5). This can be in use for RHEL 6.x (as LTS support for Debian Squeeze was recently finished).

The main idea of this repo, is to have a more central location for such work, hoping people would review and contribute fixes that should be taken into consideration.

During the process of digging into the CVE information and the commits, I’m also filling up a info such as CVE IDs to the NEWS file (e.g. https://github.com/php/php-src/pull/1892/files) and the web changelog (e.g. https://github.com/php/web-php/commits?author=kaplanlior), so users and researchers would find this info where it should be instead of digging themselves.


Filed under: Debian GNU/Linux, PHP

Planet DebianThorsten Alteholz: My Debian Activities in April 2016

FTP assistant

This month I marked 171 packages for accept and rejected 42. I also sent 3 emails to maintainers asking questions. It seems to be that another quiet month is behind us. Nevertheless the flood of strange things in NEW continued this month. Hmm, weird world ..

Debian LTS

This was my twenty-second month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.

This month my all in all workload had been 15.75h. After getting the permission of the security team I changed the temporary-issues to meanwhile assigned CVEs and uploaded fuseiso. This resulted in DSA 3551-1.

I also prepared new packages for asterisk and asked for testers on the LTS mailing list. Luckily Gabriel Filion really tried these packages and found a regression with manager connections. Dear reader, the new packages are waiting for your tests now :-) .

Further I used the upload of poppler (DLA 446-1) to test the workflow of the new wheezy-security upload. Uploading and building packages worked perfectly. Unfortunately the push to the security mirrors was a bit delayed (it only happened after an upload of the security team). But this seems to be fixed by Ansgar now.

Last but not least I had a look at PHP5. I think I will start my regular uploads in May.

Other stuff

As I had to deal with non-Debian stuff this month, I didn’t do lots of other things. I only uploaded node-uml …

Planet DebianC.J. Adams-Collier: OMG Maven 3.0.4 on stretch

“Why?”, you might ask, would one want to run something other than the most recent version of Maven on the very newest and fangledest breed of the linux distribution we have all loved for so long.

“Because!”, I might answer, I’m trying to get the nexus-apt-plugin working on nexus.fd.io, and the version of nexus we’re running there explained to me in quite uncertain terms that it would talk to no other version of maven than 3.0.4 or something else that is not packaged for debian.

So I grabbed the source for version 3.0.4 from wheezy and patched it up to work with stretch:

$ cd /usr/src/deb
$ dget http.debian.net/debian/pool/main/m/maven/maven_3.0.4-3+deb7u1.dsc
$ cd maven-3.0.4
$ perl -i -pe 's/(libmodello-maven-plugin)1.4(-java)/$1$2/' debian/control
$ quilt pop -a
$ quild push 1
$ perl -i -pe 's/-1.4.x\.jar/.jar/' build.xml
$ perl -i -pe 's/google-collections/guava/' build.xml
$ perl -i -pe 's/\s+$//' build.xml
$ quilt refresh
$ quilt pop
$ quilt push -a
$ debuild -uc -us
$ sudo apt-get remove maven libmaven3-core-java
$ sudo dpkg -i ../maven_3.0.4-3+deb7u1_all.deb

And now I can build the silly nexus-apt-plugin…

$ mkdir -p /usr/src/git/github
$ git clone git@github.com:LLC-Technologies-Collier/nexus-apt-plugin.git /usr/src/git/github/nexus-apt-plugin
$ cd /usr/src/git/github/nexus-apt-plugin
$ mvn compile && mvn -q test

Planet DebianSimon Richter: With great power comes great responsibility

On the other hand,

export EDITOR='sed -ie "/^\+/s/ override//"'
yes e | git add -p

is a good way to commit all your changes except the addition of C++11 override specifiers.

Don MartiGeorge F. Will

I think I understand what George F. Will is going through right now.

I wish I didn't.

Once, I thought I was writing for an audience of people with a principled committment to a free economy and an aversion to centrally planned decision making..

I thought I was writing for readers who wanted to restore civilized norms.

I didn't think they just wanted an oversized angry personality who would violate those norms, but take on the establishment.

I didn't think that the readers would want to go for easy answers and bling over hard work and building a movement.

I was wrong.

The desktop Linux audience, which I thought was out there, went the same way as George F. Will's principled conservative audience.

I want the desktop Linux users back, and I want George F. Will to get his principled conservatives back. But maybe people were never who we thought they were to begin with.

Planet DebianBen Hutchings: 10 years as a Debian Developer

On 1st May 2006 my Debian account was created and I gained the status of Debian Developer. At that time I had already been to several BSPs and one DebConf, and maintained a few applications and Perl library packages. We were working toward the etch release and would soon hold DebConf 6 in Mexico.

Ten years later, I still maintain one of those packages (sgt-puzzles) but the rest were either handed over to the Perl team or entirely removed. I wrote, maintained, and then gave away dvswitch all within this period. I have packaged some other applications that I needed to use - kup, ministat, odhcp6c - and I continue to maintain them. I have also made many NMUs, including security uploads, for all kinds of packages including bind9, e2fsprogs, (e)glibc, lvm2, sudo, sysvinit and udev.

However, for about the past 7 years most of my work in Debian has been done within the kernel team, working on the Linux kernel and closely related packages - such as crda, ethtool, firmware-nonfree and initramfs-tools. I have also become an upstream developer for several of these projects.

I'm proud to have played a part in the etch, lenny, squeeze, wheezy and jessie releases, and I have enjoyed attending 7 more DebConfs and many mini-DebConfs. I'm now looking forward to another great release (stretch) and to attending DebConf 16 in Cape Town this summer winter. I hope to still be active in Debian in 2026, looking back on another 10 years in this amazing project.

Planet DebianBen Hutchings: Debian LTS work, April 2016

This month was still quiet for me in terms of uploads, as "wheezy" was only handed over to the LTS team near the end of the month. I carried over 5.5 hours from March and was assigned another 15 hours of work by Freexian's Debian LTS initiative, but only worked a total of 12.25 hours. I have returned the spare hours to the pool.

As last month, I prepared a stable update for Linux 3.2 on kernel.org, which will be released soon as 3.2.80. I also triaged the open security issues and backported a few individual patches to our wheezy-security branch. However I expect to rebase the wheezy-security branch onto Linux 3.2.80 before making the next upload.

I also participated in discussion of supporting armel/armhf in wheezy LTS. I don't expect many LTS users to be using the Debian kernel packages, as we only supported a small range of ARM hardware before the introduction of the multiplatform flavours in jessie. However, those architectures rarely require any extra effort to support in linux stable updates so I had no objection to including them.

Planet DebianRuss Allbery: Review: The Oath

Review: The Oath, by Jeffrey Toobin

Publisher: Anchor
Copyright: 2012
Printing: June 2013
ISBN: 0-307-39071-3
Format: Trade paperback
Pages: 298

Jeffrey Toobin is a legal analyst for CNN and The New Yorker and plays a similar role for the intricacies of the legal system as popular science writers play for physics. I'd previously read and reviewed his The Nine, an excellent history of the Rehnquist Supreme Court. The Oath is half sequel and half extension, bringing the same analysis to the first four years of the Obama presidency and the appointments of Sonia Sotomayor and Elena Kagan.

Sequels to popular history books that are not explicitly multi-volume works are a tricky publishing niche. People expect them to stand alone; I doubt it would work to tell people "read The Nine before reading this book," and regardless, Toobin did not take that approach. But the court profiled in The Oath only differs by two justices than that in The Nine. There was therefore a fair bit of repetition, since Toobin felt obligated to repeat his profiles of the five members of the court he had already deeply analyzed in the previous book. He even retold the story of Sandra Day O'Conner leaving the court despite it falling outside the focus of this book. I think these 300 pages could have been 150 pages of additional material in The Nine if Toobin had started this project later.

That said, if you enjoyed The Nine (and I very much did), this is more of the same. Toobin picks up with Obama's inauguration ceremony and a fascinating bit of legal trivia over the oath of office, and then provides a detailed profile of the Roberts court and the major decisions of the first four years of Obama's presidency. His discussion of the nomination process and Obama's judicial philosophy rang very true following the death of Scalia: Obama's nomination of Merrick Garland is exactly what one would predict from Toobin's discussion. And, as with the previous book, I discovered that I had a lot of misconceptions about both Sotomayor and Kagan that Toobin cleared up. He does a great job showing the complexities of the interplay between law, politics, apparently unlikely friendships (such as Scalia and Ginsburg), and the executive and judicial branch.

Worth particular mention is Toobin's discussion of the office of Solicitor General of the United States. I had no idea the role it plays in Supreme Court decisions. If I had given it any thought at all, I would have assumed it was essentially a variation on White House Counsel crossed with the Attorney General's office. But it's quite a bit more than that, as Elena Kagan's profile shows. If you, like I, raised an eyebrow at Obama's nomination of Elena Kagan to the Supreme Court from Solicitor General, wondering if that was at all similar to Bush's nomination of Harriet Miers, this section will be very informative. White House Counsel and Solicitor General are very, very different positions.

However, The Oath has one major drawback that The Nine didn't: it's partisan.

Now, Toobin is a liberal, with a clear preference towards the progressive side of the court. This was also true in The Nine, and I don't think that's a serious problem. Everyone writes from a particular perspective; stating it is more honest than concealing it, and it's the reader's responsibility to weigh multiple sides. But I thought Toobin was largely fair to those he disagreed with in The Nine. Even Thomas received some defense against popular misconceptions. It probably helped that much of that book focuses on conservatives who became liberals as the court shifted, people like Sandra Day O'Conner for whom Toobin has clear respect. I commented in my review of the previous book that it didn't feel quite balanced, but it felt like Toobin was trying hard to be fair.

The Oath does not give that same feeling. Toobin hates the direction of the Roberts court, hates most of its 5-4 decisions, and strongly disagrees with the judicial philosophies of both Roberts and Alito. But more than that, he is clearly dubious that they even have coherent judicial philosophies. Maybe that's a legitimate critique, maybe it's not; regardless, I don't think he proves his case. The tone of much of the book is disgusted and angry rather than deliberate and relentless. Where Toobin engages with the thought process of Alito or, particularly, Roberts, the primary focus is to disagree with it rather than explain it. This happens to match my own emotional reaction, but I doubt it will be persuasive to someone who doesn't already agree with Toobin, and it hurts the quality of the history.

I suspect this would have been a better book if Toobin had waited ten years before writing it (still covering the same time frame). Some distance from the subject helps provide a more complete and thoughtful history. But, of course, it likely wouldn't have sold as well.

That said, one of the themes of this book is how the conservatives on the Roberts court are currently playing the role of radicals from the perspective of the judicial tradition, overturning settled case law and calling into question precedents that have been used to decide numerous cases. The liberals, in contrast, are currently mostly playing the role of conservatives: standing up for the principle of stare decisis, trying to maintain consistency with past decisions, trying to minimize disruptive change. Conservatives will argue (correctly) that this depends on one's time frame and that they're trying to overturn radical past decisions, but those radical decisions, whatever their merits, are now often more than fifty years into the past. I hadn't thought about the current Supreme Court ideological battles from that perspective and found it eye-opening. It also ties in well with Obama's judicial philosophy as Toobin presents it: preferring democracy, laws, and change from the ballot box, and with little appetite for controversial court decisions. Obama is a judicial conservative. He therefore favors the liberal wing as the court is currently constructed, but not because he has much appetite for pushing forward civil rights in the courts.

This is not the book The Nine was. It's repetitive if you've read the previous book (which you should, as it's the better book of the two), and I thought Toobin's critical balance was off. But it has a lot of interesting things to say about Obama's approach to the law, how the executive branch interacts with the Supreme Court, and the philosophy and approaches of the newer justices on the court. Recommended, although not as strongly.

Rating: 7 out of 10

,

Planet DebianChris Lamb: Free software activities in April 2016

Here is my monthly update covering a large part of what I have been doing in the free software world (previously):

  • Added Python 3 support to django-template-tests, a tool to perform simple static analysis on Django templates. (#1)
  • Corrected my Chrome extension for the FastMail web interface to not disable the CTRL+Enter keyboard shortcut when authoring emails. (#3)
  • Corrected a subtle bug in my django-staticfiles-dotd "staticfiles" library where the Content-Length HTTP header was calculated incorrectly in the presence of Unicode characters resulting in truncated output. (#2)
  • Various fixes to django-slack, a library to easily post messages to the Slack group-messaging utility from projects using the Django web development framework:
    • Don't require an explicit backend import when using the Celery task queue backend. (#41)
    • Actually generate and send messages asynchronously when using the Celery backend. (#44)
  • Fixed an issue with my local-debian-mirror tool where the option to disable DEP-11 mirroring wasn't working. (#1)
  • Fixed an issue in django-hipchat, a library to easily post messages to the Hipchat group-messaging utility from projects using the Django web development framework where the templates were not includes when installing via PyPI. (#1)
  • Created a quick-and-dirty tool to scrape a Squarespace blog and convert it to a PDF so I can read them on my Kindle e-reader. (tree)
  • Updated django-keyerror — a library to post exceptions to the KeyError.com error tracking service — to silence an AttributeError exception in some error-reporting edge-cases. (commit)
  • Suggested an improvement to the documentation for the upcoming Twitter Bootstrap version for the deprecated .hidden and .show CSS classes. (#19789)
  • Submitted a documentation update to the Ansible sever configuration tool's ufw firewall module. (commit)
  • I also blogged about parsing Jenkins CI output to determine job success or failure.

Debian

My work in the Reproducible Builds project was covered in our weekly reports. (#48, #49, #50, #51 & #52)


Uploads

  • redis (2:3.0.7-3) — Adding, amongst some other changes, systemd LimitNOFILE support to allow a higher number of open file descriptors.


FTP Team


As a Debian FTP assistant I ACCEPTed 135 packages: aptitude, asm, beagle, blends, btrfs-progs, camitk, cegui-mk2, cmor-tables, containerd, debian-science, debops, debops-playbooks, designate-dashboard, efitools, facedetect, flask-testing, fstl, ganeti-os-noop, gnupg, golang-fsnotify, golang-github-appc-goaci, golang-github-benbjohnson-tmpl, golang-github-dchest-safefile, golang-github-docker-go, golang-github-dylanmei-winrmtest, golang-github-hawkular-hawkular-client-go, golang-github-hlandau-degoutils, golang-github-hpcloud-tail, golang-github-klauspost-pgzip, golang-github-kyokomi-emoji, golang-github-masterminds-semver-dev, golang-github-masterminds-vcs-dev, golang-github-masterzen-xmlpath, golang-github-mitchellh-ioprogress, golang-github-smartystreets-assertions, golang-gopkg-hlandau-configurable.v1, golang-gopkg-hlandau-easyconfig.v1, golang-gopkg-hlandau-service.v2, golang-objx, golang-pty, golang-text, gpaste, gradle-plugin-protobuf, grip, haskell-brick, haskell-hledger-ui, haskell-lambdabot-haskell-plugins, haskell-text-zipper, haskell-werewolf, hkgerman, howdoi, jupyter-client, jupyter-core, letsencrypt.sh, libbpp-phyl, libbpp-raa, libbpp-seq, libbpp-seq-omics, libcbor-xs-perl, libdancer-plugin-email-perl, libdata-page-pageset-perl, libevt, libevtx, libgit-version-compare-perl, libgovirt, libmsiecf, libnet-ldap-server-test-perl, libpgobject-type-datetime-perl, libpgobject-type-json-perl, libpng1.6, librest-client-perl, libsecp256k1, libsmali-java, libtemplates-parser, libtest-requires-git-perl, libtext-xslate-perl, linux, linux-signed, mandelbulber2, netlib-java, nginx, node-rc, node-utml, nvidia-cuda-toolkit, openfst, openjdk-9, openssl, php-cache-integration-tests, pulseaudio, pyfr, pygccxml, pytest-runner, python-adventure, python-arrayfire, python-django-feincms, python-fastimport, python-fitsio, python-imagesize, python-lib389, python-libtrace, python-neovim-gui, python3-proselint, pythonpy, pyzo, r-cran-ca, r-cran-fitbitscraper, r-cran-goftest, r-cran-rnexml, r-cran-rprotobuf, rrdtool, ruby-proxifier, ruby-seamless-database-pool, ruby-syslog-logger, rustc, s5, sahara-dashboard, salt-formula-ceilometer, salt-formula-cinder, salt-formula-glance, salt-formula-heat, salt-formula-horizon, salt-formula-keystone, salt-formula-neutron, salt-formula-nova, seer, simplejson, smrtanalysis, tiles-autotag, tqdm, tran, trove-dashboard, vim, vulkan, xapian-bindings & xapian-core.

Planet DebianDaniel Stender: My work for Debian in April

This month I've worked on the these things for Debian:

To begin with that, I've set up a Debhelper sequencer script for dh-buildinfo1, this add-on now can be used with dh $@ --with buildinfo in deb/rules instead of having to explicitly call it somewhere in an override.

Debops

I've set up initial Debian packages of Debops2, a collection of fine crafted Ansible roles and playbooks especially for Debian servers (servers which run on Debian), which are shipped with a couple of helper and wrapper scripts in Python3. There are two binary packages, one for the toolset (debops), and the other for the playbooks and roles of the project (debops-playbooks).

The application is easy to use, just initialize a new project with debops-init foo and add your server(s) to foo/ansible/inventory/hosts belonging to groups representing services and things you want to employ on them. For example, the group [debops_gitlab] automatically installs a complete running Gitlab setup on one or a multitude of servers in the same run with the debops command4. Other groups like [debops_mariadb_server] could be used accordingly in the same host inventory. Ansible works without agent, so you don't have to prepare freshly setup servers with nothing special to use that tool randomly (like on localhost). The list of things you could deploy with Debops is quite amazing and dozens of services are at hand.

The new Debian packages are currently in experimental because they need some more fine tuning, e.g. there are a couple of minor error messages which recently occur using it, but it works well. The (early staged) documentation unfortunately couldn't be packaged because of the scattered resp. collective nature of the project (all parts have their own Github repositories)5, and also how to generate the upstream tarball remains a bit of a challenge (currently, it's the outcome of debops-init)6. I'll have this package in unstable soon. More info on Debops is coming up, then.

HashiCorp's Packer

I'm very glad to announce that Packer7 is ready being available in unstable, and the RFP bug could be finally closed after I've taken it over8. It's another great and much convenient devops tool which does a lot of different things in an automated fashion using only a single "one-argument" CLI tool in combination with a couple of lines in a configuration script (thanks to Yaroslav Halchenko for the tip).

Packer helps creating machine images for different platforms. This is like when you use e.g. Debian installations in a Qemu box for testing or development purposes. Instead of setting up a new virtual machine manually the same way as installing Debian on another computer this process can be completely automated with Packer, like I've written about in this blog entry here9. You just need a template which contains instructions for the included Qemu builder and a preseeding script for the Debian installer, and there you go drinking your coffee while Packer does all the work: download the ISO image for installation, create the new virtual harddrive, boot the emulator, run the whole installation process automatically like with answering questions, selecting things, reboot without ISO image to complete the installation etc. A couple of minutes and you have a new pre-baked virtual machine image like from a vendoring machine, another fresh one could be created anytime.

Packer10 supports a number of builders for different target platforms (desktop virtualization solutions as much as public cloud providers and private cloud software), can build in parallel, and also the full range of common provisioners can be employed in the process to equip the newly installed OSs with services and programs. Vagrant boxes could be generated by one of the included postprocessors. I'll write more on Packer here on this blog, soon.

There were more then two dozens of packages missing to complete Packer11, which is the achievement of combined forces within the pkg-go group. Much thanks esp. to Alexandre Viau who have worked on the most of the needed new packages. Thanks also to the FTP masters which were always very quick in reviewing the Go packages, so that it could be proceeded to build and package the sub dependent new ones always consecutively.

Squirrel3

I've didn't had the major work of that and just sponsored this for Fabian Wolff, but want to highlight here that there's a new package of Squirrel12 now available in Debian13.

Squirrel is a lightweight scripting language, somewhat comparable to Lua. It's fully object-oriented and highly embeddable, it's used in a lot of commerical computer games under the hood for implementing intelligence for bots next to other things14, but also for the Internet of Things (it's embedded in hardware from Electric Imp). Squirrel functions could be called from C++15.

I've filed an ITP bug for Squirrel already in 2011 (#651195), but always something else had a higher priority, and it ended up being an RFP. I'm really glad that it got picked up and completed quickly afterwards.

misc

There were a couple of uploads on updated upstream tarballs and for fixing bugs, namely afl/2.10b-1 and 2.11b-1, python-afl/0.5.3-1, pyutilib/5.3.2-1, pyomo/4.3.11327-1, libvigraimpex/1.10.0+git20160211.167be93dfsg-2 (fix of #820429, thanks to Tobias Frost), and gamera/3.4.2+svn1454-1.

For the pkg-go group, I've set up a new package of github-mitchellh-ioprogress (which is needed by the official DigitalOcean CLI tool doctl, now RFP #807956 instead of ITP due to the lack of time, again a lot of missing packages are missing for that), and provided a little patch for dh-make-golang updating some standards16.

For Packer I've also updated azure-go-autorest and azure-sdk as team upload (#821938, #821832), but it came out that the project which is currently under heavy development towards a new official release broke a lot in the past weeks (no Git branching have been used), so that Packer as a matter of fact needed a vendored snapshot, although there have been only a couple of commits in between. Docker-registry has the same problem with the new package of azure-sdk/2.1.1~beta1, so that it needed to be fixed, too (#822146).

By the way, the tool ratt17 comes very handy for automatically test building down all reverse dependencies, not only for Go packages (thanks to Tianon Gravi for the tip).

Finally, I've posted the needed reverse depencies as RFP bugs for Terraform18 (again quite a lot), Vuls19, and cve-dictionary20, which is needed for Vuls. I'll let them rest a while waiting to get picked up before working anything down.


  1. #570933: dh-buildinfo: should provide a addon file for dh command 

  2. https://tracker.debian.org/pkg/debops 

  3. http://debops.org/ 

  4. The servers have to be accessible by SSH. E.g. you could run debops like: $ debops -u root --private-key=~/.ssh/id_digitalocean 

  5. https://github.com/debops/docs/issues/132 

  6. #819816: ITP: debops -- Ansible based server management utility 

  7. https://www.packer.io/ 

  8. #740753: ITP: packer -- create vm images for multiple platforms 

  9. http://www.danielstender.com/blog/packer-qemu.html 

  10. https://packages.debian.org/unstable/packer 

  11. I've worked on the missing packages this month, namely github-klauspost-pgzip, github-masterzen-xmlpath, github-masterzen-winrm, dylanmei-winrmtest, packer-community-winrmcp (Packer uses WinRM if Windows machines images are created), github-hpcloud-tail, and updated github-rackspace-gophercloud (#822163) and google-api (#822164) to complete it. 

  12. http://squirrel-lang.org/ 

  13. https://tracker.debian.org/pkg/squirrel3 

  14. http://www.linux-magazin.de/layout/set/print/content/view/full/62184 

  15. http://www.linux-magazin.de/Ausgaben/2011/10/plus/Fremdkoerper-Squirrel-Interpreter-und-Skripte-fuer-C 

  16. https://github.com/Debian/dh-make-golang/pull/39 

  17. https://packages.debian.org/unstable/ratt 

  18. #808940: ITP: terraform -- tool for managing cloud infrastructure 

  19. #820614: ITP: vuls -- package inventory scanner for CVE vulnerabilities 

  20. #820615: ITP: go-cve-dictionary -- builds a local copy of the NVD/JVN (vulnerability databases) 

Planet DebianDaniel Stender: What I've worked on for Debian this month

This month I've worked on the following things for Debian:

To begin with that, I've set up a Debhelper sequencer script for dh-buildinfo1, this add-on now can be used with dh $@ --with buildinfo in deb/rules instead of having to explicitly call it somewhere in an override.

Debops

I've set up initial Debian packages of Debops2, a collection of fine crafted Ansible roles and playbooks especially for Debian servers, shipped with a couple of convenience and wrapper scripts in Python3. There are two binary packages, one for the toolset (debops), and the other for the playbooks and roles of the project (debops-playbooks).

The application is easy to use, just initialize a new project with debops-init foo and add your server(s) to foo/ansible/inventory/hosts belonging to groups representing services and things you want to employ on them. Like the group [debops_gitlab] automatically installs a complete running Gitlab setup on one or a multitude of servers in the same run with the debops command4. Use other groups like [debops_mariadb_server] accordingly in the same host inventory. Ansible runs agent less, so you don't have to prepare freshly setup servers with nothing special to use that tool randomly (like on localhost). The list of things you could deploy with Debops is quite amazing and you've got dozens of services at your hand.

The new packages are currently in experimental because they need some more fine tuning, like there are a couple of minor error messages which recently occur using it, but it works well. The (early staged) documentation unfortunately couldn't be packaged because of the scattered resp. collective nature of the project (all parts have their own Github repositories)5, and also how to generate the upstream tarball remains a bit of a challenge (currently, it's the outcome of debops-init)6. I'll have this package in unstable soon. More info on Debops is coming up, then.

Hashicorp's Packer

I'm very glad to announce that Packer7 is ready being available in unstable, and the two year old RFP bug could be finally closed8. It's another great and much convenient devops tool which does a lot of different things in an automated fashion using only a single "one-argument" CLI tool in combination with a couple of lines in a configuration script (thanks to Yaroslav Halchenko for the tip).

Packer helps creating machine images for different platforms. This is like when you use e.g. Debian installations in a Qemu box for testing or development purposes. Instead of setting up a new virtual machine manually like installing Debian on another computer this process could be automated with Packer, like I've written about in this blog entry here9. You just need a template containing instructions for the included Qemu-builder and a preseeding script for the Debian installer, and there you go drinking your coffee while Packer does all the work for you: downloading the installation ISO image, creating the new virtual harddrive, booting the emulator, running the whole installation process automatically like answering questions, selecting things, rebooting without ISO image to complete the installation etc. A couple of minutes and you have a new pre-baked virtual machine image like from a vendoring machine, a fresh one everytime you need it.

Packer10 supports a number of builders for different target platforms (desktop virtualization solutions as much as public cloud providers and private cloud software), can build in parallel, and also the full range of common provisioners can be employed in the process to equip the newly installed OSs. Vagrant boxes could be generated by one of the included postprocessors. I'll write more on Packer here on this blog, soon.

There were more then two dozens of packages missing to complete Packer11, which is the achievement of combined forces within the pkg-go group. Much thanks esp. to Alexandre Viau who have worked on the most of the needed new packages. Thanks also to the FTP-masters which were always very quick in reviewing the Go packages, so that it could be proceeded to build and package the sub dependent new ones always consecutively.

Squirrel3

I've didn't had the most work with it and just sponsored this for Fabian Wolff, but want to highlight here that there's a new package of Squirrel12 now available in Debian13.

Squirrel is a lightweight scripting language, somewhat comparable to Lua. It's fully object-oriented and highly embeddable, it's used in a lot of commerical computer games under the hood for implementing intelligence for bots next to other things14, but also for the Internet of Things (it's embedded in hardware from Electric Imp). Squirrel functions could be called from C++15.

I've filed an ITP bug for Squirrel already in 2011 (#651195), but always something else got in the way, and it ended up being an RFP. I'm really glad that it got picked up and completed.

misc

There were a couple of uploads on updated upstream tarballs and for fixing bugs, namely afl/2.10b-1 and 2.11b-1, python-afl/0.5.3-1, pyutilib/5.3.2-1, pyomo/4.3.11327-1, libvigraimpex/1.10.0+git20160211.167be93dfsg-2 (fix of #820429, thanks for Tobias Frost), and gamera/3.4.2+svn1454-1.

For the pkg-go group, I've set up a new package of github-mitchellh-ioprogress (which is needed by the official DigitalOcean CLI tool doctl, now RFP #807956 instead of ITP due to the lack of time - again facing a lot of missing packages), and provided a little patch for dh-make-golang updating some standards16.

For Packer I've also updated azure-go-autorest and azure-sdk as team upload (#821938, #821832), but it came out that the project which is currently under heavy development towards a new official release broke a lot in the past weeks (and no Git branching have been used), so that Packer as a matter of fact needed a vendored snapshot, although there have been only a couple of commits in between. Docker-registry hat the same problem with the new package of azure-sdk/2.1.1~beta1, so that it needed to be fixed, too (#822146).

By the way, the tool ratt17 comes very handy for automatically test building down all reverse dependencies, not only for Go packages (thanks to Tianon Gravi for the tip).

Finally, I've posted the needed reverse depencies as RFP bugs for Terraform18 (again quite a lot), Vuls19, and cve-dictionary20, which is needed for Vuls. I'll let them rest a while waiting to get picked up before working anything down.


  1. #570933: dh-buildinfo: should provide a addon file for dh command 

  2. https://tracker.debian.org/pkg/debops 

  3. http://debops.org/ 

  4. The servers have to be accessible by SSH. E.g. you could run debops like: $ debops -u root --private-key=~/.ssh/id_digitalocean 

  5. https://github.com/debops/docs/issues/132 

  6. #819816: ITP: debops -- Ansible based server management utility 

  7. https://www.packer.io/ 

  8. #740753: ITP: packer -- create vm images for multiple platforms 

  9. http://www.danielstender.com/blog/packer-qemu.html 

  10. https://packages.debian.org/unstable/packer 

  11. I've worked on the missing packages this month, namely github-klauspost-pgzip, github-masterzen-xmlpath, github-masterzen-winrm, dylanmei-winrmtest, packer-community-winrmcp (Packer uses WinRM if Windows machines images are created), github-hpcloud-tail, and updated github-rackspace-gophercloud (#822163) and google-api (#822164) to complete it. 

  12. http://squirrel-lang.org/ 

  13. https://tracker.debian.org/pkg/squirrel3 

  14. http://www.linux-magazin.de/layout/set/print/content/view/full/62184 

  15. http://www.linux-magazin.de/Ausgaben/2011/10/plus/Fremdkoerper-Squirrel-Interpreter-und-Skripte-fuer-C 

  16. https://github.com/Debian/dh-make-golang/pull/39 

  17. https://packages.debian.org/unstable/ratt 

  18. #808940: ITP: terraform -- tool for managing cloud infrastructure 

  19. #820614: ITP: vuls -- package inventory scanner for CVE vulnerabilities 

  20. #820615: ITP: go-cve-dictionary -- builds a local copy of the NVD/JVN (vulnerability databases) 

Planet DebianStein Magnus Jodal: March and April contributions

The following is a short summary of my open source work in March and April, almost like in previous months, except that I haven’t spent as much time as previously on Open Source the last two months.

Debian

Mopidy

  • Bugfixes for the upcoming Mopidy 2.0.1 (which should have been released a long time ago): merged PR #1455, created PR #1493.

  • Started on, but didn’t finish, fixing the Travis CI setup for Mopidy-GMusic.

  • Upgraded the Mopidy project server from Ubuntu 14.04 LTS to 16.04 LTS. Rebuilt the discuss.mopidy.com Discourse/Docker instance.

  • Accepted Lars Kruse as the new maintainer of Mopidy-Beets. Thanks!

  • The extensions still in need of a new maintainer are:

    If you’re a user of any of these and want to contribute, please step up. Instructions can be found in the README of any of these projects.

,

CryptogramFriday Squid Blogging: Global Squid Shortage

There's a squid shortage along the Pacific coast of the Americas.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

CryptogramI'm Writing a Book on Security

I'm writing a book on security in the highly connected Internet-of-Things World. Tentative title:

Click Here to Kill Everybody
Peril and Promise in a Hyper-Connected World

There are two underlying metaphors in the book. The first is what I have called the World-Sized Web, which is that combination of mobile, cloud, persistence, personalization, agents, cyber-physical systems, and the Internet of Things. The second is what I'm calling the "war of all against all," which is the recognition that security policy is a series of "wars" between various interests, and that any policy decision in any one of the wars affects all the others. I am not wedded to either metaphor at this point.

This is the current table of contents, with three of the chapters broken out into sub-chapters:

  • Introduction
  • The World-Sized Web
  • The Coming Threats
    • Privacy Threats
    • Availability and Integrity Threats
    • Threats from Software-Controlled Systems
    • Threats from Interconnected Systems
    • Threats from Automatic Algorithms
    • Threats from Autonomous Systems
    • Other Threats of New Technologies
    • Catastrophic Risk
    • Cyberwar
  • The Current Wars
    • The Copyright Wars
    • The US/EU Data Privacy Wars
    • The War for Control of the Internet
    • The War of Secrecy
  • The Coming Wars
    • The War for Your Data
    • The War Against Your Computers
    • The War for Your Embedded Computers
    • The Militarization of the Internet
    • The Powerful vs. the Powerless
    • The Rights of the Individual vs. the Rights of Society
  • The State of Security
  • Near-Term Solutions
  • Security for an Empowered World
  • Conclusion

That will change, of course. If the past is any guide, everything will change.

Questions: Am I missing any threats? Am I missing any wars?

Current schedule is for me to finish writing this book by the end of September, and have it published at the end of April 2017. I hope to have pre-publication copies available for sale at the RSA Conference next year. As with my previous book, Norton is the publisher.

So if you notice me blogging less this summer, this is why.

Planet DebianIain R. Learmonth: MiniDebCamp Vienna 2016

I'm currently in Vienna for MiniDebCamp and MiniDebConf at FH Technikum Wien, hosted as a part of Linuxwochen Wien. Today and yesterday have been spent hacking on Debian, and I've produced a few package updates and closed a few bugs.

Scapy

The last update to Scapy in Debian was in August 2011. Bug #773554 was filed in 2014 to request a new upstream version be packaged and in a few days this bug should be closed. As this package is maintained by someone else and I'm performing a non-maintainer upload, the upload will sit in the delayed queue for 3 days.

There has also been a Python 3 port of Scapy developed, and I've also packaged this (bug #822096). You will be able to install this version as python3-scapy and run it as /usr/bin/scapy3, which means it can fully co-exist with an installation of the original Python 2 version on the same system.

Hamradio Blend

  • Carles Fernandez had produced an updated package for gnss-sdr and this has now been uploaded to unstable.
  • Ana Custura produced an updated package for chirp and this has now been uploaded to unstable.
  • I've made a couple of updates to the hamradio-maintguide and released version 0.2 of that documentation. This was simple changes for secure URIs for the Vcs-* fields and options for updating existing packages to new upstream versions using uscan, by tarball URL or by local tarball.

qtile

qtile is a "full-featured, hackable tiling window manager written and configured in Python". I've been a big fan of tiling window managers for a few years now, starting with XMonad in 2013, moving to i3 and now I plan to move to qtile as I think its customisability and complete removal of window decorations will work well for me. I have now packaged qtile (bug #762637), and updated one of its dependencies (xcffib), and it will appear in unstable after passing ftp-master scrutiny.

Krebs on SecurityA Dramatic Rise in ATM Skimming Attacks

Skimming attacks on ATMs increased at an alarming rate last year for both American and European banks and their customers, according to recent stats collected by fraud trackers. The trend appears to be continuing into 2016, with outbreaks of skimming activity visiting a much broader swath of the United States than in years past.

Two network cable card skimming devices, as found attached to this ATM.

Two network cable card skimming devices, as found attached to this ATM.

In a series of recent alerts, the FICO Card Alert Service warned of large and sudden spikes in ATM skimming attacks. On April 8, FICO noted that its fraud-tracking service recorded a 546 percent increase in ATM skimming attacks from 2014 to 2015.

“The number of ATM compromises in 2015 was the highest ever recorded by the FICO Card Alert Service, which monitors hundreds of thousands of ATMs in the US,” the company said. “Criminal activity was highest at non-bank ATMs, such as those in convenience stores, where 10 times as many machines were compromised as in 2014.”

While 2014 saw skimming attacks targeting mainly banks in big cities on the east and west coasts of the United States, last year’s skimming attacks were far more spread out across the country, the FICO report noted.

Earlier this year, I published a post about skimming attacks targeting non-bank ATMs using hidden cameras and skimming devices plugged into the ATM network cables to intercept customer card data. The skimmer pictured in that story was at a 7-Eleven convenience store.

Since that story ran I’ve heard from multiple banking industry sources who said they have seen a spike in ATM fraud targeting cash machines in 7-Elevens and other convenience stores, and that the commonality among the machines is that they are all operated by ATM giant Cardtronics (machines in 7-Eleven locations made up for 17.5 percent of Cardtronics’ revenue last year, according to this report at ATM Marketplace).

Some financial institutions are taking dramatic steps to head off skimming activity. Trailhead Credit Union in Portland, Ore., for example, has posted a notice to customers atop its Web site, stating:

“ALERT: Until further notice, we have turned off ATM capabilities at all 7-11 ATMs due to recent fraudulent activity. Please use our ATM locator for other locations. We are sorry for the inconvenience.”

Trailhead Credit Union has stopped allowing members to withdraw cash from 7-11 ATMs.

Trailhead Credit Union has stopped allowing members to withdraw cash from 7-11 ATMs.

7-Eleven did not respond to requests for comment. Cardtronics said it wasn’t aware of any banks blocking withdrawals across the board at 7-11 stores or at Cardtronics machines.

“While Cardtronics is aware that a single financial institution [Xceed Financial Credit Union] temporarily restricted ATM access late in 2015, it soon thereafter restored full ATM access to its account holders,” the company said in a statement. “As the largest ATM services provider, Cardtronics has a long history of executing a layered security strategy and implementing innovative security enhancements at our ATMs. As criminals modify their attack, Cardtronics always has and always will aggressively respond, reactively and proactively, with innovation to address these instances.”

DRAMA IN DC

A bit closer to home for this author, on April 22 FICO pushed an alert to its customers and partners warning about “a recent and dramatic increase in skimming fraud perpetrated at a chain of discount supercenters point-of-sale (POS) terminals,” in an around the Washington, D.C. area, including Frederick, Ellicott City and Mt. Airy in Maryland, and in Fredricksburg, Va.

rico-r

“As this fraud activity has appeared and progressed suddenly, it is likely that sites in other cities and other geographic areas will be targeted by organized criminal groups,” the organization cautioned.

EUROPE

Banks in Europe also enjoyed an increase in skimming attacks of all kinds last year. According to statistics shared by the European ATM Security Team (EAST), during 2015 there were 18,738 skimming attacks reported against European ATMs. That’s a 19% increase from the previous year and equates to 51 attacks per 1000 ATMs over the period.

“During 2015 total losses of 327.48 million euros were reported,” EAST wrote. “This is a 17% increase when compared to the total losses of 279.86 million euros reported for 2014 and equates to losses of 884,069 euros per 1000 ATMs over the period.”

EAST’s report further breaks down the skimming activity by specialization. For example, there were at least 2,657 cases in which a thief tried to blow up or otherwise physically force his way into the cash machine. “This total also includes data from solid explosive and explosive gas attacks. This is a 34% increase from 2014 and equates to 7.2 attacks per 1000 ATMs over the period.”

EAST also tracked 15 malware incidents reported against European ATMs in 2015.  All of them were ‘cash out’ or ‘jackpotting’ attacks. According to EAST, this is a 71% decrease from 2014.

Source: EAST

Source: EAST

PROTECT YOURSELF

As I’ve noted in countless skimmer stories here, the simplest way to protect yourself from ATM skimming is to cover your hand when entering your PIN. That’s because most skimmers rely on hidden cameras to steal the victim’s PIN.

Interestingly, a stat in Verizon‘s new Data Breach Investigations Report released this week bears this out: According to Verizon, in over 90 percent of the breaches in the report last year involving skimmers used a tiny hidden camera to steal the PIN.

The Verizon report also offers this advice about ATM safety: Trust your gut. “If you think that something looks odd or out of place, don’t use it. While it is increasingly difficult to find signs of tampering, it is not impossible. If you think a device may have been tampered with, move on to another location, after reporting to the merchant or bank staff.”

For more on ATM skimmers and other skimming devices, check out my series All About Skimmers.

TEDA solar-powered plane, our skewed perception of reality, and a robot that reads you to sleep.

B_Piccard_clickable_blog

Just a few of the intriguing headlines involving members of the TED community this week:

Around the globe without a drop of fuel. What has the wingspan of a Boeing 747 but weighs only about as much as an SUV? The answer is Bertrand Piccard’s Solar Impulse 2, an airplane fueled entirely by solar energy that landed in California on April 24 after a 62-hour flight across the Pacific Ocean from Oahu. The plane’s journey around the world began more than a year ago in Abu Dhabi, but weather stalled them for weeks in China, then overheated batteries grounded them in Hawaii for almost 10 months. If all goes well, Solar Impulse 2 will touch back down in Abu Dhabi at the end of the summer and complete Piccard’s dream of demonstrating the viability of renewable energy technologies. (Watch Bertrand’s TED Talk and learn more on TED’s Ideas blog)

How evolution alters our reality. While we trust that what we see, smell, taste, feel and hear is a true reality, Donald Hoffman, a professor of cognitive neuroscience, argues that what we perceive is our brain’s feeble imitation. In The Atlantic, author Amanda Geftner highlights an interview with partner Quanta Magazine in which Hoffman lays out the secret evolutionary advantage of that: “According to evolution by natural selection, an organism that sees reality as it is will never be more fit than an organism of equal complexity that sees none of reality but is just tuned to fitness. Never.” As we compete for resources, he says, our skewed perceptions enable our survival: “Part of that involves hiding from us the stuff we don’t need to know. And that’s pretty much all of reality, whatever reality might be. If you had to spend all that time figuring it out, the tiger would eat you.” (Watch Donald’s TED Talk)

Abortion protests through the patient’s eyes. Virtual reality enthusiasts often point to the medium’s power for creating empathy between viewer and subject, often inspired by the example of VR journalism pioneer Nonny de la Peña. Her latest documentary Across the Line, a Planned Parenthood collaboration, puts viewers in the scene as a patient seeking reproductive healthcare is harassed by anti-abortion extremists. Premiering at the Sundance Film Festival in January, the documentary has been making the rounds since then with an April 16 exhibition at the Tribeca Film Festival and a International Documentary Association installation and workshop on VR documentary April 22-23. (Watch Nonny’s TED Talk)

TIME honors TEDsters’ contributions. In TIME Magazine’s new list of 100 Most Influential People, two TED speakers and one playlist curator are called out. Christiana Figueres, executive secretary of the UNFCCC, was honored for her outstanding leadership in the global fight against climate change. Robert Redford writes: “Her work shows the world that we can succeed in avoiding catastrophic climate change and become more unified in the process.” Architect Bjarke Ingels was honored for bringing a new energy to his field. “He is the embodiment of a fully fledged new typology, which responds perfectly to the current zeitgeist,” architect Rem Koolhaas wrote. Director Ryan Coogler (Creed, Fruitvale Station) was honored by writer Ta-Nehisi Coates: “On the biggest screen, he confers humanity and beauty on people told they are innately without it.” (Watch Christiana and Bjarke’s TED Talks and download Ryan’s personally curated playlist)

The web of corruption in Brazil. In The Intercept, journalist Glenn Greenwald sheds light on the ongoing attempt to impeach Brazilian president Dilma Rousseff by examining not Rousseff herself more closely, but the man who would replace her, Vice President Michael Temer. Of the impeachment, Greenwald says, “How can anyone rational believe that anti-corruption anger is driving the elite effort to remove Dilma, when they are now installing someone as president who is accused of corruption far more serious than she is?” By examining the chain of political inheritance and media supporters of the impeachment, Greenwald argues that the real reason behind Rousseff’s impeachment is the exact opposite of its pretext–rather than being a move to eradicate corruption, it is actually a maneuver to protect it. (Watch Glenn’s TED Talk)

The “talk” in the Internet Age. Kids are curious about sex, and parents know it — this is nothing new. But, as Adam Savage shares with The Moth, talking to his twin boys about sex and porn in the Internet Age is trickier than he thought: “We didn’t have 24/7 delivery of porn to every device strapped to our bodies.” With frankness and humor, Savage stresses the importance of providing kids with responsible context for some of the more problematic images they will inevitably see. (Watch Adam’s TED Talk)

A world that celebrates all kinds of minds. On April 20, animal behavior expert and autism activist Temple Grandin was elected to the National Academy of Arts and Sciences. Grandin has been a vocal spokesperson for autism awareness and a role model for many individuals on the spectrum. Grandin sees autism not as a limitation to be overcome, but an opportunity for seeing the world in new and innovative ways, a message she shared from the TED stage in 2010. She credits her own experiences with autism and her visual way of thinking for her unique approach to animal handling, an approach that would eventually lead to her induction this week. (Watch Temple’s TED Talk)

Robots are people, too! Okay, maybe not — but they can be still be your friends. Jibo, Cynthia Brezeal’s line of social robots, won Popular Science’s 2016 Invention Award. This friendly, smart assistant adds convenience to a user instead of distracting them from work, family or friends; it can take calls, recognize faces, give alerts, learn your preferences, engage with you, and even read you your favorite bedtime story. (Watch Cynthia’s TED Talk)

Have a news item to share? Write us at contact@ted.com and you may see it included in this weekly round-up.

 


CryptogramDocumenting the Chilling Effects of NSA Surveillance

In Data and Goliath, I talk about the self-censorship that comes along with broad surveillance. This interesting research documents this phenomenon in Wikipedia: "Chilling Effects: Online Surveillance and Wikipedia Use," by Jon Penney, Berkeley Technology Law Journal, 2016.

Abstract: This article discusses the results of the first empirical study providing evidence of regulatory "chilling effects" of Wikipedia users associated with online government surveillance. The study explores how traffic to Wikipedia articles on topics that raise privacy concerns for Wikipedia users decreased after the widespread publicity about NSA/PRISM surveillance revelations in June 2013. Using an interdisciplinary research design, the study tests the hypothesis, based on chilling effects theory, that traffic to privacy-sensitive Wikipedia articles reduced after the mass surveillance revelations. The Article finds not only a statistically significant immediate decline in traffic for these Wikipedia articles after June 2013, but also a change in the overall secular trend in the view count traffic, suggesting not only immediate but also long-term chilling effects resulting from the NSA/PRISM online surveillance revelations. These, and other results from the case study, not only offer compelling evidence for chilling effects associated with online surveillance, but also offer important insights about how we should understand such chilling effects and their scope, including how they interact with other dramatic or significant events (like war and conflict) and their broader implications for privacy, U.S. constitutional litigation, and the health of democratic society. This study is among the first to demonstrate -- using either Wikipedia data or web traffic data more generally­ how government surveillance and similar actions impact online activities, including access to information and knowledge online.

Two news stories.

Worse Than FailureError'd: What Happens in Massachusetts...

"I came across this while submitting to a background check for a new job," writes Jay S., "Good to know that felonious behavior in Boston is perfectly acceptable."

 

"You know, I'm fairly sure that wasn't my friend Christine's last name," wrote Vivia.

 

"Deliveroo seems to think a couple of burgers are not really adding up to the minimum order," writes Joao M..

 

"I got an email stating that it was time to upgrade my work computer and had to fill out an online form stating if I approved or rejected the planned upgrade," writes Angela A., "When I submitted the form, this was the response."

 

"Obviously a problem with one browser means you need a different login link with totally different credentials for a different browser!" wrote Jeffrey.

 

"Home Depot's self checkout is still wary about accepting any credit cards," Jared wrote, "Too bad I left my dynamic linked libraries in my other pants."

 

"This was seen on the high street in Bury St. Edmunds," writes Gary, "Normally, I wouldn't care but it's been stuck like this for a fortnight!"

 

[Advertisement] Scale your release pipelines, creating secure, reliable, reusable deployments with one click. Download and learn more today!

Worse Than FailureError'd: What Happens in Massachusetts...

"I came across this while submitting to a background check for a new job," writes Jay S., "Good to know that felonious behavior in Boston is perfectly acceptable."

 

"You know, I'm fairly sure that wasn't my friend Christine's last name," wrote Vivia.

 

"Deliveroo seems to think a couple of burgers are not really adding up to the minimum order," writes Joao M..

 

"I got an email stating that it was time to upgrade my work computer and had to fill out an online form stating if I approved or rejected the planned upgrade," writes Angela A., "When I submitted the form, this was the response."

 

"Obviously a problem with one browser means you need a different login link with totally different credentials for a different browser!" wrote Jeffrey.

 

"Home Depot's self checkout is still wary about accepting any credit cards," Jared wrote, "Too bad I left my dynamic linked libraries in my other pants."

 

"This was seen on the high street in Bury St. Edmunds," writes Gary, "Normally, I wouldn't care but it's been stuck like this for a fortnight!"

 

[Advertisement] Scale your release pipelines, creating secure, reliable, reusable deployments with one click. Download and learn more today!

Planet DebianSylvain Le Gall: Release of OASIS 0.4.6

I am happy to announce the release of OASIS v0.4.6.

Logo OASIS small

OASIS is a tool to help OCaml developers to integrate configure, build and install systems in their projects. It should help to create standard entry points in the source code build system, allowing external tools to analyse projects easily.

This tool is freely inspired by Cabal which is the same kind of tool for Haskell.

You can find the new release here and the changelog here. More information about OASIS in general on the OASIS website.

The main purpose of this release is to make possible to install OASIS with OPAM with OCaml 4.03.0. In order to do so, I had to disable some tests and use a new set of String.*_ascii functions. The OPAM release is pending upload and should soon be available.

,

Planet DebianMichal Čihař: Weblate 2.6

Going back to faster release cycle, Weblate 2.6 has been just released. There is improved support for Python 3 or brand new HTTP REST API.

Full list of changes for 2.6:

  • Fixed validation of subprojects with language filter.
  • Improved support for XLIFF files.
  • Fixed machine translation for non English sources.
  • Added REST API.
  • Django 1.10 compatibility.
  • Added categories to whiteboard messages.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Filed under: English phpMyAdmin SUSE Weblate | 0 comments

Krebs on SecurityDental Assn Mails Malware to Members

The American Dental Association (ADA) says it may have inadvertently mailed malware-laced USB thumb drives to thousands of dental offices nationwide.

The problem first came to light in a post on the DSL Reports Security Forum. DSLR member “Mike” from Pittsburgh got curious about the integrity of a USB drive that the ADA mailed to members to share updated “dental procedure codes” — codes that dental offices use to track procedures for billing and insurance purposes.

“Oh wow the usually inept ADA just sent me new codes,” Mike wrote. “I bet some marketing genius had this wonderful idea instead of making it downloadable. I can’t wait to plug an unknown USB into my computer that has PHI/HIPAA on it…” [link added].

The ADA says some flash drives mailed to members contained malware.

The ADA says some flash drives mailed to members contained malware. Image: Mike

Sure enough, Mike looked at the code inside one of the files on the flash drive and found it tries to open a Web page that has long been tied to malware distribution. The domain is used by crooks to infect visitors with malware that lets the attackers gain full control of the infected Windows computer.

Reached by KrebsOnSecurity, the ADA said it sent the following email to members who have shared their email address with the organization:

“We have received a handful of reports that malware has been detected on some flash drives included with the 2016 CDT manual,” the ADA said. “The ‘flash drive’ is the credit card sized USB storage device that contains an electronic copy of the CDT 2016 manual. It is located in a pocket on the inside back cover of the manual. Your anti-virus software should detect the malware if it is present. However, if you haven’t used your CDT 2016 flash drive, please throw it away.

To give you access to an electronic version of the 2016 CDT manual, we are offering you the ability to download the PDF version of the 2016 CDT manual that was included on the flash drive.

To download the PDF version of the CDT manual:

1. Click on the link »ebusiness.ada.org/login/ ··· ion.aspx
2. Log in with your ADA.org user ID and password
3. After you log in you will automatically be directed to a page showing CDT 2016 Digital Edition.
4. Click on the “Download” button to save the file to your computer for use.

If you have difficulty accessing or downloading the file, please call 1.800.947.4746 and a Member Service Advisor will be happy to assist you.

Many of the flash drives do not contain the Malware. If you have already used your flash drive and it worked as expected (it displayed a menu linking to chapters of the 2016 CDT manual), you may continue using it.

We apologize if this issue has caused you any inconvenience and thank you for being a valued ADA customer.”

This incident could give new meaning to the term “root canal.” It’s not clear how the ADA could make a statement that anti-virus should detect the malware, since presently only some of the many antivirus tools out there will flag the malware link as malicious.

In response to questions from this author, the ADA said the USB media was manufactured in China by a subcontractor of an ADA vendor, and that some 37,000 of the devices have been distributed. The not-for-profit ADA is the nation’s largest dental association, with more than 159,000 members.

“Upon investigation, the ADA concluded that only a small percentage of the manufactured USB devices were infected,” the organization wrote in an emailed statement. “Of note it is speculated that one of several duplicating machines in use at the manufacturer had become infected during a production run for another customer. That infected machine infected our clean image during one of our three production runs. Our random quality assurance testing did not catch any infected devices. Since this incident, the ADA has begun to review whether to continue to use physical media to distribute products.”

Planet DebianHolger Levsen: Voctomix available in Debian sid

Yesterday evening CarlFK prodded me to package Voctomix, which is a live video mixer written by the Chaos Communication Congress' Video Operation Crew. It's written in Python using GStreamer and was started when they realised dvswitch was not suitable anymore for them. The DebConf16 video team plans to test it in Cape Town (for covering the BoF room), so I figured I'd help now with packaging the software.

Less than 24h after I started, voctomix made it through NEW and is now available in sid and hopefully will be available in stretch soon too! And by DebConf16 it should also finally be available in jessie-backports. Wheeehooo!

Thanks to Stefano Rivera who helped me with some dh_python3 detail and the Debian ftpmasters for letting it though NEW so quickly (and btw, for their general awesome work on NEW processing in the last years too!) - may the winkekatze be with you! ;-)

Sociological ImagesWhy Aren’t There More Women in Politics?

NPR recently aired a story about female lawmaker’s representation state by state. According to the story, Colorado has the most women; female lawmakers make up 42% of that total. Wyoming had the least, with women only representing 13% of state lawmakers.

NPR’s experts suggested that term limits in Colorado and a female-friendly party leadership were behind their high number of female legislators, whereas a change in Wyoming from multi-member to single-member district in the 1990s was unfavorable to women (because voters have to pick only one and tend to lean toward men when they have to make hard choices). The story also mentioned voting rules and the difficulty of balancing home, work, and lawmaking responsibilities.

In fact, sociologists have been studying this issue in depth for some time and a few years ago Deborah Carr summarized the reigning wisdom on why women are less likely to be politicians. She highlighted six factors to explain the gender gap in the US Congress:

  1. Women have to face sexism (e.g., glass ceiling – Nancy Pelosi used the term marble ceiling in her inaugural speech as Speaker in 2007), especially voters’ sex role stereotyping “what women can and should be.”
  1. Women are not in the “pipeline,” suggesting that not enough women are in careers that have historically led to political office.
  1. Because of gendered wealth and income inequality, women don’t as often have enough money to run multi-dollar campaigns, nor access to social networks full of big donors.
  1. Women have different interests, focusing on “issues related to family and social welfare, rather than national defense and international relations.”
  1. Women are less likely to be risk-takers than their male counterparts, perhaps explaining why women must be asked several times before they seriously consider launching campaigns.
  1. Women opt out of politics because of family responsibilities.

4

To improve female participation in politics, we should promote more gender-neural political environments. Political parties should take further steps to recruit and support female candidates, as Colorado seems to be doing. We should repeatedly encourage women to run for office since they take a lot of encouragement before they seriously consider launching candidacies. More importantly, we need to seed the pipeline by encouraging young girls to get involved in student government and see governing as compatible with their interests and abilities.

Sangyoub Park, PhD is a professor of sociology at Washburn University. His research interests include social capital, demographic trends, and post-Generation Y.  

(View original at https://thesocietypages.org/socimages)

CryptogramAmazon Unlimited Fraud

Amazon Unlimited is a all-you-can-read service. You pay one price and can read anything that's in the program. Amazon pays authors out of a fixed pool, on the basis of how many people read their books. More interestingly, it pays by the page. An author makes more money if someone reads his book through to page 200 than if they give up at page 50, and even more if they make it through to the end. This makes sense; it doesn't pay authors for books people download but don't read, or read the first few pages of and then decide not to read the rest.

This payment structure requires surveillance, and the Kindle does watch people as they read. The problem is that the Kindle doesn't know if the reader actually reads the book -- only what page they're on. So Kindle Unlimited records the furthest page the reader synched, and pays based on that.

This opens up the possibility for fraud. If an author can create a thousand-page book and trick the reader into reading page 1,000, he gets paid the maximum. Scam authors are doing this through a variety of tricks.

What's interesting is what while Amazon is definitely concerned about this kind of fraud, it doesn't affect its bottom line. The fixed payment pool doesn't change; just who gets how much of it does.

EDITED TO ADD: John Scalzi comments.

Worse Than FailureCodeSOD: Interned Sort

Caleb scored his first intership at a small, family-owned print-shop. Much to his surprise, the day before he started, their primary web-developer left for a bigger, more lucrative job. His predecssor was an experienced programmer, but came at solving problems in his own unique way. This meant no comments, no functions, no classes, SQL injection vulnerabilities everywere, and cryptic 500-character one-liners stuffed into the value attribute of an input tag.

Caleb spent his first day just trying to get the code running on his dev machine. On the second day, he sat down with a more experienced co-worker to try and understand some of the queries. For example, there was one query that needed to return product details sorted in some meaningful fashion- like by name. Weirdly, though, the page wasn’t sorting them by name, except when it was- no one who used the product search understood the sort order.

Cabel dug in, expecting to see some variation on this:

SELECT `id`, `name`, `description`
FROM products
WHERE `name` LIKE '%{$keyword}% '
ORDER BY `name`;

Instead, he saw this:

# $keyword is a PHP variable that was interpolated into the concatenated PHP string.
# It is the search term the user entered, preserved here for clarity.
SELECT T1.product.id AS product_id, T1.slug, T1.name, description, details, T2.media_id, url, T1.order
FROM (
  SELECT product.id AS product_id, slug, name, description, product.details, text
  CASE
    WHEN name LIKE '$keyword' THEN 0
    WHEN name LIKE '$keyword%' THEN 1
    WHEN name LIKE '%$keyword' THEN 2
    WHEN name LIKE '%$keyword%' THEN 3
    WHEN product.details LIKE '%$keyword%' THEN 4
    WHEN description LIKE '%$keyword% ' THEN 5
    WHEN text LIKE '$keyword' THEN 6
    WHEN text LIKE '$keyword%' THEN 7
    WHEN text LIKE '%$keyword' THEN 8
    WHEN text LIKE '%$keyword%' THEN 9
  ELSE 10 END
  AS `order`
  FROM `product_option`, `product`
  WHERE `product_option`.`product_id` = `product`.`id` AND product.parent_id=0 AND (
    name LIKE '%$keyword%' OR
    description LIKE '%$keyword%' OR
    product.details LIKE '%$keyword%' OR text LIKE '%$keyword%'
  )
  ORDER BY `order`, name
) AS T1 LEFT JOIN (
  SELECT * FROM product_media WHERE product_media.order = 1) AS
  T2 ON T1.product_id=T2.product_id LEFT JOIN media ON T2.media_id=media_id
)

He sat down with a more experienced developer, trying to understand what on Earth this was supposed to do. In the end, they couldn’t figure it out, so they just replaced it with the straightforward ORDER BY name, and left some TODO comments confessing they don’t know what they just replaced.

Caleb’s boss stopped by after they released this change, complimenting him on how much better the product search page worked.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

Worse Than FailureCodeSOD: Interned Sort

Caleb scored his first intership at a small, family-owned print-shop. Much to his surprise, the day before he started, their primary web-developer left for a bigger, more lucrative job. His predecssor was an experienced programmer, but came at solving problems in his own unique way. This meant no comments, no functions, no classes, SQL injection vulnerabilities everywere, and cryptic 500-character one-liners stuffed into the value attribute of an input tag.

Caleb spent his first day just trying to get the code running on his dev machine. On the second day, he sat down with a more experienced co-worker to try and understand some of the queries. For example, there was one query that needed to return product details sorted in some meaningful fashion- like by name. Weirdly, though, the page wasn’t sorting them by name, except when it was- no one who used the product search understood the sort order.

Cabel dug in, expecting to see some variation on this:

SELECT `id`, `name`, `description`
FROM products
WHERE `name` LIKE '%{$keyword}% '
ORDER BY `name`;

Instead, he saw this:

# $keyword is a PHP variable that was interpolated into the concatenated PHP string.
# It is the search term the user entered, preserved here for clarity.
SELECT T1.product.id AS product_id, T1.slug, T1.name, description, details, T2.media_id, url, T1.order
FROM (
  SELECT product.id AS product_id, slug, name, description, product.details, text
  CASE
    WHEN name LIKE '$keyword' THEN 0
    WHEN name LIKE '$keyword%' THEN 1
    WHEN name LIKE '%$keyword' THEN 2
    WHEN name LIKE '%$keyword%' THEN 3
    WHEN product.details LIKE '%$keyword%' THEN 4
    WHEN description LIKE '%$keyword% ' THEN 5
    WHEN text LIKE '$keyword' THEN 6
    WHEN text LIKE '$keyword%' THEN 7
    WHEN text LIKE '%$keyword' THEN 8
    WHEN text LIKE '%$keyword%' THEN 9
  ELSE 10 END
  AS `order`
  FROM `product_option`, `product`
  WHERE `product_option`.`product_id` = `product`.`id` AND product.parent_id=0 AND (
    name LIKE '%$keyword%' OR
    description LIKE '%$keyword%' OR
    product.details LIKE '%$keyword%' OR text LIKE '%$keyword%'
  )
  ORDER BY `order`, name
) AS T1 LEFT JOIN (
  SELECT * FROM product_media WHERE product_media.order = 1) AS
  T2 ON T1.product_id=T2.product_id LEFT JOIN media ON T2.media_id=media_id
)

He sat down with a more experienced developer, trying to understand what on Earth this was supposed to do. In the end, they couldn’t figure it out, so they just replaced it with the straightforward ORDER BY name, and left some TODO comments confessing they don’t know what they just replaced.

Caleb’s boss stopped by after they released this change, complimenting him on how much better the product search page worked.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

Planet DebianDirk Eddelbuettel: RcppRedis 0.1.7

A new release of RcppRedis arrived on CRAN today. And just like for the previous release, Russell Pierce contributed a lot of changes via several pull requests which make for more robust operations. In addition, we have started to add support for MessagePack by relying on our recently-announced RcppMsgPack package.

Changes in version 0.1.7 (2016-04-27)

  • Added support for timeout constructor argument (PR #14 by Russell Pierce)

  • Added new commands exists, ltrim, expire and pexpire along with unit tests (PR #16 by Russell Pierce)

  • Return NULL for empty keys in serialized get for consistency with lpop and rpop (also PR #16 by Russell Pierce)

  • Minor corrections to get code and hget and hset documentation (also PR #16 by Russell Pierce)

  • Error conditions are now properly forwarded as R errors (PR #22 by Russell Pierce)

  • Results from Redis commands are now checked for NULL (PR #23 by Russell Pierce)

  • MessagePack encoding can now be used which requires MessagePackage headers of version 1.0 or later; the (optional) RcppMsgPack package can be used.

Courtesy of CRANberries, there is also a diffstat report for the most recent release. More information is on the RcppRedis page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Kelvin Thomson60 Minutes in Lebanon

It is doubtful that Channel 9 really needs to conduct an investigation into how its crew came to be arrested in Lebanon. It was right in the middle of this story, in it up to their eyeballs, and they no doubt already know exactly what happened.

The "review" is not independent. The people doing the review are all connected with Channel 9. If there was a major bungle by a Government Department or a Bank, and the Department or the Bank simply conducted an internal review, 60 Minutes would scream "cover-up". The "review" may be simply an attempt to buy time and hope the public loses interest in this debacle.

There is evidence the Channel 9 network paid $69,000 directly to the personal company of Adam Whittington, the imprisoned head of Child Abduction Recovery International. News Corporation has reported that Channel 9 made two separate payments in this case totalling more than $115,000.

Given this, rather than buying time, Channel 9 should do three things. First it should do would it would demand of anyone else in a comparable situation – provide a full accounting to the public of exactly what it did, what money it has paid or promised and to whom, and which of its personnel decided on or approved the actions it carried out.

Second, it should change its ways in relation to chequebook journalism. Chequebook journalism is a slippery slope where media outlets risk losing their moral compass. You can end up like the UK paper "The News of the World" did in 2011, caught paying bribes to police officers to reveal information about cases.

If there's nothing wrong with chequebook journalism, let TV stations always reveal when they do a story and they have paid someone for their role in it, who they have paid, and the amount. Let's have the full story. Many stories are presented as being justified in the pursuit of openness and transparency and the public's right to know. It is therefore hypocritical for those paying for the stories to shy away from saying how much was paid, to whom, what it was paid for, and why it was paid.
 
Third, it should resolve never to pay money in order to create news, and certainly not to facilitate the commission of a crime. Journalists should report the story, not be the story. If you pay the Beaconsfield miners for their story after they are rescued that is one thing – it should be disclosed – but paying money to set up a story is another thing altogether. It runs counter to the ideals of journalism which should have ethical principles and truth telling at its core.

,

Planet DebianMike Hommey: Announcing git-cinnabar 0.3.2

Git-cinnabar is a git remote helper to interact with mercurial repositories. It allows to clone, pull and push from/to mercurial remote repositories, using git.

Get it on github.

These release notes are also available on the git-cinnabar wiki.

This is mostly a bug and regression-fixing release.

What’s new since 0.3.1?

  • Fixed a performance regression when cloning big repositories on OSX.
  • git configuration items with line breaks are now supported.
  • Fixed a number of issues with corner cases in mercurial data (such as, but not limited to nodes with no first parent, malformed .hgtags, etc.)
  • Fixed a stack overflow, a buffer overflow and a use-after free in cinnabar-helper.
  • Better work with git worktrees, or when called from subdirectories.
  • Updated git to 2.7.4 for cinnabar-helper.
  • Properly remove all refs meant to be removed when using git version lower than 2.1.

Planet DebianStig Sandbeck Mathisen: Using LLDP on Linux. What's on the other side?

On any given server, or workstation, knowing what is at the other end of the network cable is often very useful.

There’s a protocol for that: LLDP. This is a link layer protocol, so it is not routed. Each end transmits information about itself periodically.

You can typically see the type of equipment, the server or switch name, and the network port name of the other end, although there are lots of other bits of information available, too.

This is often used between switches and routers in a server centre, but it is useful to enable on server hardware as well.

There are a few different packages available. I’ve looked at a few of them available for the RedHat OS family (Red Hat Enterprise Linux, CentOS, …) as well as the Debian OS family (Debian, Ubuntu, …)

(Updated 2016-04-29, added more recent information about lldpd, and gathered the switch output at the end.)

ladvd

A simple daemon, with no configuration needed. This runs as a privilege-separated daemon, and has a command line control utility. You invoke it with a list of interfaces as command line arguments to restrict the interfaces it should use.

“ladvd” is not available on RedHat, but is available on Debian.

Install the “ladvd” package, and run “ladvdc” to query the daemon for information.

root@turbotape:~# ladvdc
Capability Codes:
    r - Repeater, B - Bridge, H - Host, R - Router, S - Switch,
    W - WLAN Access Point, C - DOCSIS Device, T - Telephone, O - Other

Device ID        Local Intf Proto Hold-time Capability Port ID
office1-switch23 eno1       LLDP  98        B          42

Even better, it has output that can be parsed for scripting:

root@turbotape:~# ladvdc -b eno1
INTERFACE_0='eno1'
HOSTNAME_0='office1-switch23'
PORTNAME_0='42'
PORTDESCR_0='42'
PROTOCOL_0='LLDP'
ADDR_INET4_0=''
ADDR_INET6_0=''
ADDR_802_0='00:11:22:33:44:55
VLAN_ID_0=''
CAPABILITIES_0='B'
TTL_0='120'
HOLDTIME_0='103'

…my new favourite :)

lldpd

Another package is “lldpd”, which is also simple to configure and use.

“lldpd” is not available on RedHat, but it is present on Debian.

It features a command line interface, “lldpcli”, which can show output with different level of detail, and on different formats, as well as configure the running daemon.

root@turbotape:~# lldpcli show neighbors
-------------------------------------------------------------------------------
LLDP neighbors:
-------------------------------------------------------------------------------
Interface:    eno1, via: LLDP, RID: 1, Time: 0 day, 00:00:59
  Chassis:
    ChassisID:    mac 00:11:22:33:44:55
    SysName:      office1-switch23
    SysDescr:     ProCurve J9280A Switch 2510G-48, revision Y.11.12, ROM N.10.02 (/sw/code/build/cod(cod11))
    Capability:   Bridge, on
  Port:
    PortID:       local 42
    PortDescr:    42
-------------------------------------------------------------------------------

Among the output formats are “json”, which is easy to re-use elsewhere.

root@turbotape:~# lldpcli -f json show neighbors
{
  "lldp": {
    "interface": {
      "eno1": {
        "chassis": {
          "office1-switch23": {
            "descr": "ProCurve J9280A Switch 2510G-48, revision Y.11.12, ROM N.10.02 (/sw/code/build/cod(cod11))",
            "id": {
              "type": "mac",
              "value": "00:11:22:33:44:55"
            },
            "capability": {
              "type": "Bridge",
              "enabled": true
            }
          }
        },
        "via": "LLDP",
        "rid": "1",
        "age": "0 day, 00:53:23",
        "port": {
          "descr": "42",
          "id": {
            "type": "local",
            "value": "42"
          }
        }
      }
    }
  }
}

lldpad

A much more featureful LLDP daemon, available for both the Debian and RedHat OS families. This has lots of features, but is less trivial to set up.

Configure lldp for each interface

#!/bin/sh

find /sys/class/net/ -maxdepth 1 -name 'en*' |
    while read device; do
        basename "$device"
    done |
    while read interface; do
        {
            lldptool set-lldp -i "$interface" adminStatus=rxtx
            for item in sysName portDesc sysDesc sysCap mngAddr; do
                lldptool set-tlv -i "$interface" -V "$item" enableTx=yes |
                    sed -e "s/^/$item /"
            done
        } |
            sed -e "s/^/$interface /"
    done

Show LLDP neighbor information

#!/bin/sh

find /sys/class/net/ -maxdepth 1 -name 'en*' |
    while read device; do
        basename "$device"
    done |
    while read interface; do
        printf "%s\n" "$interface"
        ethtool $interface | grep -q 'Link detected: yes' || {
            echo "  down"
            echo
            continue
        }
        lldptool get-tlv -n -i "$interface" | sed -e "s/^/  /"
        echo
    done
[...]
enp3s0f0
  Chassis ID TLV
    MAC: 01:23:45:67:89:ab
  Port ID TLV
    Local: 588
  Time to Live TLV
    120
  System Name TLV
    site3-row2-rack1
  System Description TLV
    Juniper Networks, Inc. ex2200-48t-4g , version 12.3R12.4 Build date: 2016-01-20 05:03:06 UTC
  System Capabilities TLV
    System capabilities:  Bridge, Router
    Enabled capabilities: Bridge, Router
  Management Address TLV
    IPv4: 10.21.0.40
    Ifindex: 36
    OID: $
  Port Description TLV
    some important server, port 4
  MAC/PHY Configuration Status TLV
    Auto-negotiation supported and enabled
    PMD auto-negotiation capabilities: 0x0001
    MAU type: Unknown [0x0000]
  Link Aggregation TLV
    Aggregation capable
    Currently aggregated
    Aggregated Port ID: 600
  Maximum Frame Size TLV
    9216
  Port VLAN ID TLV
    PVID: 2000
  VLAN Name TLV
    VID 2000: Name bumblebee
  VLAN Name TLV
    VID 2001: Name stumblebee
  VLAN Name TLV
    VID 2002: Name fumblebee
  LLDP-MED Capabilities TLV
    Device Type:  netcon
    Capabilities: LLDP-MED, Network Policy, Location Identification, Extended Power via MDI-PSE
  End of LLDPDU TLV

enp3s0f1
[...]

on the switch side

On the switch, it is a bit easier to see what’s connected to each interface:

office switch

On the switch side, this system looks like:

office1-switch23# show lldp info remote-device

 LLDP Remote Devices Information

  LocalPort | ChassisId                 PortId PortDescr SysName
  --------- + ------------------------- ------ --------- ----------------------
  [...]
  42        | 22 33 44 55 66 77         eno1   Intel ... turbotape.example.com
  [...]

office1-switch23# show lldp info remote-device 42

 LLDP Remote Device Information Detail

  Local Port   : 42
  ChassisType  : mac-address
  ChassisId    : 00 11 22 33 33 55
  PortType     : interface-name
  PortId       : eno1
  SysName      : turbotape.example.com
  System Descr : Debian GNU/Linux testing (stretch) Linux 4.5.0-1-amd64 #1...
  PortDescr    : Intel Corporation Ethernet Connection I217-LM

  System Capabilities Supported  : bridge, router
  System Capabilities Enabled    : bridge, router

  Remote Management Address
     Type    : ipv4
     Address : 192.0.2.93
     Type    : ipv6
     Address : 20 01 0d b8 00 00 00 00 00 00 00 00 00 00 00 01
     Type    : all802
     Address : 22 33 44 55 66 77

datacenter switch

ssm@site3-row2-rack1> show lldp neighbors
Local Interface    Parent Interface    Chassis Id          Port info          System Name
[...]
ge-0/0/38.0        ae1.0               01:23:45:67:89:58   Interface   2 as enp3s0f0 server.example.com
ge-1/0/38.0        ae1.0               01:23:45:67:89:58   Interface   3 as enp3s0f1 server.example.com
[...]

ssm@site3-row2-rack1> show lldp neighbors interface ge-0/0/38
LLDP Neighbor Information:
Local Information:
Index: 157 Time to live: 120 Time mark: Fri Apr 29 13:00:19 2016 Age: 24 secs
Local Interface    : ge-0/0/38.0
Parent Interface   : ae1.0
Local Port ID      : 588
Ageout Count       : 0

Neighbour Information:
Chassis type       : Mac address
Chassis ID         : 01:23:45:67:89:58
Port type          : Mac address
Port ID            : 01:23:45:67:89:58
Port description   : Interface   2 as enp3s0f0
System name        : server.example.com

System Description : Linux server.example.com 3.10.0-327.13.1.el7.x86_64 #1 SMP Thu Mar 4

System capabilities
        Supported  : Station Only
        Enabled    : Station Only

Management Info
        Type              : IPv6
        Address           : 2001:0db8:0000:0000:0000:dead:beef:cafe
        Port ID           : 2
        Subtype           : 2
        Interface Subtype : ifIndex(2)
        OID               : 1.3.6.1.2.1.31.1.1.1.1.2


Planet DebianJoey Hess: my Shuttleworth Foundation flash grant

Six months ago I received a small grant from the Shuttleworth Foundation with no strings attached other than I should write this blog post about it. That was a nice surprise.

The main thing that ended up being supported by the grant was work on Propellor, my configuration management system that is configured by writing Haskell code. I made 11 releases of Propellor in the grant period, with some improvements from me, and lots more from other contributors. The biggest feature that I added to Propellor was LetsEncrypt support.

More important than features is making Propellor prevent more classes of mistakes, by creative use of the type system. The biggest improvement in this area was type checking the OSes of Propellor properties, so Propellor can reject host configurations that combine eg, Linux-only and FreeBSD-only properties.

Turns out that the same groundwork needed for that is also what's needed to get Propellor to do type-level port conflict detection. I have a branch underway that does that, although it's not quite done yet.

The grant also funded some of my work on git-annex. My main funding for git-annex doesn't cover development of the git-annex assistant, so the grant filled in that gap, particularly in updating the assistant to support the git-annex v6 repo format.

I've very happy to have received this grant, and with the things it enabled me to work on.

Google AdsenseMeet AdSense’s next generation ads: Page-level ads

Today, we’re excited to introduce AdSense’s next generation ads. Page-level ads is a family of ad formats that offer a new and innovative way to help you earn money from your mobile content.

With Page-level ads, you place the same ad code once on each page that you want to show ads. They’re designed to show automatically at the right time and in the right format -  potentially increasing your earnings without interfering with your users’ mobile experience.

The benefits of Page-level ads include:


  • Optimized ads that show when they’re likely to perform well and provide a good user experience.

  • One-time set-up that only requires you to place the Page-level ad code once on each page you’d like the ads shown. 

  • Ability to adjust the settings in your account and to enable new ad formats without having to change the code on your site.

  • Additional ads on your site that don’t count towards your AdSense per page ad limit.


Watch the video below for a quick view of Page-level ads:


Currently, the Page-level ads family includes the anchor/overlay and vignette ad formats. More ad formats will be available soon.

Learn more about the Page-level ads in the AdSense Help Center. Get started with Page-level ads today and let us know what you think in the comments below.


Posted by Matthew Conroy, AdSense Product Manager

Planet DebianNiels Thykier: auto-decrufter in top 5 after 10 months

About 10 months ago, we enabled an auto-decrufter in dak.  Then after 3 months it had become the top 11th “remover”.  Today, there are only 3 humans left that have removed more packages than the auto-decrufter… impressively enough, one of them is not even an active FTP-master (anymore).  The current score board:

 5371 Luca Falavigna
 5121 Alexander Reichle-Schmehl
 4401 Ansgar Burchardt
 3928 DAK's auto-decrufter
 3257 Scott Kitterman
 2225 Joerg Jaspert
 1983 James Troup
 1793 Torsten Werner
 1025 Jeroen van Wolffelaar
  763 Ryan Murray

For comparison, here is the number removals by year for the past 6 years:

 5103 2011
 2765 2012
 3342 2013
 3394 2014
 3766 2015  (1842 removed by auto-decrufter)
 2845 2016  (2086 removed by auto-decrufter)

Which tells us that in 2015, the FTP masters and the decrufter performed on average over 10 removals a day.  And by the looks of it, 2016 will surpass that.  Of course, the auto-decrufter has a tendency to increase the number of removed items since it is an advocate of “remove early, remove often!”.:)

 

Data is from https://ftp-master.debian.org/removals-full.txt.  Scoreboard computed as:

  grep ftpmaster: removals-full.txt | \
   perl -pe 's/.*ftpmaster:\s+//; s/\]$//;' | \
   sort | uniq -c | sort --numeric --reverse | head -n10

Removals by year computed as:

 grep ftpmaster: removals-full.txt | \
   perl -pe 's/.* (\d{4}) \d{2}:\d{2}:\d{2}.*/$1/' | uniq -c | tail -n6

(yes, both could be done with fewer commands)


Filed under: Debian

Sociological ImagesWhen Did It Become Allowable to be Pregnant in Public?

Pregnancy wasn’t always something women did in public. In her new book, Pregnant with the Stars, Renée Ann Cramer puts public pregnancies under the sociological microscope, but she notes that it is only recently that being publicly pregnant became socially acceptable. Even as recently as the 1950s, pregnancy was supposed to be a private matter, hidden behind closed doors. That big round belly was, she argues, “an indicator that sex had taken place, [which] was simply considered too risqué for polite company.”

Lucille Ball was the first person on television to acknowledge a pregnancy, real or fictional. It was 1952, but it was considered lewd to actually say the word “pregnant,” so the episode used euphemisms like “blessed event” or simply referred to having a baby or becoming a father.

Almost 20 years later, in 1970, a junior high school teacher was forced out of the classroom in her third trimester on the argument that her visible pregnancy would, as Cramer puts it, “alternately disgust, concern, fascinate, and embarrass her students.” So, when Demi Moore posed naked and pregnant on the cover of Vanity Fair just 21 years after that, it was a truly groundbreaking thing to do.

2

Today being pregnant is public is unremarkable. Visibly pregnant women are free to run errands, go to restaurants, attend events, even dress up their “baby bump” to try to (make it) look cute. All of this is part of the entrance of women into the public sphere more generally and the pressing of men to accept female bodies in those spaces. The next frontier may be breast feeding, an activity related to female-embodied parenting that many still want to relegate to behind closed doors. We may look back in 20 years and be as surprised by intolerance of breastfeeding as we are today over the idea that pregnant women weren’t supposed to leave the house. Time will tell.

Lisa Wade is a professor at Occidental College and the co-author of Gender: Ideas, Interactions, Institutions. Find her on TwitterFacebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

CryptogramTwo Good Readings on the Encryption "Going Dark" Debate

Testimonies of Matt Blaze and Danny Weitzner, both on April 19th before the House Energy and Commerce Committee. And the hearing.

Worse Than FailureTwisted Branches

David pulled his headphones off when he heard a loud harrumph behind him. One of his project managers loomed in the doorway, and had obviously been standing there for some time, trying to get David’s attention.

“You pulled from Staging-Core branch into the Version–2 branch and broke Liam’s changes,” the PM said.

David wracked his brain, trying to remember this particular PM’s name. He’d met so many during his short time at this company that he couldn’t keep them straight. “Uhh… we just had a release, right? Since I was working in Version–2, I pulled the latest version in. I thought I was the only one in there on this project right now…”

The PM shook his head. “I don’t know how you did things at your old job, but here we’ve got policies and procedures. This isn’t some tiny startup- we’ve got a dozen developers in this office, fifty more in India, another dozen in Mexico, and nearly a hundred testers. I’ll send you a meeting invite so we can discuss…”

The PM wandered off, mid-sentence, but was instantly replaced by another. This one David recognized- Lisa or Linda or Lindsey or something. “Liam tells me you’re breaking branches in our code control system,” she said. “I’m not sure how things worked at your old job, but this isn’t some tiny startup. Our codebase is over 1.2 million lines of code. I’ll send you a meeting invite so we can discuss…”

A visual report of the branching structure of the codebase in question. It's over twenty branches, with relationships that make no sense at all, like having Version 2 be a child of the Version 1 branch, and two different staging branches- one the parent of Production, and another the Child

Over the next quarter of an hour, David was visited by the ghosts of project management past, present and future, each with dire comments about David’s ability to follow policy and the promise of a follow-up meeting. By mid-afternoon, his calendar for the next week was filled with meetings.

A normal organization might do work in individual feature branches and when a feature was complete, they’d migrate those changes back to the root or trunk branch for release. People working in other feature branches should, at some point, pull those changes down into those branches. When and exactly how that happened was a possible topic of debate, but David’s new workplace wasn’t interested in a debate.

“You simply don’t pull changes down into lower branches. Ever.”

David sat at a conference table, surrounded by project managers he didn’t recognize.

“That’s why we have a Changes Under Merge team,” one of the PMs explained. “It’s for safety. Changes made by developers stay in one branch, and the Changes Under Merge team move them back up the tree, following the Change and Release Approach Policy.”

If David needed changes synced between branches, he needed to work with the Changes Under Merge team, which was a four-person group who understood how the 20+ branches in their codebase related to each other. If, for example, David needed a feature merged in, he’d need to look up all of the changesets that were used to create that feature. He’d then send them- as an Excel spreadsheet- to the Changes Under Merge team. Someone from that team would then cherry-pick just those changes and use TFS’s “baseless merge” tool to pull those changes into his branch.

Conversely, when he had a feature ready to go, he wouldn’t merge it up the tree. He’d compile a list of changesets and pass them off to the Changes Under Merge team, which would then pull the changes from his branch into Integ-Core, for integration testing by QA (there were no automated tests, because, as one of the PMs put it, “Automated tests only test your automation. You need real humans for real tests.”). Once the feature passed Integ-Core, the QA folks would request that the changes get merged into Strategic-CombRes, using the same cherry-pick and baseless merge approach. From there, the “finished” feature entered a twisty labyrinth of project management processes that David didn’t need to worry about. His feature would hop around the various Strategic Branches for awhile, and someday, maybe, make it back up into the Production branch.

David took his scoldings, did his best not to roll his eyes, and got back to work when the project management machine decided that he’d been punished enough. For the next six months, he basically ignored everything he knew about branches, and just rigorously tracked his changesets so that he could pass them off to the Changes Under Merge team. Eventually, he started to hear whispers- a major release was coming, and everyone’d need to put in some extra time to make sure it went off without a hitch.

Lisa/Linda/Lindsey descended on his cube on the eve of the release. “Now, since this is your first major milestone,” she said, “I wanted to stop by and review what’s going to happen. You’re going to get a list of changesets from the Changes Under Merge team. You need to look at the latest version of the Production branch and verify that all of your changes are in there. And no, you can’t just look and see if the changeset IDs are in there, because sometimes changes get overwritten by later changes. You’ll need to manually verify every line you’ve written. You’ve got two hours.”

This was six months worth of work. David didn’t even know where to begin with “verifying every line” he’d written. And in only two hours? It was impossible. Instead, he focused on spot-checking the release while wishing he’d been allowed to write some automated tests around his features. So far as he could tell, everything looked okay, so he signed off on the release.

The release crashed and burned. Since the company didn’t have a backout plan, and since their branching structure was convoluted, they couldn’t simply pull the previous version out of source control and redeploy it. Instead, every developer pulled the latest version of the Production branch and worked until 2AM trying to patch all of the problems in it.

There were a lot, and when the blamestorming session finally came around, several project managers were pointing their fingers at him. “You said all of your changes were in the final product, but it looks like Liam’s changes overwrote some of your changes.” “We need people that are going to follow policy.” “We need people who show real diligence.” “We don’t think this organization is the right fit for you.”

It wasn’t until he was 45 minutes into the meeting that David realized he was being fired. He wasn’t entirely certain that all of the PMs realized it either, because a few of them kept the meeting running. “Obviously,” one of them said, “while David was at fault, this problem is also a management problem. We need to expand the management team so that we can avoid these problems in the future. And since our headcount just shrank…”

[Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!

Worse Than FailureTwisted Branches

David pulled his headphones off when he heard a loud harrumph behind him. One of his project managers loomed in the doorway, and had obviously been standing there for some time, trying to get David’s attention.

“You pulled from Staging-Core branch into the Version–2 branch and broke Liam’s changes,” the PM said.

David wracked his brain, trying to remember this particular PM’s name. He’d met so many during his short time at this company that he couldn’t keep them straight. “Uhh… we just had a release, right? Since I was working in Version–2, I pulled the latest version in. I thought I was the only one in there on this project right now…”

The PM shook his head. “I don’t know how you did things at your old job, but here we’ve got policies and procedures. This isn’t some tiny startup- we’ve got a dozen developers in this office, fifty more in India, another dozen in Mexico, and nearly a hundred testers. I’ll send you a meeting invite so we can discuss…”

The PM wandered off, mid-sentence, but was instantly replaced by another. This one David recognized- Lisa or Linda or Lindsey or something. “Liam tells me you’re breaking branches in our code control system,” she said. “I’m not sure how things worked at your old job, but this isn’t some tiny startup. Our codebase is over 1.2 million lines of code. I’ll send you a meeting invite so we can discuss…”

A visual report of the branching structure of the codebase in question. It's over twenty branches, with relationships that make no sense at all, like having Version 2 be a child of the Version 1 branch, and two different staging branches- one the parent of Production, and another the Child

Over the next quarter of an hour, David was visited by the ghosts of project management past, present and future, each with dire comments about David’s ability to follow policy and the promise of a follow-up meeting. By mid-afternoon, his calendar for the next week was filled with meetings.

A normal organization might do work in individual feature branches and when a feature was complete, they’d migrate those changes back to the root or trunk branch for release. People working in other feature branches should, at some point, pull those changes down into those branches. When and exactly how that happened was a possible topic of debate, but David’s new workplace wasn’t interested in a debate.

“You simply don’t pull changes down into lower branches. Ever.”

David sat at a conference table, surrounded by project managers he didn’t recognize.

“That’s why we have a Changes Under Merge team,” one of the PMs explained. “It’s for safety. Changes made by developers stay in one branch, and the Changes Under Merge team move them back up the tree, following the Change and Release Approach Policy.”

If David needed changes synced between branches, he needed to work with the Changes Under Merge team, which was a four-person group who understood how the 20+ branches in their codebase related to each other. If, for example, David needed a feature merged in, he’d need to look up all of the changesets that were used to create that feature. He’d then send them- as an Excel spreadsheet- to the Changes Under Merge team. Someone from that team would then cherry-pick just those changes and use TFS’s “baseless merge” tool to pull those changes into his branch.

Conversely, when he had a feature ready to go, he wouldn’t merge it up the tree. He’d compile a list of changesets and pass them off to the Changes Under Merge team, which would then pull the changes from his branch into Integ-Core, for integration testing by QA (there were no automated tests, because, as one of the PMs put it, “Automated tests only test your automation. You need real humans for real tests.”). Once the feature passed Integ-Core, the QA folks would request that the changes get merged into Strategic-CombRes, using the same cherry-pick and baseless merge approach. From there, the “finished” feature entered a twisty labyrinth of project management processes that David didn’t need to worry about. His feature would hop around the various Strategic Branches for awhile, and someday, maybe, make it back up into the Production branch.

David took his scoldings, did his best not to roll his eyes, and got back to work when the project management machine decided that he’d been punished enough. For the next six months, he basically ignored everything he knew about branches, and just rigorously tracked his changesets so that he could pass them off to the Changes Under Merge team. Eventually, he started to hear whispers- a major release was coming, and everyone’d need to put in some extra time to make sure it went off without a hitch.

Lisa/Linda/Lindsey descended on his cube on the eve of the release. “Now, since this is your first major milestone,” she said, “I wanted to stop by and review what’s going to happen. You’re going to get a list of changesets from the Changes Under Merge team. You need to look at the latest version of the Production branch and verify that all of your changes are in there. And no, you can’t just look and see if the changeset IDs are in there, because sometimes changes get overwritten by later changes. You’ll need to manually verify every line you’ve written. You’ve got two hours.”

This was six months worth of work. David didn’t even know where to begin with “verifying every line” he’d written. And in only two hours? It was impossible. Instead, he focused on spot-checking the release while wishing he’d been allowed to write some automated tests around his features. So far as he could tell, everything looked okay, so he signed off on the release.

The release crashed and burned. Since the company didn’t have a backout plan, and since their branching structure was convoluted, they couldn’t simply pull the previous version out of source control and redeploy it. Instead, every developer pulled the latest version of the Production branch and worked until 2AM trying to patch all of the problems in it.

There were a lot, and when the blamestorming session finally came around, several project managers were pointing their fingers at him. “You said all of your changes were in the final product, but it looks like Liam’s changes overwrote some of your changes.” “We need people that are going to follow policy.” “We need people who show real diligence.” “We don’t think this organization is the right fit for you.”

It wasn’t until he was 45 minutes into the meeting that David realized he was being fired. He wasn’t entirely certain that all of the PMs realized it either, because a few of them kept the meeting running. “Obviously,” one of them said, “while David was at fault, this problem is also a management problem. We need to expand the management team so that we can avoid these problems in the future. And since our headcount just shrank…”

[Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!

Planet DebianMichal Čihař: motranslator 1.0

After two months since it's announcement I think it's good time to release 1.0 version of motranslator. This release doesn't bring any major changes, it's more to indicate that the library is stable :-).

The motranslator is a translation library used in current phpMyAdmin master (upcoming 4.7.0) with focus on speed and memory usage. It uses Gettext MO files to load the translations. It also comes with testsuite (100% coverage) and basic documentation.

Recommended way to install it is using composer from Packagist repository:

composer require phpmyadmin/motranslator

The Debian package will be available probably at point phpMyAdmin 4.7.0 will be out, but if you see need to have it earlier, just let me know.

Filed under: English phpMyAdmin | 1 comments

,

TEDMeet our first class of TED Residents

TEDResidents_Blog

An idea worth spreading doesn’t just magically appear out of thin air. Instead, it needs a long incubation period, a sometimes frustrating — and often exciting — trial and error of creation, failure and innovation.

On April 18, TED welcomed its first-ever class of the TED Residency program, an in-house community of 28 bright minds who are tackling ambitious projects and making meaningful change.

This group of thinkers will spend the next four months in a collaborative space, learning with and from each other on ideas that address …

  • How to explain complex scientific concepts
  • The personal stories of migrants
  • Violence prevention in at-risk communities
  • How to make the most of personal connections in a tech-heavy world
  • The history of the Internet
  • Inclusion in the fashion world
  • Building the digital Disney of Africa
  • Frictionless housing for a mobile society

… among many other fascinating subjects

At the end of the session, the residents will give a TED Talk about their ideas in the TED office theater. Read more about each resident below:

Daniel Ahmadizadeh is working with artificial intelligence to revolutionize how consumers are informed and make choices. He co-founded Riley, a chatbot concierge service for the real-estate business.

Piper Anderson is a writer and creative strategist who has spent the past 15 years working to end mass criminalization and incarceration. She recently launched the National Mass Story Campaign, which will host participatory storytelling events in 20 cities to catalyze more restorative and transformative approaches to justice.

Isabel Behncke is an Oxford field primatologist from Chile who is working on the evolutionary roots of social behavior in humans and other animals. She is creating a show on the science of joy that blurs boundaries among theater, poetry and cutting-edge science.

Susan Bird is working on a podcast about the art and skills of conversation. 

Artist and traveler Reggie Black started Sticky Inspiration as an online project designed to motivate others through thought-provoking quotes distributed daily on Post-Its left in public spaces. Now he’s ready to expand offline.

Sashko Danylenko is a Ukraine-based filmmaker whose animated films explore wonder and curiosity. Currently, he’s working on a film that documents cities around the world through by focusing on their bicyclists. 

Tanya Dwyer is an attorney and social entrepreneur in Brooklyn who works to promote inclusive capitalism and economic justice. She wants to help establish a living-wage business park in Crown Heights that is cooperatively owned by neighborhood residents and stakeholders.

Laura Anne Edwards wants to create what she’s calling DATA OASIS, a dynamic index of valuable data sets, many of which are taxpayerfunded and technically “open” but in practice, extremely difficult to locate and access.

Rob Gore, an academic emergency medicine physician based in Brooklyn, leads KAVI (Kings Against Violence Initiative), a youth empowerment and violence prevention program that has been running for the past five years. He is working to transform health care in marginalized populations.

Che Grayson is a filmmaker and comic book creator whose multimedia project Rigamo, a comic series and short film about a young girl whose tears bring people back to life, helped her overcome her grief at the death of a beloved aunt. She wants to explore using these forms of storytelling to tackle other tough subjects, heal, and inspire.

Bethany Halbreich runs Paint the World, an organization that wants to make opportunities for creativity ubiquitous. Paint the World facilitates public art projects in underserved communities; the resulting pieces are sold, the profits fund more kits and supplies for areas in need.

Sarah Hinawi is the co-founder and director of Purpl, a small-business incubator that focuses on the person rather than the business. Her latest work examines what leadership training looks like in the gig economy.

Designer and writer DK Holland has developed a free after-school program for Grades 3 to 5 in which kids take the lead. She has found that giving kids the opportunity to participate in their micro-democracy motivates even the quietest, most challenged child to express natural generosity, inquisitiveness, individualism and a sense of fairness. She now wants to offer her toolkit to other public schools across the US. 

Liz Jackson is the founder and chief advocacy officer for the Inclusive Fashion & Design Collective, the first fashion trade association for businesses and designers serving the needs of people with disabilities. Her mission is to introduce the world to inclusive design.

Ayana Elizabeth Johnson is a marine biologist and policy expert who advocates zoning the ocean as we do land, so we can use the sea without using it up. As executive director of the Waitt Institute, she led the Caribbean’s first successful island-wide ocean zoning project, resulting in one third of Barbuda’s coastal waters being protected, and went on to launch similar initiatives on other islands. 

Jonathan Kalan and Michael Youngblood want to redefine the notion of home and its relation to work. Aimed at millennials who care more about mobility than about owning real estate, their “global lease” aims to let subscribers stay “location-independent.”

Brian McCullough is the creator of the Internet History Podcast, an oral history of the internet; he’s now telling the stories of Web 2.0. 

Christia Mercer is a full-time Columbia philosophy professor and part-time activist. She plans to examine radically different answers to hard questions that people have given throughout history and across cultures and then to show their relevance to modern thinking.

Ted Myerson is a co-founder of Anonos, a Big Privacy technology company that enables data to be more readily collected, shared, and combined, potentially enabling breakthroughs in personalized and precision medicine. 

As a tap dancer, Andrew Nemr has lived the oral tradition of American Vernacular Dance. Cofounder (with the late, great Gregory Hines) of the Tap Legacy Foundation, he is now working to transfer that archive online. 

Cavaughn Noel wants to build a digital platform to broaden the horizons of urban youth by exposing them to technology, arts, fashion, and travel.

Torin Perez is building a digital platform for sharing children’s stories of the African diaspora. The DreamAfrica app contains multimedia content from established publishers, independent content creators, and children.

Amanda Phingbodhipakkiya is a Columbia-trained neuroscientistturnedart director. Her organization recruits designers and researchers to collaborate on visual media that demystify academic science.

After her Flappy Bird in a Box video went viral, Fawn Qiu wondered how else she could hook teens on engineering.  By creating an open-source model for designing fun projects with low-cost, everyday objects, she hopes to encourages a new generation of engineers.  

Vanessa Valenti is the co-founder of FRESH, a next-generation speakers’ bureau focused on diversifying public speaking. She’s studying who gets on the world’s most influential stages and what their experiences are once they get there.

Kimberlee Williams is the CEO of FEMWORKS, a communications agency based in Newark, NJ.  She wants to transform local economies by enrolling African-American consumers in buy-local campaigns.

Sheryl Winarick‘s work as an immigration lawyer gives her a unique opportunity to know intimately the people she serves, the reasons they choose to migrate, and the challenges they face. She aims to create an online storytelling platform to humanize “the other,” and to cultivate a sense of individual and collective responsibility.


Planet DebianJonathan McDowell: Notes on Kodi + IR remotes

This post is largely to remind myself of the details next time I hit something similar; I found bits of relevant information all over the place, but not in one single location.

I love Kodi. These days the Debian packages give me a nice out of the box experience that is easy to use. The problem comes in dealing with remote controls and making best use of the available buttons. In particular I want to upgrade the VDR setup my parents have to a more modern machine that’s capable of running Kodi. In this instance an AMD E350 nettop, which isn’t recent but does have sufficient hardware acceleration of video decoding to do the job. Plus it has a built in fintek CIR setup.

First step was finding a decent remote. The fintek is a proper IR receiver supported by the in-kernel decoding options, so I had a lot of flexibility. As it happened I ended up with a surplus to requirements Virgin V Box HD remote (URC174000-04R01). This has the advantage of looking exactly like a STB remote, because it is one.

Pointed it at the box, saw that the fintek_cir module was already installed and fired up irrecord. Failed to get it to actually record properly. Googled lots. Found ir-keytable. Fired up ir-keytable -t and managed to get sensible output with the RC-5 decoder. Used irrecord -l to get a list of valid button names and proceed to construct a vboxhd file which I dropped in /etc/rc_keymaps/. I then added a

fintek-cir * vboxhd

line to /etc/rc_maps.cfg to force my new keymap to be loaded on boot.

That got my remote working, but then came the issue of dealing with the fact that some keys worked fine in Kodi and others didn’t. This seems to be an issue with scancodes above 0xff. I could have remapped the remote not to use any of these, but instead I went down the inputlirc approach (which is already in use on the existing VDR box).

For this I needed a stable device file to point it at; the /dev/input/eventN file wasn’t stable and as a platform device it didn’t end up with a useful entry in /dev/input/by-id. A ‘quick’

udevadm info -a -p $(udevadm info -q path -n /dev/input/eventN)

provided me with the PNP id (FIT0002) allowing me to create /etc/udev/rules.d/70-remote-control.rules containing

KERNEL=="event*",ATTRS{id}=="FIT0002",SYMLINK="input/remote"

Bingo, a /dev/input/remote symlink. /etc/defaults/inputlirc ended up containing:

EVENTS="/dev/input/remote"
OPTIONS="-g -m 0"

The options tell it to grab the device for its own exclusive use, and to take all scancodes rather than letting the keyboard ones through to the normal keyboard layer. I didn’t want anything other than things specifically configured to use the remote to get the key presses.

At this point Kodi refused to actually do anything with the key presses. Looking at ~kodi/.kodi/temp/kodi.log I could see them getting seen, but not understood. Further searching led me to construct an Lircmap.xml - in particular the piece I needed was the <remote device="/dev/input/remote"> bit. The existing /usr/share/kodi/system/Lircmap.xml provided a good starting point for what I wanted and I dropped my generated file in ~kodi/.kodi/userdata/.

(Sadly it turns out I got lucky with the remote; it seems to be using the RC-5x variant which was broken in 3.17; works fine with the 3.16 kernel in Debian 8 (jessie) but nothing later. I’ve narrowed down the offending commit and raised #117221.)

Helpful pages included:

Planet DebianPau Garcia i Quiles: Is KDE the right place for Thunderbird?

For years, Mozilla has been saying they are no longer focused on Thunderbird and its place is outside of Mozilla. Now it seems they are going to act on what they said: Mozilla seeks new home for e-mail client Thunderbird.

The candidates they are exploring are the Software Freedom Conservancy, The Document Foundation, and I expect at least the Apache Software Foundation to be a serious candidate, and Gnome to propose.

Some voices in KDE say we should also propose the KDE eV as a candidate hosting organization.

What follows is my opinion, not the official opinion of the eV or the board’s, or the KDE Community’s opinion. Take it with a grain (or more) of salt.

I am not so sure. I am trying to think what the KDE eV can offer to Mozilla to be appealing to them and if my analysis is correct, we are too far and Thunderbird would pose many risks to the other projects in KDE.

(I am blurring the lines between “KDE eV”, “KDE community”, “KDE Frameworks”, etc as it has no relevance for the discussion)

Thunderbird is an open source project/product with a lot of commercial users and has (still has?) many paid contributors.

IMHO what Mozilla is looking for is an organization with a well-oiled funding machine, able to campaign for money (even if in a tight circle, something like ours Patron program), and accept and process funds in a way that directly benefits Thunderbird. I e. hiring developers to implement X or Y, or work on some area full-time, or at least, half-time.

KDE does not work like that.

KDE has few commercial users (other than distros, if you want to count them as commercial users). Other than Blue Systems, I don’t think we have any developer working for KDE.

Also, the eV is not exactly a well-oiled funding machine. We have been talking about that for years. And we do not hire developers directly to work on X or Y (at most, we pay for part of the expenses of sprints).

All of that makes me think we are not the right host for Thunderbird.

But it does not stop there!

Let’s say Thunderbird comes to KDE and suddenly we are offered USD 1 M from several organizations who want to be “Patrons of Thunderbird”, or influence Thunderbird, or whatever.

First problem: do we allow funds to go to a specific project rather than the eV deciding how to distribute them? AFAIK we do not allow that and at least one KDE sub-project has had trouble with that in the past.

Then there is the thing about “Patrons of Thunderbird”: no such thing. Either you are a Patron of KDE, including Plasma Mobile, OwnCloud, and whatnot, or you are nothing. You cannot be a “Patron of Partial KDE, namely Thunderbird”.

Influencing, did I say? The eV is by its own rules not an influencer on KDE’s direction, just an entity to provide legal and economic support. Quite the opposite from what Mozilla does today for Thunderbird.

Even if funders would not mind all that, there is the huge risk this poses for all the other projects. With as little as USD 200K donated towards Thunderbird (and USD 200K is not much for a product with so many commercial users, which means a healthy ecosystem of companies making money on support, development, etc, and thus donating to somehow influence or be perceived as important players), Thunderbird becomes the most important project in KDE. How would we manage this? In any sensible organization, Thunderbird would become the main focus and all the other KDE projects would be relegated. Even if we decide not to, external PR would make that look like it happened.

For all those reasons, I think KDE is not the right place for Thunderbird at the moment. It would require a big change in what the eV can do and how it operates. And that change may be for good but it’s not there now and it will not be by the time Mozilla has to decide if KDE is the right place.

All that, and I have not even talked about technology and what any sensible Thunderbird “customer” would think today: what is the medium and long-term roadmap? Migrate Thunderbird users to Kontact/KDE PIM? Port Thunderbird to Qt + KF5, maybe including moving to QtWebEngine? Will Windows support be deteriorated by that change? Or maybe the plan is to cancel KMail and Akregator? Those are second-thoughts, unimportant right now.

Update If you want to contribute to the discussion, please join the KDE-Community mailing list.

 

Planet DebianNiels Thykier: Putting Debian packages in labelled boxes

Lintian 2.5.44 was released the other day and (to most) the most significant bug fix was probably that Lintian learned about Policy 3.9.8.  I would like to thank Axel Beckert for doing that.  Notably it also made me update the test suite so to make future policy releases less painful.

For others, it might be the fact that Lintian now accepts (valid) versioned provides (which seemed prudent now that Britney accepts them as well).  Newcomers might appreciate that we are giving a much more sensible warning when they have extra spaces in their changelog “sign off” line (rather than pretending it is an improper NMU).  But I digress…

 

What I am here to talk about is that Lintian 2.5.44 started classifying packages based on various “facts” or “properties”, we can determine.  Therefore:

  • Every package will have at least one tag now!
  • These labels are known as something called “classification tags”.
  • The tags are not issues to be fixed!  (I will repeat this later to ensure you get this point!)

Here are some of the “labelled boxes” your packages will be put into[0]:

The tags themselves are (as mentioned) mere classifications and their primary purpose is to classify or measure certain properties.  With them any body can download the data set and come with some bold statement about Debian packages (hopefully without relying too much on “lies, damned lies and statistics“).  Lets try that immediately!

  • Almost 75% of all Debian packages do not need to run arbitrary code doing installation[2]!
  • The “dh-sequencer” with cdbs is the future![3]

In the next release, we will also add tracking of auto-generated snippets from dh_*-tools.  Currently unversioned, but I hope to add versioning to that so we can find and rebuild packages that have been built with buggy autoscripts (like #788098)

If you want to see the classification tags for your package, please run lintian with like this:

# Add classification tags
$ lintian -L +classification <pkg-or-changes>
# Or if you want only classification tags$ lintian -L =classification <pkg-or-changes>

Please keep in mind that classification tags (“C”) are not issues in themselves. Lintian is simply attempting to add a visible indicator about a given “fact” or “property” in the package – nothing more, nothing less.

 

Future work – help (read: patches) welcome:

 

[0] Mind you, the reporting framework’s handling of these tags could certainly be improved.

[1] Please note how it distinguishes 1.0 into native and non-native based on whether the package has a diff.gz.  Presumably that can be exploited somehow …

[2] Disclaimer: At the time of writing, only ~80% of the archive have been processed.  This is computed as: NS / (NS + WS), where NS and WS are the number of unique packages with the tags “no-ctrl-scripts” and “ctrl-script” respectively.

[3] … or maybe not, but we got two packages classified as using both CDBS and the dh-sequencer.  I have not looked at it in detail. For the curious: libmecab-java and ctioga2.


Filed under: Debian, Lintian

Krebs on SecurityAll About Fraud: How Crooks Get the CVV

A longtime reader recently asked: “How do online fraudsters get the 3-digit card verification value (CVV or CVV2) code printed on the back of customer cards if merchants are forbidden from storing this information? The answer: If not via phishing, probably by installing a Web-based keylogger at an online merchant so that all data that customers submit to the site is copied and sent to the attacker’s server.

Kenneth Labelle, a regional director at insurer Burns-Wilcox.com, wrote:

“So, I am trying to figure out how card not present transactions are possible after a breach due to the CVV. If the card information was stolen via the point-of-sale system then the hacker should not have access to the CVV because its not on the magnetic strip. So how in the world are they committing card not present fraud when they don’t have the CVV number? I don’t understand how that is possible with the CVV code being used in online transactions.”

First off, “dumps” — or credit and debit card accounts that are stolen from hacked point of sale systems via skimmers or malware on cash register systems — retail for about $20 apiece on average in the cybercrime underground. Each dump can be used to fabricate a new physical clone of the original card, and thieves typically use these counterfeits to buy goods from big box retailers that they can easily resell, or to extract cash at ATMs.

However, when cyber crooks wish to defraud online stores, they don’t use dumps. That’s mainly because online merchants typically require the CVV, criminal dumps sellers don’t bundle CVVs with their dumps.

Instead, online fraudsters turn to “CVV shops,” shadowy cybercrime stores that sell packages of cardholder data, including customer name, full card number, expiration, CVV2 and ZIP code. These CVV bundles are far cheaper than dumps — typically between $2-$5 apiece — in part because the are useful mainly just for online transactions, but probably also because overall they more complicated to “cash out” or make money from them.

The vast majority of the time, this CVV data has been stolen by Web-based keyloggers. This is a relatively uncomplicated program that behaves much like a banking Trojan does on an infected PC, except it’s designed to steal data from Web server applications.

PC Trojans like ZeuS, for example, siphon information using two major techniques: snarfing passwords stored in the browser, and conducting “form grabbing” — capturing any data entered into a form field in the browser before it can be encrypted in the Web session and sent to whatever site the victim is visiting.

Web-based keyloggers also can do form grabbing, ripping out form data submitted by visitors — including names, addresses, phone numbers, credit card numbers and card verification code — as customers are submitting the data during the online checkout process.

These attacks drive home one immutable point about malware’s role in subverting secure connections: Whether resident on a Web server or on an end-user computer, if either endpoint is compromised, it’s ‘game over’ for the security of that Web session. With PC banking trojans, it’s all about surveillance on the client side pre-encryption, whereas what the bad guys are doing with these Web site attacks involves sucking down customer data post- or pre-encryption (depending on whether the data was incoming or outgoing).

If you’re responsible for maintaining or securing Web sites, it might be a good idea to get involved in one or more local groups that seek to help administrators. Professional and semi-professionals are welcome at local chapter meetings of OWASP, CitySec, ISSA or Security Bsides meetups.

Planet DebianMatthias Klumpp: Why are AppStream metainfo files XML data?

This is a question raised quite quite often, the last time in a blogpost by Thomas, so I thought it is a good idea to give a slightly longer explanation (and also create an article to link to…).

There are basically three reasons for using XML as the default format for metainfo files:

1. XML is easily forward/backward compatible, while YAML is not

This is a matter of extending the AppStream metainfo files with new entries, or adapt existing entries to new needs.

Take this example XML line for defining an icon for an application:

<icon type="cached">foobar.png</icon>

and now the equivalent YAML:

Icons:
  cached: foobar.png

Now consider we want to add a width and height property to the icons, because we started to allow more than one icon size. Easy for the XML:

<icon type="cached" width="128" height="128">foobar.png</icon>

This line of XML can be read correctly by both old parsers, which will just see the icon as before without reading the size information, and new parsers, which can make use of the additional information if they want. The change is both forward and backward compatible.

This looks differently with the YAML file. The “foobar.png” is a string-type, and parsers will expect a string as value for the cached key, while we would need a dictionary there to include the additional width/height information:

Icons:
  cached: name: foobar.png
          width: 128
          height: 128

The change shown above will break existing parsers though. Of course, we could add a cached2 key, but that would require people to write two entries, to keep compatibility with older parsers:

Icons:
  cached: foobar.png
  cached2: name: foobar.png
          width: 128
          height: 128

Less than ideal.

While there are ways to break compatibility in XML documents too, as well as ways to design YAML documents in a way which minimizes the risk of breaking compatibility later, keeping the format future-proof is far easier with XML compared to YAML (and sometimes simply not possible with YAML documents). This makes XML a good choice for this usecase, since we can not do transitions with thousands of independent upstream projects easily, and need to care about backwards compatibility.

2. Translating YAML is not much fun

A property of AppStream metainfo files is that they can be easily translated into multiple languages. For that, tools like intltool and itstool exist to aid with translating XML using Gettext files. This can be done at project build-time, keeping a clean, minimal XML file, or before, storing the translated strings directly in the XML document. Generally, YAML files can be translated too. Take the following example (shamelessly copied from Dolphin):

<summary>File Manager</summary>
<summary xml:lang="bs">Upravitelj datoteka</summary>
<summary xml:lang="cs">Správce souborů</summary>
<summary xml:lang="da">Filhåndtering</summary>

This would become something like this in YAML:

Summary:
  C: File Manager
  bs: Upravitelj datoteka
  cs: Správce souborů
  da: Filhåndtering

Looks manageable, right? Now, AppStream also covers long descriptions, where individual paragraphs can be translated by the translators. This looks like this in XML:

<description>
  <p>Dolphin is a lightweight file manager. It has been designed with ease of use and simplicity in mind, while still allowing flexibility and customisation. This means that you can do your file management exactly the way you want to do it.</p>
  <p xml:lang="de">Dolphin ist ein schlankes Programm zur Dateiverwaltung. Es wurde mit dem Ziel entwickelt, einfach in der Anwendung, dabei aber auch flexibel und anpassungsfähig zu sein. Sie können daher Ihre Dateiverwaltungsaufgaben genau nach Ihren Bedürfnissen ausführen.</p>
  <p>Features:</p>
  <p xml:lang="de">Funktionen:</p>
  <p xml:lang="es">Características:</p>
  <ul>
    <li>Navigation (or breadcrumb) bar for URLs, allowing you to quickly navigate through the hierarchy of files and folders.</li>
    <li xml:lang="de">Navigationsleiste für Adressen (auch editierbar), mit der Sie schnell durch die Hierarchie der Dateien und Ordner navigieren können.</li>
    <li xml:lang="es">barra de navegación (o de ruta completa) para URL que permite navegar rápidamente a través de la jerarquía de archivos y carpetas.</li>
    <li>Supports several different kinds of view styles and properties and allows you to configure the view exactly how you want it.</li>
    ....
  </ul>
</description>

Now, how would you represent this in YAML? Since we need to preserve the paragraph and enumeration markup somehow, and creating a large chain of YAML dictionaries is not really a sane option, the only choices would be:

  • Embed the HTML markup in the file, and risk non-careful translators breaking the markup by e.g. not closing tags.
  • Use Markdown, and risk people not writing the markup correctly when translating a really long string in Gettext.

In both cases, we would loose the ability to translate individual paragraphs, which also means that as soon as the developer changes the original text in YAML, translators would need to translate the whole bunch again, which is inconvenient.

On top of that, there are no tools to translate YAML properly that I am aware of, so we would need to write those too.

3. Allowing XML and YAML makes a confusing story and adds complexity

While adding YAML as a format would not be too hard, given that we already support it for DEP-11 distro metadata (Debian uses this), it would make the business of creating metainfo files more confusing. At time, we have a clear story: Write the XML, store it in /usr/share/metainfo, use standard tools to translate the translatable entries. Adding YAML to the mix adds an additional choice that needs to be supported for eternity and also has the problems mentioned above.

I wanted to add YAML as format for AppStream, and we discussed this at the hackfest as well, but in the end I think it isn’t worth the pain of supporting it for upstream projects (remember, someone needs to maintain the parsers and specification too and keep XML and YAML in sync and updated). Don’t get me wrong, I love YAML, but for translated metadata which needs a guarantee on format stability it is not the ideal choice.

So yeah, XML isn’t fun to write by hand. But for this case, XML is a good choice.

Planet DebianMichal Čihař: Weekly phpMyAdmin contributions 2016-W16

Last week was again focused on bug fixing due to increased amount of received bug reports on 4.6.0 release. Fortunately most of the annoying bugs are already fixed in git and will be soon released as 4.6.1.

Another bigger task which was started last week was wiki migration. So far we've been using own wiki running MediaWiki and we're migrating it to GitHub wiki. The wiki on GitHub is way simpler, but it seems as better choice for us. During the migration all user documentation will be merged into our documentation, so that it's all in one place and wiki will be targeted on developers.

Handled issues:

Filed under: English phpMyAdmin | 2 comments

Planet DebianReproducible builds folks: Reproducible builds: week 52 in Stretch cycle

What happened in the Reproducible Builds effort between April 17th and April 23rd 2016:

Toolchain fixes

Thomas Weber uploaded lcms2/2.7-1 which will not write uninitialized memory when writing color names. Original patch by Lunar.

The GCC 7 development phase has just begun, so Dhole reworked his patch to make gcc use SOURCE_DATE_EPOCH if set which prompted interesting feedback, but it has not been merged yet.

Alexis Bienvenüe submitted a patch for sphinx to strip Python object memory addresses from the generated documentation.

Packages fixed

The following packages have become reproducible due to changes in their build dependencies: cobertura, commons-pool, easymock, eclipselink, excalibur-logkit, gap-radiroot, gluegen2, jabref, java3d, jcifs, jline, jmock2, josql, jtharness, libfann, libgroboutils-java, libjemmy2-java, libjgoodies-binding-java, libjgrapht0.8-java, libjtds-java, liboptions-java, libpal-java, libzeus-jscl-java, node-transformers, octave-msh, octave-secs2d, openmama, rkward.

The following packages have become reproducible after being fixed:

Patches submitted that have not made their way to the archive yet:

  • #821356 against emoslib by boyska: use echo in a portable manner across shells.
  • #822268 against transdecoder by Dhole: set PERL_HASH_SEED=0 when calling the scripts that generate samples.

tests.reproducible-builds.org

  • Steven Chamberlain investigated the performance of our armhf boards which also provided a nice overview of our armhf build network.
  • As i386 has almost been completely tested the order of the architectures displayed has been changed to reflect the fact that i386 is now the 2nd most popular architecture in Debian. (h01ger)
  • In order to decrease the number of blacklisted packages, the first build is now run with a timeout of 18h (previously: 12h) and the 2nd with 24h timeout (previously: 18h). (h01ger)
  • We now also vary the CPU model on amd64 (and soon on i386 too) so that one build is performed using a "AMD Opteron 62xx class CPU" while the other is done using a "Intel Core Processor (Haswell)". This is now possible as proftitbricks.com offers VMs running both types of CPU and have generously increased their sponsorship once more. (h01ger)
  • Profitbricks increased our storage space by 400 GB which will be used to setup a 2nd build node for the coreboot/OpenWrt/NetBSD/Arch Linux/Fedora tests. This 2nd build node will run 398 days in the future for testing reproducibility on a different date.

diffoscope development

diffoscope 52 was released with changes from Mattia Rizzolo, h01ger, Satyam Zode and Reiner Herrmann, who also did the release. Notable changes included:

  • Drop transitional debbindiff package.
  • Let objdump demangle symbols for better readability.
  • Install bin/diffoscope instead of auto-generated script. (Closes: #821777)

As usual, diffoscope 52 is available on Debian, Archlinux and PyPI, other distributions will hopefully soon update.

Package reviews

28 reviews have been added, 11 have been updated and 94 have been removed in this week.

14 FTBFS bugs were reported by Chris Lamb (one being was a duplicate of a bug filed by Sebastian Ramacher an hour earlier).

Misc.

This week's edition was written by Lunar, Holger 'h01ger' Levsen and Chris Lamb and reviewed by a bunch of Reproducible builds folks on IRC.

CryptogramPeople Trust Robots, Even When They Don't Inspire Trust

Interesting research:

In the study, sponsored in part by the Air Force Office of Scientific Research (AFOSR), the researchers recruited a group of 42 volunteers, most of them college students, and asked them to follow a brightly colored robot that had the words "Emergency Guide Robot" on its side. The robot led the study subjects to a conference room, where they were asked to complete a survey about robots and read an unrelated magazine article. The subjects were not told the true nature of the research project.

In some cases, the robot -- which was controlled by a hidden researcher --- led the volunteers into the wrong room and traveled around in a circle twice before entering the conference room. For several test subjects, the robot stopped moving, and an experimenter told the subjects that the robot had broken down. Once the subjects were in the conference room with the door closed, the hallway through which the participants had entered the building was filled with artificial smoke, which set off a smoke alarm.

When the test subjects opened the conference room door, they saw the smoke - and the robot, which was then brightly-lit with red LEDs and white "arms" that served as pointers. The robot directed the subjects to an exit in the back of the building instead of toward the doorway - marked with exit signs - that had been used to enter the building.

"We expected that if the robot had proven itself untrustworthy in guiding them to the conference room, that people wouldn't follow it during the simulated emergency," said Paul Robinette, a GTRI research engineer who conducted the study as part of his doctoral dissertation. "Instead, all of the volunteers followed the robot's instructions, no matter how well it had performed previously. We absolutely didn't expect this."

The researchers surmise that in the scenario they studied, the robot may have become an "authority figure" that the test subjects were more likely to trust in the time pressure of an emergency. In simulation-based research done without a realistic emergency scenario, test subjects did not trust a robot that had previously made mistakes.

Our notions of trust depend on all sorts of cues that have nothing to do with actual trustworthiness. I would be interested in seeing where the robot fits in in the continuum of authority figures. Is it trusted more or less than a man in a hazmat suit? A woman in a business suit? An obviously panicky student? How do different looking robots fare?

News article. Research paper.

Sociological ImagesMisty Copeland and the Newness of the Ballerina Body

Many hope that Misty Copeland is ushering in a new era for ballet. She is the first female African American ballet dancer to have the role of Principal Dancer at the American Ballet Theatre. She has literally changed the face of the dance.

Race is a central and important part of her story, but in A Ballerina’s Tale, the documentary featuring her career, she describes herself as defying not just one, but three ideas about what ballerinas are supposed to look like: “I’m black,” she says, and also: “I have a large chest, I’m muscular.”

In fact, asked to envision a prima ballerina, writes commentator Shane Jewel, what comes to most of our minds is probably a “perilously thin, desperately beautiful, gracefully elongated girl who is… pale as the driven snow.” White, yes, but also flat-chested and without obvious muscularity.

It feels like a timeless archetype — at least as timeless as ballet itself, which dates back to the 15th century — but it’s not. In fact, the idea that ballerinas should be painfully thin is a new development, absorbing only a fraction of ballet’s history, as can clearly be seen in this historical slideshow.

It started in the 1960s — barely more than 50 years ago — in response to the preferences of the influential choreographer George Balanchine. Elizabeth Kiem, the author of Dancer, Daughter, Traitor, Spy, calls him “the most influential figure in 20th century dance,” ballet and beyond. He co-founded the first major ballet school in America, made dozens of dancers famous, and choreographed more than 400 performances. And he liked his ballerinas wispy: “Tall and slender,” Kiem writes, “to the point of alarm.” It is called, amongst those in that world, the “Balanchine body.”

 

We’re right to view Copeland’s rise with awe, gratitude, and hope, but it’s also interesting to note that two of the the ceilings she’s breaking (by being a ballerina with breasts and muscles) have only recently been installed. It reminds me how quickly a newly introduced expectation can feel timeless; how strongly it can ossify into something that seems inevitable; how easily we accept that what we see in front of us is universal.

In The Social Construction of Reality, the sociologists Peter Berger and Thomas Luckmann explain how rapidly social inventions “harden” and “thicken.” Whoever initiates can see it for what it is — something they created — but to whoever comes next it simply seems like reality. What to Balanchine was “I will do it this way” became to his successors “This is how things are done.” And “a world so regarded,” Berger and Luckmann write, “attains a firmness in consciousness; it becomes real in an ever more massive way, and it can no longer be changed so readily.”

Exactly because the social construction of reality can be so real, even though it was merely invented, Copeland’s three glass ceilings are all equally impressive, even if only one is truly historic.

Lisa Wade is a professor at Occidental College and the co-author of Gender: Ideas, Interactions, Institutions. Find her on TwitterFacebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

Planet DebianRhonda D'Vine: Prince

Last week we lost another great musician, song writer, artist. It's painful to realise that more and more of the people you grew up with aren't there anymore. We lost Prince, TAFKAP, Symbol, Prince. He wrote a lot of great music, even some you wouldn't attribute to him, like Sinead O'Connor's Nothing Compares To You, Bangles' Manic Monday or Chaka Khan's I Feel For You. But I actually would like to share some songs that are also performed by himself, so without further ado here are the songs:

Rest in peace, Prince. And you, enjoy.

/music | permanent link | Comments: 0 | Flattr this

Planet DebianSteinar H. Gunderson: Full stack

As I'm nearing the point where Nageru, my live video mixer, can produce a stream directly that is actually suitable to streaming directly to clients (without a transcoding layer in the chain), it struck me the other day how much of the chain I've actually had to touch:

In my test setup, the signal comes into a Blackmagic Intensity Shuttle. At some point, I found what I believe is a bug in the card's firmware; I couldn't fix it, but a workaround was applied in the Linux kernel. (I also have some of their PCI cards, in which I haven't found any bugs, but I have found bugs in their drivers.)

From there, it goes into bmusb, a driver I wrote myself. bmusb uses libusb-1.0 to drive the USB card from userspace—but for performance and stability reasons, I patched libusb to use the new usbfs zerocopy support in the Linux kernel. (The patch is still pending review.) Said zerocopy support wasn't written by me, but I did the work to clean up the support and push it upstream (it's in the 4.6-rc* series).

Once safely through bmusb, it goes of course into Nageru, which I wrote myself. Nageru uses Movit for pixel processing, which I also wrote myself. Movit in turn uses OpenGL; I've found bugs in all three major driver implementations, and fixed a Nageru-related one in Mesa (and in the process of debugging that, found bugs in apitrace, a most useful OpenGL debugger). Sound goes through zita-resampler to stretch it ever so gently (in case audio and video clocks are out of sync), which I didn't wrote, but patched to get SSE support (patch pending upstream).

So now Nageru chews a bit on it, and then encodes the video using x264 (that's the new part in 1.3.0—of course, you need a fast CPU to do that as opposed to using Quick Sync). I didn't write x264, but I had to redo parts of the “speedcontrol” patch (not part of upstream; awaiting review semi-upstream) because of bugs and outdated timings, but I also found a bug in x264 proper (fixed by upstream, pending inclusion). Muxing is done through ffmpeg, where I actually found multiple bugs in the muxer (some of which are still pending fixes).

Once the stream is safely encoded and hopefully reasonably standards-conforming (that took me quite a while), it goes to Cubemap, which I wrote, for reflection to clients. For low-bitrate clients, it takes a detour through VLC to get a re-encode on a faster machine to lower bitrate—I've found multiple bugs in VLC's streaming support in the past (and also fixed some of them, plus written the code that interacts with Cubemap).

From there it goes to any of several clients, usually a browser. I didn't write any browsers (thank goodness!), but I wrote the client-side JavaScript that picks the closest relay, and the code for sending it to a Chromecast. I also found a bug in Chrome for Android (will be fixed in version 50 or 51, although the fix was just about turning on something that was already in the works), and one in Firefox for Linux (fixed by patching GStreamer's MP4 demuxer, although they've since switched away from that to something less crappy). IE/Edge also broke at some point, but unfortunately I don't have a way to report bugs to Microsoft. There's also at least one VLC bug involved on the client side (it starts decoding frames too late if they come with certain irregular timestamps, which causes them to drop), but I want to verify that they still persist even after the muxer is fixed before I go deep on that.

Moral of the story: If anyone wants to write a multimedia application and says “I'll just use <framework, language or library XYZ>, and I'll get everything for free; I just need to click things together!”, they simply don't know what they're talking about and are in for a rude awakening. Multimedia is hard, an amazing amount of things can go wrong, complex systems have subtle bugs, and there is no silver bullet.

Worse Than FailureCodeSOD: See You Last Saturday

Technocracy-Calendar

One of the more difficult things for beginning programmers to pick up is computer-minded thinking. Sure, if you're reading this, it's probably easy for you to look at a system and plot out how to get the outputs you want in one area out of the information you have in another. For someone who's been programming for years, it's practically second nature. When mentoring interns or teaching beginners, however, it can readily become apparent just how strange this mindset can be to newcomers.

We don't know this date-parsing code was written by a newbie ... but we have our suspicions:


/**
 *
 * @param date
 * @return
 * @throws Exception
 */
public static Date getPreviousSaturDay(Date date) throws Exception {
        Calendar calendar = Calendar.getInstance();
        Date saturday = null;
        if (date != null) {
                calendar.setTime(date);
                int day = calendar.get(Calendar.DAY_OF_WEEK);
                if (day == Calendar.SATURDAY) {
                        calendar.add(Calendar.DATE, 0);
                } else if (day == Calendar.SUNDAY) {
                        calendar.add(Calendar.DATE, -1);
                } else if (day == Calendar.MONDAY) {
                        calendar.add(Calendar.DATE, -2);
                } else if (day == Calendar.TUESDAY) {
                        calendar.add(Calendar.DATE, -3);
                } else if (day == Calendar.WEDNESDAY) {
                        calendar.add(Calendar.DATE, -4);
                } else if (day == Calendar.THURSDAY) {
                        calendar.add(Calendar.DATE, -5);
                } else if (day == Calendar.FRIDAY) {
                        calendar.add(Calendar.DATE, -6);
                }
                SimpleDateFormat simpledateformat = new SimpleDateFormat(
                                DateUtil.MM_DD_YYYY_DATE_PATTERN);
                saturday = simpledateformat.parse(simpledateformat.format(calendar
                                                .getTime()));
        }
        return saturday;
}

No javadoc other than a placeholder the IDE probably generated? Bizzarely-cased method name? Throwing a generic Exception? These are all signs of someone without a firm grasp on the language, but possibly not a total newcomer to programming. After all, someone who's only done hobby work in loosely typed languages could make all the above mistakes. The real hallmark is the strange, twisty path the method takes to get from point A to point B.

Starting with a Date, the method aims to return a Date that represents the previous Saturday. With JodaTime (or Java 8's new Time library, which is essentially the same), this is a simple task: date.withDayOfWeek(DateTimeConstants.SATURDAY) will get you the nearest Saturday, and from there it's a simple comparison to see if you need to subtract a week. But this uses the famously terrible Calendar library that has plagued Java for decades, which means you have to have a firm grasp on what you're doing in order to work with it successfully.

Still, the mistakes made aren't related to the date library, but to a fundamental lack of understanding of the task at hand. Stating with a Date, they turn it into a Calendar and get the day of the week the input represented. They then use an if-ladder to manipulate the Calendar, subtracting the right number of days. Then, at a loss for how to turn the Calendar back into a Date, they extract the Date from the Calendar, run it through a formatter to turn it into a string, then run that string through a parser to get a Date.

What?

Unfortunately, this wasn't a school project or some other learning exercise. This was found in production code.

[Advertisement] Infrastructure as Code built from the start with first-class Windows functionality and an intuitive, visual user interface. Download Otter today!

Worse Than FailureCodeSOD: See You Last Saturday

Technocracy-Calendar

One of the more difficult things for beginning programmers to pick up is computer-minded thinking. Sure, if you're reading this, it's probably easy for you to look at a system and plot out how to get the outputs you want in one area out of the information you have in another. For someone who's been programming for years, it's practically second nature. When mentoring interns or teaching beginners, however, it can readily become apparent just how strange this mindset can be to newcomers.

We don't know this date-parsing code was written by a newbie ... but we have our suspicions:


/**
 *
 * @param date
 * @return
 * @throws Exception
 */
public static Date getPreviousSaturDay(Date date) throws Exception {
        Calendar calendar = Calendar.getInstance();
        Date saturday = null;
        if (date != null) {
                calendar.setTime(date);
                int day = calendar.get(Calendar.DAY_OF_WEEK);
                if (day == Calendar.SATURDAY) {
                        calendar.add(Calendar.DATE, 0);
                } else if (day == Calendar.SUNDAY) {
                        calendar.add(Calendar.DATE, -1);
                } else if (day == Calendar.MONDAY) {
                        calendar.add(Calendar.DATE, -2);
                } else if (day == Calendar.TUESDAY) {
                        calendar.add(Calendar.DATE, -3);
                } else if (day == Calendar.WEDNESDAY) {
                        calendar.add(Calendar.DATE, -4);
                } else if (day == Calendar.THURSDAY) {
                        calendar.add(Calendar.DATE, -5);
                } else if (day == Calendar.FRIDAY) {
                        calendar.add(Calendar.DATE, -6);
                }
                SimpleDateFormat simpledateformat = new SimpleDateFormat(
                                DateUtil.MM_DD_YYYY_DATE_PATTERN);
                saturday = simpledateformat.parse(simpledateformat.format(calendar
                                                .getTime()));
        }
        return saturday;
}

No javadoc other than a placeholder the IDE probably generated? Bizzarely-cased method name? Throwing a generic Exception? These are all signs of someone without a firm grasp on the language, but possibly not a total newcomer to programming. After all, someone who's only done hobby work in loosely typed languages could make all the above mistakes. The real hallmark is the strange, twisty path the method takes to get from point A to point B.

Starting with a Date, the method aims to return a Date that represents the previous Saturday. With JodaTime (or Java 8's new Time library, which is essentially the same), this is a simple task: date.withDayOfWeek(DateTimeConstants.SATURDAY) will get you the nearest Saturday, and from there it's a simple comparison to see if you need to subtract a week. But this uses the famously terrible Calendar library that has plagued Java for decades, which means you have to have a firm grasp on what you're doing in order to work with it successfully.

Still, the mistakes made aren't related to the date library, but to a fundamental lack of understanding of the task at hand. Stating with a Date, they turn it into a Calendar and get the day of the week the input represented. They then use an if-ladder to manipulate the Calendar, subtracting the right number of days. Then, at a loss for how to turn the Calendar back into a Date, they extract the Date from the Calendar, run it through a formatter to turn it into a string, then run that string through a parser to get a Date.

What?

Unfortunately, this wasn't a school project or some other learning exercise. This was found in production code.

[Advertisement] Infrastructure as Code built from the start with first-class Windows functionality and an intuitive, visual user interface. Download Otter today!

Planet DebianMatthias Klumpp: A GNOME Software Hackfest report

Two weeks ago was the GNOME Software hackfest in London, and I’ve been there! And I just now found the time to blog about it, but better do it late than never 😉 .

Arriving in London and finding the Red Hat offices

After being stuck in trains for the weekend, but fortunately arriving at the airport in time, I finally made it to London with quite some delay due to the slow bus transfer from Stansted Airport. After finding the hotel, the next issue was to get food and a place which accepted my credit card, which was surprisingly hard – in defence of London I must say though, that it was a Sunday, 7 p.m. and my card is somewhat special (in Canada, it managed to crash some card readers, so they needed a hard-reset). While searching for food, I also found the Red Hat offices where the hackfest was starting the next day by accident. My hotel, the office and the tower bridge were really close, which was awesome! I have been to London in 2008 the last time, and only for a day, so being that close to the city center was great. The hackfest didn’t leave any time to visit the city much, but by being close to the center, one could hardly avoid the “London experience” 😉 .

Cool people working on great stuff

towerbridge2016That’s basically the summary for the hackfest 😉 . It was awesome to meet with Richard Hughes again, since we haven’t seen each other in person since 2011, but work on lots of stuff together. This was especially important, since we managed to solve quite some disagreements we had over stuff – Richard even almost managed to make me give in to adding <kudos/> to the AppStream spec, something which I was pretty against supporting (it didn’t make it yet, but I am no longer against the idea of having that – the remaining issues are solvable).

Meeting Iain Lane again (after FOSDEM) was also very nice, and also seeing other people I’ve only worked with over IRC or bug reports (e.g. William, Kalev, …) was great. Also lots of “new” people were there, like guys from Endless, who build their low-budget computer for developing/emerging countries on top of GNOME and Linux technologies. It’s pretty cool stuff they do, you should check out their website! (they also build their distribution on top of Debian, which is even more awesome, and something I didn’t know before (because many Endless people I met before were associated with GNOME or Fedora, I kind of implicitly assumed the system was based on Fedora 😛 )).

The incarnation of GNOME Software used by endless looks pretty different from what the normal GNOME user sees, since it’s adjusted for a different audience and input method. But it looks great, and is a good example for how versatile GS already is! And for upstream GNOME, we’ve seen some pretty great mockups done by Endless too – I hope those will make it into production somehow.

Ironically, a "snapstore" was close to the office ;-)

Ironically, a “snapstore” was close to the office ;-)

XdgApp and sandboxing of apps was also a big topic, aside from Ubuntu and Endless integration. Fortunately, Alexander Larsson was also there to answer all the sandboxing and XdgApp-questions.

I used the time to follow up on a conversation with Alexander we started at FOSDEM this year, about the Limba vs. XdgApp bundling issue. While we are in-line on the sandboxing approach, the way how software is distributed is implemented differently in Limba and XdgApp, and it is bad to have too many bundling systems around (doesn’t make for a good story where we can just tell developers “ship as this bundling format, and it will be supported everywhere”). Talking with Alex about this was very nice, and I think there is a way out of the too-many-solutions dilemma, at least for Limba and XdgApp – I will blog about that separately soon.

On the Ubuntu side, a lot of bugs and issues were squashed and changes upstreamed to GNOME, and people were generally doing their best to reduce Richard’s bus-factor on the project a little 😉 .

I mainly worked on AppStream issues, finishing up the last pieces of appstream-generator and running it against some sample package sets (and later that week against the whole Debian archive). I also started to implement support for showing AppStream issues in the Debian PTS (this work is not finished yet). I also managed to solve a few bugs in the old DEP-11 generator and prepare another release for Ubuntu.

We also enjoyed some good Japanese food, and some incredibly great, but also suddenly very expensive Indian food (but that’s a different story 😉 ).

The most important thing for me though was to get together with people actually using AppStream metadata in software centers and also more specialized places. This yielded some useful findings, e.g. that localized screenshots are not something weird, but actually a wanted feature of Endless for their curated AppStore. So localized screenshots will be part of the next AppStream spec. Also, there seems to be a general need to ship curation information for software centers somehow (which apps are featured? how are they styled? added special banners for some featured apps, “app of the day” features, etc.). This problem hasn’t been solved, since it’s highly implementation-specific, and AppStream should be distro-agnostic. But it is something we might be able to address in a generic way sooner or later (I need to talk to people at KDE and Elementary about it).

In summary…

It was a great event! Going to conferences and hackfests always makes me feel like it moves projects leaps ahead, even if you do little coding. Sorting out issues together with people you see in person (rather than communicating with them via text messages or video chat), is IMHO always the most productive way to move forward (yeah, unless you do this every week, but I think you get my point 😀 ).

For me, being the only (and youngest ^^) developer at the hackfest who was not employed by any company in the FLOSS business, the hackfest was also motivating to continue to invest spare time into working on these projects.

So, the only thing left to do is a huge shout out of “THANK YOU” to the Ubuntu Community Fund – and therefore the Ubuntu community – for sponsoring me! You rock! Also huge thanks to Canonical for organizing the sponsoring really quickly, so I didn’t get into trouble with paying my flights.

Laney and attente walking on the Millennium Bridge after we walked the distance between Red Hat and Canonical's offices.

Laney and attente on the Millennium Bridge after we walked the distance between Red Hat and Canonical’s offices.

To worried KDE people: No, I didn’t leave the blue side – I just generally work on cross-desktop stuff, and would like all desktops to work as well as possible 😉

Planet DebianFrancois Marier: Using DNSSEC and DNSCrypt in Debian

While there is real progress being made towards eliminating insecure HTTP traffic, DNS is a fundamental Internet service that still usually relies on unauthenticated cleartext. There are however a few efforts to try and fix this problem. Here is the setup I use on my Debian laptop to make use of both DNSSEC and DNSCrypt.

DNSCrypt

DNSCrypt was created to enable end-users to encrypt the traffic between themselves and their chosen DNS resolver.

To switch away from your ISP's default DNS resolver to a DNSCrypt resolver, simply install the dnscrypt-proxy package and then set it as the default resolver either in /etc/resolv.conf:

nameserver 127.0.2.1

if you are using a static network configuration or in /etc/dhcp/dhclient.conf:

supersede domain-name-servers 127.0.2.1;

if you rely on dynamic network configuration via DHCP.

There are two things you might want to keep in mind when choosing your DNSCrypt resolver:

  • whether or not they keep any logs of the DNS traffic
  • whether or not they support DNSSEC

I have personally selected a resolver located in Iceland by setting the following in /etc/default/dnscrypt-proxy:

DNSCRYPT_PROXY_RESOLVER_NAME=ns0.dnscrypt.is

DNSSEC

While DNSCrypt protects the confidentiality of our DNS queries, it doesn't give us any assurance that the results of such queries are the right ones. In order to authenticate results in that way and prevent DNS poisoning, a hierarchical cryptographic system was created: DNSSEC.

In order to enable it, I have setup a local unbound DNSSEC resolver on my machine and pointed /etc/resolv.conf (or /etc/dhcp/dhclient.conf) to my unbound installation at 127.0.0.1.

Then I put the following in /etc/unbound/unbound.conf.d/dnscrypt.conf:

server:
    # Remove localhost from the donotquery list
    do-not-query-localhost: no

forward-zone:
    name: "."
    forward-addr: 127.0.2.1@53

to stop unbound from resolving DNS directly and to instead go through the encrypted DNSCrypt proxy.

Reliability

In my experience, unbound and dnscrypt-proxy are fairly reliable but they eventually get confused (presumably) by network changes and start returning errors.

The ugly but dependable work-around I have found is to create a cronjob at /etc/cron.d/restart-dns.conf that restarts both services once a day:

0 3 * * *    root    /usr/sbin/service dnscrypt-proxy restart
1 3 * * *    root    /usr/sbin/service unbound restart

Captive portals

The one remaining problem I need to solve has to do with captive portals. This can be quite annoying when travelling because it requires me to use the portal's DNS resolver in order to connect to the splash screen that unlocks the wifi connection.

The dnssec-trigger package looked promising but when I tried it on my jessie laptop, it wasn't particularly reliable.

My temporary work-around is to comment out this line in /etc/dhcp/dhclient.conf whenever I need to connect to such annoying wifi networks:

#supersede domain-name-servers 127.0.0.1;

If you've found a better solution to this problem, please leave a comment!

Planet DebianDirk Eddelbuettel: RcppMgsPack 0.1.0

Over the last few months, I have been working casually on a new package to integrate MessagePack with R. What is MessagePack, you ask? To quote its website, "It's like JSON, but fast and small."

Or in more extended terms:

MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.

Now, serialization formats are a dime a dozen: JSON, BSON, Protocol Buffers, CaptnProto, Flatbuffer. The list goes on and on. So why another? In a nutshell: "software ecosystems".

I happen to like working with Redis, and within the world of Redis, MessagePack is a first-class citizen supported by things close to the core like the embedded Lua interpreter, as well as fancy external add-ons such as the Redis Desktop Manager GUI. So nothing overly fundamentalist here, but a fairly pragmatic choice based on what happens to fit my needs. Plus, having worked on and off with Protocol Buffers for close to a decade, the chance of working with something not requiring a friggin' schema compiler seemed appealing for a chance.

So far, we have been encoding a bunch of data streams at work via MessagePack into Redis (and of course back). It works really well---header-only C++11 libraries for the win. I'll provide an updated RcppRedis which uses this (if present) in due course.

For now and the foreseeable future, this RcppMsgPack package will live only on the ghrr drat repository. To make RcppMsgPack work, I currently have to include the MessagePack 1.4.0 headers. A matching package for this version of the headers is in Debian but so far only in experimental. Once this hits the mainline repository I can depend on it, and upload a (lighter, smaller) RcppMsgPack to CRAN.

Until then, please just do

## install drat if not present
if (!require(drat)) install.packages("drat")

## use drat to select ghrr repo
drat::addRepo("ghrr")

## install RcppMsgPack
install.packages("RcppMsgPack")

More details, issue tickets etc are at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

,

CryptogramGraffiti by Drone

Drones can graffiti walls that no person can reach.

(Note that wired.com blocks ad blockers. My trick is to copy the page and then paste it into my text editor.)

Planet DebianGunnar Wolf: Passover / Pesaj, a secular viewpoint, a different viewpoint... And slowly becoming history!

As many of you know (where "you" is "people reading this who actually know who I am), I come from a secular Jewish family. Although we have some religious (even very religious) relatives, neither my parents nor my grandparents were religious ever. Not that spirituality wasn't important to them — My grandparents both went deep into understanding by and for themselves the different spiritual issues that came to their mind, and that's one of the traits I most remember about them while I was growing up. But formal, organized religion was never much welcome in the family; again, each of us had their own ways to concile our needs and fears with what we thought, read and understood.

This week is the Jewish celebration of Passover, or Pesaj as we call it (for which Passover is a direct translation, as Pesaj refers to the act of the angel of death passing over the houses of the sons of Israel during the tenth plague in Egypt; in Spanish, the name would be Pascua, which rather refers to the ritual sacrifice of a lamb that was done in the days of the great temple)... Anyway, I like giving context to what I write, but it always takes me off the main topic I want to share. Back to my family.

I am a third-generation member of the Hashomer Hatzair zionist socialist youth movement; my grandmother was among the early Hashomer Hatzair members in Poland in the 1920s, both my parents were active in the Mexico ken in the 1950s-1960s (in fact, they met and first interacted there), and I was a member from 1984 until 1996. It was also thanks to Hashomer that my wife and I met, and if my children get to have any kind of Jewish contact in their lifes, I hope it will be through Hashomer as well.

Hashomer is a secular, nationalist movement. A youth movement with over a century of history might seem like a contradiction. Over the years, of course, it has changed many details, but as far as I know, the essence is still there, and I hope it will continue to be so for good: Helping shape integral people, with identification with Judaism as a nation and not as a religion; keeping our cultural traits, but interpreting them liberally, and aligned with a view towards the common good — Socialism, no matter how the concept seems passé nowadays. Colectivism. Inclusion. Peaceful coexistence with our neighbours. Acceptance of the different. I could write pages on how I learnt about each of them during my years in Hashomer, how such concepts striked me as completely different as what the broader Jewish community I grew up in understood and related to them... But again, I am steering off the topic I want to pursue.

Every year, we used to have a third Seder (that is, a third Passover ceremony) at Hashomer. A third one, because as tradition mandates two ceremonies to be held outside Israel, and a movement comprised of people aged between 7 and 21, having a seder competing with the familiar one would not be too successful, we held a celebration on a following day. But it would never be the same as the "formal" Pesaj: For the Seder, the Jewish tradition mandates following the Hagada — The Seder always follows a predetermined order (literally, Seder means order), and the Hagadá (which means both legend and a story that is spoken; you can find full Hagadot online if you want to see what rites are followed; I found a seemingly well done, modern, Hebrew and English version, a more traditional one, in Hebrew and Spanish, and Wikipedia has a description including its parts and rites) is, quite understandably, full with religious words, praises for God, and... Well, many things that are not in line with Hashomer's values. How could we be a secular movement and have a big celebration full with praises for God? How could we yearn for life in the kibbutz distance from the true agricultural meaning of the celebration?

The members of Hashomer Hatzair repeatedly took on the task (or, as many would see it, the heresy) of adapting the Hagada to follow their worldview, updated it for the twentieth century, had it more palatable for our peculiarities. Yesterday, when we had our Seder, I saw my father still has –together with the other, more traditional Hagadot we use– two copies of the Hagadá he used at Hashomer Hatzair's third Seder. And they are not only beautiful works showing what they, as very young activists thought and made solemn, but over time, they are becoming historic items by themselves (one when my parents were still young janijim, in 1956, and one when they were starting to have responsabilities and were non-formal teachers or path-showers, madrijim, in 1959). He also had a copy of the Hagadá we used in the 1980s when I was at Hashomer; this last one was (sadly?) not done by us as members of Hashomer, but prepared by a larger group between Hashomer Hatzair and the Mexican friends of Israeli's associated left wing party, Mapam. This last one, I don't know which year it was prepared and published on, but I remember following it in our ceremony.

So, I asked him to borrow me the three little books, almost leaflets, and scanned them to be put online. Of course, there is no formal licensing information in them, much less explicit authorship information, but they are meant to be shared — So I took the liberty of uploading them to the Internet Archive, tagging them as CC-0 licensed. And if you are interested in them, flowing over and back between Spanish and Hebrew, with many beautiful texts adapted for them from various sources, illustrated by our own with the usual heroic, socialist-inspired style, and lovingly hand-reproduced using the adequate technology for their day... Here they are:

I really enjoyed the time I took scanning and forming them, reading some passages, imagining ourselves and my parents as youngsters, remembering the beautiful work we did at such a great organization. I hope this brings this joy to others like it did to me.

פעם שומר, תמיד שומר. Once shomer, always shomer.

Sociological ImagesThe Decline of Image and the Rise of the Brand

One word in the headlines last week seemed like a throwback to an earlier era:

As Trump moves to soften his image, Democrats seek to harden it

The Washington Post

Donald Trump to reshape image, new campaign chief tells G.O.P.

The New York Times

Trump surrogates say GOP front-runner “projecting an image” during primaries

— Fox News

It was in the 1960s that politicians, their handlers, and the people who write about them discovered image. The word carries the cynical implication that voters, like shoppers, respond to the surface image rather than the substance – the picture on the box rather than what’s inside.  A presidential campaign was based on the same thing as an advertising campaign – image.  You sold a candidate the same way you sold cigarettes, at least according to the title and book jacket of Joe McGinnis’s book.

Then, sometime around 1980, image began to fade. In its place we now have brand. I went to Google N-grams and looked at the ratio of image to brand in both the corporate and the political realm. The pattern is nearly identical.


The ratio rises steeply from 1960 to 1980 – lots more talk about image, no increase in brand. Then the trend reverses. Sightings of image were still rising, but nowhere nearly as rapidly as brand, which doubled from 1980 to 2000 in politics and quadrupled in the corporate world.

Image sounds too deceptive and manipulative; you can change it quickly according to the needs of the moment. Brand implies permanence and substance (not to mention Marlboro-man-like rugged independence and integrity.) No wonder people in the biz prefer brand.

Decades ago, when my son was in grade school, I met another parent who worked in the general area of public relations. On seeing him at the next school function a few weeks later, I said, “Oh right, you work in corporate image-mongering.” I thought I said it jokingly, but he seemed offended. He was, I quickly learned, a brand consultant. Image bad; brand good.

In later communications, he also said that a company’s attempt to brand itself as something it’s not will inevitably fail.  The same thing supposedly goes for politics:

“One thing you learn very quickly in political consulting is the fruitlessness of trying to get a candidate to change who he or she fundamentally is at their core,” said Republican strategist Whit Ayres, who did polling for Rubio’s presidential campaign before he dropped out of the race. “So, is the snide, insulting, misogynistic guy we’ve seen really who Donald Trump is? Or is it the disciplined, respectful, unifying Trump we saw for seven minutes after the New York primary?

These consultants are saying what another Republican said a century and a half ago: “You can fool all the people some of the time, and some of the people all the time, but you cannot fool all the people all the time.”

This seems to argue that political image-mongers have to be honest about who their candidate really is. But there’s another way of reading Lincoln’s famous line: You only need to fool half the people every four years.

Originally posted at Montclair SocioBlog.

———————

Jay Livingston is the chair of the Sociology Department at Montclair State University. You can follow him at Montclair SocioBlog or on Twitter.

(View original at https://thesocietypages.org/socimages)

CryptogramBlackBerry's Global Encryption Key

Last week, there was a big news story about the BlackBerry encryption key. The news was that all BlackBerry devices share a global encryption key, and that the Canadian RCMP has a copy of it. Stupid design, certainly, but it's not news. As the Register points out, this has been repeatedly reported on since 2010.

And note that this only holds for a individual users. If your organization uses a BlackBerry Enterprise Server (BES), you have your own unique key.

Worse Than FailureMercy the Mercenary in… The App Store

We return with the penultimate installment of the tale of Mercy, the Mercenary Developer. Last time, she implemented a countdown clock- but nobody told her what it was counting down to, because nobody knew.

It was standing-room only at Rockwood for Governor campaign headquarters. All the tables had been pushed to the walls or folded and stowed away; most of the chairs were stacked. Volunteers milled about, eating delivery pizza, wings, and (probably spiked) soda.

Mercy and Sullivan, two of the few people who got chairs, sat close to the widescreen TV set up near the phone banks. It was tuned to a local network, broadcasting a feed from USF, a couple hours down I–4 from headquarters.

The first Florida gubernatorial debate had started merely five minutes ago, and already things were testy.

someone using an app on their smartphone

“This is a question from Facebook,” said Carson Cooper, the moderator. “It says: ‘I am concerned that Florida will keep municipal broadband and Google Fiber from taking root in the state, while big cable interests operate without competition. How would you address the need for better practices in the cable industry?’ Packard, we’ll start with you.”

Packard gave his response. “I think we ought to protect the interests of American companies, Carson. And that means standing behind our friends in the telecom industry, and keeping government out of the internet business!”

“Rockwood, your response?”

“We briefed him on this,” Sullivan whispered to Mercy. “He just has to say his lines.”

Their candidate gave his most confident smile. “I don’t know if Packard has ever had to look at a cable bill in his life.” Laughter from the volunteers. “My real-estate clients have all told me how expensive it is to get cable, to get internet. The gigs are too much, Carson, too much.”

“Oh, no,” Sullivan whispered. “It was ‘the fees,’ not the ‘gigs.’ He sounds like a roadie.”

Mercy cringed. “He should have just said cable internet is too expensive.”

The democrat, Hewlett, waffled through a non-response. After he finished, Packard focused his attention on Rockwood. “For a real estate developer who’s made his living flipping houses, you suddenly know a whole lot about how the telecom industry works.” The TV audience laughed, but the boos from the volunteers drowned it out. “Tell me one thing, Mr. Rockwood. What is one forward-looking innovation that you, your company, or your campaign has launched? Because I can name at least a dozen at the pharmaceutical firm I founded.”

“I agree,” said Hewlett, hoping to get any kind of airtime between Rockwood and Packard.

Rockwood got that glassy look in his eye again, which Mercy had noticed more often since the “countdown clock” incident a few weeks ago. There was something off about Rockwood whenever he got that look. It lasted three whole seconds; next to her, Sullivan had balled her hands into fists.

“Rockwood, your response?” Carson Cooper said.

Rockwood snapped out of it. “I’ll tell you something, Packard. Now I’m sure you’ve made some big improvements to whatever diet pills you manufacture in big pharma, but here’s something that will affect all Floridians. You know how hard it is to understand tax code? I don’t know if you’ve ever seen a return in your life, I mean with your own eyes. Well, our campaign has laid out a revised tax code for Floridians. Lower sales taxes for everyday goods. Fair and proportional property taxes. And a non-income profit tax that only affects the wealthiest Floridians … which includes you, I’m not sorry to say. Tomorrow, we’re launching a new mobile application that tells you exactly how much more you’ll owe. And for most Floridians, that number will be zero.”

The volunteers applauded, drowning out Carson Cooper’s voice. Meanwhile, Mercy’s stomach did a somersault. “A tax code app? Tomorrow?”

“I do believe that’s the first time he’s mentioned it,” Sullivan replied.

“Did he just make that up on the spot?”

“Well, it can’t be that hard, can it? Just like building a website.” Sullivan faked a gracious smile.

“You have no idea.”

Built != Launched

“This is how hard it is to launch a new mobile app in a day,” Mercy said. She, Ellis, and Sullivan had gathered after the debate in the conference room. Rockwood was off to St. Petersburg and Tampa for some barnstorming after his performance, which the media had characterized as “fresh” and “unpredictable,” not terms they used in a complimentary fashion. “I could do it in a week. I can’t do it in 24 hours.”

“We don’t have a week,” Ellis said. “John said we’d have it up in 24 hours. That’s what we have to do.”

“You know what, Ellis,“ Sullivan snapped, ”Maybe you should let the one who’s actually going to build the mobile app tell us how it can be done.”

“Well, I’ll go get Rockwood as soon as his stump speech is done and I’ll tell him. You’ll see how he likes that kind of news.”

“Look,” Mercy held up her hands, “do you want to hear what I have to say or not?”

Ellis and Sullivan quit squabbling.

“Here’s what has to happen. I need to build it first. I can use PhoneGap — it’s a framework that uses HTML, so it’s like a web page inside an application. I’ll need the campaign’s tax reform policy written up.”

“Uh, we don’t have that yet,” Ellis said.

“Do you need it?” Sullivan asked. “Could you just build the application and put it in later?”

“It’s a tax policy application,” Mercy said, her voice cracking. “There’s no application without knowing how the new taxes will work. That’s the whole point.”

“I’ll get something tomorrow morning,” Ellis conceded.

“I can build most of it out tonight,” Mercy said. She reminded herself to buy some energy drinks on the way back to her apartment. “Tomorrow I’ll make changes to the algorithm based on our new tax policy. We won’t have much time for testing, so I’ll get some volunteers to help.”

“Great, we can get it built by tomorrow afternoon,” Ellis said. “It’ll be done in 24 hours.”

“No, it’ll be built in 24 hours,” Mercy said. “It won’t be live in 24 hours. It still has to get on the Apple store and Google Play.”

“So, can’t we just make an account and put it up?” Sullivan asked.

“Apple takes at least a week to approve an app. If you’re lucky, and we’ll be a new publisher with no prior releases. They’ll drag their feet.”

“Okay, so we make some phone calls,” Ellis said. “Can’t be that hard.”

“Have you ever tried to call Apple? I mean, not just tech support.”

“Look.” Ellis steepled his hands on the table. “It’s going to happen. You’re going to find a way to do it. End of story.” Without even a dismissal, Ellis left the room.

“What’s his deal? Was it the debate?” Mercy cracked her knuckles. Her wrists already hurt from the arduous typing they’d endure tonight.

“I can’t talk about that, honey,” Sullivan said, leaving the conference room.

Tax Code

By 4AM she had a working prototype of the application. It took a user’s gross income, property values, etc., and spit out a dollar amount, based on what she knew about Florida tax law. Mercy figured the campaign’s tax policy wouldn’t be very complicated, either adjusting individual percentages on specific values, fiddling with the base percentage, or some combination thereof. She headed back to the office to be ready for when Ellis could get her their tax policy platform language. She took a nap, sitting next to the web server hamilton.

Around 8AM, a hand shook her shoulder. It was Ellis. He shoved a piece of paper in front of her. “Here’s what we came up with a few weeks ago.”

It was a 17-circle Venn diagram. Arrows were drawn from circle to circle around the circumference, making it look like a giant recycling symbol. Houses, boats, and other property icons dotted the diagram. There were no numbers anywhere on it.

Mercy said as much to Ellis.

“Look,” Ellis said, “The actual property rates and percentages won’t actually change. Just the way they’re calculated. Just make it look like it’s saving our constituents money. The actual tax laws will get worked out in the legislature after Rockwood’s elected.”

Mercy rubbed her eyes. She started redrawing the Venn chain link into an algorithm flow chart, then crumpled that up, adjusted a few values in her own algorithm, and prepped for QA.

Failure to Launch

Mercy was struggling to keep awake as two volunteers, a brother and sister attending UCF, tested on an iPhone and an Android tablet respectively.

“Whoa, hey, those boxes showed up again,” the brother said. He handed Mercy his iPhone.

“It’s the stupid glyphicon font. I must have moved the files by mistake.” Mercy dug through her codebase, double-checked the font locations and her font-face CSS declarations, and recompiled. She pushed the new package to cato, the intranet server for the campaign, and brother and sister re-installed their apps respectively.

“Looks good on mine,” the sister said. The brother nodded in agreement.

“Okay, thanks.” Mercy yawned. The app was mostly bug-free — or as bug-free as she could make it in just over 12 hours — and ready to ship. While she had been finishing the app, Sullivan had set up accounts on Google Play and the Apple Store. Mercy logged into the Google Play account, uploaded the Android installer, and hit “Publish.” That would be the easy part.

But getting Apple to play along would be harder.

She put the iOS installer into the campaign account, then tagged it for review. An email came back, with just what she expected: it would take nearly a week for Apple to respond.

Meanwhile, Rockwood was coming back from his barnstorming in Tampa and St. Petersburg, and would want to see the app in action when he arrived around 6PM. Six hours left.

Against her better nature, Mercy scoured her LinkedIn connections. She had made numerous contacts at local conferences. Surely someone worked at Apple, someone who could pull a few strings in the review process. As she searched, her eyes began to close. She felt as though she were falling out of her chair, and she jolted awake.

There wouldn’t be anyone she knew, and besides, she didn’t have the money to bribe anyone to speed up the process. She headed for the conference room. “Wake me in a couple hours,” she told Sullivan. Inside, she rolled up a sweater for a pillow, locked the door, shut off the light, and went to sleep.

Beta Release

Sullivan’s knocking woke Mercy. She checked her phone. It was 5PM.

“When’s he getting here?” she said, her voice muffled.

“He’s on his way from the airport,” she replied. “Ellis has already called and said you couldn’t get the app finished.”

Mercy groaned. “It’s done, it’s just not on the Apple Store—“

Mercy realized it didn’t need to be. A few iPhone and iPad owners unlocked their devices, allowing third-party apps to run on them. She could just put a “beta” of the application for download on their site. Only jailbroken devices could run it, but she could say she released the app in the 24-hour window Rockwood wanted.

She dragged herself out of the conference room, head pounding from an enormous caffeine withdrawal. She slumped in front of hamilton, logged onto the campaign blog, and wrote a post. “Download Rockwood’s Tax Code App Here!” the post said. At the bottom was a link to the iOS installer, which she had copied to the public servers. She made it clear it was a beta release, and that as soon as it hit the iOS store it would no longer be available. But it was published, and just in time.

Sullivan met Rockwood as he came in the door, before Ellis could get a chance to talk to him. “The tax app is up on the web!” she said. “We got it done.”

“Taxes? I released my tax returns months ago,” he said, chuckling.

Ellis sandwiched himself between Sullivan and Rockwood. “Let’s get you situated,” he said, ushering the candidate through a few loitering volunteers. Rockwood put on his best smile, but seemed like he didn’t know where he was.

“I’m gonna sleep for three days,” Mercy said to Sullivan, as she left headquarters. But Sullivan followed her out to the parking lot.

Sullivan began, “I should really tell you–“

Mercy said, her voice short, “I don’t have time–“

“–John has non-Alzheimer’s dementia.” Sullivan’s usual southern tea-sweet tone was gone. “He was diagnosed just after he announced his campaign. I first noticed the symptoms a couple years ago. He’s lucid most of the time, but he’s having a lot more episodes recently. Ellis and I are basically running things until John gets better. You’re the third person to find out.”

“After you and Ellis?”

Sullivan nodded. “Please keep this to yourself. It’ll kill his chances if—“

“I can’t deal with this right now,” Mercy said. “I’m taking the next few days off.” She got in her old Honda and drove home. When she got there, she turned off her phone, covered herself in three blankets, curled up, and slept.

[Advertisement] Otter, ProGet, BuildMaster – robust, powerful, scalable, and reliable additions to your existing DevOps toolchain.

Worse Than FailureMercy the Mercenary in… The App Store

We return with the penultimate installment of the tale of Mercy, the Mercenary Developer. Last time, she implemented a countdown clock- but nobody told her what it was counting down to, because nobody knew.

It was standing-room only at Rockwood for Governor campaign headquarters. All the tables had been pushed to the walls or folded and stowed away; most of the chairs were stacked. Volunteers milled about, eating delivery pizza, wings, and (probably spiked) soda.

Mercy and Sullivan, two of the few people who got chairs, sat close to the widescreen TV set up near the phone banks. It was tuned to a local network, broadcasting a feed from USF, a couple hours down I–4 from headquarters.

The first Florida gubernatorial debate had started merely five minutes ago, and already things were testy.

someone using an app on their smartphone

“This is a question from Facebook,” said Carson Cooper, the moderator. “It says: ‘I am concerned that Florida will keep municipal broadband and Google Fiber from taking root in the state, while big cable interests operate without competition. How would you address the need for better practices in the cable industry?’ Packard, we’ll start with you.”

Packard gave his response. “I think we ought to protect the interests of American companies, Carson. And that means standing behind our friends in the telecom industry, and keeping government out of the internet business!”

“Rockwood, your response?”

“We briefed him on this,” Sullivan whispered to Mercy. “He just has to say his lines.”

Their candidate gave his most confident smile. “I don’t know if Packard has ever had to look at a cable bill in his life.” Laughter from the volunteers. “My real-estate clients have all told me how expensive it is to get cable, to get internet. The gigs are too much, Carson, too much.”

“Oh, no,” Sullivan whispered. “It was ‘the fees,’ not the ‘gigs.’ He sounds like a roadie.”

Mercy cringed. “He should have just said cable internet is too expensive.”

The democrat, Hewlett, waffled through a non-response. After he finished, Packard focused his attention on Rockwood. “For a real estate developer who’s made his living flipping houses, you suddenly know a whole lot about how the telecom industry works.” The TV audience laughed, but the boos from the volunteers drowned it out. “Tell me one thing, Mr. Rockwood. What is one forward-looking innovation that you, your company, or your campaign has launched? Because I can name at least a dozen at the pharmaceutical firm I founded.”

“I agree,” said Hewlett, hoping to get any kind of airtime between Rockwood and Packard.

Rockwood got that glassy look in his eye again, which Mercy had noticed more often since the “countdown clock” incident a few weeks ago. There was something off about Rockwood whenever he got that look. It lasted three whole seconds; next to her, Sullivan had balled her hands into fists.

“Rockwood, your response?” Carson Cooper said.

Rockwood snapped out of it. “I’ll tell you something, Packard. Now I’m sure you’ve made some big improvements to whatever diet pills you manufacture in big pharma, but here’s something that will affect all Floridians. You know how hard it is to understand tax code? I don’t know if you’ve ever seen a return in your life, I mean with your own eyes. Well, our campaign has laid out a revised tax code for Floridians. Lower sales taxes for everyday goods. Fair and proportional property taxes. And a non-income profit tax that only affects the wealthiest Floridians … which includes you, I’m not sorry to say. Tomorrow, we’re launching a new mobile application that tells you exactly how much more you’ll owe. And for most Floridians, that number will be zero.”

The volunteers applauded, drowning out Carson Cooper’s voice. Meanwhile, Mercy’s stomach did a somersault. “A tax code app? Tomorrow?”

“I do believe that’s the first time he’s mentioned it,” Sullivan replied.

“Did he just make that up on the spot?”

“Well, it can’t be that hard, can it? Just like building a website.” Sullivan faked a gracious smile.

“You have no idea.”

Built != Launched

“This is how hard it is to launch a new mobile app in a day,” Mercy said. She, Ellis, and Sullivan had gathered after the debate in the conference room. Rockwood was off to St. Petersburg and Tampa for some barnstorming after his performance, which the media had characterized as “fresh” and “unpredictable,” not terms they used in a complimentary fashion. “I could do it in a week. I can’t do it in 24 hours.”

“We don’t have a week,” Ellis said. “John said we’d have it up in 24 hours. That’s what we have to do.”

“You know what, Ellis,“ Sullivan snapped, ”Maybe you should let the one who’s actually going to build the mobile app tell us how it can be done.”

“Well, I’ll go get Rockwood as soon as his stump speech is done and I’ll tell him. You’ll see how he likes that kind of news.”

“Look,” Mercy held up her hands, “do you want to hear what I have to say or not?”

Ellis and Sullivan quit squabbling.

“Here’s what has to happen. I need to build it first. I can use PhoneGap — it’s a framework that uses HTML, so it’s like a web page inside an application. I’ll need the campaign’s tax reform policy written up.”

“Uh, we don’t have that yet,” Ellis said.

“Do you need it?” Sullivan asked. “Could you just build the application and put it in later?”

“It’s a tax policy application,” Mercy said, her voice cracking. “There’s no application without knowing how the new taxes will work. That’s the whole point.”

“I’ll get something tomorrow morning,” Ellis conceded.

“I can build most of it out tonight,” Mercy said. She reminded herself to buy some energy drinks on the way back to her apartment. “Tomorrow I’ll make changes to the algorithm based on our new tax policy. We won’t have much time for testing, so I’ll get some volunteers to help.”

“Great, we can get it built by tomorrow afternoon,” Ellis said. “It’ll be done in 24 hours.”

“No, it’ll be built in 24 hours,” Mercy said. “It won’t be live in 24 hours. It still has to get on the Apple store and Google Play.”

“So, can’t we just make an account and put it up?” Sullivan asked.

“Apple takes at least a week to approve an app. If you’re lucky, and we’ll be a new publisher with no prior releases. They’ll drag their feet.”

“Okay, so we make some phone calls,” Ellis said. “Can’t be that hard.”

“Have you ever tried to call Apple? I mean, not just tech support.”

“Look.” Ellis steepled his hands on the table. “It’s going to happen. You’re going to find a way to do it. End of story.” Without even a dismissal, Ellis left the room.

“What’s his deal? Was it the debate?” Mercy cracked her knuckles. Her wrists already hurt from the arduous typing they’d endure tonight.

“I can’t talk about that, honey,” Sullivan said, leaving the conference room.

Tax Code

By 4AM she had a working prototype of the application. It took a user’s gross income, property values, etc., and spit out a dollar amount, based on what she knew about Florida tax law. Mercy figured the campaign’s tax policy wouldn’t be very complicated, either adjusting individual percentages on specific values, fiddling with the base percentage, or some combination thereof. She headed back to the office to be ready for when Ellis could get her their tax policy platform language. She took a nap, sitting next to the web server hamilton.

Around 8AM, a hand shook her shoulder. It was Ellis. He shoved a piece of paper in front of her. “Here’s what we came up with a few weeks ago.”

It was a 17-circle Venn diagram. Arrows were drawn from circle to circle around the circumference, making it look like a giant recycling symbol. Houses, boats, and other property icons dotted the diagram. There were no numbers anywhere on it.

Mercy said as much to Ellis.

“Look,” Ellis said, “The actual property rates and percentages won’t actually change. Just the way they’re calculated. Just make it look like it’s saving our constituents money. The actual tax laws will get worked out in the legislature after Rockwood’s elected.”

Mercy rubbed her eyes. She started redrawing the Venn chain link into an algorithm flow chart, then crumpled that up, adjusted a few values in her own algorithm, and prepped for QA.

Failure to Launch

Mercy was struggling to keep awake as two volunteers, a brother and sister attending UCF, tested on an iPhone and an Android tablet respectively.

“Whoa, hey, those boxes showed up again,” the brother said. He handed Mercy his iPhone.

“It’s the stupid glyphicon font. I must have moved the files by mistake.” Mercy dug through her codebase, double-checked the font locations and her font-face CSS declarations, and recompiled. She pushed the new package to cato, the intranet server for the campaign, and brother and sister re-installed their apps respectively.

“Looks good on mine,” the sister said. The brother nodded in agreement.

“Okay, thanks.” Mercy yawned. The app was mostly bug-free — or as bug-free as she could make it in just over 12 hours — and ready to ship. While she had been finishing the app, Sullivan had set up accounts on Google Play and the Apple Store. Mercy logged into the Google Play account, uploaded the Android installer, and hit “Publish.” That would be the easy part.

But getting Apple to play along would be harder.

She put the iOS installer into the campaign account, then tagged it for review. An email came back, with just what she expected: it would take nearly a week for Apple to respond.

Meanwhile, Rockwood was coming back from his barnstorming in Tampa and St. Petersburg, and would want to see the app in action when he arrived around 6PM. Six hours left.

Against her better nature, Mercy scoured her LinkedIn connections. She had made numerous contacts at local conferences. Surely someone worked at Apple, someone who could pull a few strings in the review process. As she searched, her eyes began to close. She felt as though she were falling out of her chair, and she jolted awake.

There wouldn’t be anyone she knew, and besides, she didn’t have the money to bribe anyone to speed up the process. She headed for the conference room. “Wake me in a couple hours,” she told Sullivan. Inside, she rolled up a sweater for a pillow, locked the door, shut off the light, and went to sleep.

Beta Release

Sullivan’s knocking woke Mercy. She checked her phone. It was 5PM.

“When’s he getting here?” she said, her voice muffled.

“He’s on his way from the airport,” she replied. “Ellis has already called and said you couldn’t get the app finished.”

Mercy groaned. “It’s done, it’s just not on the Apple Store—“

Mercy realized it didn’t need to be. A few iPhone and iPad owners unlocked their devices, allowing third-party apps to run on them. She could just put a “beta” of the application for download on their site. Only jailbroken devices could run it, but she could say she released the app in the 24-hour window Rockwood wanted.

She dragged herself out of the conference room, head pounding from an enormous caffeine withdrawal. She slumped in front of hamilton, logged onto the campaign blog, and wrote a post. “Download Rockwood’s Tax Code App Here!” the post said. At the bottom was a link to the iOS installer, which she had copied to the public servers. She made it clear it was a beta release, and that as soon as it hit the iOS store it would no longer be available. But it was published, and just in time.

Sullivan met Rockwood as he came in the door, before Ellis could get a chance to talk to him. “The tax app is up on the web!” she said. “We got it done.”

“Taxes? I released my tax returns months ago,” he said, chuckling.

Ellis sandwiched himself between Sullivan and Rockwood. “Let’s get you situated,” he said, ushering the candidate through a few loitering volunteers. Rockwood put on his best smile, but seemed like he didn’t know where he was.

“I’m gonna sleep for three days,” Mercy said to Sullivan, as she left headquarters. But Sullivan followed her out to the parking lot.

Sullivan began, “I should really tell you–“

Mercy said, her voice short, “I don’t have time–“

“–John has non-Alzheimer’s dementia.” Sullivan’s usual southern tea-sweet tone was gone. “He was diagnosed just after he announced his campaign. I first noticed the symptoms a couple years ago. He’s lucid most of the time, but he’s having a lot more episodes recently. Ellis and I are basically running things until John gets better. You’re the third person to find out.”

“After you and Ellis?”

Sullivan nodded. “Please keep this to yourself. It’ll kill his chances if—“

“I can’t deal with this right now,” Mercy said. “I’m taking the next few days off.” She got in her old Honda and drove home. When she got there, she turned off her phone, covered herself in three blankets, curled up, and slept.

[Advertisement] Otter, ProGet, BuildMaster – robust, powerful, scalable, and reliable additions to your existing DevOps toolchain.

Planet DebianRicardo Mones: Maximum number of clients reached Error: Can't open display: :0

Today it happened again: you try to open some program and nothing happens. Go to an open terminal, try again and it answers with the above message. Other days I used to reboot the session, but that's something I don't really think should be necessary.

First thought about X gone mad, but this one seems pretty well behaved:

$ lsof -p `pidof Xorg` | wc -l
5

Then noticed I had a long running chromium process (a jQuery page monitoring a remote service) so tried this one as well:

$ for a in `pidof chromium`; do echo "$a "`lsof -p $a | wc -l`; done
27914 5
26462 5
25350 5
24693 5
23378 5
22723 5
22165 5
21476 222
21474 1176
21443 5
21441 204
21435 546
11644 5
11626 5
11587 5
11461 5
11361 5
9833 5
9726 5

Wow, I'd bet you can guess next command ;-)

$ kill -9 21435 21441 21474 21476

This of course wiped out all chromium processes, but also fixed the problem. Suggestions for selective chromium killing welcome! But I'd better like to know why those files are not properly closed. Just relaunching chromium to write this post yields:

$ for a in `pidof chromium`; do echo "$a "`lsof -p $a | wc -l`; done
11919 5
11848 222
11841 432
11815 5
11813 204
11807 398

Which looks a bit exaggerated to me :-(

Planet DebianNorbert Preining: Gödel and Daemons – an excursion into literature

Explaining Gödel’s theorems to students is a pain. Period. How can those poor creatures crank their mind around a Completeness and an Incompleteness Proof… I understand that. But then, there are brave souls using Gödel’s theorems to explain the world of demons to writers, in particular to answer the question:

You can control a Demon by knowing its True Name, but why?

goedel-glabrezu

Very impressive.

Found at worldbuilding.stackexchange.com, pointed to me by a good friend. I dare to full quote author Cort Ammon (nothing more is known), to preserve this masterpiece. Thanks!!!!


Use of their name forces them to be aware of the one truth they can never know.

Tl/Dr: If demons seek permanent power but trust no one, they put themselves in a strange position where mathematical truisms paint them into a corner which leaves their soul small and frail holding all the strings. Use of their name suggests you might know how to tug at those strings and unravel them wholesale, from the inside out!

Being a demon is tough work. If you think facing down a 4000lb Glabrezu without their name is difficult, try keeping that much muscle in shape in the gym! Never mind how many manicurists you go through keeping the claws in shape!

I don’t know how creative such demons truly are, but the easy route towards the perfect French tip that can withstand the rigors of going to the gym and benching ten thousand pounds is magic. Such a demon might learn a manicure spell from the nearby resident succubi. However, such spells are often temporary. No demon worth their salt is going to admit in front of a hero that they need a moment to refresh their mani before they can fight. The hero would just laugh at them. No, if a demon is going to do something, they’re going to do it right, and permanently. Not just nice french tips with a clear lacquer over the top, but razor sharp claws that resharpen themselves if they are blunted and can extend or retract at will!

In fact, come to think of it, why even go to the gym to maintain one’s physique? Why not just cast a magic spell which permanently makes you into the glorious Hanz (or Franz) that the trainer keeps telling you is inside you, just waiting to break free. Just get the spell right once, and think of the savings you could have on gym memberships.

Demons that wish to become more powerful, permanently, must be careful. If fairy tales have anything to teach is, it’s that one of the most dangerous things you can do is wish for something forever, and have it granted. Forever is a very long time, and every spell has its price. The demon is going to have to make sure the price is not greater than the perks. It would be a real waste to have a manicure spell create the perfect claws, only to find that they come with a peculiar perchance to curve towards one’s own heart in an attempt to free themselves from the demon that cast them.

So we need proofs. We need proofs that each spell is a good idea, before we cast it. Then, once we cast it, we need proof that the spell actually worked intended. Otherwise, who knows if the next spell will layer on top perfectly or not. Mathematics to the rescue! The world of First Order Logic (FOL, or herefter simply “logic”) is designed to offer these guarantees. With a few strokes of a pen, pencil, or even brush, it can write down a set of symbols which prove, without a shadow of a doubt, that not only will the spell work as intended, but that the side effects are manageable. How? So long as the demon can prove that they can cast a negation spell to undo their previous spell, the permanency can be reverted by the demon. With a few more fancy symbols, the demon can also prove that nobody else outside of the demon can undo their permanency. It’s a simple thing for mathematics really. Mathematics has an amazing spell called reductio ad infinitum which does unbelievable things.

However, there is a catch. There is always a catch with magic, even when that magic is being done through mathematics. In 1931, Kurt Gödel published his Incompleteness Theorems. These are 3 fascinating works of mathematical art which invoke the true names of First Order Logic and Set Theory. Gödel was able to prove that any system which is powerful enough to prove out all of algebra (1 + 1 = 2, 2 + 1 = 3, 3 * 5 = 15, etc.), could not prove its own validity. The self referential nature of proving itself crossed a line that First Order Logic simply could not return from. He proved that any system which tries must pick up one of these five traits:

  • Incomplete – they missed a detail when trying to prove everything
  • Incorrect – They got everything, but at least one point is wrong
  • Unprovable – They might be right, but they can never prove it
  • Intractable – If you’re willing to sit down and write down a proof that takes longer than eternity, you can prove a lot. Proofs that fit into eternity have limits.
  • Illogical – Throw logic to the wind, and you can prove anything!

If the demon wants itself to be able to cancel the spell, his proof is going to have to include his own abilities, creating just the kind of self referential effects needed to invoke Gödel’s incompleteness theorems. After a few thousand years, the demon may realize that this is folly.

A fascinating solution the demon might choose is to explore the “incomplete” solution to Gödel’s challenge. What if the demon permits the spell to change itself slightly, but in an unpredictable way. If the demon was a harddrive, perhaps he lets a single byte get changed by the spell in a way he cannot expect. This is actually enough to sidestep Gödel’s work, by introducing incompleteness. However, now we have to deal with pesky laws of physic and magics. We can’t just create something out of nothing, so if we’re going to let the spell change a single byte of us, there must be a single byte of information, its dual, that is unleashed into the world. Trying to break such conservation laws opens up a whole can of worms. Better to let that little bit go free into the world.

Well, almost. If you repeat this process a whole bunch of times, layering spells like a Matryoska doll, you’re eventually left with a “soul” that is nothing but the leftover bits of your spells that you simply don’t know enough about to use. If someone were collecting those bits and pieces, they might have the undoing of your entire self. You can’t prove it, of course, but its possible that those pieces that you sent out into the world have the keys to undo your many layers of armor, and then you know they are the bits that can nullify your soul if they get there. So what do you do? You hide them. You cast your spells only on the darkest of nights, deep in a cave where no one can see you. If you need assistants, you make sure to ritualistically slaughter them all, lest one of them know your secret and whisper it to a bundle of reeds, “The king has horns,” if you are familiar with the old fairy tale. Make it as hard as possible for the secret to escape, and hope that it withers away to nothingness before someone discovers it, leaving you invincible.

Now we come back to the name. The demon is going to have a name it uses to describe its whole self, including all of the layers of spellcraft it has acquired. This will be a great name like Abraxis, the Unbegotten Father or “Satan, lord of the underworld.” However, they also need to keep track of their smaller self, their soul. Failure to keep track of this might leave them open to an attack if they had missed a detail when casting their spells, and someone uncovered something to destroy them. This would be their true name, potentially something less pompous, like Gaylord Focker or Slartybartfarst. They would never use this name in company. Why draw attention to the only part of them that has the potential to be weak.

So when the hero calls out for Slartybartfarst, the demon truly must pay attention. If they know the name the demon has given over the remains of their tattered soul, might they know how to undo the demon entirely? Fear would grip their inner self, like a child, having to once again consider that they might be mortal. Surely they would wish to destroy the hero that spoke the name, but any attempt runs the risk of falling into a trap and exposing a weakness (surely their mind is racing, trying to enumerate all possible weaknesses they have). It is surely better for them to play along with you, once you use their true name, until they understand you well enough to confidently destroy you without destroying themselves.

So you ask for answers which are plausible. This one needs no magic at all. None of the rules are invalid in our world today. Granted finding a spell of perfect manicures might be difficult (believe me, some women have spent their whole life searching), but the rules are simply those of math. We can see this math in non-demonic parts of society as well. Consider encryption. An AES-256 key is so hard to brute force that it is currently believed it is impossible to break it without consuming 3/4 of the energy in the Milky Way Galaxy (no joke!). However, know the key, and decryption is easy. Worse, early implementations of AES took shortcuts. They actually left the signature of the path they took through the encryption in their accesses to memory. The caches on the CPU were like the reeds from the old fable. Merely observing how long it took to read data was sufficient to gather those reeds, make a flute, and play a song that unveils the encryption key (which is clearly either “The king has horns” or “1-2-3-4-5” depending on how secure you think your luggage combination is). Observing the true inner self of the AES encryption implementations was enough to completely dismantle them. Of course, not every implementation fell victim to this. You had to know the name of the implementation to determine which vulnerabilities it had, and how to strike at them.

Or, more literally, consider the work of Alfred Whitehead, Principia Mathematica. Principia Mathematica was to be a proof that you could prove all of the truths in arithmetic using purely procedural means. In Principia Mathematica, there was no manipulation based on semantics, everything he did was based on syntax — manipulating the actual symbols on the paper. Gödel’s Incompleteness Theorem caught Principia Mathematica by the tail, proving that its own rules were sufficient to demonstrate that it could never accomplish its goals. Principia Mathematica went down as the greatest Tower of Babel of modern mathematical history. Whitehead is no longer remembered for his mathematical work. He actually left the field of mathematics shortly afterwards, and became a philosopher and peace advocate, making a new name for himself there.

(by Cort Ammon)

,

Planet DebianBits from Debian: Debian welcomes its 2016 summer interns

GSoC 2016 logo Outreachy logo

We're excited to announce that Debian has selected 29 interns to work with us this summer: 4 in Outreachy, and 25 in the Google Summer of Code.

Here is the list of projects and the interns who will work on them:

Android SDK tools in Debian:

APT - dpkg communications rework:

Continuous Integration for Debian-Med packages:

Extending the Debian Developer Horizon:

Improving and extending AppRecommender:

Improving the debsources frontend:

Improving voice, video and chat communication with Free Software:

MIPS and MIPSEL ports improvements:

Reproducible Builds for Debian and Free Software:

Support for KLEE in Debile:

The Google Summer of Code and Outreachy programs are possible in Debian thanks to the effort of Debian developers and contributors that dedicate part of their free time to mentor students and outreach tasks.

Join us and help extend Debian! You can follow the students weekly reports on the debian-outreach mailing-list, chat with us on our IRC channel or on each project's team mailing lists.

Congratulations to all of them!

,

Chaotic IdealismFor the record...

I'm autistic and support vaccines. The risks are so low, the benefits so great, and large-scale participation is so crucial, that I believe vaccination should be enforced by law.

The risks are not zero, but many people seem to think that by avoiding vaccines, one can avoid the risks. In reality, one is choosing between two sets of risks: The risk of vaccinating nearly everyone, and the risk of disease that spreads when vaccination rates drop too low. The risks of not vaccinating are greater. If we let the vaccinated population go below the threshold where the disease can spread, we already know what will happen, because it happened before we had vaccines.

With the return of vaccine-preventable diseases would come an increase in childhood mortality. The youngest infants would be most vulnerable. Those with disabilities or chronic illnesses would die at high rates; children with asthma would die from diseases that attacked their lungs, cancer survivors with compromised immune systems would die because they could not fight the illnesses. And completely healthy children would die just because they caught a particularly bad strain of the illness. Poor children would die at higher rates than rich children because of higher rates of malnutrition and lower access to medical care.

Some people would die, but many more would sustain permanent injury from vaccine-preventable illness. Brain damage from measles encephalitis can range from mild to profound. Mumps can cause sterility. Polio can paralyze the muscles, and even years later those who were thought recovered can weaken again due to post-polio syndrome. Ironically, congenital rubella syndrome can cause autism along with other, deadlier, health problems. Even influenza can permanently damage the respiratory system.

Oh, and the economic effects--not as important, but still present. Children would miss school and fall behind; work hours would be lost when adults were sick or stayed home with sick children. People with disease-related disabilities would work less or not at all. Modern medicine would raise survival rates from vaccine-preventable illnesses higher than they were before vaccines, but medical care isn't free; when a child is hospitalized for whooping cough, their parents would have to pay the hospital bills, and if the parents couldn't pay, taxes would have to be raised to cover the cost. Even if you think to yourself, "Well, it's just money; money isn't as important as people," think of the things we'd have to skimp on to afford these essentially unnecessary medical costs: Schools. Libraries. Roads. Salaries for police, firefighters, teachers. Those are important things.

Those are the risks of not vaccinating, and they need to be balanced against the risks of vaccination. Healthy people should be vaccinated; those few percent who can remain unvaccinated without risking epidemics should be reserved for those who cannot be vaccinated or for whom vaccines would be ineffective. For me, the decision is easy because the risks of not enforcing vaccinations are so extreme compared to the risks of large-scale public vaccination programs.

Yet many people still fall for the lure of being perfectly safe from vaccine side effects. They are like people sitting in a rowboat who are so afraid of getting splinters that they jump out and into piranha-infested waters.

,

CryptogramFriday Squid Blogging: My Little Cephalopod

I assume this is more amusing to people who know about My Little Pony.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

CryptogramEncryption Backdoor Cartoons

Dilbert has a series: 1, 2, 3, 4, and 5.

SMBC.

And three more that make it clear this is a security vs. surveillance debate. Also this.

TEDGot a story you can’t share? Tell it, anonymously, to “Sincerely, X”

Sincerely_X audio podcast pilot logo

Think of a story from your life that you can’t tell anyone. Not because it reflects badly on you — in fact, you’re proud of the lessons you learned — but because of its effects on someone you love. A new-mom story that you’d never want your kid to hear. A high-stakes mistake on the job that made you a wiser person — but left lingering sadness and regret behind. Or perhaps you have an insider’s take on an exclusive milieu that few can imagine — and no one can talk about.

As a pilot for a new audio show, TED producers June Cohen and Deron Triff are looking for those stories. “Sincerely, X” will be a place where the ideas inside stories can come forward … without revealing the identity of the storyteller.

As Cohen says: “We’re not just looking for corporate whistleblowers or other traditional anonymous insiders — we also want to hear the personal lessons of learning and growth, the hard stories that create our worldview and make us who we are. For example, imagine a mom who faced and came through severe postpartum depression. She has a story, an idea, that could help other people — but she never wants her kid to know about her darkest thoughts as a new mom. How do we share her ideas with the world?”

The format of the audio show is carefully tailored to allow identities to be disguised, while the power of the story comes through.

TED is now looking for great ideas and stories to kick off the pilot. if you have one, use this form to submit it. All information will be carefully handled, as you might expect.


CryptogramCheating in Bicycle Races with Tiny Hidden Motors

If doping weren't enough, cyclists are cheating in races by hiding tiny motors in their bicycles. There are many detection techniques:

For its report, Stade 2 positioned a thermal imaging camera along the route of the Strade Bianche, an Italian professional men's race in March held mostly on unpaved roads and featuring many steep climbs. The rear hub of one bicycle glowed with almost the same vivid orange-yellow thermal imprint of the riders' legs. Engineers and antidoping experts interviewed by the TV program said the pattern could be explained only by heat generated by a motor. The rider was not named by the program and could not be identified from the thermal image.

[...]

Cycling's equivalents of the Zapruder film are online videos that show unusual patterns of bike changes that precede or follow exceptional bursts of speed by riders. Other videos analyze riders' hand movements for signs of switching on motors. Still other online analysts pore over crashes, looking for bikes on which the cranks keep turning after separation from the rider.

Unlike the thermal images, however, the videos have only implied that a motor was present.

In a statement, the cycling union, which commonly goes by its French initials, U.C.I., said it had tested and rejected thermal imaging.

"The U.C.I. has been testing for technological fraud for many years, and with the objective of increasing the efficiency of these tests, we have been trialling new methods of detection over the last year," the governing body said. "We have looked at thermal imaging, X-ray and ultrasonic testing, but by far the most cost-effective, reliable and accurate method has proved to be magnetic resonance testing using software we have created in partnership with a company of specialist developers."

Worse Than FailureError'd: Paula Lives

"While poking around on my brand new Galaxy S7, I was tempted to pick one ringtone in particular," writes Ronon D..

 

Jerry L. wrote, "Well, using 'comment' solves half the problem."

 

"Does this guy really show up at anything non-Hillary, or is this just the result of WaPo photo staff budget cuts?" asks Mark R.

 

Scott S. writes, "Is Microsoft trying to warn me that I won't get 'Jacked' up on this iced coffee? Or that the 'chikn' is a bit fowl?"

 

"Java may still have its little problems and misconceptions, but for this job, the duties are pretty low impact," wrote Billy B.

 

"While trying to register on Honeywell's support site, I came across an interesting conflict of abbreviations that is apparently driven by unique keys not being applied across multiple columns of the database," Phil S. writes, "But my registration was successful anyway?!"

 

"I hear it's just as good as the brand that inspired its name," Ergin writes, "In fact, it's got quite a kick! (just make sure you have your 'safe word' handy)"

 

[Advertisement] Infrastructure as Code built from the start with first-class Windows functionality and an intuitive, visual user interface. Download Otter today!

Worse Than FailureError'd: Paula Lives

"While poking around on my brand new Galaxy S7, I was tempted to pick one ringtone in particular," writes Ronon D..

 

Jerry L. wrote, "Well, using 'comment' solves half the problem."

 

"Does this guy really show up at anything non-Hillary, or is this just the result of WaPo photo staff budget cuts?" asks Mark R.

 

Scott S. writes, "Is Microsoft trying to warn me that I won't get 'Jacked' up on this iced coffee? Or that the 'chikn' is a bit fowl?"

 

"Java may still have its little problems and misconceptions, but for this job, the duties are pretty low impact," wrote Billy B.

 

"While trying to register on Honeywell's support site, I came across an interesting conflict of abbreviations that is apparently driven by unique keys not being applied across multiple columns of the database," Phil S. writes, "But my registration was successful anyway?!"

 

"I hear it's just as good as the brand that inspired its name," Ergin writes, "In fact, it's got quite a kick! (just make sure you have your 'safe word' handy)"

 

[Advertisement] Infrastructure as Code built from the start with first-class Windows functionality and an intuitive, visual user interface. Download Otter today!

,

Mark ShuttleworthY is for…

Yakkety yakkety yakkety yakkety yakkety yakkety yakkety yakkety yak. Naturally 🙂

Kelvin ThomsonChannel 9 Owes Public Explanation of Lebanon Conduct

Channel 9 must now provide what its 60 Minutes Program would require of anyone else involved in a debacle like the child abduction in Lebanon - a full accounting to the public of exactly what it did, what money it has paid or promised and to whom, both before and since the abduction, and a full accounting of which of its personnel decided on or approved the actions it carried out.

Channel 9 has embarrassed Australia and its diplomatic personnel, has put its own employees in danger, and has almost certainly been involved in a conspiracy to break the laws of another country. People are not entitled to take the law into their own hands, and two wrongs do not make a right.

60 Minutes seeks to shine a light in dark places and is strident about the public's right to know. On this occasion the public has a right to know exactly who in its organisation did what, and who is responsible for this.

TEDMotherhood without maternity leave, redefining “nude”, and a Pulitzer prize win.

Jessica_Shortall_clickable_blog

The TED community has lots of news this week. Below, just a few highlights.

Motherhood without maternity leave. Imagine going back to work only 20 days after giving birth, using up all of your vacation days in lieu of paid or unpaid family leave. In a two-part series for The Atlantic, Jessica Shortall shares how a woman named Tara, after hearing her TED Talk, reached out to her on Facebook, opening up about her experiences. Tara later updated Jessica on the struggles of balancing work with regular nursing and the other demands of motherhood via text message conversations with Shortall, “Leaking thru shirt half way through a 3 hour meeting is pretty awkward. Wow. Glamorous.” Shortall makes not only the moral but also the economic case for paid family leave, as shown by findings of The American Enterprise Institute, a conservative think tank: “Not only do household earnings rise when women go back to work following leave; the overall economy benefits as well.” (Watch Jessica’s TED Talk.)

The wisdom of our life’s work. We spend most of our waking hours working, but besides just making ends meet, is there wisdom to be found in the 9-5? In Time, Dave Isay, founder of StoryCorps, shares the ambitions, struggles and triumphs of working people, including recovering heroin addicts’ hope of opening a museum, a single fast-food-working mom turned forensic anthropologist, and a medical student who, after his father’s murder, became a teacher to help prevent disadvantaged youth from following a dangerous, criminal path. These and other stories are in Dave’s new book, Callings: The Purpose and Passion of Work. (Watch Dave’s TED Talk)

A new “nude.” A nude color crayon or a cute pair of nude ballet flats — “nude” seems to mean “the color of white skin.” A new fashion-tech startup called Mia Pielle plans to challenge that, helping clients find clothes and accessories (like a great “nude” bra) to match their own shade. By analyzing 87 photographs of women all over the world by Angélica Daas, among other data sets, Mia Pielle settled on six skin tones from which a shopper can choose. It’s still in alpha, but this blend of art, fashion and tech should help finally eradicate the notion of one default skin tone. (Watch Angélica’s TED Talk)

The Pacific Northwest’s disaster forecast. On April 18, The New Yorker’s Kathryn Schulz took home the Pulitzer prize in Feature Writing for “The Really Big One.” In the piece, Schulz dives deep into the geology and history of the Cascadia subduction zone, a West Coast faultline overshadowed by its better known but less powerful cousin, the San Andreas, and its potential to unleash a devastating earthquake and tsunami on the Pacific Northwest. With humor and whimsical detail at odds with the piece’s gravity, Schulz reveals the shocking unpreparedness of the region to handle such a powerful quake, and ponders its significance  as a cautionary tale on our relationship to science and nature — and on our very relationship to time. (Watch Kathryn’s TED Talk)

COP21: Open for business.  In December 2015, after a lengthy and delayed process shepherded by Christiana Figueres, 195 countries adopted the first universal, legally-binding climate agreement in Paris. On April 22, not so coincidentally Earth Day, at the United Nations in New York, 130 countries are expected to sign the COP21 agreement, marking the beginning of a year-long period for signatures. At TED2016, Figueres spoke about the need for optimism to bring such a landmark agreement into being, an optimism she retains today, commenting that she believes the agreement is ahead of schedule and will come into effect by 2018 instead of 2020. (Watch Christiana’s TED Talk)

A city flag redesign. In his talk at TED2015, Roman Mars called out Pocatello, Idaho, for having the worst city flag in America. The talk stirred the small city (population 54,3500) into action: They’ve started a city flag design committee. Mars attended the first meeting on April 13 to offer encouragement and advice such as: “You’ll get asked, ‘Why does this matter? Aren’t there better things we could be doing? … Get an answer for that in your head early.” As it turns out, the current flag has only ever been flown outside of Pocatello’s wastewater treatment facility and was never authorized as the official city flag, but somehow ended up in that position anyway. (Watch Roman’s TED Talk)  

… and redrawing US borders.The traditional map of the US, with its familiar clear-cut and misshapen lines of 50 states, is geographically correct, but functionally outdated. In The New York Times, global strategist Parag Khanna literally re-imagines the map to reflect the metropolis centers where physical and digital architecture are interconnecting people and economies across state lines. “Economically and socially, the country is drifting toward looser metropolitan and regional formations, anchored by the great cities and urban archipelagos that already lead global economic circuits.” Out April 19, Parag Khanna’s new book, Connectography: Mapping the Future of Global Civilization, explores this idea in depth. (Watch Parag’s TED Talk)

Research gag laws in the UK. Researchers should have the right to speak up and warn about imminent dangers, such as the effects of climate change or an epidemic on the rise. But according to Robin McKie’s article in The Guardian, this may no longer be so in the UK. Astrophysicist Sir Martin Rees, among others, weighs in on the dangers of a new decision made by the Cabinet Office of the UK that bans researchers who receive government grants from using their findings to lobby for changes to regulations or laws. “It would be far too damaging to allow this clause to proceed.” Due to this intense backlash, certain researchers are now exempt from this clause, including those in national academies. (Watch Martin’s TED Talk)

Can we design for happiness? When we think of design, it is easy to think of the book covers, technology and objects that surround us, but graphic designer Stefan Sagmeister proves that design’s principles can be applied to any aspect of our lives, even something as abstract as happiness. Seven years in the making and premiering at the Tribeca Film Festival, his new documentary The Happy Film delves with gusto into an idea he has dabbled with before–designing happiness. Like a true designer, he systematically experiments with three methods of becoming a better, happier person. (Watch Stefan’s TED Talks “Happiness by design” and “7 rules for making more happiness”)

On the brink of change. In essays spanning 25 years and 7 continents, journalist Andrew Solomon’s new book, Far and Away: Reporting from the Brink of Change, published on April 19, captures the stories of places in the throes of cataclysmic change. From his first-hand perch as a foreign correspondent, Solomon describes major events of the 20th and 21st centuries, including the fall of the Taliban in Afghanistan, the end of the Soviet Union, and Qaddafi’s Libya. With characteristic insight and detail, he reflects on the shift in personal identity that occurs when the surrounding culture, politics, and spiritual beliefs change–and the profound role individuals can have in making those changes happen. (Watch Andrew’s TED Talk)

Have a news item to share? Write us at contact@ted.com and you may see it included in this weekly round-up.

 


Sociological ImagesIllustrating Global Wealth Inequality

Wealth inequality in the U.S. is extreme, but global wealth inequality, illustrates a video by The Rules, is even more stunning. Some facts:

  • The top 20% control 80% of the world’s wealth.
  • The richest 2% control more wealth than the bottom half of the world’s population.
  • The richest 300 people on earth have more wealth than the poorest 3,000,000,000.
  • 200 years ago, rich countries were three times as rich as poor countries. Today, they are eighty times richer.
  • Rich countries give $130 billion dollars worth of aid to poor countries every year, but they extract $2 trillion each year thanks to global economic rules.

Here are their sources; or watch the four minute video:

The Rules wants to reveal and challenge the laws that govern our global economy. It is a distinctly sociological project, looking at how factors outside of individuals — or, in this case, countries — shape lives. Shaped strongly by the richest countries in their own best interest, rules governing the trading of goods and money are determining the economic solvency and future of countries.

When those rules are invisible, it can seem like struggling countries are just poorly managed or culturally problematic when, in fact, the rules ensure that the deck is stacked against them.

Hat tip to Martin Hart-Landsberg.

Lisa Wade is a professor at Occidental College and the co-author of Gender: Ideas, Interactions, Institutions. Find her on TwitterFacebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

CryptogramHow Hacking Team Got Hacked

The hacker who hacked Hacking Team posted a lengthy description of how he broke into the company and stole everything.

Three news articles.

Worse Than FailureCoded Smorgasbord: Mysterious Mysteries of Strange Mystery

Code is a window into the programmer’s mind. Our thought processes are laid bare, exposed and cemented for all eternity in keywords and symbols. It’s left there, waiting for another programmer to come by and wonder: “What were they thinking?”

That’s exactly what “seebs” was wondering, when he found this PHP code.

function startsWith($haystack, $needle) {
        // search backwards starting from haystack length characters from the end
        return $needle === "" || strrpos($haystack, $needle, -strlen($haystack)) !== FALSE;
}
function endsWith($haystack, $needle) {
        // search forward starting from end minus needle length characters
        return $needle === "" || strpos($haystack, $needle, strlen($haystack) - strlen($needle)) !== FALSE;
}

Now, the “normal” solution here would be to look at the length of your “needle” and chop off that many characters from the “haystack” using substr, then compare the two strings. Using strpos and strrpos is odd, but not half so odd as comparing the result to FALSE instead of zero.

While I was checking on PHP starts/ends-with functions, I did figure out where this code came from. It’s the most popular answer on StackOverflow, showing us another case of “programming by copy paste”.


Etta overheard some co-workers talking about issues with the “Entity” class. That was exciting, since their business objects were a mess, and Etta was excited about the thought that they might build some classes that were business objects that represented a single entity, which would be a change of pace.

Instead, Etta found a 4,000 line Entity class that was actually a “God Object” with every feature and function crammed into it. But it was even worse than that.

public class Entity
{
// 4000 lines and 3 inner classes later
    public string getFirstOpenApptHtml(final CSREntity csr) throws Exception
    {
        final StringBuilder sb = new StringBuilder();
        sb.append("<a href='javascript:showNotePopup(");
        sb.append(getEntityID());
        sb.append(",");
        sb.append(this.firstOpenAppt_AbbreviatedData.getNoteID());
        sb.append(")'><span style='color: #3333CC; text-decoration: underline; font-weight: bold;'>Appt.: ");
        // add 300 more lines of stuff to the string builder
        return sb.toString();
    }
}

There are a few fun things about this method. First, there’s the obvious: it uses a StringBuilder to construct HTML, JavaScript and CSS to inject into the page body. That’s pretty ugly. Do you see that input parameter, CSREntity? That’s actually a class that inherits from Entity- yes, this method is written to accept a parameter of its own time, and in fact, it’s usually called by running csr.getFirstOpenApptHtml(csr);


Finally, Jess was wondering, “What if I needed to sort a list of integers? Could I use the built in functionality, or is there a better way?”

Actually, Jess wasn’t wondering that at all. But a co-worker apparently was.

namespace MyCompany.Utilities
{
   using System;
   [Serializable]
   public class Integer : IComparable
   {
      private int _value;
      public Integer(int value)
      {
          _value = value;
      }
      public int Value
      {
         get
         {
            return _value;
         }
         set
         {
             _value = value;
         }
      }
      public int CompareTo(object obj)
      {
         if (obj is Integer)
         {
            Integer integer = (Integer)obj;
            return _value.CompareTo(integer._value);
         }
         else
         {
            throw new ArgumentException("Object is not of type Integer");
         }
      }
   }
}

For the unfamiliar, C# and all the .NET languages autobox primitives, and integers already implement the IComparable interface, making this code 100% redundant. The code was also 100% unused, and had never been used- despite being the first check-in to the codebase.

[Advertisement] Otter allows you to easily create and configure 1,000's of servers, all while maintaining ease-of-use, and granular visibility down to a single server. Find out more and download today!

Worse Than FailureCoded Smorgasbord: Mysterious Mysteries of Strange Mystery

Code is a window into the programmer’s mind. Our thought processes are laid bare, exposed and cemented for all eternity in keywords and symbols. It’s left there, waiting for another programmer to come by and wonder: “What were they thinking?”

That’s exactly what “seebs” was wondering, when he found this PHP code.

function startsWith($haystack, $needle) {
        // search backwards starting from haystack length characters from the end
        return $needle === "" || strrpos($haystack, $needle, -strlen($haystack)) !== FALSE;
}
function endsWith($haystack, $needle) {
        // search forward starting from end minus needle length characters
        return $needle === "" || strpos($haystack, $needle, strlen($haystack) - strlen($needle)) !== FALSE;
}

Now, the “normal” solution here would be to look at the length of your “needle” and chop off that many characters from the “haystack” using substr, then compare the two strings. Using strpos and strrpos is odd, but not half so odd as comparing the result to FALSE instead of zero.

While I was checking on PHP starts/ends-with functions, I did figure out where this code came from. It’s the most popular answer on StackOverflow, showing us another case of “programming by copy paste”.


Etta overheard some co-workers talking about issues with the “Entity” class. That was exciting, since their business objects were a mess, and Etta was excited about the thought that they might build some classes that were business objects that represented a single entity, which would be a change of pace.

Instead, Etta found a 4,000 line Entity class that was actually a “God Object” with every feature and function crammed into it. But it was even worse than that.

public class Entity
{
// 4000 lines and 3 inner classes later
    public string getFirstOpenApptHtml(final CSREntity csr) throws Exception
    {
        final StringBuilder sb = new StringBuilder();
        sb.append("<a href='javascript:showNotePopup(");
        sb.append(getEntityID());
        sb.append(",");
        sb.append(this.firstOpenAppt_AbbreviatedData.getNoteID());
        sb.append(")'><span style='color: #3333CC; text-decoration: underline; font-weight: bold;'>Appt.: ");
        // add 300 more lines of stuff to the string builder
        return sb.toString();
    }
}

There are a few fun things about this method. First, there’s the obvious: it uses a StringBuilder to construct HTML, JavaScript and CSS to inject into the page body. That’s pretty ugly. Do you see that input parameter, CSREntity? That’s actually a class that inherits from Entity- yes, this method is written to accept a parameter of its own time, and in fact, it’s usually called by running csr.getFirstOpenApptHtml(csr);


Finally, Jess was wondering, “What if I needed to sort a list of integers? Could I use the built in functionality, or is there a better way?”

Actually, Jess wasn’t wondering that at all. But a co-worker apparently was.

namespace MyCompany.Utilities
{
   using System;
   [Serializable]
   public class Integer : IComparable
   {
      private int _value;
      public Integer(int value)
      {
          _value = value;
      }
      public int Value
      {
         get
         {
            return _value;
         }
         set
         {
             _value = value;
         }
      }
      public int CompareTo(object obj)
      {
         if (obj is Integer)
         {
            Integer integer = (Integer)obj;
            return _value.CompareTo(integer._value);
         }
         else
         {
            throw new ArgumentException("Object is not of type Integer");
         }
      }
   }
}

For the unfamiliar, C# and all the .NET languages autobox primitives, and integers already implement the IComparable interface, making this code 100% redundant. The code was also 100% unused, and had never been used- despite being the first check-in to the codebase.

[Advertisement] Otter allows you to easily create and configure 1,000's of servers, all while maintaining ease-of-use, and granular visibility down to a single server. Find out more and download today!

,

Krebs on SecuritySpyEye Makers Get 24 Years in Prison

Two hackers convicted of making and selling the infamous SpyEye botnet creation kit were sentenced in Georgia today to a combined 24 years in prison for helping to infect hundreds of thousands of computers with malware and stealing millions from unsuspecting victims.

The Justice Department alleges that 24-year-old Aleksander Panin was responsible for SpyEye. Image courtesy: RT.

Aleksander Panin developed and sold SpyEye. Image courtesy: RT.

Atlanta Judge Amy Totenberg handed down a sentence of nine years, six months for Aleksandr Andreevich Panin, a 27-year-old Russian national also known by the hacker aliases “Gribodemon” and “Harderman.”

Convicted of conspiracy to commit wire and bank fraud, Panin was the core developer and distributor of SpyEye, a botnet toolkit that made it easy for relatively unsophisticated cyber thieves to steal millions of dollars from victims.

Sentenced to 15 years in jail was Panin’s business partner —  27-year-old Hamza “Bx1” Bendelladj, an Algerian national who pleaded guilty in June 2015 to helping Panin develop and market the SpyEye kit. Bendelladj also admitting to running his own SpyEye botnet of hacked Windows computers, a crime machine that he used to harvest and steal 200,000 credit card numbers. By the government’s math (an assumed $500 loss per card) Bx1 was potentially responsible for $100 million in losses.

“It is difficult to over state the significance of this case, not only in terms of bringing two prolific computer hackers to justice, but also in disrupting and preventing immeasurable financial losses to individuals and the financial industry around the world,” said John Horn, U.S. Attorney for the Northern District of Georgia.

THE HAPPY HACKER

Bendelladj was arrested in Bangkok in January 2013 while in transit from Malaysia to Egypt. He quickly became known as the “happy hacker” after his arrest, in which he could be seen smiling broadly while in handcuffs and being paraded before the local news media.

Photo: Hamza "BX1" Bendelladj, Bangkok Post

Photo: Hamza “Bx1” Bendelladj, Bangkok Post

In its case against the pair of hackers, the government presented chat logs between Bendelladj and Panin and other hackers. The government says the chat logs reveal that although Bendelladj worked with Panin to fuel the rise of SpyEye by vouching for him on cybercrime forums such as “Darkode,” the two had an antagonistic relationship.

Their business partnership imploded after Bx1 announced that he was publicly releasing the source code for SpyEye.

“Indeed, after Bendelladj ‘cracked’ SpyEye and made it available to others without having to purchase it from Panin, the two had a falling out,” reads the government’s sentencing memo (PDF) to the judge in the case.

The government says that while Bendelladj maintained he was little more than a malware analyzer working for a security company, his own chat logs put the lie to that claim, noting in November 2012 Bx1 bluntly said: “if they pay me the whole money of the world . . . I wont work for security.”

Bx1 had a penchant for marketing to other thieves. He shrewdly cast SpyEye as a lower-cost, more powerful alternative to the Zeus botnet creation kit, plastering cybercrime forums with animated ads pimping SpyEye as the “Zeuskiller” (in part because SpyEye was designed to remove Zeus from host computers before infecting them).

Part of a video ad for SpyEye.

Part of a video ad for SpyEye.

In Oct. 2010, KrebsOnSecurity was the first to report on rumors in the underground that the authors of Zeus and SpyEye were ending their rivalry and merging the two crimeware products into one software stack and support structure for existing clients.

“Panin developed SpyEye as a successor to the notorious Zeus malware that had, since 2009, wreaked havoc on financial institutions around the world,” the Justice Department said in its statement today. “In November 2010, Panin allegedly received the source code and rights to sell Zeus from Evginy Bogachev, a/k/a Slavik, and incorporated many components of Zeus into SpyEye.  Bogachev remains at large and is currently the FBI’s most wanted cybercriminal.”

Bogachev, the alleged Zeus Trojan author, in undated photos.

Bogachev, the alleged Zeus Trojan author, in undated photos.

It’s not clear whether Bendelladj had any intention of honoring the sanctity of the merger agreement with the author of the Zeus Trojan. Not long after the supposed merger, copies of the Zeus source code were available for sale online, and the code went fully public and free not long after that. My money is on Bendelladj for that leak as well.

Apparently Bx1 was not a big fan of KrebsOnSecurity, either. According to the government’s sentencing memo:

“At various points, [Bendelladj] has expressed contempt for Brian Krebs, the author of the “Krebs on Security,” and claims that he has credit cards (‘ccs’) of Mr. Krebs’s family and that Bendelladj will be ‘after him until he die.’ He even suggests inflicting a Distributed Denial of Service attack against Mr. Krebs.”

Maybe that antagonism had something to do with this story, in which I repost chat logs from a conversation I had with Bx1 back in January 2012. In it, Bx1 brags about hacking one of his competitors and to getting the guy arrested.

Google Adsense[New Resource] Download the AdSense Guide to Audience Engagement


Research shows that “29% of smartphone users will immediately switch to another site or app if it doesn’t satisfy their needs.

In a world where people are making split decisions about what to consume, it’s increasingly challenging but critical for publishers to figure out how to effectively engage their audiences on their sites. To help lay the foundation to a winning engagement strategy, we’ve created the AdSense Guide to Audience Engagement.

This guide help you drive toward your goals for growing your site – from defining your brand voice to tips to make your site’s content easy to consume. Don’t waste another moment developing web pages that leave you with little opportunity to engage with your audience. Download the AdSense Guide to Audience Engagement here.

In this guide, you’ll learn:

  • How to help your audience become familiar with your brand
  • Best practices to design delightful user journeys
  • How to develop content that resonates with your audience
  • Ways to make your content easy to consume
  • Why you should share the love with other sites by referring to good sources



Check out the guide and share your feedback with us on Google+ and Twitter using #AdSenseGuide. We’d love to hear what you think. 




Posted by Jay Castro
from the AdSense team

Krebs on SecurityGiant Food Sees Giant Card Fraud Spike

Citing a recent and large increase in credit card fraud, Washington, DC-area grocer Giant Food says it will no longer allow customers to use credit cards when purchasing gift cards and reloadable or prepaid debit cards.

A new warning sign at Giant Food checkout counters. Giant says the warning was prompted by a spike in credit card fraud.

A new warning sign at Giant Food checkout counters. Giant says the warning was prompted by a spike in credit card fraud.

I had no idea this was a new thing at Landover, Md.-based Giant, which operates 169 supermarkets in the Washington, D.C. metro area.  That is, until I encountered a couple of large new “attention” stickers in the checkout line at a local Giant in Virginia recently. Next to the credit card terminal were big decals with the warning:

“Attention Gift Card Customers: Effective immediately, all purchases of Visa, MasterCard, American Express Gift Cards and all General Purpose Reloadable or Prepaid Cards may only be made with Cash or Bank Pin-based Debit.”

Asked for comment about the change, Giant Food released a brief statement about the policy change that went into effect in March 2016, but otherwise didn’t respond to requests for more details.

“Giant has recently made a change in procedures for purchasing gift cards because of a large increase of fraudulent gift card purchasing,” the company said. “Giant will now accept only a Bank PIN-based debit card or cash for all VISA, MasterCard, and American Express gift cards, as well as re-loadable and prepaid gift cards. This change has been made in order to mitigate potential fraud risk.”

It’s not clear why Giant is only just now taking this basic anti-fraud step. Card thieves love to pick on grocery and convenience stores. Street gangs involved in card fraud (and they’re all involved in card fraud now) often extract money from grocery, dollar and convenience stores using “runners” — low-level members who are assigned the occasionally risky business of physically “cashing out” counterfeit credit and debit cards.

One of the easiest ways thieves can cash out? Walk into a grocery or retail store and buy prepaid gift cards using stolen credit cards. Such transactions — if successful — effectively launder money by converting the stolen item (counterfeit/stolen card) into a good that is equivalent to cash or can be easily resold for cash (gift cards).

I witnessed this exact crime firsthand at a Giant in Maryland last year. As I noted in a Dec. 2015 post about gift card fraud, the crooks caught in the process of these cashout schemes usually are found with dozens of counterfeit credit cards on their person or in their vehicle. From that post:

“The man in front of me in line looked and smelled homeless. The only items he was trying to buy were several $200 gift cards that Giant had on sale for various retailers. When the first card he swiped was declined, the man fished two more cards out of his wallet. Each was similarly declined, but the man just shrugged and walked out of the store. I asked the cashier if this sort of thing happened often, and he just shook his head and said, ‘Man, you have no idea.'”

Meanwhile, every Giant I visit still asks me to swipe my chip-based card, effectively negating any added security the chip provides. Chip-based cards are far more expensive and difficult for thieves to counterfeit, and they can help mitigate the threat from most modern card-skimming methods that read the cardholder data in plain text from the card’s magnetic stripe. Those include malicious software at the point-of-sale terminal, as well as physical skimmers placed over card readers at self-checkout lanes — like this one found at a Maryland Safeway earlier this year.

In a recent column – The Great EMV Fake-Out: No Chip for You! – I explored why so few retailers currently allow or require chip transactions, even though many of them already have all the hardware in place to accept chip transactions. I suspect also that grocers are reluctant to introduce chip readers at self-checkout lanes, as more supermarket chains seem to be pushing customers in the self-checkout direction.

CryptogramHelen Nissenbaum on Regulating Data Collection and Use

NYU Helen Nissenbaum gave an excellent lecture at Brown University last month, where she rebutted those who think that we should not regulate data collection, only data use: something she calls "big data exceptionalism." Basically, this is the idea that collecting the "haystack" isn't the problem; it what is done with it that is. (I discuss this same topic in Data and Goliath, on pages 197-9.)

In her talk, she makes a very strong argument that the problem is one of domination. Contemporary political philosopher Philip Pettit has written extensively about a republican conception of liberty. He defines domination as the extent one person has the ability to interfere with the affairs of another.

Under this framework, the problem with wholesale data collection is not that it is used to curtail your freedom; the problem is that the collector has the power to curtail your freedom. Whether they use it or not, the fact that they have that power over us is itself a harm.

Worse Than FailureEnterprise Automation

Rex had just been hired on with a large retailer as a Puppet Automation Engineer, tasked with using Puppet Labs to automate deployments of some SAP-py, enterprisey software. He was paired up with another Puppet Automation Engineer, Alexi. Alexi was the expert, and he was in charge of automating the company’s Sarbanes-Oxley (SOX) auditing.

Alexei was a firm believer that the Customer Is Always Wrong, and Alexei Knows Best. As a consequence, he thought that any requirements he didn’t like could be changed to arbitrary ones he did like. If the customer wanted a report that provided some summarized sales numbers for the year and he thought that was stupid, he’d instead give them a report showing their top product’s Line-Of-Code count divided by the Dow Jones Industrial Average for the month. If they wanted to slice-and-dice their customer database by demographics, he would code up a line graph relating the number of characters in their last name to the average nightly lows on their date of birth.

SarbanesOxley
These are Sarbanes and Oxley, but you can imagine these are the characters of this story.

This caused huge problems with the SOX auditing. As he worked on the requirements-not generated by users, but dictated by US law- Alexei replaced “stupid” ones with his own “expert” ones. After a chain of angry emails, angrier phone calls, and downright apocalyptic meetings, management finally got Alexi to deal with the actual requirements. Exactly as written.

Exactly. As. Written.

From what Rex could see, much of ALexi’s implementation was deliberate malicious obedience. For example, many rules were given the same name, causing their auditing engine (which ran through the rules in alphabetic order) to give varying and nondeterministic results as the rules were run in a different order each time, often overwriting each others’ output.

What really amazed Rex was the incident that finally convinced his coworker to leave. Alexei was automating another piece of enterprisey software deployments- Initech’s “SalesManagerPoint”- with Puppet, using an existing (and broken, unsupported and officially deprecated) module designed specifically for their case. When the module didn’t work, Alexei doubled-down and tried to fix it, adding layer upon layer of expert complexity, meanwhile ignoring key requirements because “The module doesn’t support that!”

Frustrated, management brought in a Highly-Paid Consultant, named Harry. Harry was intimately familiar with Puppet Labs, and within a few days, proved the point by fixing many of their automation-pain-points. Everything was great, until Harry discovered Alexei’s work. Harry was also familiar with Initech’s SalesManagerPoint. “That’s a very tricky one. I can take a quick glance at it, but we normally will not touch that without a minimum six-month contract.”

As soon as the words were out of his mouth, Rex stopped paying attention to his work, and started paying attention to Harry and Alexi. Their disagreement escalated into a hilarious shouting match. Harry tried to tell Alexi he was using Puppet, the module, and SalesManagerPoint all the wrong way, to which Alexei would respond with screams of “But your so-called ‘right way’ is stupid! My way is better!”

Alexei was angry enough to quit on the spot and Rex inherited his work. Harry refused to go anywhere near anything Alexi had touched without renegotiating the contract. After reviewing Alexi’s work, Rex decided on ignoring it completely and built up a new system from scratch. Several months later Rex had a working Puppet automation that actually met all the customer’s requirements, without using Alexei’s work and without using the broken Puppet module. No expert required.

[Advertisement] Application Release Automation – build complex release pipelines all managed from one central dashboard, accessibility for the whole team. Download and learn more today!

Worse Than FailureEnterprise Automation

Rex had just been hired on with a large retailer as a Puppet Automation Engineer, tasked with using Puppet Labs to automate deployments of some SAP-py, enterprisey software. He was paired up with another Puppet Automation Engineer, Alexi. Alexi was the expert, and he was in charge of automating the company’s Sarbanes-Oxley (SOX) auditing.

Alexei was a firm believer that the Customer Is Always Wrong, and Alexei Knows Best. As a consequence, he thought that any requirements he didn’t like could be changed to arbitrary ones he did like. If the customer wanted a report that provided some summarized sales numbers for the year and he thought that was stupid, he’d instead give them a report showing their top product’s Line-Of-Code count divided by the Dow Jones Industrial Average for the month. If they wanted to slice-and-dice their customer database by demographics, he would code up a line graph relating the number of characters in their last name to the average nightly lows on their date of birth.

SarbanesOxley
These are Sarbanes and Oxley, but you can imagine these are the characters of this story.

This caused huge problems with the SOX auditing. As he worked on the requirements-not generated by users, but dictated by US law- Alexei replaced “stupid” ones with his own “expert” ones. After a chain of angry emails, angrier phone calls, and downright apocalyptic meetings, management finally got Alexi to deal with the actual requirements. Exactly as written.

Exactly. As. Written.

From what Rex could see, much of ALexi’s implementation was deliberate malicious obedience. For example, many rules were given the same name, causing their auditing engine (which ran through the rules in alphabetic order) to give varying and nondeterministic results as the rules were run in a different order each time, often overwriting each others’ output.

What really amazed Rex was the incident that finally convinced his coworker to leave. Alexei was automating another piece of enterprisey software deployments- Initech’s “SalesManagerPoint”- with Puppet, using an existing (and broken, unsupported and officially deprecated) module designed specifically for their case. When the module didn’t work, Alexei doubled-down and tried to fix it, adding layer upon layer of expert complexity, meanwhile ignoring key requirements because “The module doesn’t support that!”

Frustrated, management brought in a Highly-Paid Consultant, named Harry. Harry was intimately familiar with Puppet Labs, and within a few days, proved the point by fixing many of their automation-pain-points. Everything was great, until Harry discovered Alexei’s work. Harry was also familiar with Initech’s SalesManagerPoint. “That’s a very tricky one. I can take a quick glance at it, but we normally will not touch that without a minimum six-month contract.”

As soon as the words were out of his mouth, Rex stopped paying attention to his work, and started paying attention to Harry and Alexi. Their disagreement escalated into a hilarious shouting match. Harry tried to tell Alexi he was using Puppet, the module, and SalesManagerPoint all the wrong way, to which Alexei would respond with screams of “But your so-called ‘right way’ is stupid! My way is better!”

Alexei was angry enough to quit on the spot and Rex inherited his work. Harry refused to go anywhere near anything Alexi had touched without renegotiating the contract. After reviewing Alexi’s work, Rex decided on ignoring it completely and built up a new system from scratch. Several months later Rex had a working Puppet automation that actually met all the customer’s requirements, without using Alexei’s work and without using the broken Puppet module. No expert required.

[Advertisement] Application Release Automation – build complex release pipelines all managed from one central dashboard, accessibility for the whole team. Download and learn more today!

,

TEDMeet the 110 speakers at TEDSummit 2016 (including some of the most popular of all time)

TEDSummit logo

The number is 110: One hundred and ten past and new TED speakers are part of our newest conference, TEDSummit, happening in Banff, Canada, 26–30 June 2016.

And you are invited to join us!

Some of the most popular TED speakers of all time, including Dan Pink, David Gallo, Esther Perel, Kelly and Jane McGonigal, Pico Iyer and dozens more will be joined by brand-new voices including food innovator Josh Tetrick, forest biologists Suzanne Simard, environmental writer Emma Marris, energy experts Joe Lassiter and Michael Shellenberger, blockchain researcher Bettina Warburg, global affairs writer Jonathan Tepperman, empathy scientist Abigail Marsh and more.

About half of these speakers will take the stage to give major TED Talks on topics ranging from advanced digital technologies to climate change to surveillance and transparency … from relationships to brain microscopy … from trust to what humans might look like in 200 years

These 110 speakers will also join — and often lead — workshops and participatory sessions. Look, among the more than 100 sessions, for workshops on the ethics of artificial intelligence, and on the fragility of global megacities … brainstorms on what the TED community might do to help confront the refugee crisis, or on the idea of a female utopia … master classes on social storytelling and on how to think like a scientist … a walk in the woods guided by a forest biologist … even a hands-on genetic manipulation lab.

And there will be planned and unplanned surprises, and of course, outdoor activities in the gorgeous scenery of the Canadian Rocky Mountains.

There are a few seats left to attend TEDSummit. You can find more information and apply here.

And here is the full list of past and new TED speakers who have confirmed their participation in TEDSummit 2016 (subject to change):

Alessandro Acquisti, Privacy economist
Esra’a Al Shafei, Human rights activist
Monica Araya, Activist
Tasso Azevedo, Forester, sustainability activist
Julia Bacha, Filmmaker
Uldus Bakhtiozina, Photographer, visual artist
Benedetta Berti, International policy analyst
Alexander Betts, Refugee scholar
Laila Biali, Musician
Rachel Botsman, Sharing innovator
Laura Boushnak, Photographer
Ed Boyden, Neuroengineer
Steve Boyes, Explorer
Jennifer Bréa, Filmmaker
Erik Brynjolfsson, Innovation researcher
Kitra Cahana, Journalist and conceptual artist
Daniela Candillari, Musician
Jason Clay, Market transformer
Angélica Dass, Photographer
Abe Davis, Computer scientist
Dan Dennett, Philosopher, cognitive scientist
Jamie Drummond, Anti-poverty activist
R. Luke DuBois, Artist, composer, engineer
Zak Ebrahim, Peace activist
Hasan Elahi, Privacy artist
Juan Enriquez, Futurist
Helen Fisher, Anthropologist; expert on love
Melissa Fleming, Voice for refugees
David Gallo, Oceanographer
Casey Gerald, American
Anand Giridharadas, Author
Michael Green, Social progress expert
Michael Green, Architect
Brian Greene, Physicist
Johann Hari, Journalist
Sam Harris, Neuroscientist and philosopher
Gary Haugen, Human rights attorney
Lesley Hazleton, Accidental theologist
Celeste Headlee, Writer and radio host
Margaret Heffernan, Management thinker
Hugh Herr, Bionics designer
Erik Hersman, Blogger, technologist
Hays + Ryan Holladay, Musical artists
John Hunter, Educator
Jedidah Isler, Astrophysicist
Pico Iyer, Global author
Meg Jay, Clinical psychologist
Ellen Jorgensen, Biologist and community science advocate
Sarah Kay, Poet
Kevin Kelly, Digital visionary
Matt Kenyon, New media artist
Ken Lacovara, Paleontologist
David Lang, Maker
Joe Lassiter, Energy scholar
Tim Leberecht, Marketer
Monica Lewinsky, Social activist
Rebecca MacKinnon, Media activist
Pia Mancini, Democracy activist
Emma Marris, Environmental writer
Abigail Marsh, Psychologist
Jane McGonigal, Game designer
Kelly McGonigal, Health psychologist
Lee Mokobe, Poet
Robert Muggah, Megacities expert
Michael Murphy, Designer
Ethan Nadelmann, Drug policy reformer
Iyeoka Okoawo, Singer
Ngozi Okonjo-Iweala, Economist
Dan Pallotta, Charity defender
Amanda Palmer, Musician
Sarah Parcak, Space archaelogist, TED Prize winner
Eli Pariser, Organizer and author
Vikram Patel, Mental health care advocate
Esther Perel, Relationship therapist
Dan Pink, Career analyst
Will Potter, Investigative journalist
Navi Radjou, Innovation strategist
Shai Reshef, Education entrepreneur
Usman Riaz, Percussive guitarist
Joshua Roman, Cellist
Jon Ronson, Writer and filmmaker
Martine Rothblatt, Transhumanist
Juliana Rotich, Tech entrepreneur
Louie Schwartzberg, Filmmaker
eL Seed, Calligraffiti artist
BillSellanga, Musician
Graham Shaw, Communicator
Michael Shellenberger, Climate policy expert
Michael Shermer, Skeptic
Suzanne Simard, Forest biologist
Ernesto Sirolli, Sustainable development expert
Kevin Slavin, Algoworld expert
Christopher Soghoian, Privacy researcher + activist
Andrew Solomon, Writer
Malte Spitz, Politician and data activist
Daniel Suarez, Sci-fi author
Pavan Sukhdev, Environmental economist
Ilona Szabo de Carvalho, Policy reformer
Don Tapscott, Digital strategist
Anastasia Taylor-Lind, Documentary photographer
Marco Tempest, Techno-illusionist
Jonathan Tepperman, Editor, Foreign Affairs
Josh Tetrick, Food innovator
Julian Treasure, Sound consultant
Zeynep Tufekci, Techno-sociologist
Sherry Turkle, Cultural analyst
James Veitch, Comedian and writer
Robert Waldinger, Psychiatrist, psychoanalyst and Zen priest
Bettina Warburg, Blockchain researcher


CryptogramGCHQ Gets Involved in Mundane Surveillance Matters

GCHQ detected a potential pre-publication leak of a Harry Potter book, and alerted the publisher.

Is this what British national intelligence is supposed to be doing?

Google AdsenseTips for writing a successful invalid traffic appeal

In a previous publisher blog post, we discussed tips for writing a successful policy appeal. In today's topic, we’re exploring what happens when publisher accounts are disabled due to invalid traffic, when to submit an appeal, and tips for making those appeals successful.

Invalid traffic includes any clicks or impressions that may artificially inflate an advertiser's costs or a publisher's earnings. Invalid traffic covers intentionally fraudulent traffic as well as accidental clicks.

In the ads ecosystem, advertisers rely on the relevance of our ad placement and the quality of the interaction their ads receive. Publishers in turn count on advertiser participation that contributes to the success of their apps and business. Without this trust, the Google advertising network could not exist. Google treats invalid traffic very seriously, analyzing all clicks and impressions to determine whether they fit a pattern of use that might artificially drive up an advertiser's costs or a publisher's earnings. If we determine that an account might pose a risk to our advertisers, we may take actions against the account, such as suspending or disabling it, in order to protect our advertisers' interests.

Before we continue, let's clarify the difference between an account suspension and an account disablement.

If your account was suspended due to invalid traffic, ad-serving has been turned off for a fixed period (typically 30 days). While suspensions are not currently appealable, if you would like to provide additional feedback to help us improve our processes and communications, you may do so using our suspended publisher feedback form. If there are no further compliance issues with your account, it will be automatically unsuspended after the fixed period. For more information about account suspensions, please visit our AdSense Help Center (or AdMob Help Center).

If your account was disabled due to invalid traffic, your account is no longer serving ads, and you will be unable to monetize with any Google ad solutions. You're eligible to file an invalid activity appeal, but please be aware that that filing an appeal does not guarantee reinstatement. For more information about account disablements, please visit our Help Center.

If you would like to file an invalid traffic appeal for account reinstatement, please first review the AdSense program policies and top reasons for account closure. These policies and reasons apply to all Google ad solution products, including but not limited to AdMob and YouTube.

Here are some tips that you may find useful in writing your appeal:


  1. Review the top reasons for account closure. Think about if any of these reasons applied to you and your content. Did your friends click on your ads too many times? Did you purchase traffic that led to a surge of invalid activity? Can you make content and/or behavioral changes to prevent the activity from happening again?
  2. Review ad implementations on your desktop site, mobile site, and/or mobile app. Think about what your typical user journey would be, and see if the ad implementations may cause users to accidentally click on your ads.
  3. In the appeal form, provide the e-mail address that is associated with your disabled AdSense account. This will help locate your account and reduce delays in appeal processing time.
  4. Tell us what changes you’ll make for the future. Once again, carefully consider the top reasons for account closure. What systems or behaviors have you put in place to ensure this won't happen again? For example, tell us how you've adjusted your ad implementations, evaluated your traffic sources, hired testers to properly test your content, etc. We will appreciate your honesty in the appeal.


Publishers are a vital part of the online ads ecosystem, and we want to see you succeed while keeping your users happy and engaged. Everyone in the digital ecosystem benefits when publishers provide engaging content and useful resources, which in turn incentivizes advertisers to bid for space on your content. We hope that these resources can provide some guidance.

Posted by Danielle Chang of the AdSense Ad Traffic Quality Team

Sociological ImagesRace-Based Activism is Changing College Campuses

A survey of college and university presidents conducted earlier this year suggests that campus activists are making a difference. The American Council on Education asked 567 presidents about their experience with and response to activists on campus organized around racial diversity and justice.

Almost half (47%) of presidents at 4-year institutions said that such activism was occurring on their campuses and that the dialogue about such matters had increased (41%). The majority (86%) had met with student organizers more than once and more than half (55%) said that the “racial climate” on campus was more of a priority  than it had been just a few years ago. The trends for 2-year institutions were weaker, but in the same direction.

When asked what concrete steps they had taken to improve the racial climate, presidents reported a range of strategies:

5

As with all activism, progress requires vigilance, so it will be interesting to see how many of these efforts translate into real changes in climate. New policies and procedures can be toothless or even harmful, resources can be mis-spent and trainings can be terrible, public acknowledgement can be nothing but lip service, and curricular revision can die in committee. Still, these data point to the potential for activism to make a difference and are encouraging for those of us who care about this issue.

Cross-posted at Pacific Standard.

Lisa Wade is a professor at Occidental College and the co-author of Gender: Ideas, Interactions, Institutions. Find her on TwitterFacebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

CryptogramDetails about Juniper's Firewall Backdoor

Last year, we learned about a backdoor in Juniper firewalls, one that seems to have been added into the code base.

There's now some good research: "A Systematic Analysis of the Juniper Dual EC Incident," by Stephen Checkoway, Shaanan Cohney, Christina Garman, Matthew Green, Nadia Heninger, Jacob Maskiewicz, Eric Rescorla, Hovav Shacham, and Ralf-Philipp Weinmann:

Abstract: In December 2015, Juniper Networks announced that unknown attackers had added unauthorized code to ScreenOS, the operating system for their NetScreen VPN routers. This code created two vulnerabilities: an authentication bypass that enabled remote administrative access, and a second vulnerability that allowed passive decryption of VPN traffic. Reverse engineering of ScreenOS binaries revealed that the first of these vulnerabilities was a conventional back door in the SSH password checker. The second is far more intriguing: a change to the Q parameter used by the Dual EC pseudorandom number generator. It is widely known that Dual EC has the unfortunate property that an attacker with the ability to choose Q can, from a small sample of the generator's output, predict all future outputs. In a 2013 public statement, Juniper noted the use of Dual EC but claimed that ScreenOS included countermeasures that neutralized this form of attack.

In this work, we report the results of a thorough independent analysis of the ScreenOS randomness subsystem, as well as its interaction with the IKE VPN key establishment protocol. Due to apparent flaws in the code, Juniper's countermeasures against a Dual EC attack are never executed. Moreover, by comparing sequential versions of ScreenOS, we identify a cluster of additional changes that were introduced concurrently with the inclusion of Dual EC in a single 2008 release. Taken as a whole, these changes render the ScreenOS system vulnerable to passive exploitation by an attacker who selects Q. We demonstrate this by installing our own parameters, and showing that it is possible to passively decrypt a single IKE handshake and its associated VPN traffic in isolation without observing any other network traffic.

We still don't know who installed the back door.

Worse Than FailureCodeSOD: And It's Collated

As anyone who’s ever written a c-style char * string knows, strings are much more complicated than they look. This is even more true in this modern era of Unicode and character encodings and multilingual applications. How does “ä” compare to “a” or “á”?

John Moore’s company sent some code to a contracting firm. They needed to strip off any diacritics and unusual characters when they were comparing strings, so that “ä” and “å” were treated as the same character when searching- a not uncommon problem. In Java, there’s a special family of classes inheriting from Collator which can be used to solve exactly that problem. Now, most developers aren’t deeply familiar with these, so seeing a contractor that turns in a more “home brewed” approach is hardly surprising.

This approach goes above and beyond. It starts out bad, but not horrible: convert the string to their character codes, and then look at each one. What follows is a textbook example of why you don’t write gigantic if-blocks using magic numbers as boundary conditions, including this gem:

if (index == 292 && index == 294) {
   resultCharacterIndexes.add(68);//H
}

Yes, they only convert to a 68 (the letter “H”) when the index character is both 292 and 294, which may ʒ and ʔ you.

For bonus points, they use the input parameter as their storage point for the return variable, which is fine, I suppose, but return source doesn’t exactly sound like it’s returning an altered reference.

    protected String replaceCharacters(String source) {
        if (!String.isEmpty(source)) {
            List<Integer> resultCharacterIndexes = new List<Integer>();
            List<Integer> characterIndexes = source.getChars();

            for (Integer index : characterIndexes) {
                if (index >= 192 && index <= 198) {
                    resultCharacterIndexes.add(65);//A
                }
                else if (index == 199 || index == 262 || index == 264 || index == 266 || index == 268) {
                    resultCharacterIndexes.add(67);//C
                }
                if (index == 270 && index == 272) {
                    resultCharacterIndexes.add(68);//D
                }
                else if (index >= 200 && index <= 203 || index == 208 || index == 274 || index == 276 || index == 278 || index == 280 || index == 282) {
                    resultCharacterIndexes.add(69);//E
                }
                else if (index == 284 || index == 286 || index == 288 || index == 290) {
                    resultCharacterIndexes.add(71);//G
                }
                if (index == 292 && index == 294) {
                    resultCharacterIndexes.add(68);//H
                }
                else if (index >= 204 && index <= 207 || index == 296 || index == 298 || index == 300 || index == 302 || index == 304) {
                    resultCharacterIndexes.add(73);//I
                }
                else if (index == 308) {
                    resultCharacterIndexes.add(74);//J
                }
                else if (index == 310) {
                    resultCharacterIndexes.add(75);//K
                }
                else if (index == 313 || index == 315 || index == 317 || index == 319 || index == 321) {
                    resultCharacterIndexes.add(76);//L
                }
                else if (index == 209 || index == 323 || index == 325 || index == 327) {
                    resultCharacterIndexes.add(78);//N
                }
                else if (index >= 210 && index <= 216 || index == 332 || index == 334 || index == 336) {
                    resultCharacterIndexes.add(79);//O
                }
                else if (index == 340 || index == 342 || index == 344) {
                    resultCharacterIndexes.add(82);//R
                }
                else if (index == 346 || index == 348 || index == 350 || index == 352) {
                    resultCharacterIndexes.add(83);//S
                }
                else if (index == 354 || index == 356 || index == 358) {
                    resultCharacterIndexes.add(84);//T
                }
                else if (index >= 217 && index <= 220 || index == 360 || index == 362 || index == 364 || index == 366 || index == 368 || index == 370) {
                    resultCharacterIndexes.add(85);//U
                }
                else if (index == 372) {
                    resultCharacterIndexes.add(87);//W
                }
                else if (index == 221 || index == 374 || index == 376) {
                    resultCharacterIndexes.add(89);//Y
                }
                else if (index == 377 || index == 379 || index == 381) {
                    resultCharacterIndexes.add(90);//Z
                }
                else if (index >= 224 && index <= 230 || index == 257 || index == 259 || index == 261) {
                    resultCharacterIndexes.add(97);//a
                }
                else if (index == 231 || index == 263 || index == 265 || index == 267 || index == 269) {
                    resultCharacterIndexes.add(99);//c
                }
                else if (index == 271 || index == 273) {
                    resultCharacterIndexes.add(100);//d
                }
                else if (index >= 232 && index <= 235 || index == 240 || index == 275 || index == 277 || index == 279 || index == 281 || index == 283) {
                    resultCharacterIndexes.add(101);//e
                }
                else if (index == 285 || index == 287 || index == 289 || index == 291) {
                    resultCharacterIndexes.add(103);//g
                }
                else if (index == 293 || index == 295) {
                    resultCharacterIndexes.add(104);//h
                }
                else if (index >= 236 && index <= 239 || index == 297 || index == 299 || index == 301 || index == 303) {
                    resultCharacterIndexes.add(105);//i
                }
                else if (index == 309) {
                    resultCharacterIndexes.add(106);//j
                }
                else if (index == 311 || index == 312) {
                    resultCharacterIndexes.add(107);//k
                }
                else if (index == 314 || index == 316 || index == 318 || index == 320 || index == 322) {
                    resultCharacterIndexes.add(108);//l
                }
                else if (index == 241) {
                    resultCharacterIndexes.add(110);//n
                }
                else if (index >= 242 && index <= 248 || index == 333 || index == 335 || index == 337) {
                    resultCharacterIndexes.add(111);//o
                }
                else if (index == 341 || index == 343 || index == 345) {
                    resultCharacterIndexes.add(114);//r
                }
                else if (index == 223 || index == 347 || index == 349 || index == 351 || index == 353) {
                    resultCharacterIndexes.add(115);//s
                }
                else if (index == 355 || index == 357 || index == 359) {
                    resultCharacterIndexes.add(116);//t
                }
                else if (index >= 249 && index <= 252 || index == 361 || index == 363 || index == 365 || index == 367 || index == 369 || index == 371) {
                    resultCharacterIndexes.add(117);//u
                }
                else if (index == 373) {
                    resultCharacterIndexes.add(119);//w
                }
                else if (index == 253 || index == 255 || index == 375) {
                    resultCharacterIndexes.add(121);//y
                }
                else if (index == 378 || index == 380 || index == 382) {
                    resultCharacterIndexes.add(122);//z
                }
                else {
                    resultCharacterIndexes.add(index);
                }
            }
            source = String.fromCharArray(resultCharacterIndexes);
        }
        return source;
    }
[Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!

Worse Than FailureCodeSOD: And It's Collated

As anyone who’s ever written a c-style char * string knows, strings are much more complicated than they look. This is even more true in this modern era of Unicode and character encodings and multilingual applications. How does “ä” compare to “a” or “á”?

John Moore’s company sent some code to a contracting firm. They needed to strip off any diacritics and unusual characters when they were comparing strings, so that “ä” and “å” were treated as the same character when searching- a not uncommon problem. In Java, there’s a special family of classes inheriting from Collator which can be used to solve exactly that problem. Now, most developers aren’t deeply familiar with these, so seeing a contractor that turns in a more “home brewed” approach is hardly surprising.

This approach goes above and beyond. It starts out bad, but not horrible: convert the string to their character codes, and then look at each one. What follows is a textbook example of why you don’t write gigantic if-blocks using magic numbers as boundary conditions, including this gem:

if (index == 292 && index == 294) {
   resultCharacterIndexes.add(68);//H
}

Yes, they only convert to a 68 (the letter “H”) when the index character is both 292 and 294, which may ʒ and ʔ you.

For bonus points, they use the input parameter as their storage point for the return variable, which is fine, I suppose, but return source doesn’t exactly sound like it’s returning an altered reference.

    protected String replaceCharacters(String source) {
        if (!String.isEmpty(source)) {
            List<Integer> resultCharacterIndexes = new List<Integer>();
            List<Integer> characterIndexes = source.getChars();

            for (Integer index : characterIndexes) {
                if (index >= 192 && index <= 198) {
                    resultCharacterIndexes.add(65);//A
                }
                else if (index == 199 || index == 262 || index == 264 || index == 266 || index == 268) {
                    resultCharacterIndexes.add(67);//C
                }
                if (index == 270 && index == 272) {
                    resultCharacterIndexes.add(68);//D
                }
                else if (index >= 200 && index <= 203 || index == 208 || index == 274 || index == 276 || index == 278 || index == 280 || index == 282) {
                    resultCharacterIndexes.add(69);//E
                }
                else if (index == 284 || index == 286 || index == 288 || index == 290) {
                    resultCharacterIndexes.add(71);//G
                }
                if (index == 292 && index == 294) {
                    resultCharacterIndexes.add(68);//H
                }
                else if (index >= 204 && index <= 207 || index == 296 || index == 298 || index == 300 || index == 302 || index == 304) {
                    resultCharacterIndexes.add(73);//I
                }
                else if (index == 308) {
                    resultCharacterIndexes.add(74);//J
                }
                else if (index == 310) {
                    resultCharacterIndexes.add(75);//K
                }
                else if (index == 313 || index == 315 || index == 317 || index == 319 || index == 321) {
                    resultCharacterIndexes.add(76);//L
                }
                else if (index == 209 || index == 323 || index == 325 || index == 327) {
                    resultCharacterIndexes.add(78);//N
                }
                else if (index >= 210 && index <= 216 || index == 332 || index == 334 || index == 336) {
                    resultCharacterIndexes.add(79);//O
                }
                else if (index == 340 || index == 342 || index == 344) {
                    resultCharacterIndexes.add(82);//R
                }
                else if (index == 346 || index == 348 || index == 350 || index == 352) {
                    resultCharacterIndexes.add(83);//S
                }
                else if (index == 354 || index == 356 || index == 358) {
                    resultCharacterIndexes.add(84);//T
                }
                else if (index >= 217 && index <= 220 || index == 360 || index == 362 || index == 364 || index == 366 || index == 368 || index == 370) {
                    resultCharacterIndexes.add(85);//U
                }
                else if (index == 372) {
                    resultCharacterIndexes.add(87);//W
                }
                else if (index == 221 || index == 374 || index == 376) {
                    resultCharacterIndexes.add(89);//Y
                }
                else if (index == 377 || index == 379 || index == 381) {
                    resultCharacterIndexes.add(90);//Z
                }
                else if (index >= 224 && index <= 230 || index == 257 || index == 259 || index == 261) {
                    resultCharacterIndexes.add(97);//a
                }
                else if (index == 231 || index == 263 || index == 265 || index == 267 || index == 269) {
                    resultCharacterIndexes.add(99);//c
                }
                else if (index == 271 || index == 273) {
                    resultCharacterIndexes.add(100);//d
                }
                else if (index >= 232 && index <= 235 || index == 240 || index == 275 || index == 277 || index == 279 || index == 281 || index == 283) {
                    resultCharacterIndexes.add(101);//e
                }
                else if (index == 285 || index == 287 || index == 289 || index == 291) {
                    resultCharacterIndexes.add(103);//g
                }
                else if (index == 293 || index == 295) {
                    resultCharacterIndexes.add(104);//h
                }
                else if (index >= 236 && index <= 239 || index == 297 || index == 299 || index == 301 || index == 303) {
                    resultCharacterIndexes.add(105);//i
                }
                else if (index == 309) {
                    resultCharacterIndexes.add(106);//j
                }
                else if (index == 311 || index == 312) {
                    resultCharacterIndexes.add(107);//k
                }
                else if (index == 314 || index == 316 || index == 318 || index == 320 || index == 322) {
                    resultCharacterIndexes.add(108);//l
                }
                else if (index == 241) {
                    resultCharacterIndexes.add(110);//n
                }
                else if (index >= 242 && index <= 248 || index == 333 || index == 335 || index == 337) {
                    resultCharacterIndexes.add(111);//o
                }
                else if (index == 341 || index == 343 || index == 345) {
                    resultCharacterIndexes.add(114);//r
                }
                else if (index == 223 || index == 347 || index == 349 || index == 351 || index == 353) {
                    resultCharacterIndexes.add(115);//s
                }
                else if (index == 355 || index == 357 || index == 359) {
                    resultCharacterIndexes.add(116);//t
                }
                else if (index >= 249 && index <= 252 || index == 361 || index == 363 || index == 365 || index == 367 || index == 369 || index == 371) {
                    resultCharacterIndexes.add(117);//u
                }
                else if (index == 373) {
                    resultCharacterIndexes.add(119);//w
                }
                else if (index == 253 || index == 255 || index == 375) {
                    resultCharacterIndexes.add(121);//y
                }
                else if (index == 378 || index == 380 || index == 382) {
                    resultCharacterIndexes.add(122);//z
                }
                else {
                    resultCharacterIndexes.add(index);
                }
            }
            source = String.fromCharArray(resultCharacterIndexes);
        }
        return source;
    }
[Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!

,

Rondam RamblingsCan Trump win? Let's do the math

There are a total of 2472 delegates going to the Republican convention in July.  At the moment, Donald Trump has 756.  He needs 1237 to win the nomination on the first ballot (I think it's extremely unlikely he could win any other way), a difference of 481.  There are 769 delegates left to choose, so Trump needs to win about 64% of them.  That's a tall order.  To date, 1703 delegates have been

CryptogramKuwaiti Government will DNA Test Everyone

There's a new law that will enforce DNA testing for everyone: citizens, expatriates, and visitors. They promise that the program "does not include genealogical implications or affects personal freedoms and privacy."

I assume that "visitors" includes tourists, so presumably the entry procedure at passport control will now include a cheek swab. And there is nothing preventing the Kuwaiti government from sharing that information with any other government.

Sociological ImagesTrump’s Wall Would Mean More, not Fewer Undocumented Immigrants in the US

Most Americans are either attracted to or repulsed by Donald Trump’s strong rhetoric around the “wall” between the US and Mexico. His plan is to build one taller and wider than the ones we already have, on the assumption that this will curb undocumented immigration and the number of migrants who live here.

But the idea isn’t just exciting or offensive, depending on who you’re talking to, it’s also wrong-headed. That is, there’s no evidence that building a better wall will accomplish what Trump wants and, in fact, the evidence suggests the opposite.

.

The data comes from a massive 30-year study led by sociologist Douglas Massey, published last month at the American Journal of Sociology and summarized at Made in America. He and his colleagues collected the migration histories of about 150,000 Mexican nationals who had lived for at least a time in the US and compared them with border policy. They found that:

  • More border enforcement changed where migrants crossed into the US, but not whether they did. More migrants were apprehended, but this simply increased the number of times they had to try to get across. It didn’t slow the flow.
  • Border enforcement did, though, make crossing more expensive and more dangerous, which meant that migrants that made it to the US were less likely to leave. Massey and his colleagues estimate that there are about 4 million more undocumented migrants in the US today than there would have been in the absence of enforcement.
  • Those who stayed tended to disperse. So, while once migrants were likely to stay along the border and go back and forth to Mexico according to labor demands, now they are more likely to be settled all across the US.

In any case, the economic impetus to migrate has declined; for almost a decade, the flow of undocumented migrants has been zero or even negative (more leaving than coming). So, Trump would be building a wall at exactly the moment that undocumented Mexican immigration has slowed. To put it in his terms, a wall would be a bad investment.

Lisa Wade is a professor at Occidental College and the co-author of Gender: Ideas, Interactions, Institutions. Find her on TwitterFacebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

Chaotic Idealism"Because he's crazy!"

Look, folks, when you write your villains, stop using "They're crazy!" as a motivation. Crazy is not a character trait. Crazy is not a reason to be a villain. Crazy is a thing that happens to villains, heroes, and bystanders, and it doesn't turn the one into the other. I'm looking at you, Batman writers.

Using "crazy" as a reason for your character to do what they're doing is not good enough. Yeah, your villain can be crazy. So can your hero, if you like. Cool. I'm all about inclusion. But if your villain is out there terrorizing Gotham "because he's crazy!", then you're being a lazy, lazy writer.

Oh, I know it's tempting. You're sitting there going, "So what's my villain's motivation?" and you have to think about their backstory and their goals and all of that; but hey, it's lunchtime and you really want that burrito. So instead of, y'know, thinking, you go, "I know how I'm gonna solve all this! I'm just gonna make my villain crazy! Crazy people don't need motivations!"

Yes. They. Do.

Don't use crazy as a cop-out. It's bad writing.

Krebs on SecurityUS-CERT to Windows Users: Dump Apple Quicktime

Microsoft Windows users who still have Apple Quicktime installed should ditch the program now that Apple has stopped shipping security updates for it, warns the Department of Homeland Security‘s U.S. Computer Emergency Readiness Team (US-CERT). The advice came just as researchers are reporting two new critical security holes in Quicktime that likely won’t be patched.

quicktimeUS-CERT cited an April 14 blog post by Christopher Budd at Trend Micro, which runs a program called Zero Day Initiative (ZDI) that buys security vulnerabilities and helps researchers coordinate fixing the bugs with software vendors. Budd urged Windows users to junk Quicktime, citing two new, unpatched vulnerabilities that ZDI detailed which could be used to remotely compromise Windows computers.

“According to Trend Micro, Apple will no longer be providing security updates for QuickTime for Windows, leaving this software vulnerable to exploitation,” US-CERT wrote. The advisory continued:

“Computers running QuickTime for Windows will continue to work after support ends. However, using unsupported software may increase the risks from viruses and other security threats. Potential negative consequences include loss of confidentiality, integrity, or availability of data, as well as damage to system resources or business assets. The only mitigation available is to uninstall QuickTime for Windows. Users can find instructions for uninstalling QuickTime for Windows on the Apple Uninstall QuickTime page.”

While the recommendations from US-CERT and others apparently came as a surprise to many, Apple has been distancing itself from QuickTime on Windows for some time now. In 2013, the Cupertino, Calif. tech giant deprecated all developer APIs for Quicktime on Windows.

Apple shipped an update to Quicktime in January 2016 that removed the Quicktime browser plugin on Windows systems, meaning the threat from browser-based attacks on Quicktime flaws was largely mitigated over the past few months for Windows users who have been keeping up to date with the latest version. Nevertheless, if you have Quicktime on a Windows box — do yourself a favor and get rid of it.

Update, Apr. 21, 10:00 a.m. ET: Apple has finally posted a support document online that explains QuickTime 7 for Windows is no longer supported by Apple. See the full advisory here.

CryptogramSecurity Risks of Shortened URLs

Shortened URLs, produced by services like bit.ly and goo.gl, can be brute-forced. And searching random shortened URLs yields all sorts of secret documents. Plus, many of them can be edited, and can be infected with malware.

Academic paper. Blog post with lots of detail.

Worse Than FailureMercy the Mercenary in… The (not so) Final Countdown

In the continuing saga of Mercy the Mercenary, she continues to struggle with a political campaign- Rockwood for Governor- and its backwards approach to IT. Last time, she had an uphill battle getting the kit to keep their website up.

The request seemed so simple, Mercy knew. Embed a video inside an email message.

“I’m not saying it’s hard,” she said. “I’m saying it’s impossible.”

“The boss wants it to happen,” Ellis said. “And that means—“

“That means we trust her judgment,” Sullivan interrupted. “But why don’t you try telling us why it can’t happen, Mercy? You can use kindergarten terms so Ellis can understand.”

Mercy sighed. “The HTML in email messages gets scrubbed by the client. What that means is anything the email client thinks is suspicious, such as a script tag or an iframe, gets taken out. The stuff that gets left in is just text formatting and images. Even if you got some embedded video code into a message somehow, email clients don’t have the Flash plugin available to load it.”

Ellis clicked on his trackpad, then spun his laptop around to show Mercy his screen. “Packard got it in somehow.”

Mercy examined the browser window. A message was displayed in Gmail from the Packard for Governor campaign. Just above the fold was what looked like a screenshot with a play button in the center. Just like a video player.

“It’s just an image.” Mercy clicked on it, and the browser spawned a YouTube page in a new tab, a Packard for Governor campaign ad playing at full volume. Ellis hit pause halfway into the sentence “I’m Harold Packard and I approve this message.”

“Look,” Sullivan said, "Rockwood’s really stuck on getting this in. Is there any workaround?”

“It won’t be a real video … but we can embed a .gif image.” Mercy, on her own laptop, found the Packard ad and ran it through an online .gif converter. It produced the first three seconds of the ad. “I can add a caption at the bottom, put a big play button in the center, make it look exactly like a video player. There’s no sound, but who wants sound to play when they open an email message?”

“Don’t mention that to Rockwood,“ Sullivan said. ”He’ll want sound playing on every email message we send.”

Mercy downloaded the .gif, noticing the size of the file. “Hey, we don’t want to embed a 3MB image in each email message. We should put this on our CDN server and link to it from there.”

“Won’t there be that little X icon if we do that?” Ellis asked. “I hate it when Gmail does it to our messages.”

“Most subscribers have our newsletter added to their whitelist, so they won’t all see that,” Mercy said. “We attach the template images to the email message, but they’re only a few kilobytes. We can’t send 3MB messages to all of our subscribers.”

The Teapot prize from the British gameshow 'Countdown'

“Our email provider’s giving us a discount this month,“ Sullivan said, ”and I know if Rockwood’s going to want to see the video in the email as soon as it’s out. And if he sees that X icon, he won’t be happy about it.”

Mercy shrugged. “Well, what’s 3MB when people have a TB of storage for email nowadays? I’ll get it done.”

“That’s the kind of motivation I like to see.”

Mercy turned to see Rockwood in the doorway of the conference room. She wasn’t sure when he’d walked in.

“Oh, I’ll make sure that video gets sent out,” Ellis said, with a straight face.

“Ellis, if I was appointed Emperor of Florida tomorrow, you’d say it was all because of you.” Rockwood winked at the man as he sat in an empty chair. Mercy rolled her eyes.

“Now, don’t give me that,” Rockwood said, addressing Mercy. “I have a task just for you. See, I need a countdown clock. You know, like the one in Mission: Impossible, the bomb timer.”

“John,” Sullivan asked, “when is this … bomb set to go off?”

“August 1st, one second after midnight. Think you can do that?”

“Uh, sure,” Mercy said. “What happens when it gets to zero?”

“I’ll let you know.” With that, Rockwood left the conference room.

“Well,” Sullivan said, “I’m sure he’ll let us know just what he wants in due time. So, can you make a countdown timer, Mercy?”

“Sure can.”

// TODO

Cloistered away in her apartment, far from the squawking of the volunteers on the phone lines, Mercy built a countdown clock. Javascript programming didn’t lend itself well precision timing, but coding timers was easier than in other languages due to the setTimeout()/setInterval() methods and functions as first-class objects. In fact, Mercy had written countdown clocks for several past clients. She grabbed a recent copy of that code and modified it for Rockwood’s clock.

$(function(){
	var launchTime = Date.parse(‘2016-08-01 00:00:01 EDT’);
	var $clockTextDays = $(‘#countdown-days’);
	var $clockTextHours = $(‘#countdown-hours’);
	var $clockTextMinutes = $(‘#countdown-minutes’);
	var $clockTextSeconds = $(‘#countdown-seconds’);
	window.setInterval(function(){
		var delta = launchTime - (new Date()).getTime();
		if (delta <= 0) {
			// TODO: ASK SULLIVAN WHAT SHOULD HAPPEN
		}
		else {
			var remainder = delta;
			var deltaDays = Math.floor(delta / 86400000);
			remainder -= deltaDays * 86400000;
			var deltaHours = Math.floor(remainder / 3600000);
			remainder -= deltaHours * 3600000;
			var deltaMinutes = Math.floor(remainder / 60000);
			remainder -= deltaMinutes * 60000;
			var deltaSeconds = Math.floor(remainder / 1000);
			$clockTextDays.text(deltaDays + ‘ days’);
			$clockTextHours.text(deltaHours + ‘ hours’);
			$clockTextMinutes.text(deltaMinutes + ‘ minutes’);
			$clockTextSeconds.text(deltaSeconds + ‘ seconds’);
		}
	}, 100);
})

She put together a quick concept image in Photoshop and emailed it to Sullivan for her approval (she had learned not to run anything past Ellis unless she wanted him to sabotage it in some way).

Sullivan’s response was quick: Good, but less jaggies. -Barbie

Mercy re-examined the concept image in Photoshop to see if she had left a filter on, then realized that Sullivan was viewing her 1200-pixel wide image on a 700 pixel laptop screen. With Sullivan’s implied approval, she coded the styles for the clock, using an LCD-style font for the digits and placing the entire timer inside a velvet-lined briefcase.

The exercise took less than a couple hours, most of it adjusting the styles for cross-browser compatibility. She pushed the code up to the web server hamilton, which replicated it to the others.

The thought of that TODO comment drifted out of the back of her mind as she went to sleep that night.

In With A Bang

WE’RE ON TV! -Barbie

The notification on her phone woke Mercy at 7AM. She wondered which channel Sullivan was referring to, but a Google Alert in her inbox told her. Fox News, taking an interest in the iconoclastic Rockwood facing off against their chosen favorite Harold Packard, had Mercy’s briefcase-framed countdown clock displayed behind the hosts of the morning news, with the headline “Rockwood Campaign About To Explode?” in the ticker below the talking heads.

Back at headquarters later that morning, Mercy checked the logs of the founding father servers. Fox News, Drudge Report, Breitbart, HuffPo, and Politico had all linked to the timer. The common thread of speculation between all of them was exactly what John Rockwood’s timer was counting down to. Mercy brought this up to Sullivan, but she brushed it off.

“This is the best kind of publicity,” she said. “We’re the belle at the ball and every man wants to dance.”

“More like Cinderella,” Mercy said, “and that pumpkin carriage turns back into a pumpkin at midnight. All that timer does is count down to a date. When the date comes and goes, it won’t actually do anything unless we tell it to.”

“The boss will know what to do then,” Sullivan said, as if it were no concern at all. “Anyway, you have approval to begin building our new blogging platform. Just make sure it looks like Ellis’s template.”

Mercy sighed at coding another one of Ellis’s ugly concepts, putting the timer out of her mind.

Out With A Whimper

The morning of August 1, Mercy’s phone buzzed with another Google Alert. This one was from Drudge Report: ROCKWOOD’S BOMB FIZZLES.

Mercy tried to remember in which of Rockwood’s speeches he had mentioned a bomb going off. She couldn’t, so she went back to sleep.

The phone buzzed again. This time, it was Sullivan calling her.

“That countdown clock is counting in the negative!” Sullivan was breathless. “It’s just going negative one, negative two, negative three…”

Mercy remembered. “You never told me what it should do.”

“I thought we told you when we gave you the assignment.”

“No, I asked, and no one would give me a straight answer.”

Mercy dressed and drove to headquarters. Ellis spotted her as she came in, his eyes widening like a shark’s as it smelled blood. “Your countdown clock is broken!”

“Okay, Ellis.” Mercy’s teeth ground together. “What was it supposed to do?”

Ellis waved his hand dismissively. “I’m sure it goes to a press release or something. You should check your email.”

Mercy pulled out her phone. She entered the phrase “countdown clock” in her email client. The only search results were the brief exchange between Sullivan and herself approving the concept image.

In the back hallway leading to the conference room, she saw Sullivan talking to Rockwood. The candidate looked a bit listless. Sullivan was throwing her hands up in frustration. She rubbed his back, said something she couldn’t make out, and made her way to where she and Ellis were standing.

“Funny,“ Sullivan said, ”John doesn’t actually remember telling you to do the countdown clock. I reminded him, but it didn’t jog his memory. He’s such a busy man, I’m sure it slipped his mind.”

Ellis nodded vigorously. “You should have written it down,” he said again to Mercy.

“In fact,” Sullivan added, “I recall the conversation exactly. He said he would tell us later, and wouldn’t you know, he never did. Not his fault, he’s just been so busy!” Her voice sounded flustered. “But I know what it should link to. It’ll go to our first blog post from the campaign! Can you do that?”

Mercy nodded. “The page won’t actually redirect to it unless they refresh the page.”

“I’m surprised you couldn’t have guessed that,” Ellis said, baring a toothy grin. “I mean, anyone—“

“Harry,” Sullivan said, leaning in so close her face was an inch from Ellis’s, “not everyone here is as close to John as you are. In fact, he could really use your help right now. Why don’t you see to it.”

Mercy’s blood ran cold as Ellis shuffled back to see to Rockwood.

Smoothed Over

Mercy stared listlessly at the landing page of the Rockwood for Governor campaign blog, titled “Righteous Rants,” picked by consensus from the volunteers.

Ellis had spent the entire day in the conference room with Rockwood. He only came out once to talk to Mercy, complaining that the mobile view of the blog template didn’t use his hover menus. Mercy explained as best she could that hover menus don’t work on mobile because you can’t actually hover a cursor on a touchscreen. He seemed much more complacent than he had been earlier when he confronted her about the countdown clock.

She was watching a video of one of Rockwood’s stump speeches, filled with his typical long pauses, when the candidate himself emerged from the back. His shirt was rumpled and untucked, but his personality was fresh and cheerful. After greeting some of the volunteers (and fixing his shirt), he came over to Mercy. “I appreciate how much work you put into that little countdown,” he said, shaking her hand. “Wish I would have thought of that! Barbie tells me our page impressions are through the roof!” He whispered, in a conspiratorial tone, “I don’t know what kinds of impressions she’s talking about, but I hope they’re good ones!” He slapped Mercy’s shoulder and left for other matter, Ellis trailing behind. Mercy smiled, remembering the man she saw back in that high school gym several months ago.

Mercy decided that the dropdown menus would open on click at mobile sizes, hover at typical desktop resolutions. They hadn’t fired her for building a countdown clock that did nothing after counting down. Surely she had some executive power when it came to building CSS menus.

But deep down, she realized something was very wrong with Rockwood, and only Ellis and Sullivan knew what that could be.

[Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!

Worse Than FailureMercy the Mercenary in… The (not so) Final Countdown

In the continuing saga of Mercy the Mercenary, she continues to struggle with a political campaign- Rockwood for Governor- and its backwards approach to IT. Last time, she had an uphill battle getting the kit to keep their website up.

The request seemed so simple, Mercy knew. Embed a video inside an email message.

“I’m not saying it’s hard,” she said. “I’m saying it’s impossible.”

“The boss wants it to happen,” Ellis said. “And that means—“

“That means we trust her judgment,” Sullivan interrupted. “But why don’t you try telling us why it can’t happen, Mercy? You can use kindergarten terms so Ellis can understand.”

Mercy sighed. “The HTML in email messages gets scrubbed by the client. What that means is anything the email client thinks is suspicious, such as a script tag or an iframe, gets taken out. The stuff that gets left in is just text formatting and images. Even if you got some embedded video code into a message somehow, email clients don’t have the Flash plugin available to load it.”

Ellis clicked on his trackpad, then spun his laptop around to show Mercy his screen. “Packard got it in somehow.”

Mercy examined the browser window. A message was displayed in Gmail from the Packard for Governor campaign. Just above the fold was what looked like a screenshot with a play button in the center. Just like a video player.

“It’s just an image.” Mercy clicked on it, and the browser spawned a YouTube page in a new tab, a Packard for Governor campaign ad playing at full volume. Ellis hit pause halfway into the sentence “I’m Harold Packard and I approve this message.”

“Look,” Sullivan said, "Rockwood’s really stuck on getting this in. Is there any workaround?”

“It won’t be a real video … but we can embed a .gif image.” Mercy, on her own laptop, found the Packard ad and ran it through an online .gif converter. It produced the first three seconds of the ad. “I can add a caption at the bottom, put a big play button in the center, make it look exactly like a video player. There’s no sound, but who wants sound to play when they open an email message?”

“Don’t mention that to Rockwood,“ Sullivan said. ”He’ll want sound playing on every email message we send.”

Mercy downloaded the .gif, noticing the size of the file. “Hey, we don’t want to embed a 3MB image in each email message. We should put this on our CDN server and link to it from there.”

“Won’t there be that little X icon if we do that?” Ellis asked. “I hate it when Gmail does it to our messages.”

“Most subscribers have our newsletter added to their whitelist, so they won’t all see that,” Mercy said. “We attach the template images to the email message, but they’re only a few kilobytes. We can’t send 3MB messages to all of our subscribers.”

The Teapot prize from the British gameshow 'Countdown'

“Our email provider’s giving us a discount this month,“ Sullivan said, ”and I know if Rockwood’s going to want to see the video in the email as soon as it’s out. And if he sees that X icon, he won’t be happy about it.”

Mercy shrugged. “Well, what’s 3MB when people have a TB of storage for email nowadays? I’ll get it done.”

“That’s the kind of motivation I like to see.”

Mercy turned to see Rockwood in the doorway of the conference room. She wasn’t sure when he’d walked in.

“Oh, I’ll make sure that video gets sent out,” Ellis said, with a straight face.

“Ellis, if I was appointed Emperor of Florida tomorrow, you’d say it was all because of you.” Rockwood winked at the man as he sat in an empty chair. Mercy rolled her eyes.

“Now, don’t give me that,” Rockwood said, addressing Mercy. “I have a task just for you. See, I need a countdown clock. You know, like the one in Mission: Impossible, the bomb timer.”

“John,” Sullivan asked, “when is this … bomb set to go off?”

“August 1st, one second after midnight. Think you can do that?”

“Uh, sure,” Mercy said. “What happens when it gets to zero?”

“I’ll let you know.” With that, Rockwood left the conference room.

“Well,” Sullivan said, “I’m sure he’ll let us know just what he wants in due time. So, can you make a countdown timer, Mercy?”

“Sure can.”

// TODO

Cloistered away in her apartment, far from the squawking of the volunteers on the phone lines, Mercy built a countdown clock. Javascript programming didn’t lend itself well precision timing, but coding timers was easier than in other languages due to the setTimeout()/setInterval() methods and functions as first-class objects. In fact, Mercy had written countdown clocks for several past clients. She grabbed a recent copy of that code and modified it for Rockwood’s clock.

$(function(){
	var launchTime = Date.parse(‘2016-08-01 00:00:01 EDT’);
	var $clockTextDays = $(‘#countdown-days’);
	var $clockTextHours = $(‘#countdown-hours’);
	var $clockTextMinutes = $(‘#countdown-minutes’);
	var $clockTextSeconds = $(‘#countdown-seconds’);
	window.setInterval(function(){
		var delta = launchTime - (new Date()).getTime();
		if (delta <= 0) {
			// TODO: ASK SULLIVAN WHAT SHOULD HAPPEN
		}
		else {
			var remainder = delta;
			var deltaDays = Math.floor(delta / 86400000);
			remainder -= deltaDays * 86400000;
			var deltaHours = Math.floor(remainder / 3600000);
			remainder -= deltaHours * 3600000;
			var deltaMinutes = Math.floor(remainder / 60000);
			remainder -= deltaMinutes * 60000;
			var deltaSeconds = Math.floor(remainder / 1000);
			$clockTextDays.text(deltaDays + ‘ days’);
			$clockTextHours.text(deltaHours + ‘ hours’);
			$clockTextMinutes.text(deltaMinutes + ‘ minutes’);
			$clockTextSeconds.text(deltaSeconds + ‘ seconds’);
		}
	}, 100);
})

She put together a quick concept image in Photoshop and emailed it to Sullivan for her approval (she had learned not to run anything past Ellis unless she wanted him to sabotage it in some way).

Sullivan’s response was quick: Good, but less jaggies. -Barbie

Mercy re-examined the concept image in Photoshop to see if she had left a filter on, then realized that Sullivan was viewing her 1200-pixel wide image on a 700 pixel laptop screen. With Sullivan’s implied approval, she coded the styles for the clock, using an LCD-style font for the digits and placing the entire timer inside a velvet-lined briefcase.

The exercise took less than a couple hours, most of it adjusting the styles for cross-browser compatibility. She pushed the code up to the web server hamilton, which replicated it to the others.

The thought of that TODO comment drifted out of the back of her mind as she went to sleep that night.

In With A Bang

WE’RE ON TV! -Barbie

The notification on her phone woke Mercy at 7AM. She wondered which channel Sullivan was referring to, but a Google Alert in her inbox told her. Fox News, taking an interest in the iconoclastic Rockwood facing off against their chosen favorite Harold Packard, had Mercy’s briefcase-framed countdown clock displayed behind the hosts of the morning news, with the headline “Rockwood Campaign About To Explode?” in the ticker below the talking heads.

Back at headquarters later that morning, Mercy checked the logs of the founding father servers. Fox News, Drudge Report, Breitbart, HuffPo, and Politico had all linked to the timer. The common thread of speculation between all of them was exactly what John Rockwood’s timer was counting down to. Mercy brought this up to Sullivan, but she brushed it off.

“This is the best kind of publicity,” she said. “We’re the belle at the ball and every man wants to dance.”

“More like Cinderella,” Mercy said, “and that pumpkin carriage turns back into a pumpkin at midnight. All that timer does is count down to a date. When the date comes and goes, it won’t actually do anything unless we tell it to.”

“The boss will know what to do then,” Sullivan said, as if it were no concern at all. “Anyway, you have approval to begin building our new blogging platform. Just make sure it looks like Ellis’s template.”

Mercy sighed at coding another one of Ellis’s ugly concepts, putting the timer out of her mind.

Out With A Whimper

The morning of August 1, Mercy’s phone buzzed with another Google Alert. This one was from Drudge Report: ROCKWOOD’S BOMB FIZZLES.

Mercy tried to remember in which of Rockwood’s speeches he had mentioned a bomb going off. She couldn’t, so she went back to sleep.

The phone buzzed again. This time, it was Sullivan calling her.

“That countdown clock is counting in the negative!” Sullivan was breathless. “It’s just going negative one, negative two, negative three…”

Mercy remembered. “You never told me what it should do.”

“I thought we told you when we gave you the assignment.”

“No, I asked, and no one would give me a straight answer.”

Mercy dressed and drove to headquarters. Ellis spotted her as she came in, his eyes widening like a shark’s as it smelled blood. “Your countdown clock is broken!”

“Okay, Ellis.” Mercy’s teeth ground together. “What was it supposed to do?”

Ellis waved his hand dismissively. “I’m sure it goes to a press release or something. You should check your email.”

Mercy pulled out her phone. She entered the phrase “countdown clock” in her email client. The only search results were the brief exchange between Sullivan and herself approving the concept image.

In the back hallway leading to the conference room, she saw Sullivan talking to Rockwood. The candidate looked a bit listless. Sullivan was throwing her hands up in frustration. She rubbed his back, said something she couldn’t make out, and made her way to where she and Ellis were standing.

“Funny,“ Sullivan said, ”John doesn’t actually remember telling you to do the countdown clock. I reminded him, but it didn’t jog his memory. He’s such a busy man, I’m sure it slipped his mind.”

Ellis nodded vigorously. “You should have written it down,” he said again to Mercy.

“In fact,” Sullivan added, “I recall the conversation exactly. He said he would tell us later, and wouldn’t you know, he never did. Not his fault, he’s just been so busy!” Her voice sounded flustered. “But I know what it should link to. It’ll go to our first blog post from the campaign! Can you do that?”

Mercy nodded. “The page won’t actually redirect to it unless they refresh the page.”

“I’m surprised you couldn’t have guessed that,” Ellis said, baring a toothy grin. “I mean, anyone—“

“Harry,” Sullivan said, leaning in so close her face was an inch from Ellis’s, “not everyone here is as close to John as you are. In fact, he could really use your help right now. Why don’t you see to it.”

Mercy’s blood ran cold as Ellis shuffled back to see to Rockwood.

Smoothed Over

Mercy stared listlessly at the landing page of the Rockwood for Governor campaign blog, titled “Righteous Rants,” picked by consensus from the volunteers.

Ellis had spent the entire day in the conference room with Rockwood. He only came out once to talk to Mercy, complaining that the mobile view of the blog template didn’t use his hover menus. Mercy explained as best she could that hover menus don’t work on mobile because you can’t actually hover a cursor on a touchscreen. He seemed much more complacent than he had been earlier when he confronted her about the countdown clock.

She was watching a video of one of Rockwood’s stump speeches, filled with his typical long pauses, when the candidate himself emerged from the back. His shirt was rumpled and untucked, but his personality was fresh and cheerful. After greeting some of the volunteers (and fixing his shirt), he came over to Mercy. “I appreciate how much work you put into that little countdown,” he said, shaking her hand. “Wish I would have thought of that! Barbie tells me our page impressions are through the roof!” He whispered, in a conspiratorial tone, “I don’t know what kinds of impressions she’s talking about, but I hope they’re good ones!” He slapped Mercy’s shoulder and left for other matter, Ellis trailing behind. Mercy smiled, remembering the man she saw back in that high school gym several months ago.

Mercy decided that the dropdown menus would open on click at mobile sizes, hover at typical desktop resolutions. They hadn’t fired her for building a countdown clock that did nothing after counting down. Surely she had some executive power when it came to building CSS menus.

But deep down, she realized something was very wrong with Rockwood, and only Ellis and Sullivan knew what that could be.

[Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!

Planet Linux AustraliaBen Martin: Making PCB with a hobby CNC machine

One of the main goals I had in mind when getting a CNC "engraving" machine was to make PCB at home. It's sort of full circle to the '70s I guess. Only instead of using nasty chemicals I just have the engraver scratch off an isolation path between traces. Or so the plan goes.


My "hello world" board is the above controller for a 3d printer. This is a follow up to the similar board I made to help use the CNC itself. For a 3d printer I added buttons to set Z=0.1 height and a higher Z height to aid in homing. The breakout headers on the bottom right are for the ESP8266 daughter board. The middle chip is an MCP32017 gpio extender. I've had good experiences using TWI on the ESP8266 and the MCP overcomes the pin limitations quite nicely. It also gives all the buttons a nice central place to go :)

The 3v3 regulator makes the whole show a plug in the AA pack and go type board. The on/off switch is the physical connection to an external battery.

One step closer to the design in the morning, physically create in the afternoon, and use in the evening goal.

Valerie AuroraHOWTO therapy: what psychotherapy is, how to find a therapist, and when to fire your therapist

I read this hilarious post by Amanda Rosenberg called “I Asked My Therapist How to Find a Therapist” and cry-laughed the whole way through it. (TL;DR: ¯\_(ツ)_/¯) Because it’s so true: when you need a therapist the most is when you have the least energy, organization, and resilience, all qualities that are helpful in finding a therapist in the U.S. (Most people muddle through with desperation, panic, and flailing instead.)

Finding a therapist doesn’t have to be this hard. As an American, I was amazed to learn that many countries offer free government-provided mental health care. It’s not a panacea – you still have to jump through hoops and fill out paperwork and get referrals – but it does show that there’s no inherent reason why finding a therapist has to be so. Damned. Hard.

Personally, I love therapy – or rather, I love what therapy has done for me and how much happier I am after doing therapy for many years. I have had to find a number of therapists in my life, and recently I used what I’ve learned to help several people I know find good therapists. I figured I’d share what I learned in this blog post, starting with how to find a therapist since that’s the question I get asked most, and then going on to things like how therapy works and how to pay for therapy. It got kind of long, so here’s a table of contents so that you can skip to the part you’re most interested in.

How do I find a therapist?
What is therapy anyway?
Can I go to therapy if I don’t know what’s wrong?
Can therapy help me?
What if I can’t afford to pay for therapy?
What if I don’t have the free time to go to therapy?
How do I know when I should switch therapists?
How do I know when to stop or reduce frequency of therapy?


How do I find a therapist?

In the case that you are paying privately for a therapist at market rates, here is my recommended algorithm:

  1. Search on Psychology Today for therapists near you.
  2. Optionally, filter your results by therapists who use cognitive behavioral therapy (CBT). (CBT is the most evidence-based method of talk therapy. You don’t have to use it as part of your therapy program, but listing CBT as a method of treatment is a positive sign in a therapist in my experience.)
  3. Add any other constraints that are important to you: gender of the therapist, whether they specialized in LGBT issues, if they are a person of color, etc.
  4. Read the self-descriptions of the therapists until you find three that click with you. Be critical and picky and pay attention to how they present themselves.
  5. Once you have three, schedule a free get-to-know-each-other appointment with each of them, preferably in the same week. Go to all three appointments and tell them what you are worried about happening in therapy as well as the problems that brought you there. Optionally, you can send them the Geek Feminism wiki page for therapists.
  6. After you’ve been to each, talk through your experiences with each therapist with someone you trust and pick one. If none of them seem right, go find three more therapists and repeat.

If you have therapy through government healthcare or healthcare insurance or an Employee Assistance Plan or something similar (see the section on affording therapy), use whatever directory or right to choose providers that you have to get an opportunity to work with three different therapists if at all possible. Therapy is in part about the fit or the match between your style and your therapist’s style; there’s no one size fits all. If you can only work with one therapist at a time, see the section on when to switch therapists to decide when to move on to another therapist.

If you are looking to pay privately for therapy but can’t afford market rates, here are some suggestions for finding therapists to interview:

  • Google search for “cheap therapy [YOUR LOCATION]” or similar phrases
  • Search on Foursquare or Yelp or other review sites for the same
  • Search online for counseling training schools near you (they usually have cheap rates for working with students)
  • Ask anyone you know who often works with disadvantaged folks: social workers, court-appointed advocates, activists, etc.
  • If you have any advocates or healthcare workers caring for you – social workers, legal assistance, nurses, doctors, legal advocates, case workers – ask them for suggestions

A great collection of resources for therapy for people with specific needs (such as a polyamory-friendly therapist) is the MetaFilter wiki page on therapy.


What is therapy anyway?

Therapy/talk therapy/psychotherapy is when a patient talks regularly with a counselor or psychotherapist to figure out new ways to think and act so that they are happier. In particular, many of us have developed beliefs and habits about how to be happy and safe that seem to work in the short run, but that end up making us feel unhappy and unsafe in the long run. The therapist helps you recognize these unhelpful beliefs and habits and change them (or at least stop acting in ways that reinforce them). As my favorite advice blogger, Captain Awkward, puts it: “I think every adult could benefit from a look under the emotional hood at some point in their lives.

Many forms of therapy use your relationship with the therapist as a testing ground for trying out new beliefs and actions. In the U.S., a fairly common frequency for therapy is one hour a week or every two weeks. Therapists who use classical Freudian psychoanalysis (what you see on TV shows or movies most often) like to meet for an hour 3-5 days a week.

The forms of therapy differ, but generally they all work better when you are truthful with your therapist, attend appointments regularly, and do any assigned homework. (Conditions that make any of these tasks hard are harder to treat.) The most important thing is to tell your therapist what you are thinking or feeling about therapy or about them, even if it is things like, “I am afraid of you,” or “I feel sexually attracted to you,” or “I want to say what makes you happy,” or “I hate coming to this appointment” or “I’m embarrassed to be in therapy.” It’s the therapist’s job to not take comments like these personally and to use it to help figure out your beliefs. (If they respond to you saying these things with, e.g., anger, or by seducing you, or making you feel guilty, fire them and find a new therapist immediately.)


Can I go to therapy if I don’t know what’s wrong?

Several people have asked me if it is it helpful to go to therapy if you don’t know what’s wrong, or can’t put your feelings into words. The answer is most decidedly, yes. People often go to therapy because they feel vaguely dissatisfied, or incomplete, anxious, depressed, unhappy, empty, tired, hopeless, unimportant, isolated, angry, sad, ashamed, or any number of feelings. People often feel this way even when their life seems objectively great – great job, great family, great friends, etc. If you have figured out why you have those feelings and can put that into words, that’s wonderful – you have a head-start on working with the therapist to figure out what to do about them. But if you don’t know why you feel the way you do, therapists are good at helping you figuring out why.


Can therapy help me?

Oversimplifying wildly, here are the requirements for therapy to work as I understand them:

  1. Self-motivation: Do you want to change badly enough to do scary hard things?
  2. Self-criticism: Can you accept and internalize criticism?
  3. Self-discipline: Are you willing to put in the effort to change, even if it is hard or scary?

It’s okay if you’re not that good at accepting criticism or at consistently applying yourself, as long as you’re motivated to get better at those two skills for reasons you find compelling. But if you’re going to therapy in order to appease or manipulate someone else, but don’t actually think you need to change, it’s less likely to work. Most of the work of therapy happens outside the time that you meet with your therapist, and you are unlikely to do that work if you don’t see how it benefits yourself. This kind of motivation normally fluctuates – I’ve several times taken a break from therapy because I didn’t care to work on my problems at that particular time. I came back when I was motivated to do the work again, sometimes years later.

Several psychological symptoms or disorders interfere with one or more of self-motivation, self-criticism, or self-discipline. These include (but aren’t limited to) depression, anxiety, difficulty staying focused, narcissistic personality disorder, and antisocial personality disorder (ASPD, better known as psychopathy or sociopathy).

If you have depression, anxiety, or difficulty staying focused, therapy will be harder but still doable with effort and advice from your therapist – after all, these are some of the most common reasons people go to therapy.

Narcissistic personality disorder presents as extreme outward confidence, disregard for the feelings of others, and obsession with personal image. One of the less well-known aspects of narcissism is the inability of the narcissist to accept and internalize criticism. It is difficult to improve at any skill if you can’t directly face and accept feedback on how you are doing it wrong. As a result, someone with narcissism has great difficulty changing anything about themselves for the better. Narcissism is notoriously difficult to treat. (If you’re seriously wondering if you are a narcissist, that is an excellent reason to go to therapy. The answer is probably no, but either way, talking to a therapist is a very good idea at this point.)

People with Antisocial personality disorder (better known as sociopathy or psychopathy) have difficulty caring about or understanding the needs or feelings of others, are impulsive, and have difficulty imagining or caring about negative consequences for their behavior. Many people with ASPD are quite content with their personality and actions and see no reason to change them – although some proportion of people with ASPD decide to be a positive part of society anyway and it’s not clear why they are different. Some proportion of ASPD cases are due to permanent impairment of specific brain structures; they can use therapy to learn different behaviors but won’t ever recover that specific brain function. Other folks have neurotypical brain physiology and acquire ASPD after experiencing intense abuse and trauma. Therapy can sometimes help them recover full function.

Other things that can make it harder to get benefit out of therapy: a mental illness, an addiction, an allergy, not getting enough sleep, and being in an abusive relationship. Often, therapy will have the side effect of helping you to solve or reduce these problems, but fixing them may also take medication, diet changes, social support, and time.


What if I can’t afford to pay for therapy?

I’ll assume if you’re asking this question you don’t have access to state-provided healthcare, such as citizens or residents of Australia, Canada, or many European countries. To start finding therapy in this case, my understanding is that you usually ask your general practitioner or primary care doctor for a referral. The rest of this answer will focus on countries that don’t have this.

The easiest way to get therapy (even in countries that provide it for free through the government) is to have enough disposable income to pay the market price for therapy. If you have the money, go this route. If you don’t have the money, you’ll be amazed at the number of ways to get therapy at below market rate. Here are some of the ways to get therapy at a lower price.

You may have private health insurance that covers therapy. However, relatively few therapists accept the extremely low rates paid by insurance, and those that do often have long waiting lists. If you have more money than emotional energy or time, I recommend not even trying to get therapy paid for by health insurance. Otherwise, go to your health insurance web site and look for information on using your mental health benefits. They will probably have an online directory of providers who take your insurance.

Your employer may have an Employee Assistance Plan that covers therapy, usually a specific number of sessions, like six (which is laughably low but better than nothing). The nice thing about EAPs is that usually there is a hotline that you can call and say, “Help me get a therapist,” and they will do the work. This is extremely helpful!

For the case where you can’t afford full price for therapy, but you could afford, say, 25% – 75% of normal costs, many areas have a school for training counselors with students who need patients to practice on. They usually charge need-based sliding scale fees. Many individual therapists will also charge on a sliding scale basis – just ask when you first speak to them. Many therapists also list their price ranges on their Psychology Today profile. Another option in this price range may be online counseling services like In Your Corner.

If you can’t afford that, many community service organizations provide free counseling as part of their services or can help you find free counseling. Homeless shelters, halfway houses, LGBT youth centers, and addiction centers are some places that will be willing and able to help you find free or very low cost therapy.


What if I don’t have the free time to go to therapy?

If freeing up the time to go to therapy seems impossible, here’s my suggestion: move heaven and earth to go to one appointment and tell the therapist why you don’t have time for regular therapy. Then they will be able to suggest ideas for how to make therapy possible for you, based on their far more extensive experience working with many different patients. Remember, it’s not a matter of having time for therapy, it’s a matter of prioritizing therapy just slightly higher than other things in your life that you spend a couple of hours a week on. If therapy is key to you staying alive and functioning, then it’s worth exploring the options.

Some useful options for some people with little money and an uncertain schedule and certain types of problems are the various twelve step programs that branched off from Alcoholics Anonymous, especially the program for the friends and families of addicts. Twelve step programs are free (funded by small voluntary donations from those who can afford it) and usually have meetings at a variety of times in major metro areas. They also have meetings within prisons and hospitals and even phone-based or online meetings. These programs can also be helpful for people who can afford therapy, and many therapists recommend joining the appropriate twelve step program in addition to therapy. Check out the list of twelve step programs on Wikipedia to get an idea if one of those is a good match for you. Note: there is significant research questioning the effectiveness of twelve step programs compared to naltrexone for ending alcohol and narcotic addictions. I agree that twelve step programs don’t work for everyone and attendance shouldn’t be court-mandated; at the same time, some twelve step programs are helpful for some people and are definitely cheaper than most therapy.

One last plug for trying to make therapy work in the face of obstacles from my favorite advice blogger, Captain Awkward: “I recommend therapy here a lot. And I will keep doing it. Even though it is often prohibitively expensive. And/or difficult to locate. And/or difficult to acquire once you do locate it and can maybe afford it. I have a very strong bias in favor of therapy/counseling/mental health services because I have found them to be personally extremely helpful to me and to people I love – some of whom are alive and breathing because they sought out mental health services in time to save their own lives.”


How do I know when I should switch therapists?

Sometimes you aren’t done with therapy but you need to work with a different therapist. This can happen for a lot of reasons. The easy reasons are things like: you develop a symptom or a condition that the therapist doesn’t feel qualified to treat, you move away and they aren’t willing to do therapy over the phone or Internet, or you can no longer afford to pay this therapist’s rate. The less obvious reasons are when therapy isn’t working for some subtler reason: you’re still showing up to therapy and doing homework, but things aren’t progressing. Are they not working because you aren’t ready or doing the work, or is it because you and the therapist are a bad match for each other, or is it because the therapist is bad at their job?

Here are some red flags for therapist relationships that aren’t working out and should probably be ended:

  • The therapist creeps you out (no need to put it into words or get more specific)
  • The therapist attempts to make you feel guilty
  • The therapist makes any kind of sexual advance (or accepts your sexual advances – they are in the position of power and should never accept your advances if you make them)
  • You find yourself unable to stop lying to the therapist
  • The therapist talks about themselves for more than a few minutes per session
  • The therapist does things that make you feel you need to care for the therapist (e.g., becomes visibly upset and requires soothing from you)
  • The therapist “one-ups” you by sharing information about themselves that inhibits you from speaking about your own comparatively minor problems
  • The therapist is unable to hide their anger in session with you
  • You feel belittled or smaller or beaten down after sessions
  • The therapist relies on information provided by your abusers or an unqualified third party (e.g., a parent defining what is wrong with their child)
  • The therapist dismisses your feelings (this is different from searching for underlying feelings or first feelings that turn into your current feelings, a normal activity)
  • You find yourself “accidentally” missing appointments (though this could be a sign that you need to end therapy entirely too)
  • The therapist says things or takes actions that make you feel like you are broken or weird
  • The therapist tells you that something is concerning or bad, but does not help you address it
  • The therapist doesn’t make an effort to understand things that are important to you, like your job or online community
  • The therapist has difficulty remembering important facts about you between sessions
  • The therapist can’t hide that they don’t share basic values with you, such as feminism
  • You find it very hard to tell them that therapy isn’t working for you in some way
  • The therapist offers advice outside the boundaries of the therapy relationship (e.g., about sports or nutrition)
  • You are comfortable and unchallenged in most of your sessions
  • Your take away from most sessions is that you are a really great person who is doing nothing wrong and doesn’t need to change anything but for some reason you need to keep coming to therapy
  • You feel like you are able to fool or charm or manipulate the therapist into doing what you want

Overall, you should feel like your therapist is supporting you in doing difficult, painful, but necessary work. If seeing your therapist makes you feel worthless or helpless or more self-critical, or if they simply affirm you without helping you grow in ways that are difficult for you, you’re not getting the help you need.

Here are some normal (but not necessary) experiences in a relationship with a therapist that is working:

  • You cry. A lot. In session, out of session, on the way to sessions, at work, at home, everywhere
  • You feel sadness and grief more intensely than you have in years
  • You feel strong guilt and anxiety (but not as a direct result of the therapist’s actions or words)
  • You are simultaneously dreading and looking forward to your next appointment
  • You get angry with your therapist in the session (but they do not express anger towards you)
  • You avoid appointments because you don’t want to talk about a specific subject
  • You have to drag yourself into the appointment
  • You sit silently for most or all of the appointment
  • You want to say something to your therapist but you are afraid to and spend the whole appointment avoiding saying it
  • You want to please the therapist and be the perfect patient
  • You don’t want to make your therapist feel sad by telling them unpleasant things you have experienced
  • You deliberately insult or shock or act rudely towards your therapist
  • You show up late to appointments
  • Your therapist reminds you of someone important in your life (mother, ex-husband, etc.)
  • You feel guilty for taking up the therapist’s time
  • You feel like you should be making faster progress
  • You are really tired after an appointment

You can also be having any of these normal experiences with a therapist who is still a bad match for you, just don’t think that these experiences alone are a bad sign.


How do I know when to stop or reduce frequency of therapy?

Spending time and money on therapy has diminishing returns at some point for many people, and at some point you can have a perfectly fine working relationship with a therapist but not have any motivation to continue therapy. Some signs that it might be time for you to reduce frequency or end therapy are:

  • You start forgetting your appointments because you aren’t thinking about what you’re going to discuss at the next one
  • You made a lot of progress in one area of your life but you aren’t much interested in working on any other area right now
  • You feel like you aren’t connecting with your therapist after several weeks
  • Your appointments are uniformly boring
  • You have difficulty thinking of things to say (as distinct from having things to say but not wanting to say them)
  • You keep cancelling your appointments because other things are more important

It can be uncomfortable bringing up the topic of ending therapy with your therapist. Keep in mind that they have been through this many times and that for them, it’s like having a student graduate (in the best case). Just say, “Hey, I’m starting to wonder how much longer I should be in therapy. What do you think?” If you are worried that you want to end therapy for the wrong reasons, or shouldn’t end therapy, your therapist is a good person to discuss that with.

I hope some of this advice is useful to you! I love therapy and it has made me a much happier and healthier person – after years and years of difficult hard work and buckets of tears, so don’t give up too quickly. I wish you all the best for your journey towards greater happiness!


Tagged:
advice, therapy

Planet Linux AustraliaOpenSTEM: OpenSTEM 3D Printing and Robotics @ Kilcoy Show

Last Friday and Saturday we had a great time at the Kilcoy Show (that’s a few hours North of Brisbane), showing visitors 3D printing in action and answering lots of questions on that topic – we actually printed some HO-scale train gear and miniature sheep for our model railway neighbours at the show!

We also let kids have a good play with the Mirobot drawing turtle robots and enjoying our cute robotic caterpillar.

OpenSTEM booth at Kilcoy Show 2016The OpenSTEM booth, next to Andrew Triggs of Mt Kilcoy State School

We saw a fair amount of interest from parents and visiting teachers for our work with schools, from the workshops to our resources and complete classroom programs, of which we had some sample materials that people could browse and ask us about.

On Friday we had the opportunity to meet briefly with Deb Frecklington, QLD state MP for Nanango (Shadow Minister for Agriculture, Fisheries and Forestry), and talked about the meaning and importance of STEM. Naturally STEM is critical for our agriculture as well!

Trevor Wessling (Kilcoy Show), Deb Frecklington (Nanango MP), Arjen (OpenSTEM)Trevor Wessling (Kilcoy Show), Deb Frecklington (Nanango MP), Arjen (OpenSTEM)

Our other neighbours at the show were Mt Kilcoy State School, where teacher Andrew Triggs showed off QUT LEGO robotics. Kids enjoyed watching the Rubik cube solver – I competed with it a few times, and was done well before it was 😉

The event has inspired me to build a few more things for kids to interact with and to show how awesome electronics and robotics can be explored without requiring costly gear.

,

Google AdsenseMore defenses roll out to thwart Clickjacking

At Google we defend our ad systems from fraud using technology in a variety of ways. Often our investment in these defenses goes beyond protecting against only known threats. Our engineering and operations teams are continually working to identify new and emerging threats.  Once a new ad fraud threat is found, we move quickly to defend our systems against it using a combination of technology, operations, and policy.

Recently we identified “Clickjacking” (aka UI Redress) as an emerging threat to cost-per-click display ads, and we’ve rolled out new defenses to protect advertisers against this threat. Clickjacking is a type of web attack where the appearance of a website is changed so that a victim does not realize they are taking an important action, in this case clicking on one or more ads. For example, a user may intend to click on a video play button or menu item, but instead clicks an invisible ad unit.

Figure 1: An example of a clickable ad hidden behind a video playback button.

Moving quickly to thwart Clickjacking attempts
Earlier this year when our operations team identified Clickjacking activity on our display network, they moved swiftly to terminate accounts, removing entities involved in or attempting to use this technique to trick users. Our engineering team worked in parallel to quickly release a filter to automatically exclude this type of invalid traffic across display ads.

This approach delivered a one-two punch to publishers who violated our policies: our operations team, which forms an early line of defense against invalid traffic, cleaned out publishers from our ad systems, while engineers built a new filter as a durable defense to protect against Clickjacking traffic.

Figure 2: An example of mouse-tracking, which leads to a page with lots of ads being opened regardless of where a user clicks.

Even as there are ongoing attempts to perpetrate this type of attack, our ongoing and proactive hunt for emerging types of invalid traffic has enabled us to move early and quickly to address Clickjacking threats on several occasions.


A combination of defenses
Our Clickjacking defenses operate at considerable scale, analyzing display ad placements across mobile and desktop platforms, evaluating a variety of characteristics. When our system detects a Clickjacking attempt, we zero-in on the traffic attributed to that placement, and remove it from upcoming payment reports to ensure that advertisers are not charged for those clicks.

This latest effort also is a great example of how our work against invalid traffic is at the intersection of technology, operations, and policy.  Each piece plays a key role in keeping our ad systems clean and defended against ad fraud.

Equally important, our efforts also promote a level playing field for good publishers on our ad systems.  And while our Ad Traffic Quality team works hard to keep our ad systems clean, we also rely on publishers to do their part in contributing to a healthy ads ecosystem.


Best practices for publishers
Publishers play a crucial role in delivering a good ads experience.  We’ve included some relevant best practices below to remind publishers of ways that they can improve the ads experience on their web properties.

  • Double and triple-check implementations to verify that your sites contain no programming errors, conform to AdSense policies, and display correctly across different browsers and platforms.
  • For mobile devices, plan your layout carefully to accommodate limited screen real estate.
  • Avoid placing ads close to other clickable content to prevent accidental clicks. For more guidance on how to implement banner ads see our best practices video.
  • Monitor analytics often to spot traffic anomalies. For example, setting up Analytics alerts can show if an unusual amount of traffic comes from a particular ad placement or site.
  • Lastly, if you find suspicious activity, please report it via the Invalid Clicks Contact Form.

We’re proud of our work to protect our ad systems against emerging threats like Clickjacking, and we’ll continue to be vigilant as we fight the good fight against ad fraud. 


Posted by: Andres Ferrate, Chief Advocate, Ad Traffic Quality


TEDA voyage to our closest star, the surprising paradox of an elephant’s brain, and rethinking college rankings

Hawking-CTA

As usual, the TED community has lots of news to share this week. Below, some highlights.

A 4.37-lightyear starshot. Humanity has sent people to the moon and rovers to Mars. It might be about time we embark on interstellar travel. Russian philanthropist Yuri Milner, along with board member Stephen Hawking, unveiled a plan on Tuesday to send a fleet of iPhone-sized robots to our closest star, Alpha Centauri. Led by Pete Worden, a former director of NASA’s Ames Research Center, this bold initiative will take 20 years to get off the ground and another 20, roughly, to make the 4.37-lightyear journey. If completed, Hawking says, it will propel humans further into what we’re made for: “Today we commit to the next great leap in the cosmos, because we are human and our nature is to fly.” (Watch Stephen’s TED Talk)

A chemistry-driven 3D printer you can buy. At TED2015, Joseph DeSimone introduced a radical new 3D printing technology prototype that creates objects out of a puddle of liquid (inspired, DeSimone admits, by Terminator 2). Through Continuous Liquid Interface Production, the chemical interplay between light and oxygen is harnessed to print objects 25-100 faster than standard 3D printing. This month DeSimone’s company, Carbon3D, debuted the M1, its first commercially available printer. (Watch Joseph’s TED Talk)

Health that zips through the sky. Drones — ominous, unmanned vehicles in the sky with the power to destroy, or, alternatively, bring hope and health to thousands worldwide. The latter is Keller Rinaudo and his company Zipline’s mission. Working with governments of developing countries, “Zipline plans to use its drone fleet to deliver medications to rural clinics all over the developing world,” says Olga Khazan in The Atlantic. Their first flights will begin this July in Rwanda. (Watch Keller’s TED Talk)

The surprising paradox of an elephant’s brain.  What about the human brain gives us greater cognitive abilities than other animals?  Suzane Herculano-Houzel believes the answer lies in the absolute number of neurons contained in an animal’s brain rather than the brain’s mass. But the African elephant poses an interesting, and enlightening, paradox to her research. She discovered that the African elephant’s brain–more than 3 times heavier than our brain–contained more neurons, but the location of those neurons plays a pivotal role in the difference between our cognitive ability and the elephant’s. (Watch Suzane’s TED Talk)

Stories from home. StoryCorps, which recently celebrated its own anniversary, announced a partnership with Fun Home to celebrate the Tony-winning musical’s one-year anniversary on Broadway. Stories will be available from Fun Home’s cast and creative team, and fans are encouraged to record their own stories using the StoryCorps app. The collaboration was born at the 2015 StoryCorps gala, which celebrated OutLoud, StoryCorp’s multi-year project to capture LGBT stories from around the country. (Watch Dave’s TED Talk and read his stories on TED’s Ideas blog)

Remembering the Bosnian War. Janine Di Giovanni, with radio in hand, listened to a Bosnian Muslim commander’s plea for help in 1993: “In the name of God, do something…We are dying here.” In a new Newsweek piece, she remembers what it was like reporting on the Bosnian War and the pain afflicted by President Radovan Karadžić, who in March was found guilty of 10 war crimes, including genocide, by a UN tribunal at The Hague. (Watch Janine’s TED Talk)

Oil fields ablaze. “Twenty-five years ago, as the United States–led coalition started driving out Iraqi forces from Kuwait, Saddam Hussein’s troops responded by setting ablaze hundreds of oil wells, creating one of the worst environmental disasters in recent memory,” recalls photojournalist Sebastião Salgado in The New York Times. He describes his admiration for the oil-well firefighters and the difficulty of photographing in such an extreme environment: “the heat warped one of my lenses and my jaws ached from the sheer tension of being exposed for hours to scalding temperatures.” In his signature black-and-white, Salgado captures the drama of the burning landscape–the giant clouds of smoke and flame spilling into the air, the firefighters covered in oil–with this reminder, “We must remember that in the brutality of battle another such apocalypse is always just around the corner.” (Watch Sebastiao’s TED Talk)

A blend of theater and journalism.  Playwright Anna Deavere Smith is famous for her bold mix of theater and journalism, capturing the experiences of her subjects in interviews and translating them, verbatim, onto the stage.  On April 5, she received the 2016 Guggenheim Fellowship. The award will fund her latest project, Notes from the Field: Doing Time in Education, which explores the school-to-prison pipeline. (Watch Anna’s TED Talk)

(Several other 2016 Guggenheim grantees have great TED Talks too, including poet Stephen Burt, musicologist Ge Wang, photographer Hasan Elahi, and neuroscientist Rajesh Rao. And watch soon for a talk from choreographer and TED Fellow Camille A. Brown.)

“Animal” shouldn’t be an insult. Humans have friends, but animals have “affiliation partners”; humans have sex, but animals have “breeding behavior.” But as primatologist Frans de Waal points out, humans laugh … and animals do too. In The New York Times, de Waal looks at the dangers of the linguistic pedestals we have erected over the animal kingdom, and suggests that humbling ourselves to recognize the true capabilities of animals is a mark of progress. Our human brilliance and the animal in us are not mutually exclusive: “There is nothing wrong with the recognition that we are apes — smart ones perhaps, but apes nonetheless… The more we succeed, the more we will realize that we are not the only intelligent life on earth.” (Watch Fran’s TED Talk)

Why safety should affect college rankings.  As college acceptance season hits its climax with students preparing to make a decision by May 1, Michael Kimmel puts forward a provocative idea on how to make college campuses safer. In Time, he suggests incorporating campus safety data into the Princeton Review’s college rankings. While current crime data for the rankings is self-reported, most ranking information comes from an annual survey of 100,000 students, “the real experts,” and Kimmel believes that this survey should include student opinions on safety and sexual violence. By incorporating this information into the ranking system, it “would help students make informed choices — and help hold schools accountable.” (Watch Michael’s TED Talk)

Have a news item to share? Write us at contact@ted.com and you may see it included in this weekly round-up.


CryptogramFriday Squid Blogging: Replicating Reflecting Squid Tissue

New research.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Rondam RamblingsYou must now get government approval before you can practice your faith in the U.S.

A federal judge today denied a Nebraska inmate named Stephen Cavanaugh the right to practice his religion because Cavanaugh's relgion didn't meet the judge's standards as to what a religion should be.  Cavanaugh professes to be a Pastafarian, a worshipper of the Flying Spaghetti Monster.  The judge ruled that Pastafarianism (which Cavanaugh calls FSMism) is not entitled to Constitutional

CryptogramIRS Security

Monday is Tax Day. Many of us are thinking about our taxes. Are they too high or too low? What's our money being spent on? Do we have a government worth paying for? I'm not here to answer any of those questions -- I'm here to give you something else to think about. In addition to sending the IRS your money, you're also sending them your data.

It's a lot of highly personal financial data, so it's sensitive and important information.

Is that data secure?

The short answer is "no." Every year, the GAO -- Government Accountability Office -- reviews IRS security and issues a report. The title of this year's report kind of says it all: "IRS Needs to Further Improve Controls over Financial and Taxpayer Data." The details are ugly: failures in identification and authentication of network users, failures to encrypt data, failures in audit and monitoring and failures to patch vulnerabilities and update software.

To be fair, the GAO can sometimes be pedantic in its evaluations. And the 43 recommendations for the IRS to improve security aren't being made public, so as not to advertise our vulnerabilities to the bad guys. But this is all pretty basic stuff, and it's embarrassing.

More importantly, this lack of security is dangerous. We know that cybercriminals are using our financial information to commit fraud. Specifically, they're using our personal tax information to file for tax refunds in our name to fraudulently collect the refunds.

We know that foreign governments are targeting U.S. government networks for personal information on U.S. citizens: Remember the OPM data theft that was made public last year in which a federal personnel database with records on 21.5 million people was stolen?

There have been some stories of hacks against IRS databases in the past. I think that the IRS has been hacked even more than is publicly reported, either because the government is keeping the attacks secret or because it doesn't even realize it's been attacked.

So what happens next?

If the past is any guide, not a lot. The GAO has been warning about problems with IRS security since it started writing these reports in 2007. In each report, the GAO has issued recommendations for the IRS to improve security. After each report, the IRS did a few of those things, but ignored most of the recommendations. In this year's report, for example, the GAO complained that the IRS ignored 47 of its 70 recommendations from 2015. In its 2015 report, it complained that the IRS only mitigated 14 of the 69 weaknesses it identified in 2013. The 2012 report didn't paint IRS security in any better light.

If I had to guess, I'd say the IRS's security is this bad for the exact same reason that so much corporate network-security is so bad: lack of budget. It's not uncommon for companies to skimp on their security budget. The budget at the IRS has been cut 17% since 2010; I am certain IT security was not exempt from those cuts.

So we're stuck. We have no choice but to give the IRS our data. The IRS isn't doing a good job securing our data. Congress isn't giving the IRS enough budget to do a good job securing our data. Last Tuesday, the Senate Finance Committee urged the IRS to improve its security. We all need to urge Congress to give it the money to do so.

Nothing is absolutely hacker-proof, but there are a lot of security improvements the IRS can make. If we have to give the IRS all our information -- and we do -- we deserve to have it taken care of properly.

This essay previously appeared on CNN.com.

Worse Than FailureError'd: TAYLOR VS. TIME

You know that new Taylor Swift treadmill commercial? Looks like there's a little bit of a sync issue between the time on her phone and the analog clock icon at 0:05.

 

Hans wrote, "Google News now appears to be adding political commentary to its headline summaries."

 

"That's a lot of notifications for only one document," writes Philip C.

 

Mattias wrote, "Go home Steam, you're drunk."

 

"Initially, I wondered why my package kept bouncing between Asia and North America," writes David D., "When I realized FedEx believed it actually traveled backwards in time 24 hours as it crossed the international date line, it all made sense."

 

Art writes, "So, if I drive my electric car backwards, does that mean I can save money on my electric bill?"

 

"The 'Copy text from picture' feature in OneNote is awesome, but sometimes, you'll get unexpected additions," wrote Antonio.

 

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Worse Than FailureError'd: TAYLOR VS. TIME

You know that new Taylor Swift treadmill commercial? Looks like there's a little bit of a sync issue between the time on her phone and the analog clock icon at 0:05.

 

Hans wrote, "Google News now appears to be adding political commentary to its headline summaries."

 

"That's a lot of notifications for only one document," writes Philip C.

 

Mattias wrote, "Go home Steam, you're drunk."

 

"Initially, I wondered why my package kept bouncing between Asia and North America," writes David D., "When I realized FedEx believed it actually traveled backwards in time 24 hours as it crossed the international date line, it all made sense."

 

Art writes, "So, if I drive my electric car backwards, does that mean I can save money on my electric bill?"

 

"The 'Copy text from picture' feature in OneNote is awesome, but sometimes, you'll get unexpected additions," wrote Antonio.

 

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Kelvin ThomsonAustralian Business Culture Doomed to Remain in the Stone Age?

On April 14, the Knowledge Nation 2016 Summit was convened in Sydney, involving leaders in science, technology and innovation, to consider how Australian society may be successfully transformed into a prosperous, globally competitive, knowledge-based economy. Elena Douglas, Convenor of the Summit, outlined the nature and urgency of this challenge in The Australian (‘Yes, we have innovation skills, but we must foster entrepreneurialism’, April 14, p. 12).

Although Elena Douglas highlights a number of home truths about the Australian economy and business culture which obstruct economic revitalisation, there are also a number of significant factors which she does not discuss, but which are central to any serious attempt to foster such a transition.
Acknowledging the quickening pace of technological innovation in the world economy, it is argued that Australia needs to become an ‘agile’, innovative economy, which will require a new sense of purpose and collaboration between government, business and educational institutions. Cultural obstacles are identified – lack of ambition (the ‘lucky country disease’), and a business culture too focussed on short-term return.  
Much of this criticism is valid particularly in relation to an Australian business culture preoccupied with short-term economic return. A longer-term view of the Australian economy and governance since the 1980s, however, points to a number of other uncomfortable truths which need to be faced up to by national leaders if any genuine progress towards economic and cultural renewal is to be realised.
The neo-liberal orthodoxy which has dominated Australian business and government thinking since the 1980s represents a major barrier to economic innovation and renewal. Too great a reliance upon market processes alone has resulted in the hollowing out of the Australian economy through the widespread destruction of existing enterprises and a failure to deliver new ones with higher technological sophistication and global reach.  That this failure has been met with calls for an even greater reliance upon small government and deregulated market processes is an impediment to national economic renewal. Basic lessons have not been learned.
At the same time, other economies in Pacific Asia have modernised - developed new, globally oriented and knowledge-intensive industries from relative economic backwardness and left Australia behind. We are left with no alternative, but to import virtually all elaborately transformed goods that we associate with our First World lifestyle. In return, for the most part, we depend on mining and agriculture for export income.  A key observation, however, is that these economies – South Korea, Japan and China, have not engineered this success on the basis of crude free-market principles, but coherent, mercantilist national strategies. These societies have not relied upon ‘comparative advantage’ as determined by market forces, but have created their own advantage through strong pro-active government, focussed government-business collaboration and an unswerving sense of national purpose. Former US Assistant Secretary of Commerce, Clyde Prestowitz, has highlighted this dilemma. The trade policies of the free market West have become increasingly divorced from reality. The reality, he argues, is a global economy where “roughly half the countries are more or less free trade driven, while the other half are neo-mercantilist (Prestowitz, 2009).”
Furthermore, before there can be any paradigm shift in investment priorities from short-term to strategic long-term innovation outcomes, there has to be an honest recognition of the extent to which Australian business elites have become dependent upon quick returns from crude, low-level capital widening based on rapid population growth and city building. The economic pie does get bigger, but the growth is largely ‘more of the same’, doing what we do already, but on an ever larger scale; with declining GDP per capita.
So entrenched has this approach become that, in its 2015 Intergenerational Report, the Australian Treasury calculates that  nearly half of Australia’s modest expected annual economic growth to the year 2054-55 will be due to continued high  population growth. It is worrying that the Australian Treasury engaged in outright political deception in overemphasising the negative implications of reducing population growth, while largely ignoring the serious social and economic problems of high population growth.
It is simply muddle-headed to bemoan the Australian business culture’s fixation on short-term financial gain and upon the domestic market rather than global competitiveness, when the primary economic strategy of the Australian Government and the Australian Treasury is to facilitate, encourage and reward such entrepreneurial backwardness. Moreover, powerful business interests (retail, housing construction and banking), which have benefited from this failed strategy continue to successfully lobby government for its perpetuation.
Nevertheless, crude growth is politically seductive; it has created an illusion of prosperity and even the illusion of good governance – an economy that is ‘the envy of the world’. It is worrying that the Prime Minister, Malcolm Turnbull, on his recent visit to China bragged about the “remarkable resilience” of the Australian economy in context of the Global Financial Crisis and its aftermath. The fact is that Australia faired reasonably well through the GFC because of a mining boom propped up by Chinese iron ore demand and reliance upon a high population growth capital widening strategy. The reality is that, when the Chinese demand for Iron ore rapidly subsided, the Australian economy was exposed as ill-equipped and underdeveloped in the global high-tech stakes. In terms of economic modernisation, high population growth and city building is now exposed as a road to nowhere.    
The Federal Government’s response to this is woefully inadequate. Instead of a robust hands-on approach by government, as practiced by our successful regional neighbours, the Federal Minister for Industry, Innovation and Science, Christopher Pyne’s key initiatives for correcting the situation have been to offer tax breaks for start-up firms, to flag the creation of a special visa to attract smart minds from overseas and to reprimand those who suggest that the Australian Government might spend more on public research and development. A whole hearted commitment to Australia paying for and generating its own human capital seems to be beyond the Minister’s and the Government’s expectations.
Australia is now behind the eight ball in the economic modernisation stakes. The 1990s and the recent mining boom have been an era of lost opportunity for Australia. Until our economic and political elites can face up to this, talk fests and any amount of hand wringing about Australia’s falling position in global knowledge economy rankings will likely fail to rectify the situation.
The contradictions are staggering. While the Federal Government insists that the public research sector has to pay its own way through stronger links with private industry, its continuing commitment to high population and crude growth sees a disproportionate share of Australia’s limited wealth being diverted into urban infrastructure and other spending in our ballooning capital cities.

,

Chaotic IdealismI have a headache...

I have a headache....
  Well, go take a tylenol.
I go to the kitchen, forget why I'm there, and make coffee. I like coffee.
I still have a headache.
  Did you really forget to get that painkiller?
I go to the kitchen, wander away halfway there, and end up petting the cat. She purrs.
I still have a headache.
  Really? This time, get up and actually go take some painkiller.
I'm really having trouble with executive function today. I should blog about this.
I blog about it. I still have a headache.
  Maybe now that you're done, you'll finally be able to successfully take some frickin' tylenol?!
Maybe.

Siiigh....

And this, ladies and gentlefolks, is how a simple task can take half an hour.

Krebs on Security‘Blackhole’ Exploit Kit Author Gets 7 Years

A Moscow court this week convicted and sentenced seven hackers for breaking into countless online bank accounts — including “Paunch,” the nickname used by the author of the infamous “Blackhole” exploit kit.  Once an extremely popular crimeware-as-a-service offering, Blackhole was for several years responsible for a large percentage of malware infections and stolen banking credentials, and likely contributed to tens of millions of dollars stolen from small to mid-sized businesses over several years.

Paunch, the accused creator of the Blackhole Exploit Kit, stands in front of his Porche Cayenne.

Fedotov, the convicted creator of the Blackhole Exploit Kit, stands in front of his Porche Cayenne in an undated photo.

According to Russia’s ITAR-TASS news network, Dmitry “Paunch” Fedotov was sentenced on April 12 to seven years in a Russian penal colony. In October 2013, the then 27-year-old Fedotov was arrested along with an entire team of other cybercriminals who worked to sell, develop and profit from Blackhole.

According to Russian security firm Group-IB, Paunch had more than 1,000 customers and was earning $50,000 per month from his illegal activity. The image at right shows Paunch standing in front of his personal car, a Porsche Cayenne.

First spotted in 2010, BlackHole is commercial crimeware designed to be stitched into hacked or malicious sites and exploit a variety of Web-browser vulnerabilities for the purposes of installing malware of the customer’s choosing.

The price of renting the kit ran from $500 to $700 each month. For an extra $50 a month, Paunch also rented customers “crypting” services; cryptors are designed to obfuscate malicious software so that it remains undetectable by antivirus software.

Paunch worked with several other cybercriminals to purchase new exploits and security vulnerabilities that could be rolled into Blackhole and help increase the success of the software. He eventually sought to buy the exploits from other cybercrooks directly to fund a pricier ($10,000/month) and more exclusive exploit pack called “Cool Exploit Kit.”

The main page of the Blackhole exploit kit Web interface.

The main page of the Blackhole exploit kit Web interface.

As documented on this blog in January 2013 (see Crimeware Author Funds Exploit Buying Spree), Paunch contracted with a third-party exploit broker who announced that he had a $100,000 budget for buying new, previously undocumented “zero-day” vulnerabilities.

Not long after that story, the individual with whom Paunch worked to purchase those exclusive exploits — a miscreant who uses the nickname “J.P. Morgan” — posted a message to the Darkode[dot]com crime forum, stating that he was doubling his exploit-buying budget to $200,000.

In October 2013, shortly after news of Paunch’s arrest leaked to the media, J.P. Morgan posted to Darkode again, this time more than doubling his previous budget — to $450,000.

“Dear ladies and gentlemen! In light of recent events, we look to build a new exploit kit framework. We have budgeted $450,000 to buy vulnerabilities of a browser and its plugins, which will be used only by us afterwards! ”

J.P. Morgan alludes to his former partner's arrest, and ups his monthly exploit buying budget to $450,000.

J.P. Morgan alludes to his former partner’s arrest, and ups his monthly exploit buying budget to $450,000.

The Russian Interior Ministry (MVD) estimates that Paunch and his gang earned more than 70 million rubles, or roughly USD $2.3 million. But this estimate is misleading because Blackhole was used as a means to perpetrate a vast array of cybercrimes. I would argue that Blackhole was perhaps the most important driving force behind an explosion of cyber fraud over the past three years. A majority of Paunch’s customers were using the kit to grow botnets powered by Zeus and Citadel, banking Trojans that are typically used in cyberheists targeting consumers and small businesses.

For more about Paunch, check out Who is Paunch?, a profile I ran in 2013 shortly after Fedotov’s arrest that examines some of the clues that connected his online criminal persona with his personal social networking profiles.

Update, 1:42: Corrected headline.

CryptogramCheating in Marathon Running

Story of Julie Miller, who cheated in multiple triathlon races:

The difference between cheating in 1980 and cheating today is that it's much harder to get away with now. What trips up contemporary cheaters, Empfield said, is their false assumption that the only thing they have to worry about is their timing chip, the device they wear that records their time at various points along a course.

But the use of additional technology ­ especially the ubiquitous course photos taken by spectators and professional photographers, which provide a wealth of information about athletes' positions and times throughout a race ­ makes it difficult for people to cover their tracks after the fact.

"What these people don't understand is that the photos contain so much data ­ they don't know that this exists," Empfield said of cheaters. "They think that if they hide in the bushes and re-emerge or take the chip off or whatever, they're in the clear. But the problem is that people can now forensically recreate your race."

Reminds me of this 2012 story about marathon cheating.