Planet Russell

,

Worse Than FailureIt's Dark In Here

Gustave Doré - Dante Alighieri - Inferno - Plate 8 (Canto III - Abandon all hope ye who enter here)

September 17, 20XX

Dear Susanna,

I hope the fresh start of a new college semester finds you well. I have great news! I've just signed a 2-year contract with one of the world's largest investment banks, and will be helping to maintain a next-generation trading platform.

This is the sort of programming assignment I've dreamt of since being in your shoes, imagining what post-college life might bring. In these first few years, my naiveté was rewarded with arcane VB6 apps and moldering SharePoint servers—but no more! I can't wait to start.

I shall log my experiences for your edification. Whenever you feel discouraged, remember my example and know it won't be much longer before you, too, realize your dreams.

-Frederic


Day 1. I expected the words to flow from my fingers with electric thrill, dear sister. Instead, I write to you now in a mood of deep bemusement.

The morning began well enough. My cubicle, laptop, phone, and network credentials were furnished to me straight away: a true rarity in the business world. Then my new boss took me to meet the genius behind the cutting-edge trading platform I'd been told so much about. He's a man they simply call The Architect.

The moldy smell of expired sandwiches hung about his cubicle. The man himself had sharp, beady eyes and a patronizing smile. He's either plumbed the universe's darkest machinations firsthand, or he believes he has.

My boss asked The Architect to give me a quick preview of the trading platform. I was genuinely excited to see what cutting-edge, next-gen software looked like.

The Architect obliged, remoting into the server hosting the application and launching it. On his monitor, the splash screen appeared, displaying a Greek soldier posing like Zeus. I realized he looked familiar.

It was The Architect's face Photoshopped onto a warrior's body.

A spine-curling cringe settled over me, which I struggled to suppress. This was no sign of world-class software, rather the opposite. I don't recall anything else. Now I'm alone in my cube, shuddering in dread of all that's yet to come.

No, I mustn't succumb to pessimism. Perhaps The Architect simply has a bad sense of humor. Worry not for me, Susanna. The next time I write, it will be in much higher spirits.


Day 2. Can this be real? I've sat down with my first coffee and have barely begun to look into the code base, and I'm already horrified by what I see. The main file, which is just for bootstrapping, contains over 10,000 lines of spaghetti code. The comments and log files contain multiple instances of Who cares?, Bah, and other expressions of futility.

The other developers are starting to filter into the office now. Peeking over the cubicle walls, I see plainly the hopelessness in their eyes.

I feel as though I stand at the edge of an abyss. Though I can't see the vast chasm below, I'm aware it's terribly far down, and the darkness stretching in between is fraught with uncertainty.


Day 5. It gets worse. Further dealings with The Architect's code have revealed more horrors, Susanna, things you must promise me never to permit within your own software projects:

  • The Listener interface has a publish() method.
  • There is a class named NothingIsSomethingElse with a single method, wireUp().
  • Many functions and constructors require 256 parameters- or more!
  • The container classes follow no standard convention. Map has <V, K> as the template arguments.
  • There are templated functions that take up to 12 template arguments.
  • Instead of using conventional terms like "source" and "sink," The Architect insists upon terms like "faucet" and "basin."

And the trading platform is every bit as horrific as the code lurking beneath. Running in production, each process requires 48 GB of RAM, consumes 350% CPU when the system is idle, and spawns anywhere from 10,000-20,000 threads. During full garbage collection, a process may stall for as long as 3 minutes.

People depend upon this monstrosity for their financial futures? God help them. God help us.

I've spoken to some of the other developers. They say The Architect's code is so complicated that only he can modify it. They're resigned to the application being this way.

I'm not. For my own sanity's sake, I shall start listing all the things we can improve, and take it to the next status meeting.


Day 10. Alas, all hope is lost. When I'm not looking at eye-bleeding code, I'm looking at my inbox—currently full of emails from The Architect, scathing emails shaming any member of the dev team who dares to make a mistake or question him. One of these emails is addressed to me. Another is addressed to an unfortunate soul who'd dared to use inheritance in her latest work.

720 more days of this nightmare. It's too late for me, Susanna, but not for you!

Change your major!

SAVE YOURSELF!

[Advertisement] Onsite, remote, bare-metal or cloud – create, configure and orchestrate 1,000s of servers, all from the same dashboard while continually monitoring for drift and allowing for instantaneous remediation. Download Otter today!

Planet DebianJoerg Jaspert: New gnupg-agent in Debian

In case you just upgraded to the latest gnupg-agent and used gnupg-agent as your ssh-agent you may find that ssh refuses to work with a simple but not helpful

sign_and_send_pubkey: signing failed: agent refused operation

This seems to come from systemd starting the agent, no longer a script at the start of the X session. And so it ends up with either no or an unusable tty. A simple

gpg-connect-agent updatestartuptty /bye

updates that and voila, ssh agent functionality is back in.

Note: This assumes you have “enable-ssh-support” in your ~/.gnupg/gpg-agent.conf

Planet DebianNorbert Preining: Gaming: Deus Ex Go

Long flights and lazy afternoons relaxing from teaching, I tried out another game on my Android device, Deus Ex Go. It is a turn based game in the style of the Deus Ex series (long years ago I was beta tester for the Deus Ex version of LGP). A turn based game where you have to pass through a series of levels, each one consisting of an hexagonal grid with an entry and exit point, and some nasty villains or machines trying to kill you.

Deus-ex-go

Without any explanations given you are thrown into the game and it takes a few iterations until you understand what kind of attacks you are facing, but once you have figured that out, it is a more or less simple game of combination how to manage to get to the exit. I played through the around 50 levels of the story mode and I think it was only in the last five that I once or twice had to actually try and think hard to find a solution.

I found the game quite amusing at the beginning, but soon it became repetitive. But since you can play through the whole story mode in probably one long afternoon, that is not so much of a problem. More a problem is the apparently incredible battery usage of this game. Playing without checking for some time leaves you soon with a near empty battery.

Graphically well done, with more or less interesting gameplay, it still does not stand up to Monument Valley.

Planet DebianFrancois Marier: Debugging gnome-session problems on Ubuntu 14.04

After upgrading an Ubuntu 14.04 ("trusty") machine to the latest 16.04 Hardware Enablement packages, I ran into login problems. I could log into my user account and see the GNOME desktop for a split second before getting thrown back into the LightDM login manager.

The solution I found was to install this missing package:

apt install libwayland-egl1-mesa-lts-xenial

Looking for clues in the logs

The first place I looked was the log file for the login manager (/var/log/lightdm/lightdm.log) where I found the following:

DEBUG: Session pid=12743: Running command /usr/sbin/lightdm-session gnome-session --session=gnome
DEBUG: Creating shared data directory /var/lib/lightdm-data/username
DEBUG: Session pid=12743: Logging to .xsession-errors

This told me that the login manager runs the gnome-session command and gets it to create a session of type gnome. That command line is defined in /usr/share/xsessions/gnome.desktop (look for Exec=):

[Desktop Entry]
Name=GNOME
Comment=This session logs you into GNOME
Exec=gnome-session --session=gnome
TryExec=gnome-shell
X-LightDM-DesktopName=GNOME

I couldn't see anything unexpected there, but it did point to another log file (~/.xsession-errors) which contained the following:

Script for ibus started at run_im.
Script for auto started at run_im.
Script for default started at run_im.
init: Le processus gnome-session (GNOME) main (11946) s'est achevé avec l'état 1
init: Déconnecté du bus D-Bus notifié
init: Le processus logrotate main (11831) a été tué par le signal TERM
init: Le processus update-notifier-crash (/var/crash/_usr_bin_unattended-upgrade.0.crash) main (11908) a été tué par le signal TERM

Seaching for French error messages isn't as useful as searching for English ones, so I took a look at /var/log/syslog and found this:

gnome-session[4134]: WARNING: App 'gnome-shell.desktop' exited with code 127
gnome-session[4134]: WARNING: App 'gnome-shell.desktop' exited with code 127
gnome-session[4134]: WARNING: App 'gnome-shell.desktop' respawning too quickly
gnome-session[4134]: CRITICAL: We failed, but the fail whale is dead. Sorry....

It looks like gnome-session is executing gnome-shell and that this last command is terminating prematurely. This would explain why gnome-session exits immediately after login.

Increasing the amount of logging

In order to get more verbose debugging information out of gnome-session, I created a new type of session (GNOME debug) by copying the regular GNOME session:

cp /usr/share/xsessions/gnome.desktop /usr/share/xsessions/gnome-debug.desktop

and then adding --debug to the command line inside gnome-debug.desktop:

[Desktop Entry]
Name=GNOME debug
Comment=This session logs you into GNOME debug
Exec=gnome-session --debug --session=gnome
TryExec=gnome-shell
X-LightDM-DesktopName=GNOME debug

After restarting LightDM (service lightdm restart), I clicked the GNOME logo next to the password field and chose GNOME debug before trying to login again.

This time, I had a lot more information in ~/.xsession-errors:

gnome-session[12878]: DEBUG(+): GsmAutostartApp: starting gnome-shell.desktop: command=/usr/bin/gnome-shell startup-id=10d41f1f5c81914ec61471971137183000000128780000
gnome-session[12878]: DEBUG(+): GsmAutostartApp: started pid:13121
...
/usr/bin/gnome-shell: error while loading shared libraries: libwayland-egl.so.1: cannot open shared object file: No such file or directory
gnome-session[12878]: DEBUG(+): GsmAutostartApp: (pid:13121) done (status:127)
gnome-session[12878]: WARNING: App 'gnome-shell.desktop' exited with code 127

which suggests that gnome-shell won't start because of a missing library.

Finding the missing library

To find the missing library, I used the apt-file command:

apt-file update
apt-file search libwayland-egl.so.1

and found that this file is provided by the following packages:

  • libhybris
  • libwayland-egl1-mesa
  • libwayland-egl1-mesa-dbg
  • libwayland-egl1-mesa-lts-utopic
  • libwayland-egl1-mesa-lts-vivid
  • libwayland-egl1-mesa-lts-wily
  • libwayland-egl1-mesa-lts-xenial

Since I installed the LTS Enablement stack, the package I needed to install to fix this was libwayland-egl1-mesa-lts-xenial.

I filed a bug for this on Launchpad.

,

Chaotic IdealismQ&A: Faking Normal

Q: Why do you "bash" autistic people who want to learn to be normal? Why do you encourage them to be autistic?

A: We encourage people to “be autistic” because we’ve tried faking normal ourselves, and it led to a lot of pain. We want to spare them that.

I was raised by a mom who was totally in denial about my autism. She taught me to believe that I was not really autistic, that I was actually lazy, strong-willed, and bad-tempered. And she taught me that the only way to accomplish things was to try harder. If you couldn’t do it, you weren’t trying hard enough. She would look at me, laugh, and say "Just do it!" as though I were pretending I couldn't. Sometimes she said, "You're so smart." She meant, "You're too smart to have an excuse for not being able to do this." And every time I took advantage of a good day and managed to do something that was difficult for me, as an autistic person, to do, she took it as proof that I could do that thing whenever I wanted to, and was just being contrary when I couldn't do it on command.

Well, I got out on my own and I wasn’t ready to take care of myself. I could neither use a bus nor drive. I couldn’t order at a restaurant. I couldn’t cook for myself. My sleep schedule was completely out of whack. I didn’t take regular showers. And I had never made a friend. I'd made friendly contact with others; many people were kind to me when I was a child. But I had never actually made a friend.

According to what I had been taught, the solution was to try harder to be as normal as possible, to tell myself that if I wasn’t fitting in, it was my own fault and I needed to change. Well, I tried. I tried to take care of myself, hold jobs, go to college. I pretended I was just lazy, strong-willed, and bad-tempered. I was burning out, but I didn’t know what to do other than try harder. I got to the point that I broke down mentally and ended up in the hospital. Twice.

Autistic brains are not meant to operate the way neurotypical brains are, and doing things the NT way is often not the way that works best for us. Forcing ourselves to go to crowded social events is not going to help us look normal; it’s just going to make us shut down. Whereas, conducting business one-on-one or even by e-mail is much more natural and easier for someone on the spectrum, and that way we actually get things done. Forcing ourselves to “sit still and stop fidgeting” can handicap our ability to think and process information; letting ourselves stim can free our minds to work efficiently. And so on.

For those of us who are “high-functioning” and can theoretically look normal for a few minutes or hours at a time, it’s a lot like trying to force a profoundly deaf child to lip-read and speak. Oh, they can learn it; the trouble is, it takes so long to learn it that they have no time to be a child. Even once they have learned, they’ll always have a harder time reading lips than a hearing person will have with listening to speech. Sign language is much more natural for that deaf child, even though it’s not the typical way people speak.

Now imagine being forced to do the equivalent of lip-reading in every area of life. There’s a reason autism is called a pervasive developmental disorder: Not just language but every little part of how you think and act and communicate is atypical in autism. You can try to mimic normal, but it’s always going to be slow, difficult, and exhausting. Or you can do things the way your brain was meant to do them, be your own person, and reach your own potential in your own way. Focusing on what works should be the goal of autism therapy and education, and what works is often as atypical as our minds are.

TEDBoyd Varty on Nelson Mandela and tracking your life’s purpose

December 5, 2013, was one of the most memorable days at TEDWomen — and everywhere else in the world, too. The world lost one of its great leaders, Nelson Mandela, and I will never forget the way we heard the news at the TEDWomen gathering in San Francisco.

A young South African, Boyd Varty, was scheduled to give his TED Talk that day, and as he came backstage to get miked, the news came through on our phones and computers: Mandela had died. I knew that Boyd and his family were close to him. Mandela had visited the Varty family’s game preserve, Londolozi, on one of his first retreats after being released from his long prison term.

I saw the tears well up as Boyd absorbed the sad news. I suggested that we rearrange the schedule so he could take a break and deliver his TED Talk later in the day. But he assured me he was ready to go, and asked if he could mention the news to the audience. Of course, I said yes. Who better than someone who knew him personally to share this tragic news of the passing of the great South African leader admired by the world for leading his country from the violent policies of apartheid through truth and reconciliation trials to the vibrant country that it is today?

Boyd stepped on stage into the red TED circle and, his voice shaking, told the audience the news. I was quite worried that he wouldn’t be able to deliver the TED Talk he had prepared to give, but he did brilliantly. In fact, his talk, which was posted immediately on TED.com, has been viewed more than 1.5 million times since.

Boyd shared childhood memories of Mandela’s visits to Londolozi, connecting the values he observed in Mandela to the values that are the foundation of his own life’s work protecting the natural resources of his homeland. One of South Africa’s greatest resources is its natural environment and the big animals that are endangered by hunting and poaching. Boyd and his family are committed to preserving these great resources so that generations to come can visit Africa and witness the majesty of its animals in the wild.

Boyd spoke about growing up in the Bush and the lessons he learned from tracking the animals there — lessons he recounts in his book, “Restoring Eden,” and lessons he is now applying to some exciting new work. We met recently for coffee at one of my favorite places in the world, Londolozi game preserve in the Sabi Sands — coincidentally on Mandela Day — to talk about the responses to his TEDTalk and to get an update on what’s he up to now.

2016-08-13-1471099070-8791115-patmitchellboydvarty.jpg

Boyd says he gets emails and comments every day about his TED Talk, and he noticed a theme emerging: an emotional connection people from all over the world were making with his stories about animals and tracking. So he decided to explore how his skills as a highly trained ranger and wildlife tracker might be applied to life tracking.

In his new “Track Your Life” retreats, he guides small groups of men on a “shared endeavor in the wild” to teach them “the ancient and powerful art of animal tracking.”

He’s already led a few of these workshops with men of all ages from diverse backgrounds and will be coming to the US with more soon.

Tracking a lion through the bush in Sabi Sands might seem a long way from tracking one’s life and career, but I’m sure that many men — and perhaps he will offer the course to women soon, too — will want to follow Boyd Varty on this special kind of “learning journey.”

I’m certainly not predicting that this year’s TEDWomen will have such a moment as the one that Boyd Varty shared with us that day, but I can predict that this year’s lineup will yield ideas and TED Talks that will shift perspectives for those present and the millions more who view them on TED.com.

A few main theater passes are still available for TEDWomen 2016, to be held October 26-28 in San Francisco. Find out more about TEDWomen 2016: It’s About Time.

——-
Photo credits: (top) Boyd Varty at TEDWomen 2013, courtesy of TED Talks. (middle) Pat visiting with Boyd Varty and his fiancée, Joelle Simpson, in South Africa, credit Scott Seydel.


Sociological ImagesTotem Vodka and Indigenous Cultural Appropriation

Cultural appropriation generally refers to the adoption of traditional practices, objects, or images by a person or group that is not part of the originating culture. Cultural appropriation can become problematic when it is done without permission, serves to benefit the dominant group, and erases or further marginalizes the oppressed group. In this way, cultural appropriation can recreate larger structures of inequality.

On a recent stroll through a duty-free shop, I was introduced to one of these problematic examples in the form of a new Canadian product named “Totem Vodka,” packaged in a bottle resembling a totem pole. Totem Vodka is not a product of Indigenous entrepreneurship. Instead it is a form of problematic cultural appropriation. Here’s why:

First Nations Erasure

Totem poles are important symbolic creations of some First Nations families in Canada’s Pacific Northwest. They are symbols of family lineage that serve to document stories or histories of people, communities or clans. The Totem Vodka bottle and marketing images erases these families, while appropriating their symbols.


The bottle stopper is shaped like a Thunderbird, a supernatural bird who causes thunder and lightning according to First Nations mythology. The Thunderbird crest is traditionally carved on the totem poles of people from the Thunderbird clan of the Kwakwaka’wakw nations (on Vancouver Island). The origin of the Thunderbird (and totem poles) within Pacific Northwest First Nations communities is absent from the company’s description of the bottle’s design and construction. Instead, the bottle is superficially connected to a wide-array of global references; the bottle was “designed on the West Coast of Canada, moulded by French glassmakers and topped with an Italian-made custom stopper.”

Significantly, the individuals featured in pictures on the company’s Twitter account include few or no indigenous people.

First Nations Exploitation

The owner of Totem Distilleries is a wealthy white entrepreneur and proceeds from the vodka help support a wildlife rescue association without any First Nations connection.

Settler societies have, paradoxically, both outlawed the sacred work of totem pole carving by indigenous peoples and exploited it for their own profit. In this case, the totem pole is used as an aesthetic tool to distinguish the vodka as authentically “Canadian,” while reproducing an abstracted, exotified, and ultimately false vision of indigeneity. First Nations people in Canada have rarely been either credited or compensated for the use of their cultural symbol.

The example of Totem vodka fits within a larger pattern of racism and colonial exploitation of indigenous people. We can look to the historical effects of colonization in Canada to see how attempts to erase Indigenous culture, while simultaneously exploiting it for the benefit of colonizers, has led to systemic discrimination, exclusionary policies and neglect that continue into the present day. Using a totem pole as a vodka bottle symbolizes this larger, patterned systems of inequality.

Alexandra Rodney is a PhD candidate in the Department of Sociology at the University of Toronto. She teaches Cultural Sociology and researches in the areas of food, gender and health. You can read more of Alexandra’s work on her website or follow her on Twitter

A special thank-you to Josée Johnston and Samantha Maskwa for their feedback on this post. Samantha is of Cree, Ojibway and Celtic ancestry. Her family is from the Rice Lake area and the southern part of Turtle Island and she is Bear clan. In addition to her midwifery degree, she is also completing a minor in Sociology and an Aboriginal Knowledges and Experiences certificate at Ryerson University in Toronto.

(View original at https://thesocietypages.org/socimages)

Planet DebianDon Armstrong: H3ABioNet Hackathon (Workflows)

I'm in Pretoria, South Africa at the H3ABioNet hackathon which is developing workflows for Illumina chip genotyping, imputation, 16S rRNA sequencing, and population structure/association testing. Currently, I'm working with the imputation stream and we're using Nextflow to deploy an IMPUTE-based imputation workflow with Docker and NCSA's openstack-based cloud (Nebula) underneath.

The OpenStack command line clients (nova and cinder) seem to be pretty usable to automate bringing up a fleet of VMs and the cloud-init package which is present in the images makes configuring the images pretty simple.

Now if I just knew of a better shared object store which was supported by Nextflow in OpenStack besides mounting an NFS share, things would be better.

You can follow our progress in our git repo: [https://github.com/h3abionet/chipimputation]

CryptogramInteresting Internet-Based Investigative Techniques

In this article, detailing the Australian and then worldwide investigation of a particularly heinous child-abuse ring, there are a lot of details of the pedophile security practices and the police investigative techniques. The abusers had a detailed manual on how to scrub metadata and avoid detection, but not everyone was perfect. The police used information from a single camera to narrow down the suspects. They also tracked a particular phrase one person used to find him.

This story shows an increasing sophistication of the police using small technical clues combined with standard detective work to investigate crimes on the Internet. A highly painful read, but interesting nonetheless.

TEDHave an anonymous TED Talk? We want to hear it.

Cross-posted from Chris Anderson on Medium

Today you may have heard that TED announced a rather unusual experiment with Audible. I’m pretty excited about what we’re doing here and want to share some thoughts.

Broadcast journalist Jad Abumrad once said that the most powerful thing about audio is what it lacks … that is: pictures. When a human voice describes something, the listener’s brain is wired to connect images and assign meaning to that voice. This is true for the many creative and expanding possibilities that digital audio now offers.

This act of co-authorship — between the speaker and the listener — to fill the gap of “picturelessness” does something really interesting. It connects us, perhaps more intimately than any other medium. We’ve certainly learned how this rings true for audio content TED puts out to the world.

And here’s something else audio can do that is quite special. A voice disconnected from visual identity provides anonymity to the speaker — while at the same time, letting their ideas reach millions of people.

And so, through this partnership with Audible, we’re creating a platform for TED Talks to be given anonymously. Why is this important? We’ve made it our mission at TED to track down a special breed of under-celebrated hero: People who have knowledge that matters. We find them, and invite them to share their knowledge on a global platform that gets billions of views.

But what if that exposure — the very spotlight that until now has defined the TED Talk experience was actually the reason some people chose not to submit their ideas? How many people have an important message but refrain from “going public” out of fear of losing their jobs or hurting loved ones? How many ideas worth spreading remain hidden because some speakers simply can’t publicly be associated with the very thing the world needs to hear?

Our best guess? A lot.

“Sincerely, X from TED and Audible” is an original audio series that will feature talks from speakers whose ideas need to be heard, but whose identities must remain hidden. Sincerely, X lets us share important ideas without sacrificing the privacy of the speakers or those close to them. In other words, this thrilling project opens up a category of talks that simply haven’t been possible previously.

Imagine ghostwriters, witnesses, wise souls who’ve survived something profound. A public figure living with mental illness. Someone who secretly gave up a child for adoption. A teenager who fought back against bullying and won. A parent who found a way to balance the needs of an autistic child and a neurologically normal one. A doctor living with a life and death mistake. An illegal immigrant with ideas on how to change the system. A CEO who know exactly how and when companies go wrong. Someone living a double life.

We’re curating talks from those who need to separate their professional ideas from their personal lives; people who want to share an idea, but fear it would hurt others in their family or company if they did so publicly; perhaps even those who are just scared to death of public speaking.

There won’t be a stage, and there won’t be any standing ovations. But those aren’t the essence of TED Talks. What matters is only what can be shared: an idea that matters.

And so I am asking you to help the world bring these ideas out of hiding. Do you have an important idea too important to stay secret? We want to hear about it. Perhaps it will change someone else’s life — perhaps it will even have a shot at shaping a better global conversation and a better future for all of us.

Here is the form to submit your proposal for an anonymous TED Talk. Only our internal team will see what you write. (Please don’t leave your proposal as a comment on this page, for obvious reasons!)


Worse Than FailureRepresentative Line: Accuracy in Comments

Comments are rough. I always take the stance that code should always be clear enough to explain what it does, but you’ll may need a comment to explain why it does that. I recently attended a talk by Sean Griffin (maintainer of Rails) who argued that commit messages should accomplish that goal, since they can contain far more content than a code comment, and while code comments and code can drift apart and cease to be accurate, commit messages are always linked to the point-in-time when they were made. Donald Knuth, on the other hand, might argue that code should annotate comments instead of the opposite.

Regardless of the method we use, I think most of us would agree that code needs some documentation in the same way it needs tests: it should exist, but we don’t want to have to create it.

Stephania found herself in the situation where she was creating the documentation. In this case, I don’t think we have to worry about the comment ceasing to be an accurate description of the code. This comment doesn’t need to be linked to a specific point in time- it tells us everything we need to know about the entire codebase.

# Note: The parameters "backup_freq" below do not actually refer to how frequently the backup script runs.  
# It's just a tag so that the retention scripts know what kind of backup the created snapshot is.
# How often the script runs is determined by the name of the generated output file.


[Advertisement] Universal Package Manager - ProGet easily integrates with your favorite Continuous Integration and Build Tools, acting as the central hub to all your essential components. Learn more today!

Sam VargheseOld is gold, but not when it involves rugby backs

It’s funny that none of the rugby scribes around wrote a single word about the selection of 34-year-old Matt Giteau and 28-year-old Will Genia in the Australian side to face New Zealand in the first of the annual internationals.

In the normal course of things, one would assume that the coach of any team that has a chance of winning the World Cup would like to start aiming for that target right at the start of the four-year cycle. Australia made it to the last World Cup final and have won the Cup twice, so they are one of the nations that can reasonably entertain hopes of winning again.

But you can’t do that with a 38-year-old centre which is what Giteau will be in 2019 when the next rugby World Cup rolls around. And you wouldn’t want a 32-year-old scrum-half either.

Is one to believe that Nick Phipps, who performed the job at the base of the scrum adequately in the last World Cup, was not good enough for the Australian coach Michael Cheika? Indeed, Phipps showed his prowess by coming on and playing on the right wing after Australia lost three backs, including Giteau, to injury and also scoring the lone try that the home team got as it suffered a big defeat against New Zealand.

Is one to also believe that among the five teams that do duty in the Super Rugby tournament there is not one individual who can fit in as a centre and that Cheika’s only option was to call in a 34-year-old with injury issues to face up to what is arguably the fittest and strongest rugby team in the world?

From the moment that Cheika announced these selections, it was obvious that he was more interested in pleasing his masters at the Australian Rugby Union than building a team for the next World Cup. New Zealand has held the Bledisloe Cup since 2003 when Reuben Thorne’s side won it back from Australia, and winning that trophy would have pleased the local big-wigs.

Of course, Cheika is not the only one who is looking to the past when trying to fill the ranks. His South African counterpart Allister Coetzee displayed similar thinking by playing Bryan Habana on the wing against Argentina on the same weekend. Habana is 33 and I am yet to see a 37-year-old winger play in a team in the World Cup. South Africa is also a team that would be in contention in any World Cup, having won the Cup twice, once admittedly under rather dubious circumstances. So why Habana? South Africa has one of the largest pools of players to pick from and someone like Courtnall Skosan would have benefitted from the exposure.

On the other hand, Steve Hansen, the coach of New Zealand, has brought in new players instead of depending on any old hands. He lost much more experience compared to the others because Richie McCaw, Dan Carter, Ma’a Nonu, Conrad Smith, and Keven Mealamu all ended their international careers after the last World Cup.

Hansen has retained two older players in Kieran Read and Jerome Kaino; the latter will be 37 when the next World Cup comes around but is one of the fittest and strongest players in world rugby and is unlikely to be a liability in the team. Remember, he is a forward and does not have to be a strong runner – even though he does a fair bit of scoring in internationals.

Planet DebianZlatan Todorić: Take that boredom

While I was bored on Defcon, I took the smallest VPS in DO offering (512MB RAM, 20GB disk), configured nginx on it, bought domain zlatan.tech and cp'ed my blog data to blog.zlatan.tech. I thought it will just be out of boredom and tear it apart in a day or two but it is still there.

Not only that, the droplet came with Debian 8.5 but I just added unstable and experimental to it and upgraded. Just to experiment and see what time will I need to break it. To make it even more adventurous (and also force me to not take it too much serious, at least at this point) I did something on what Lars would scream - I did not enable backups!

While having fun with it I added letsencrypt certificate to it (wow, that was quite easy).

Then I installed and configured Tor. Ende up adding an .onion domain for it! It is: pvgbzphm622hv4bo.onion

My main blog is still going to be zgrimshell.github.io (for now at least) where I push my Nikola (static site generator written in python) generated content as git commits. To my other two domains (on my server) I just rsync the content now. Simple and efficient.

I must admit I like my blog layout. It is simple, easy to read, efficient and fast, I don't bother with comments and writing a blog in markdown (inside terminal as all good behaving hacker citizen) while compiling it with Nikola is breeze (and yes, I did choose Nikola because of Nikola Tesla and python). Also I must admit that nginx is pretty nice webserver, no need to explain the beauty of git but I can't recommend enough of rsync.

If anyone is interested in doing the same I am happy to talk about it but these tools are really simple (as I enjoy simple things and by simple I mean small tools, no complicated configs and easy execution).

Planet Linux AustraliaMaxim Zakharov: Small fix for AMP WordPress plugin

If you use AMP plugin for WordPress to make AMP (Accelerated Mobile Pages) version of your posts and have some troubles validating them on AMP validator, you may try this fix for AMP plugin to make those pages valid.

Krebs on SecurityA Life or Death Case of Identity Theft?

Identity thieves have perfected a scam in which they impersonate existing customers at retail mobile phone stores, pay a small cash deposit on pricey new phones, and then charge the rest to the victim’s account. In most cases, switching on the new phones causes the victim account owner’s phone(s) to go dead. This is the story of a Pennsylvania man who allegedly died of a heart attack because his wife’s phone was switched off by ID thieves and she was temporarily unable to call for help.

On Feb. 20, 2016, James William Schwartz, 84, was going about his daily routine, which mainly consisted of caring for his wife, MaryLou. Mrs. Schwartz was suffering from the end stages of endometrial cancer and wasn’t physically mobile without assistance. When Mr. Schwartz began having a heart attack that day, MaryLou went to use her phone to call for help and discovered it was completely shut off.

Little did MaryLou know, but identity thieves had the day before entered a “premium authorized Verizon dealer” store in Florida and impersonated the Schwartzes. The thieves paid a $150 cash deposit to “upgrade” the elderly couple’s simple mobiles to new iPhone 6s devices, with the balance to be placed on the Schwartz’s account.

“Despite her severely disabled and elderly condition, MaryLou Schwartz was finally able to retrieve her husband’s cellular telephone using a mechanical arm,” reads a lawsuit (PDF) filed in Beaver County, Penn. on behalf of the Schwartz’s two daughters, alleging negligence by the Florida mobile phone store. “This monumental, determined and desperate endeavor to reach her husband’s working telephone took Mrs. Schwartz approximately forty minutes to achieve due to her condition. This vital delay in reaching emergency help proved to be fatal.”

By the time paramedics arrived, Mr. Schwartz was pronounced dead. MaryLou Schwartz died seventeen days later, on March 8, 2016. Incredibly, identity thieves would continue robbing the Schwartzes even after they were both deceased: According to the lawsuit, on April 14, 2016 the account of MaryLou Schwartz was again compromised and a tablet device was also fraudulently acquired in MaryLou’s name.

The Schwartz’s daughters say they didn’t learn about the fraud until after both parents passed away. According to them, they heard about it from the guy at a local Verizon reseller that noticed his longtime customers’ phones had been deactivated. That’s when they discovered that while their mother’s phone was inactive at the time of her father’s death, their father’s mobile had inexplicably been able to make but not receive phone calls.

KNOW YOUR RIGHTS AND OPTIONS

Exactly what sort of identification was demanded of the thieves who impersonated the Schwartzes is in dispute at the moment. But it seems clear that this is a fairly successful and common scheme for thieves to steal (and, in all likelihood, resell) high-end phones.

Lorrie Cranor, chief technologist for the U.S. Federal Trade Commission, was similarly victimized this summer when someone walked into a mobile phone store, claimed to be her, asked to upgrade her phones and walked out with two brand new iPhones assigned to her telephone numbers.

“My phones immediately stopped receiving calls, and I was left with a large bill and the anxiety and fear of financial injury that spring from identity theft,” Cranor wrote in a blog on the FTC’s site.  Cranor’s post is worth a read, as she uses the opportunity to explain how she recovered from the identity theft episode.

She also used her rights under the Fair Credit Reporting Act, which requires that companies provide business records related to identity theft to victims within 30 days of receiving a written request. Cranor said the mobile store took about twice that time to reply, but ultimately explained that the thief had used a fake ID with Cranor’s name but the impostor’s photo.

“She had acquired the iPhones at a retail store in Ohio, hundreds of miles from where I live, and charged them to my account on an installment plan,” Cranor wrote. “It appears she did not actually make use of either phone, suggesting her intention was to sell them for a quick profit. As far as I’m aware the thief has not been caught and could be targeting others with this crime.”

Cranor notes that records of identity thefts reported to the FTC provide some insight into how often thieves hijack a mobile phone account or open a new mobile phone account in a victim’s name.

“In January 2013, there were 1,038 incidents of these types of identity theft reported, representing 3.2% of all identity theft incidents reported to the FTC that month,” she explained. “By January 2016, that number had increased to 2,658 such incidents, representing 6.3% of all identity thefts reported to the FTC that month.  Such thefts involved all four of the major mobile carriers.”

The reality, Cranor said, is that identity theft reports to the FTC likely represent only the tip of a much larger iceberg. According to data from the Identity Theft Supplement to the 2014 National Crime Victimization Survey conducted by the U.S. Department of Justice, less than 1% of identity theft victims reported the theft to the FTC.

While dealing with diverted calls can be a hassle, having your phone calls and incoming text messages siphoned to another phone also can present new security problems, thanks to the growing use of text messages in authentication schemes for financial services and other accounts.

Perhaps the most helpful part of Cranor’s post is a section on the security options offered by the four major mobile providers in the U.S. For example, AT&T offers an “extra security” feature that requires customers to present a custom passcode when dealing with the wireless provider via phone or online.

“All of the carriers have slightly different procedures but seem to suffer from the same problem, which is that they’re relying on retail stores relying on store employee to look at the driver’s license,” Cranor told KrebsOnSecurity. “They don’t use services that will check the information on the drivers license, and so that [falls to] the store employee who has no training in spotting fake IDs.”

Some of the security options offered by the four major providers. Source: FTC.

Some of the security options offered by the four major providers. Source: FTC.

It’s important to note that secret passcodes often can be bypassed by determined attackers or identity thieves who are adept at social engineering — that is, tricking people into helping them commit fraud.

I’ve used a six-digit passcode for more than two years on my account with AT&T, and last summer noticed that I’d stopped receiving voicemails. A call to AT&T’s customer service revealed that all voicemails were being forwarded to a number in Seattle that I did not recognized or authorize.

Since it’s unlikely that the attackers in this case guessed my six-digit PIN, they likely tricked a customer service representative at AT&T into “authenticating” me via other methods — probably by offering static data points about me such as my Social Security number, date of birth, and other information that is widely available for sale in the cybercrime underground on virtually all Americans over the age of 35. In any case, Cranor’s post has inspired me to exercise my rights under the FCRA and find out for certain.

Vineetha Paruchuri, a masters in computer science student at Dartmouth College, recently gave a talk at the Bsides security conference in Las Vegas on her research into security at the major U.S. mobile phone providers. Paruchuri said all of the major mobile providers suffer from a lack of strict protocols for authenticating customers, leaving customer service personnel exposed to social engineering.

“As a computer science student, my contention was that if we take away the control from the humans, we can actually make this process more secure,” Paruchuri said.

Paruchuri said perhaps the most dangerous threat is the smooth-talking social engineer who spends time collecting information about the verbal shorthand or mobile industry patois used by employees at these companies. The thief then simply phones up customer support and poses as a mobile store technician or employee trying to assist a customer. This was the exact approach used in 2014, when young hooligans tricked my then-ISP Cox Communications into resetting the password for my Cox email account.

I suppose one aspect of this problem that makes the lack of strong customer authentication measures by the mobile industry so frustrating is that it’s hard to imagine a device which holds more personal and intimate details about you than your wireless phone. After all, your phone likely knows where you were last night, when you last traveled, the phone number you last called and numbers you most frequently text.

And yet, the best the mobile providers and their fleet of reseller stores can do to tell you apart from an ID thief is to store a PIN that could be bypassed by clever social engineers (who may or may not be shaving yet).

A NOTE FOR AT&T READERS

By the way, readers with AT&T phones may have received a notice this week that AT&T is making some changes to “authorized users” allowed on accounts. The notice advised that starting Sept. 1, 2016, customers can designate up to 10 authorized users per account.

“If your Authorized User does not know your account passcode or extra security passcode, your Authorized User may still access your account in a retail store using a Forgotten Passcode process. Effective Nov. 5, 2016, Authorized Users and those persons who call into Customer Service and provide sufficient account information (“Authenticated Callers”) Will have the ability to add a new line of service to your account. Such requests, whether made by you, an Authorized User, an Authenticated Caller or someone with online access to your account, will trigger a credit check on you.”

AT&T's message this week about upcoming account changes.

AT&T’s message this week about upcoming account changes.

I asked AT&T about what need this new policy was designed to address, and the company responded that AT&T has made no changes to how an authorized user can be added to an account. AT&T spokesman Jim Greer sent me the following:

“With this notice, we are simply increasing the number of authorized users you may add to your account and giving them the ability to add a line in stores or over the phone. We made this change since more customers have multiple lines for multiple people. Authorized users still cannot access the account holder’s sensitive personal information.”

“Over the past several years, the authentication process has been strengthened. In stores, we’re safeguarding customers through driver’s license or other government issued ID authentication.  We use a two-factor authentication when you contact us online or by phone that requires a one-time PIN. We’re continuing our efforts to better protect customers, with additional improvements on the horizon.”

“You don’t have to designate anyone to become an authorized user on your account. You will be notified if any significant changes are made to your account by an authorized user, and you can remove any person as an authorized user at any time.”

The rub is what AT&T does — or more specifically, what the AT&T customer representative does — to verify your identity when the caller says he doesn’t remember his PIN or passcode. If they allow PIN-less authentication by asking for your Social Security number, date of birth and other static information about you, ID thieves can defeat that easily.

Has someone fraudulently ordered phone service or phones in your name? Sound off in the comments below.

If you’re wondering what you can do to shield yourself and your family against identity theft, check out these primers:

How I Learned to Stop Worrying and Embrace the Security Freeze (this primer goes well beyond security freezes and includes a detailed Q&A as well as other tips to help prevent and recover from ID theft).

Are Credit Monitoring Services Worth It? 

What Tax Fraud Victims Can Do

The Lowdown on Freezing Your Kid’s Credit

,

LongNowJonathan Rose Seminar Tickets

 

The Long Now Foundation’s monthly

Seminars About Long-term Thinking

Jonathan Rose on The Well Tempered City

Jonathan Rose on “The Well Tempered City”

TICKETS

Tuesday September 20, 02016 at 7:30pm Herbst Theater

Long Now Members can reserve 2 seats, join today! General Tickets $15

 

About this Seminar:

Cities and urban regions can make coherent sense, can metabolize efficiently, can use their very complexity to solve problems, and can become so resilient they “bounce forward” when stressed.

In this urbanizing century ever more of us live in cities (a majority now; 80% expected by 2100), and cities all over the world are learning from each other how pragmatic governance can work best. Jonathan Rose argues that the emerging best methods focus on deftly managing “cognition, cooperation, culture, calories, connectivity, commerce, control, complexity, and concentration.”

Unlike most urban theorists and scholars, Rose is a player. A third-generation Manhattan real estate developer, in 1989 he founded and heads the Jonathan Rose Company, which does world-wide city planning and investment along with its real estate projects–half of the work for nonprofit clients. He is the author of the new book, THE WELL-TEMPERED CITY: What Modern Science, Ancient Civilizations, and Human Nature Teach Us About the Future of Urban Life.

Planet DebianReproducible builds folks: Reproducible Builds: week 69 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday August 14 and Saturday August 20 2016:

Fasten your seatbelts

Important note: we enabled build path variation for unstable now, so your package(s) might become unreproducible, while previously it was said to be reproducible… given a specific build path it probably still is reproducible but read on for the details below in the tests.reproducible-builds.org section! As said many times: this is still research and we are working to make it reality.

Media coverage

Daniel Stender blogged about python packaging and explained some caveats regarding reproducible builds.

Toolchain developments

Thomas Schmitt uploaded xorriso which now obeys SOURCE_DATE_EPOCH. As stated in its man pages:

ENVIRONMENT
[...]
SOURCE_DATE_EPOCH  belongs to the specs of reproducible-builds.org.  It
is supposed to be either undefined or to contain a decimal number which
tells the seconds since january 1st 1970. If it contains a number, then
it is used as time value to set the  default  of  --modification-date=,
--gpt_disk_guid,  and  --set_all_file_dates.  Startup files and program
options can override the effect of SOURCE_DATE_EPOCH.

Packages reviewed and fixed, and bugs filed

The following packages have become reproducible after being fixed:

The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

  • vulkan/1.0.21.0+dfsg1-1 by Timo Aaltonen.

The following 2 packages were not changed, but have become reproducible due to changes in their build-dependencies: tagsoup tclx8.4.

Some uploads have addressed some reproducibility issues, but not all of them:

Patches submitted that have not made their way to the archive yet:

Bug tracker house keeping:

  • Chris Lamb pinged 164 bugs he filed more than 90 days ago which have a patch and had no maintainer reaction.

Reviews of unreproducible packages

55 package reviews have been added, 161 have been updated and 136 have been removed in this week, adding to our knowledge about identified issues.

2 issue types have been updated:

Weekly QA work

FTBFS bugs have been reported by:

  • Chris Lamb (16)
  • Santiago Vila (2)

diffoscope development

Chris Lamb, Holger Levsen and Mattia Rizzolo worked on diffoscope this week.

Improvements were made to SquashFS and JSON comparison, the https://try.diffoscope.org/ web service, documentation, packaging, and general code quality.

diffoscope 57, 58, and 59 were uploaded to unstable by Chris Lamb. Versions 57 and 58 were both broken, so Holger set up a job on jenkins.debian.net to test diffoscope on each git commit. He also wrote a CONTRIBUTING document to help prevent this from happening in future.

From these efforts, we were also able to learn that diffoscope is now reproducible even when built across multiple architectures:

< h01ger> | https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope.html shows these packages were built on amd64:
< h01ger> |  bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> |  366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger> | and on i386:
< h01ger> |  bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> |  366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger> | and on armhf:
< h01ger> |  bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> |  366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb

And those also match the binaries uploaded by Chris in his diffoscope 59 binary upload to ftp.debian.org, yay! Eating our own dogfood and enjoying it!

tests.reproducible-builds.org

Debian related:

  • show percentage of results in the last 24/48h (h01ger)
  • switch python database backend to SQLAlchemy (Valerie)
  • vary build path varitation for unstable and experimental for all architectures. (h01ger)

The last change probably will have an impact you will see: your package might become unreproducible in unstable and this will be shown on tracker.debian.org, while it will still be reproducible in testing.

We've done this, because we think reproducible builds are possible with arbitrary build paths. But: we don't think those are a realistic goal for stretch, where we still recommend to use ´.buildinfo´ to record the build patch and then do rebuilds using that path.

We are doing this, because besides doing theoretical groundwork we also have a practical goal: enable users to independently verify builds. And if they only can do this with a fixed path, so be it. For now :)

To be clear: for Stretch we recommend that reproducible builds are done in the same build path as the "original" build.

Finally, and just for our future references, when we enabled build path variation on Saturday, August 20th 2016, the numbers for unstable were:

suite all reproducible unreproducible ftbfs depwait not for this arch blacklisted
unstable/amd64 24693 21794 (88.2%) 1753 (7.1%) 972 (3.9%) 65 (0.2%) 95 (0.3%) 10 (0.0%)
unstable/i386 24693 21182 (85.7%) 2349 (9.5%) 972 (3.9%) 76 (0.3%) 103 (0.4%) 10 (0.0%)
unstable/armhf 24693 20889 (84.6%) 2050 (8.3%) 1126 (4.5%) 199 (0.8%) 296 (1.1%) 129 (0.5%)

Misc.

Ximin Luo updated our git setup scripts to make it easier for people to write proper descriptions for our repositories.

This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

CryptogramPrivacy Implications of Windows 10

The EFF has a good analysis of all the ways Windows 10 violates your privacy.

Worse Than FailureYour Code Might Be Unmaintainable…

Let’s talk about maintainability.

Those of you that know me know that in my civillian identity, I work as a SQA professional. QA gets a bum rap sometimes; manual functional testing can be one of the most boring parts of software engineering, but while there’s plenty of button-pushers who will be happy to poke at an application for minimum wage, there’s a lot more to quality than simply functionality. One of the commonly overlooked aspects is maintainability: the ease with which changes can be made to the software system.

Jeff Foxworthy at Resch Center

Now, maintainability can be measured. You can track how long it takes to discover the root cause of an issue, or how long it takes to work a simple enhancement request. You can track the number of groans or “WTFs” per minute in code review. You can track the cylcomatic complexity of the codebase or, if you’re a masochist, the Halstead complexity. But there’s a number of informal, “gut feel” warning signs you can use to tell if your application is maintainable. Here’s a sampling I’ve collected from various programmers in the industry. I like to call it Your code might be unmaintainable if…

Your code might be unmaintainable if the programmers give it nicknames like “the monster” or “the barge” or utter the words “there be dragons”

Your code might be unmaintainable if you can tell how old a file is by what revision of the coding standards it follows – within 3 seconds of opening the file

Your code might be unmaintainable if requesting a dependency map from the database server chugs for 30 minutes then crashes with an “out of memory” exception

Your code might be unmaintainable if you’re the most senior dev on the team… six months out of uni

Your code might be unmaintainable if you can’t localize the text of dropdowns for a new locale because that would break the existing 400-line if-else chain that lists out every localized string and keys functionality based on it

Your code might be unmaintainable if management dictates the design of the codebase – down to the nitty-gritty tactical level. Bonus points if the manager is a non-technical VP who responds to criticism by firing the developer on the spot.

Your code might be unmaintainable if half the variable names are in a different language and nobody’s quite sure which

Your code might be unmaintainable if you’re serving 20MB of CSS files on every request

Your code might be unmaintainable if your dependency graph is unreadable when printed on a single sheet of 8.5“ x 11” paper. Or a single sheet of A4. Or a single sheet of A3.

Your code might be unmaintainable if your variable naming “convention” is u734, u1234–2, u623, etc… and variables all change names between files

Your code might be unmaintainable if you have a single function webservice with 17 layers of abstraction between the entry point and where the business logic lies. Apparently the developer had never heard of YAGNI

Your code might be unmaintainable if you re-order the properties of an object and the code breaks

Your code might be unmaintainable if Microsoft Access forms a critical part of your business workflow. Or Microsoft Excel. Bonus points for unreadable Excel macros that fall victim to other traps in this article

Your code might be unmaintainable if it includes the line #defined ONE_HUNDRED 100 unironically

Your code might be unmaintainable if you find the source code! …in a folder marked DEV_BACKUP_2013

Your code might be unmaintainable if you can’t find the malfunctioning bit of code because it’s monkeypatched onto a class you control from god knows where

Your code might be unmaintainable if you go to debug a production issue and not only can you not reproduce, the entire functionality isn’t present in the codebase on the development servers.

Your code might be unmaintainable if you go to debug a production issue and not only can you not reproduce in dev, you can’t find the functionality even in production. Bonus points if you finally chase down a series of redirects and end up on some server somewhere, named after a flower, that was meant to have been decomissioned years ago. Double bonus points if you’re pretty sure it shells out to a Minecraft server in the process.

Your code might be unmaintainable if it’s full of comments that simply read //BUGBUG. Or //TODO (with no further explanation)

Your code might be unmaintainable if there’s a header at the top of every page served up in production that reads


// [Decompiled](http://thedailywtf.com/articles/variables-everywhere-but-not-a-stop-to-think) with JetBrains decompiler
// Type: FloorPlanCLR.EventManagementService.Triggers
// Assembly: FloorPlanCLR, Version=1.0.0, Culture=neutral, PublicKeyToken=null
// MVID: 55C2F875-6F43-4FDE-A2C6-6CD7F464A43B
// AssemblyLocation: C:\Users\SomeGuy\Pictures\FloorPlanCLR.dll

Your code might be unmaintainable if it’s in a proprietary langauge your company invented. Or a blend of two such languages.

[Advertisement] Otter enables DevOps best practices by providing a visual, dynamic, and intuitive UI that shows, at-a-glance, the configuration state of all your servers. Find out more and download today!

,

Planet DebianSylvain Le Gall: Release of OASIS 0.4.7

I am happy to announce the release of OASIS v0.4.7.

Logo OASIS small

OASIS is a tool to help OCaml developers to integrate configure, build and install systems in their projects. It should help to create standard entry points in the source code build system, allowing external tools to analyse projects easily.

This tool is freely inspired by Cabal which is the same kind of tool for Haskell.

You can find the new release here and the changelog here. More information about OASIS in general on the OASIS website.

Pull request for inclusion in OPAM is pending.

Here is a quick summary of the important changes:

  • Drop support for OASISFormat 0.2 and 0.1.
  • New plugin "omake" to support build, doc and install actions.
  • Improve automatic tests (Travis CI and AppVeyor)
  • Trim down the dependencies (removed ocaml-gettext, camlp4, ocaml-data-notation)

Features:

  • findlib_directory (beta): to install libraries in sub-directories of findlib.
  • findlib_extra_files (beta): to install extra files with ocamlfind.
  • source_patterns (alpha): to provide module to source file mapping.

This version contains a lot of changes and is the achievement of a huge amount of work. The addition of OMake as a plugin is a huge progress. The overall work has been targeted at making OASIS more library like. This is still a work in progress but we made some clear improvement by getting rid of various side effect (like the requirement of using "chdir" to handle the "-C", which leads to propage ~ctxt everywhere and design OASISFileSystem).

I would like to thanks again the contributor for this release: Spiros Eliopoulos, Paul Snively, Jeremie Dimino, Christopher Zimmermann, Christophe Troestler, Max Mouratov, Jacques-Pascal Deplaix, Geoff Shannon, Simon Cruanes, Vladimir Brankov, Gabriel Radanne, Evgenii Lepikhin, Petter Urkedal, Gerd Stolpmann and Anton Bachin.

Sociological ImagesLessons from Hurricane Katrina for the Child Victims of the Louisiana Floods of 2016

The great Louisiana Floods of 2016 have led to the closure of at least 22 of the state’s 70 public school districts, with additional districts calling off classes as a precaution given the immense devastation. This means that as many as one-third of the state’s public school students were out of school last week ,and potentially for many weeks to come. That equates to more than 241,000 children who are not in classrooms where they belong; and these figures do not even account for the many thousands of private and charter school students also out of school across the water-logged state.

2

Almost exactly 11 years ago, Hurricane Katrina disrupted some 370,000 school-age children. For our book, Children of Katrina, we spent nearly a decade examining how their lives unfolded in the years after the catastrophe. We focused on education as a key “sphere” of children’s lives. It is a special sphere in that it is unique to children and youth and it has specific time parameters: when the window for schooling is gone, children cannot get it back. Missing school means missing critical stages in cognitive and social development and likely suffering irreparable harm in terms of their intellectual growth, development, and future educational goals.

The school sphere, as with the other spheres of children’s lives, is marked by inequality, with some students having access to greater advantages than others. Some school districts, often segregated by race and class, have more resources and support than others; some families have the ability to enroll children in private schools that require tuition or arrange to be in a high-quality school district, while other families do not have those options.

Keeping this in mind, and recognizing the importance of education during displacement and recovery, there are many things that can and should be done, to support disaster affected children and youth and their educational process. These include:

  • Reopening schools (including childcare centers and pre-schools) as quickly as possible after a disaster; this means allocating proper resources to repair, rebuild, and/or revive schools in disaster zones;
  • In receiving communities that receive large numbers of displaced children and youth, providing pathways for their rapid enrollment;
  • Offering emotional support through optional peer-oriented and/or peer-led support groups as well as licensed professional counselors, social workers, and school therapists;
  • Training all school staff—from upper-level administrators, to teachers, to custodians—how to be supportive of children and youth who have been affected by disaster as well as those who are in receiving communities who are now welcoming disaster-affected youth into their classrooms;
  • Designing and implementing disaster preparedness, response, and recovery curriculum within classrooms;
  • Providing opportunities for children to help their schools’ and classmates’ recovery; this could, for example, come in the form of service learning, fundraising, mentoring programs, or community action activities;
  • Offering immediate and long-term support for teachers, who are often recovering from disaster themselves; this may include financial, professional, and emotional support;
  • Intervening against bullying and stigma that may be attached to “disaster survivor” status for youth; reminding these professionals that bullying may be exacerbated based on region of origin, gender, age, race, or other characteristics;
  • Integrating displaced children in classrooms with familiar faces if possible;
  • Making school days as predictable as possible and re-establishing routines within classrooms and schools;
  • Allowing children and youth the opportunity to work on projects that help them process their disaster experience;
  • Funding school programs in arts, music, drama, and creative writing to encourage expression and foster healing.

Alice Fothergill, PhD is an associate professor of sociology at the University of Vermont. She is a member of the Social Science Research Council Research Network on Persons Displaced by Katrina. Fothergill’s book, Heads Above Water: Gender, Class, and Family in the Grand Forks Flood, examines women’s experiences in the 1997 flood in North Dakota. She is also co-editor of Social Vulnerability to Disasters.

Lori Peek, PhD is an associate professor of sociology and Co-Director of the Center for Disaster and Risk Analysis at Colorado State University. She also serves as the Associate Chair for the SSRC Task Force on Hurricane Katrina and Rebuilding the Gulf Coast and is a member of the SSRC Research Network on Persons Displaced by Katrina. Peek is the author of the award-winning book Behind the Backlash: Muslim Americans after 9/11 and co-editor of the volume Displaced: Life in the Katrina Diaspora.

Together, Fothergill and Peek are the authors of the award-winning book, Children of Katrina, the longest-term ethnographic study of children in disaster.

(View original at https://thesocietypages.org/socimages)

Planet DebianLuciano Prestes Cavalcanti: AppRecommender - Last GSoC Report

My work on Google Summer of Code is to create a new strategy on AppRecommender, where this strategy should be able to get a referenced package, or a list of referenced packages, then analyze the packages that the user has already installed and make a recommendation using the referenced packages as a base, for example: if the user runs "$ sudo apt install vim", the AppRecommender uses "vim" as the referenced package, and should recommend packages with relation between "vim" and the other packages that the user has installed. This work is done and added to the official AppRecommender repository.
 
During the GSoC program, more contributions were done with the AppRecommender project helping the system to improve the recommendations, installation and configurations to help Debian package.
 
The following link contains my commits on AppRecommender:
 
During the period destined to students get to know the community of the project, I talked with the Debian community about my project to get feedback and ideas. When talking to the Debian community on the IRC channels, we came up with the idea of using the popularity-contest data to improve the recommendations. I talked with my mentors, who approved the idea, then we increased the project scope to use the popularity-contest data to improve the AppRecommender recommendations.
 
The popularity-contest has several privacy political terms, then we did a research and published, on the Debian Planeta post that explains why we need the popularity-contest data to improve the recommendations and how we use this data. This post also contains an explanation about the risks and the measures taken to minimize them.
 
Then two activities were added to be made. One of them is to create a script to be added on popularity-contestThis script is destined to get the popularity-contest data, which is the users' packages, and generate clusters that group these packages analyzing similar users. The other activity is to add collaborative data into the AppRecommender, where this will download the clusters data and use it to improve the recommendations.
 
The popularity-contest cluster script was done and reviewed by my mentor, but was not integrated into popularity-contest yetThe usage of clusters data into AppRecommender has been already implemented, but still not added on official repository because it is waiting the cluster cript's acceptance into the popularity-contest. This work is not complete, but I will continue working with AppRecommender and Debian community, and with my mentorshelp, I will finish this work.
 
The following link contains my commits on repository with the popularity-contest cluster script's feature, as well as other scripts that I used to improve my work, but the only script that will be sent to popularity-contest is the create_popcon_clusters.py:
 
The following link contains my commits on repository with the AppRecommender collaborative data feature: 
 
Google Drive folder with the patch:

Planet DebianLars Wirzenius: Linux 25 jubilee symposium

I gave a talk about the early days of Linux at the jubilee symposium arranged by the University of Helsinki CS department. Below is an outline of what I meant to speak about, but the actual talk didn't follow it exactly. You can compare these to the video once it comes online.

  • Linus and I met at uni, the only 2 Swedish speaking new students that year, so we naturally migrated towards each other.
  • After a year away for military service, got back in touch, summer of
    1. .
  • C & Unix course fall of 1990; Minix.
  • Linus didn't think atime updates in real time were plausible, but I showed him; funnily enough, atime updates have been an issue in Linux until fairly recently, since they slow things down (without being particularly useful)
  • Jan 5, 1991 bought his first PC (i386 + i387 + 4 MiB RAM and a small hard disk); he had a Sinclair QL before that.
  • Played Prince of Persia for a couple of months.
  • Then wanted to learn i386 assembly and multitasking.
  • A/B threading demo.
  • Terminal emulation, Usenet access from home.
  • Hard disk driver, mistaking hard disk for a modem.
  • More ambition, announced Linux to the world for the first time
  • first ever Linux installation.
  • Upload to ftp.funet.fi, directory name by Ari Lemmke.
  • Originally not free software, licence changed early 1992.
  • First mailing list was created and introduced me to a flood of email (managed with VAX/VMS MAIL and later mush on Unix).
  • I talked a lot with Linus about design at this time, but never really participated in the kernel work (partly because disagreeing with Linus is a high-stress thing).
  • However, I did write the first sprintf for the kernel, since Linus hadn't learnt about varargs functions in C; he then ruined it and added the comment "Wirzenius wrote this portably..." (add google hit count for wirzenius+fucked).
  • During 1992 Linux grew fast, and distros happened, and a lot of packaging and porting of software; porting was easier because Linus was happy to add/change things in the kernel to accomodate software
  • A lot of new users during 1992 as well.
  • End of 1992 I and a few others founded the Linux Documentation Project to help all the new users, some of who didn't come from a Unix background.
  • In fact, things progressed so fast in 1992 that Linus thought he'd release 1.0 very soon, resulting in a silly sequence of version numbers: 0.12, 0.95, 0.96, 0.96b, 0.96c, 0.96c++2.
  • X server ported to Linux; almost immediate prediction of the year of the Linux desktop never happening unless ALL the graphics cards were supported immediately.
  • Linus was of the opinion that you needed one process (not thread) per window in X; I taught him event driven programming.
  • Bug in network code, resulting in ban on uni network.
  • Pranks in the shared office room.
  • We released 1.0 in an event at the CS dept in March, 1994; this included some talks and a ritual compilation of the release version during the event.

Planet DebianSatyam Zode: Google Summer of Code 2016 : Final Report

Project Title : Improving diffoscope tool and reproducibility of Debian packages

Project details

This project aims to improve diffoscope tool and fix Debian packages which are unreproducible in Reproducible builds testing framework. diffoscope recursively unpack archives of many kinds and transform various binary formats into more human readable form to compare them. As a part of this project I worked on argument completion feature and ignoring .buildinfo feature. This project is a part of Reproducible Builds effort

Mentor and Co-Mentor

  • Jérémy Bobbio (Lunar) : Mentor
  • Reiner Herrmann (deki) : Co-Mentor
  • Holger Levsen (h01ger) : Co-Mentor
  • Mattia Rizzolo (mapreri) : Co-Mentor

Project Discussion

Project Implementation

Challenges and Work Left

Future work

Acknowledgement

I would like to express my deepest gratitude to Lunar for mentoring me throughout Google Summer of Code program and for being cool. Lunar’s deep knowledge regarding diffoscope and Python skills helped me a lot throughout the project and we literally had great discussions. I would also like to thank Debaian community and Google for giving me this opportunity. Special thanks to Reproducible Builds folks for all the guidance!

Planet DebianDebConf team: Proposing speakers for DebConf17 (Posted by DebConf17 team)

As you may already know, next DebConf will be held at Collège de Maisonneuve in Montreal from August 6 to August 12, 2017. We are already thinking about the conference schedule, and the content team is open to suggestions for invited speakers.

Priority will be given to speakers who are not regular DebConf attendees, who are more likely to bring diverse viewpoints to the conference.

Please keep in mind that some speakers may have very busy schedules and need to be booked far in advance. So, we would like to start inviting speakers in the middle of September 2016.

If you would like to suggest a speaker to invite, please follow the procedure described on the Inviting Speakers page of the DebConf wiki.


DebConf17 team

Planet DebianVincent Sanders: Down the rabbit hole

My descent began with a user reporting a bug and I fear I am still on my way down.

Like Alice I headed down the hole. https://commons.wikimedia.org/wiki/File:Rabbit_burrow_entrance.jpg
The bug was simple enough, a windows bitmap file caused NetSurf to crash. Pretty quickly this was tracked down to the libnsbmp library attempting to decode the file. As to why we have a heavily used library for bitmaps? I am afraid they are part of every icon file and many websites still have favicons using that format.

Some time with a hex editor and the file format specification soon showed that the image in question was malformed and had a bad offset header entry. So I was faced with two issues, firstly that the decoder crashed when presented with badly encoded data and secondly that it failed to deal with incorrect header data.

This is typical of bug reports from real users, the obvious issues have already been encountered by the developers and unit tests formed to prevent them, what remains is harder to produce. After a debugging session with Valgrind and electric fence I discovered the crash was actually caused by running off the front of an allocated block due to an incorrect bounds check. Fixing the bounds check was simple enough as was working round the bad header value and after adding a unit test for the issue I almost moved on.

Almost...

american fuzzy lop are almost as cute as cats https://commons.wikimedia.org/wiki/File:Rabbit_american_fuzzy_lop_buck_white.jpg
We already used the bitmap test suite of images to check the library decode which was giving us a good 75% or so line coverage (I long ago added coverage testing to our CI system) but I wondered if there was a test set that might increase the coverage and perhaps exercise some more of the bounds checking code. A bit of searching turned up the american fuzzy lop (AFL) projects synthetic corpora of bmp and ico images.

After checking with the AFL authors that the images were usable in our project I added them to our test corpus and discovered a whole heap of trouble. After fixing more bounds checks and signed issues I finally had a library I was pretty sure was solid with over 85% test coverage.

Then I had the idea of actually running AFL on the library. I had been avoiding this because my previous experimentation with other fuzzing utilities had been utter frustration and very poor return on investment of time. Following the quick start guide looked straightforward enough so I thought I would spend a short amount of time and maybe I would learn a useful tool.

I downloaded the AFL source and built it with a simple make which was an encouraging start. The library was compiled in debug mode with AFL instrumentation simply by changing the compiler and linker environment variables.

$ LD=afl-gcc CC=afl-gcc AFL_HARDEN=1 make VARIANT=debug test
afl-cc 2.32b by <lcamtuf@google.com>
afl-cc 2.32b by <lcamtuf@google.com>
COMPILE: src/libnsbmp.c
afl-cc 2.32b by <lcamtuf@google.com>
afl-as 2.32b by <lcamtuf@google.com>
[+] Instrumented 751 locations (64-bit, hardened mode, ratio 100%).
AR: build-x86_64-linux-gnu-x86_64-linux-gnu-debug-lib-static/libnsbmp.a
COMPILE: test/decode_bmp.c
afl-cc 2.32b by <lcamtuf@google.com>
afl-as 2.32b by <lcamtuf@google.com>
[+] Instrumented 52 locations (64-bit, hardened mode, ratio 100%).
LINK: build-x86_64-linux-gnu-x86_64-linux-gnu-debug-lib-static/test_decode_bmp
afl-cc 2.32b by <lcamtuf@google.com>
COMPILE: test/decode_ico.c
afl-cc 2.32b by <lcamtuf@google.com>
afl-as 2.32b by <lcamtuf@google.com>
[+] Instrumented 65 locations (64-bit, hardened mode, ratio 100%).
LINK: build-x86_64-linux-gnu-x86_64-linux-gnu-debug-lib-static/test_decode_ico
afl-cc 2.32b by <lcamtuf@google.com>
Test bitmap decode
Tests:606 Pass:606 Error:0
Test icon decode
Tests:392 Pass:392 Error:0
TEST: Testing complete

I stuffed the AFL build directory on the end of my PATH, created a directory for the output and ran afl-fuzz

afl-fuzz -i test/bmp -o findings_dir -- ./build-x86_64-linux-gnu-x86_64-linux-gnu-debug-lib-static/test_decode_bmp @@ /dev/null

The result was immediate and not a little worrying, within seconds there were crashes and lots of them! Over the next couple of hours I watched as the unique crash total climbed into the triple digits.

I was forced to abort the run at this point as, despite clear warnings in the AFL documentation of the demands of the tool, my laptop was clearly not cut out to do this kind of work and had become distressingly hot.

AFL has a visualisation tool so you can see what kind of progress it is making which produced a graph that showed just how fast it managed to produce crashes and how much the return plateaus after just a few cycles. Although it was finding a new unique crash every ten minutes or so when aborted.

I dove in to analyse the crashes and it immediately became obvious the main issue was caused when the test tool attempted allocations of absurdly large bitmaps. The browser itself uses a heuristic to determine the maximum image size based on used memory and several other values. I simply applied an upper bound of 48 megabytes per decoded image which fits easily within the fuzzers default heap limit of 50 megabytes.

The main source of "hangs" also came from large allocations so once the test was fixed afl-fuzz was re-run with a timeout parameter set to 100ms. This time after several minutes no crashes and only a single hang were found which came as a great relief, at which point my laptop had a hard shutdown due to thermal event!

Once the laptop cooled down I spooled up a more appropriate system to perform this kind of work a 24way 2.1GHz Xeon system. A Debian Jessie guest vm with 20 processors and 20 gigabytes of memory was created and the build replicated and instrumented.

AFL master node display
To fully utilise this system the next test run would utilise AFL in parallel mode. In this mode there is a single "master" running all the deterministic checks and many "secondary" instances performing random tweaks.

If I have one tiny annoyance with AFL, it is that breeding and feeding a herd of rabbits by hand is annoying and something I would like to see a convenience utility for.

The warren was left overnight with 19 instances and by morning had generated crashes again. This time though the crashes actually appeared to be real failures.

$ afl-whatsup sync_dir/
Summary stats
=============

Fuzzers alive : 19
Total run time : 5 days, 12 hours
Total execs : 214 million
Cumulative speed : 8317 execs/sec
Pending paths : 0 faves, 542 total
Pending per fuzzer : 0 faves, 28 total (on average)
Crashes found : 554 locally unique

All the crashing test cases are available and a simple file command immediately showed that all the crashing test files had one thing in common the height of the image was -2147483648 This seemingly odd number is actually meaningful to a programmer, it is the largest negative number which can be stored in a 32bit integer (INT32_MIN) I immediately examined the source code that processes the height in the image header.

if ((width <= 0) || (height == 0))          
return BMP_DATA_ERROR;
if (height < 0) {
bmp->reversed = true;
height = -height;
}

The bug is where the height is made a positive number and results in height being set to 0 after the existing check for zero and results in a crash later in execution. A simple fix was applied and test case added removing the crash and any possible future failure due to this.

Another AFL run has been started and after a few hours has yet to find a crash or non false positive hang so it looks like if there are any more crashes to find they are much harder to uncover.

Main lessons learned are:
  • AFL is an easy to use and immensely powerful and effective tool. State of the art has taken a massive step forward.
  • The test harness is part of the test! make sure it does not behave in a poor manner and cause issues itself.
  • Even a library with extensive test coverage and real world users can benefit from this technique. But it remains to be seen how quickly the rate of return will reduce after the initial fixes.
  • Use the right tool for the job! Ensure you head the warnings in the manual as AFL uses a lot of resources including CPU, disc and memory.
I will of course be debugging any new crashes that occur and perhaps turning my sights to all the projects other unit tested libraries. I will also be investigating the generation of our own custom test corpus from AFL to replace the demo set, this will hopefully increase our unit test coverage even further.

Overall this has been my first successful use of a fuzzing tool and a very positive experience. I would wholeheartedly recommend using AFL to find errors and perhaps even integrate as part of a CI system.

CryptogramResearch on the Timing of Security Warnings

fMRI experiments show that we are more likely to ignore security warnings when they interrupt other tasks.

A new study from BYU, in collaboration with Google Chrome engineers, finds the status quo of warning messages appearing haphazardly­ -- while people are typing, watching a video, uploading files, etc.­ -- results in up to 90 percent of users disregarding them.

Researchers found these times are less effective because of "dual task interference," a neural limitation where even simple tasks can't be simultaneously performed without significant performance loss. Or, in human terms, multitasking.

"We found that the brain can't handle multitasking very well," said study coauthor and BYU information systems professor Anthony Vance. "Software developers categorically present these messages without any regard to what the user is doing. They interrupt us constantly and our research shows there's a high penalty that comes by presenting these messages at random times."

[...]

For part of the study, researchers had participants complete computer tasks while an fMRI scanner measured their brain activity. The experiment showed neural activity was substantially reduced when security messages interrupted a task, as compared to when a user responded to the security message itself.

The BYU researchers used the functional MRI data as they collaborated with a team of Google Chrome security engineers to identify better times to display security messages during the browsing experience.

Research paper. News article.

CryptogramTerrorist False Alarm at JFK Airport Demonstrates How Unprepared We Really Are

The detailed accounts of the terrorist-shooter false-alarm at Kennedy Airport in New York last week illustrate how completely and totally unprepared the airport authorities are for any real such event.

I have two reactions to this. On the one hand, this is a movie-plot threat -- the sort of overly specific terrorist scenario that doesn't make sense to defend against. On the other hand, police around the world need training in these types of scenarios in general. Panic can easily cause more deaths than terrorists themselves, and we need to think about what responsibilities police and other security guards have in these situations.

Worse Than FailureCodeSOD: An Angular Watch

Let’s talk a little bit about front-end development. Even at its best, it’s terrible- decades of kruft mixed with standards and topped off with a pile of frameworks that do their best to turn this mess into a cohesive whole.

Jameson is suffering through this, and his suffering is the special level of front-end suffering known as “Angular”. Angular bolts Model-View-Controller semantics on top of HTML/JS/CSS, and its big selling point is that it makes two-way data-binding trivially easy.

Under the hood, that two-way data-binding is implemented using a concept of “watchers”. Essentially, these abstract out the event handling and allow Angular- or your own custom code- easily detect changes in the various UI widgets. These watchers also implement nice features, like automatically detecting if a form field is “$pristine” or if the form (or any given field) happens to be “$valid”.

So, for example, if you wanted to have a submit button automatically disable itself if the form were untouched or invalid, you might do something like this:

        <button type="submit" ng-disabled="companyProfileForm.$pristine || companyProfileForm.$invalid">Save</button>

Of course, that’s only if you wanted to actually get some benefit out of the ten-thousand line framework you just baked into your application. Jameson’s fellow developers have a very different approach:

var watchers =
        'paymentOptions.PrimaryCreditCardActive,' + //0
        'paymentOptions.PrimaryCardOwnerName,' + //1
        'paymentOptions.PrimaryCardNumber,' + //2
        'paymentOptions.PrimaryCardMonth,' + //3
        'paymentOptions.PrimaryCardYear,' + //4

        'paymentOptions.SecondaryCreditCardActive,' + //5
        'paymentOptions.SecondaryCardOwnerName,' + //6
        'paymentOptions.SecondaryCardNumber,' + //7
        'paymentOptions.SecondaryCardMonth,' + //8
        'paymentOptions.SecondaryCardYear,' + //9

        'paymentOptions.ECheckDirectWithdrawlActive,' + //10
        'paymentOptions.ECheckBankAccountNumber,' + //11
        'paymentOptions.ECheckBankRoutingNumber'; //12

$scope.$watchCollection('[' + watchers + ']', function(newValues){
        $scope.companyProfileForm.$pristine = true;

        if(newValues[0] && newValues[1] && newValues[2] && newValues[3] && newValues[4]){
                $scope.companyProfileForm.$pristine = false;
        }

        if(newValues[5] && newValues[6] && newValues[7] && newValues[8] && newValues[9]){
                $scope.companyProfileForm.$pristine = false;
        }

        if(newValues[10] && newValues[11] && newValues[12]){
                $scope.companyProfileForm.$pristine = false;
        }

        if(newValues[0] && ( !newValues[1] || !newValues[2] || !newValues[3] || !newValues[4])){
                $scope.companyProfileForm.$pristine = true;
        }

        if(newValues[5] && ( !newValues[6] || !newValues[7] || !newValues[8] || !newValues[9])){
                $scope.companyProfileForm.$pristine = true;
        }

        if(newValues[10] && ( !newValues[11] || !newValues[12])){
                $scope.companyProfileForm.$pristine = true;
        }
});

The initial watchers string represents an “Angular expression” which is a fancy way of saying, “JavaScript scope is too complicated, so we’re just not going to do it and require developers to use our own custom expression language”. This string is passed to $watchCollection, which will execute the callback function if the value of any one of those fields changes.

Then, inside of a batch of cryptic if statements, it makes its own decisions about whether or not the Angular-controlled $pristine property should be a certain value or not- decisions that aren’t in any way based on the requirements for this application.

Jameson fixed the code to look something more like my suggested version.

[Advertisement] Atalasoft’s imaging SDKs come with APIs & pre-built controls for web viewing, browser scanning, annotating, & OCR/barcode capture. Try it for 30 days with included support.

Planet DebianMichal Čihař: Continuous integration on multiple platforms

Over the weekend I've played with continuous integration for Gammu to make it run on more platforms. I had to remember many things from the Windows world on the way and the solution is not yet complete, but the basic build is working, the only problematic part are external dependencies.

First of all we already have Linux builds on Travis CI. These cover compilation with both GCC and Clang compilers, hopefully covering most of the possible problems.

Recently I've added OS X builds on Travis CI, what was pretty much painless and worked out of the box.

The next major architecture to support is Windows. Once I've discovered AppVeyor I thought it might be the way to go. The have free plans for open-source projects (though it has only one parallel build compared to four provided by Travis CI).

As our build system is cross platform based on CMake, it should work pretty much out of the box, right? Well almost, tweaking the basics took some time (unfortunately there is no CMake support on AppVeyor, so you have to script it a bit).

The most painful things on the way:

  • finding our correct way to invoke build and testsuite
  • our code was broken on Windows, making the testsuite to fail
  • how to work with power shell (no, I'm not going to like it)
  • how to download and install executable to PATH
  • test output integration with AppVeyor - done using XSLT transformation and uploading test results manually
  • 32-bit / 64-bit mess, CMake happily finds 32-bit libs during the 64-bit build and vice versa, what makes the build fail later when linking - fixed by trying if code can be built with given library
  • 64-bit code crashes in dummy driver, causing testsuite failures (this has to be something Windows specific as the code works fine on 64-bit Linux) - this seems to be caused by too big allocations on stack, moving them to heap will fix this

You can check our current appveyor.yml in case you're going to try something similar. Current build results are on AppVeyor.

As a nice side effect, we now have up to date Windows binaries for Gammu.

Filed under: Debian English Gammu | 0 comments

Planet DebianNOKUBI Takatsugu: The 9th typhoon looks like Debian swirl logo

According to my follower’s tweet:

The typhoon image and horizontal flipped Debian logo looks same.

Planet DebianZlatan Todorić: When you wake up with a feeling

I woke up at 5am. Somehow made myself to soon go back to sleep again. Woke up at 6am. Such is the life of jet-lag. Or I am just getting old for it.

But the truth wouldn't be complete with only those assertion. I woke inspired and tired and the same time. Tired because I am doing very time consumable things. Also in the same time very emotional things. AND at the exact same time things that inspire me.

On paper, I am technical leader of Purism. In reality, I have insanely good relations with my CEO for such a short time. So good that I for months were not leading the technical shift only, but also I overtook operations (getting orders and delivering them while working with our assembly line to automate most of the tasks in this field). I was playing also as first line of technical support (forums, IRC and email). Actually I was pretty much the only line of support for few months. I was doing some website changes: change some wording, updating bunch of plugins and making it sure all works, resolved (hopefully) Tor and Cloudflare issues for it, annoying caching system for forums, stopped forum spam and so on. I worked on better messaging for Purism public relations. I thought my team to use keys for signing and encryption. I interviewed (and read all mails) for people that were interested in working or helping Purism. In process of doing all that, I maybe wasn't the most speedy person for all our users needs but I hope they understand and forgive me.

I was doing all that while I was researching and developing tablets (which ended up not being the most successful campaign but we now do have them as product). I was doing all that while seeing (and resolving) that our kernel builds were failing. Worked on pushing touchpad (not so good but we are still working on) patches upstream (and they ended being upstreamed). While seeing repos being down because of our host. Repos being down because of broken sync with Debian. Repos being down because of our key mis-management. Metadata not working well. PureBrowser getting broken all the time. Tor browser out of date. No real ISO updates. Wrong sources.list entries and so on.

And the hardest part on work was, I was doing all this with very limited scope and even more limited resources. So what kept me on, what is pushing me forward and what am I doing?

One philosophy - Free software. Let me not explain it as a technical debt. Let me explain it as social movement. In age, where people are "bombed" by media, by all-time lying politicians (which use fear of non-existent threats/terror as model to control population), in age where proprietary corporations are selling your freedom so you can gain temporary convenience the term Free software is like Giordano Bruno in age of Inquisitions. Free software does not only preserve your Freedom to software source usage but it preserves your Freedom to think and think out of the box and not being punished for that. It preserves the Freedom to live - to choose what and when to do, without having the negative impact on your or others people lives. The Freedom to be transparent and to share. Because not only ideas grow with sharing, but we, as human beings, grow as we share. The Freedom to say "NO".

NO. I somehow learnt, and personally think, that the Freedom to say NO is the most important Freedom in our lives. No I will not obey some artificially created master that think they can plan and choose my life decision. No I will not negotiate my Freedom for your convenience (also, such Freedom is anyway not real and it is matter of time where you will be blown away by such illusion). No I will not accept your credit because it has STRINGS attached to it which you either don't present or you blur it in mountain of superficial wording. No I will not implant a chip inside me for sake of your research or my convenience. No I will not have social account on media where majority of people are. No, I will not have pacemaker which is a blackbox with proprietary (buggy) software and it harvesting my data without me being able to look at it.

Yin-Yang. Yes, I want to collaborate on making world better place for us all. I don't agree with most of people, but that doesn't make them my enemies (although media would like us to feel and think like that). I will try to preserve everyones Freedom as much as I can. Yes I will share with my community and friends. Yes I want to learn from better than I am. Yes I want to have awesome mentors. Yes, I will try to be awesome mentor. Yes, I choose to care and not ignore facts and actions done by me and other people. Yes, I have the right to be imperfect and do mistakes as long as I will aknowledge and work on them. Bugfixing ourselves as humans is the most important task in our lives. As in software, it is very time consumable but also as in software, it is improvement and incredible satisfaction to see better version of yourself, getting more and more features (even if that sometimes means actually getting read of other/bad features).

This all is blending with my work at Purism. I spend a lot of time thinking about projects, development and future. I must do that in order not to make grave mistakes. Failing hardware and software is not grave mistake. Serious, but not grave. Grave is if we betray ourselves and our community in pursue for Freedom. We are trying to unify many things - we want to give you security, privacy and FREEDOM with convenience. So I am pushing myself out of comfort zones and also out of conventional and sometimes even my standard way of thinking. I have seen that non-existing infrastructure for PureOS is hurting is a lot but I needed to cope with it to the time where I will be able to say: not anymore, we are starting to build our own infrastructure. I was coping with Cloudflare being assholes to Tor users but now we also shifting away from them. I came to team where people didn't properly understand what and why are we building this. Came to very small and not that efficient team.

Now, we employed a dedicated and hard working person on operations (Goran) which I trust. We have dedicated support person (Mladen) which tries hard to work with people. A very creative visual mastermind (Francois). We have a capable Debian Developer (Matthias Klumpp) working on PureOS new infra. We have a capable and dedicated sysadmins (Theo and Stelio) which we didn't even have in past. We are trying to LEVEL UP Free software and unify them in convenient solution which is lead by Joey Hess. We have a hard-working PureOS developer (Hema) who is coping with current non-existent PureOS infra. We have GNOME Boards of Directors person (Jeff) who is trying to light up our image in world (working with James, to try bring some lights into our shadows caused by infinite supply chain delays). We have created Advisory Board for Freedom, Privacy and Security which I don't want to name now as we are preparing to announce soon that (and trust me, we have good people in here).

But, the most important thing here is not that they are all capable or cool people. It is the core value in all of them - they care about Freedom and I trust them on their paths. The trust is always important but in Purism it is essential for our work. I built the workflow without time management (everyone spends their time every single day as they see it fit as long as the work gets done). And we don't create insane short deadlines because everyone else thinks it is important (and rarely something is more important than our time freedom). So the trust is built out of knowledge and the knowledge I have about them and their works is because we freely share with no strings attached.

Because of them, and other good people from our community I have the energy to sacrifice my entire time for Purism. It is not white and black: CEO and me don't always agree, some members of my team don't always agree with me or I with them, some people in community are very rude, impolite and don't respect our work but even with disagreement everyone in Purism finds agreement at the end (we use facts in our judgments) and all the people who just try to disturb my and mine teams work aren't as efficient as all the lovely words of people who believe in us, who send us words of support and who share ideas and their thoughts with us. There is no more satisfaction for me than reading a personal mail giving us kudos for the work and their understanding of underlaying amount of work and issues.

While we are limited with resources we had an occasional outcry from community to help us. Now I want to help them to help me (you see the Freedom of sharing here?). PureOS has now a wiki. It will be a community wiki which is endorsed by Purism as company. Yes you read it right, Purism considers its community part of company (you don't need to get paycheck to be Purism member). That is why a call upon contributors (technical but mostly non-technical too) to help us make PureOS wiki the best resource on net for our needs. Write tutorials for others, gather and put info on wiki, create an ideas page and vote on them so we can see what community wants to see, chat with us so we all understand what, why and how are we working on things. Make it as transparent as possible. Everyone interested please get in touch with our teams by either poking us online (IRC, social accounts) or via emails (our personal or [hr, pr, feedback]@puri.sm.

To finish this writing (as it is 8am here and I still want to rest a bit because I will have meetings for 6 hours straight today) - I wanted to share some personal insight into few things from my point of view. I wanted to say despite all the troubles and people who tried to make our time even harder (and it is already hard by all the limitation which come naturally today with our kind of work), we still create products, we still ship them, we still improved step by step, we still hired and we are still building. Keeping all that together and making progress is for me a milestone greater than just creating a technical product. I just hope we will continue and improve our pace so we can start progressing towards my personal great goal - integrate and cooperate with most of FLOSS ecosystem.

P.S. yes, I also (finally!) became an official Debian Developer - still didn't have time to sit and properly think and cry (as every good men) about it.

Planet DebianChristian Perrier: [LIFE] Running activities - Ultra Trail du Mont-Blanc

Hello dear readers,

It's been ages since I last blogged. Being far less active in Debian than I've been in the past, I guess this is a logical consequence.

However, I'm still active as you may witness if you read the debian-boot mailing list : I still consider myself part of the D-I team and I'm maintaining a few sports-related packages.

Most know what has taken precedence over Debian development, namely trail and ultra-trail running. And, well, it hasn't decreased, far from that : I ran about 10 races already this year....6 of them being above 50km and I ran my favourite 100km moutain race in early July for the second year in a row.

So, the upcoming week, I'll be trying to reach what is usually considered as the Grail of ultra-trail runners : the Ultra-Trail du Mont-Blanc race in Chamonix.

The race is fairly simple : run all around the Mont-Blanc summits, for a 160km race with a bit less than 10,000 meters positive climb. The race itself takes place between 800 and 2700 meters (so no "high mountain") and I expect to complete it (if I succeed) in about 40 hours.

I'm very confident (maybe too much?) as I successfully completed a much more difficult race last year (only 144km, but over 11,000 meters positive climb and a much more difficult path...it took me over 50 hours to complete it).

You can follow me on the live tracking site. The race starts on Friday August 26th, 18:00 CET DST.

I everything goes well, I have great projects for next year, including a 100-mile race in Colorado in August (we'll be traveling in USA for over 3 weeks, peaking with the solar eclipse of August 21st in Kansas City).

Planet DebianPaul Tagliamonte: go-wmata - golang bindings to the DC metro system

A few weeks ago, I hacked up go-wmata, some golang bindings to the WMATA API. This is super handy if you are in the DC area, and want to interface to the WMATA data.

As a proof of concept, I wrote a yo bot called @WMATA, where it returns the closest station if you Yo it your location. For hilarity, feel free to Yo it from outside DC.

For added fun, and puns, I wrote a dbus proxy for the API as weel, at wmata-dbus, so you can query the next train over dbus. One thought was to make a GNOME Shell extension to tell me when the next train is. I’d love help with this (or pointers on how to learn how to do this right).

,

Planet DebianCyril Brulebois: Freelance Debian consultant: running DEBAMAX

Executive summary

Since October 2015, I've been running a FLOSS consulting company, specialized on Debian, called DEBAMAX.

DEBAMAX logo

Longer version

Everything started two years ago. Back then I blogged about one of the biggest changes in my life: trying to find the right balance between volunteer work as a Debian Developer, and entrepreneurship as a Freelance Debian consultant. Big change because it meant giving up the comfort of the salaried world, and figuring out whether working this way would be sufficient to earn a living…

I experimented for a while under a simplified status. It comes with a number of limitations but that’s a huge win compared to France’s heavy company-related administrativia. Here’s what it looked like, everything being done online:

  • 1 registration form to begin with: wait a few days, get an identifier from INSEE, mention it in your invoices, there you go!

  • 4 tax forms a year: taxes can be declared monthly or quarterly, I went for the latter.

A number of things became quite clear after a few months:

  • I love this new job! Sharing my Debian knowledge with customers, and using it to help them build/improve/stabilise their products and their internal services feels great!

  • Even if I wasn't aware of that initially, it seems like I've got a decent network already: Debian Developers, former coworkers, and friends thought about me for their Debian-related tasks. It was nice to hear about their needs, say yes, sign paperwork, and start working right away!

  • While I'm trying really hard not to get too optimistic (achieving a given turnover on the first year doesn't mean you're guaranteed to do so again the following year), it seemed to go well enough for me to consider switching from this simplified status to a full-blown company.

Thankfully I was eligible to being accompanied by the local Chamber of Commerce and Industry (CCI Rennes), which provides teaching sessions for new entrepreneurs, coaching, and meeting opportunities (accountants, lawyers, insurance companies, …). Summer in France is traditionally rather quiet (read: almost everybody is on vacation), so DEBAMAX officially started operating in October 2015. Besides different administrative and accounting duties, running this company doesn't change the way I've been working since July 2014, so everything is fine!

As before, I won't be writing much about it through my personal blog, except for an occasional update every other year; if you want to follow what's happening with DEBAMAX:

  • Website: debamax.com — in addition to the usual company, services, and references sections, it features a blog (with RSS) where some missions are going to be detailed (when it makes sense to share and when customers are fine with it). Spoiler alert: Tails is likely to be the first success story there. ;)
  • Twitter: @debamax — which is going to be retweeted for a while from my personal account, @CyrilBrulebois.

Planet DebianGregor Herrmann: RC bugs 2016/30-33

not much to report but I got at least some RC bugs fixed in the last weeks. again mostly perl stuff:

  • #759979 – src:simba: "simba: FTBFS: RoPkg::Rsync ...failed! (needed)"
    keep ExtUtils::AutoInstall from downlaoding stuff, upload to DELAYED/7
  • #817549 – src:libropkg-perl: "libropkg-perl: Removal of debhelper compat 4"
    use debhelper compatibility level 5, upload to DELAYED/7
  • #832599 – iodine: "Fails to start after upgrade"
    update service file and use deb-systemd-helper in postinst
  • #832832 – src:perlbrew: "perlbrew: FTBFS: Tests failures"
    add patch to deal with removed old perl version (pkg-perl)
  • #832833 – src:libtest-valgrind-perl: "libtest-valgrind-perl: FTBFS: Tests failures"
    upload new upstream release (pkg-perl)
  • #832853 – src:libmojomojo-perl: "libmojomojo-perl: FTBFS: Tests failures"
    close, the underlying problem is fixed (pkg-perl)
  • #832866 – src:libclass-c3-xs-perl: "libclass-c3-xs-perl: FTBFS: Tests failures"
    upload new upstream release (pkg-perl)
  • #834210 – libdancer-plugin-database-core-perl: "libdancer-plugin-database-perl: FTBFS: Failed 1/5 test programs. 6/45 subtests failed."
    upload new upstream release (pkg-perl)
  • #834793 – libgit-wrapper-perl: "libgit-wrapper-perl: FTBFS: t/basic.t whitespace changes"
    add patch from upstream bug (pkg-perl)

Planet DebianDavid Moreno: WIP: Perl bindings for Facebook Messenger

A couple of weeks ago I started looking into wrapping the Facebook Messenger API into Perl. Since all the calls are extremely simple using a REST API, I thought it could be easier and simpler even, to provide a small framework to hook bots using PSGI/Plack.

So I started putting some things together and with a very simple interface you could do a lot:

use strict;
use warnings;
use Facebook::Messenger::Bot;

my $bot = Facebook::Messenger::Bot->new({
    access_token   => '...',
    app_secret     => '...',
    verify_token   => '...'
});

$bot->register_hook_for('message', sub {
    my $bot = shift;
    my $message = shift;

    my $res = $bot->deliver({
        recipient => $message->sender,
        message => { text => "You said: " . $message->text() }
    });
    ...
});

$bot->spin();

You can hook a script like that as a .psgi file and plug it in to whatever you want.

Once you have some more decent user flow and whatnot, you can build something like:



…using a simple script like this one.

The work is not finished and not yet CPAN-ready but I’m posting this in case someone wants to join me in this mini-project or have suggestions, the work in progress is here.

Thanks!

Planet DebianDavid Moreno: Cosmetic changes to my posts archive

I’ve been doing a lot of cosmetic/layout changes to the nearly 750 posts in my blog’s archive. I apologize if this has broken some feed readers or aggregators. It appears like Hexo still needs better syndication support.

Planet DebianDavid Moreno: Running find with two or more commands to -exec

I spent a couple of minutes today trying to understand how to make find (1) to execute two commands on the same target.

Instead of this or any similar crappy variants:

$ find . -type d -iname "*0" -mtime +60 -exec scp -r -P 1337 "{}" "meh.server.com:/mnt/1/backup/storage" && rm -rf "{}" \;

Try something like this:

$ find . -type d -iname "*0" -mtime +60 -exec scp -r -P 1337 "{}" "meh.server.com:/mnt/1/backup/storage" \; -exec rm -rf "{}" \;

Which is:

$ find . -exec command {} \; -exec other command {} \;

And you’re good to go.

Planet DebianDirk Eddelbuettel: RcppEigen 0.3.2.9.0

A new upstream release 3.2.9 of Eigen is now reflected in a new RcppEigen release 0.3.2.9.0 which got onto CRAN late yesterday and is now going into Debian. Once again, Yixuan Qiu did the heavy lifting of merging upstream (and two local twists we need to keep around). Another change is by James "coatless" Balamuta who added a row exporter.

The NEWS file entry follows.

Changes in RcppEigen version 0.3.2.9.0 (2016-08-20)

  • Updated to version 3.2.9 of Eigen (PR #37 by Yixuan closing #36 from Bob Carpenter of the Stan team)

  • An exporter for RowVectorX was added (thanks to PR #32 by James Balamuta)

Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

,

Planet DebianDaniel Stender: Collected notes from Python packaging

Here are some collected notes on some particular problems from packaging Python stuff for Debian, and more are coming up like this in the future. Some of the issues discussed here might be rather simple and even benign for the experienced packager, but maybe this is be helpful for people coming across the same issues for the first time, wondering what's going wrong. But some of the things discussed aren't easy. Here are the notes for this posting, there is no particular order.

UnicodeDecoreError on open() in Python 3 running in non-UTF-8 environments

I've came across this problem again recently packaging httpbin 0.5.0. The build breaks the following way e.g. trying to build with sbuild in a chroot, that's the first run of setup.py with the default Python 3 interpreter:

I: pybuild base:184: python3.5 setup.py clean 
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    os.path.join(os.path.dirname(__file__), 'README.rst')).read()
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2386: ordinal not in range(128)
E: pybuild pybuild:274: clean: plugin distutils failed with: exit code=1: python3.5 setup.py clean 

One comes across UnicodeDecodeError-s quite oftenly on different occasions, mostly related to Python 2 packaging (like here). Here it's the case that in setup.py the long_description is tried to be read from the opened UTF-8 encoded file README.rst:

long_description = open(
    os.path.join(os.path.dirname(__file__), 'README.rst')).read()

This is a problem for Python 3.5 (and other Python 3 versions) when setup.py is executed by an interpreter run in a non-UTF-8 environment1:

$ LANG=C.UTF-8 python3.5 setup.py clean
running clean
$ LANG=C python3.5 setup.py clean
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    os.path.join(os.path.dirname(__file__), 'README.rst')).read()
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2386: ordinal not in range(128)
$ LANG=C python2.7 setup.py clean
running clean

The reason for this error is, the default encoding for file object returned by open() e.g. in Python 3.5 is platform dependent, so that read() fails on that when there's a mismatch:

>>> readme = open('README.rst')
>>> readme
<_io.TextIOWrapper name='README.rst' mode='r' encoding='ANSI_X3.4-1968'>

Non-UTF-8 build environments because $LANG isn't particularly set at all or set to C are common or even default in Debian packaging, like in the continuous integration resp. test building for reproducible builds the primary environment features that (see here). That's also true for the base system of the sbuild environment:

$ schroot -d / -c unstable-amd64-sbuild -u root
(unstable-amd64-sbuild)root@varuna:/# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

A problem like this is solved mostly elegant by installing some workaround in debian/rules. A quick and easy fix is to add export LC_ALL=C.UTF-8 here, which supersedes the locale settings of the build environment. $LC_ALL is commonly used to change the existing locale settings, it overrides all other locale variables with the same value (see here). C.UTF-8 is an UTF-8 locale which is available by default in a base system, it could be used without installing the locales package (or worse, the huge locales-all package):

(unstable-amd64-sbuild)root@varuna:/# locale -a
C
C.UTF-8
POSIX

This problem ideally should be taken care of upstream. In Python 3, the default open() is io.open(), for which the specific encoding could be given, so that the UnicodeDecodeError vanishes. Python 2 uses os.open() for open(), which doesn't support that, but has io.open(), too. A fix which works for both Python branches goes like this:

import io
long_description = io.open(
    os.path.join(os.path.dirname(__file__), 'README.rst'), encoding='utf-8').read()

non-deterministic order of requirements in egg-info/requires.txt

This problem appeared in prospector/0.11.7-5 in the reproducible builds test builds, that was the first package of Prospector running on Python 32. It was revealed that there were differences in the sorting order of the [with_everything] dependencies resp. requirements in prospector-0.11.7.egg-info/requires.txt if the package was build on varying systems:

$ debbindiff b1/prospector_0.11.7-6_amd64.changes b2/prospector_0.11.7-6_amd64.changes 
{...}
├── prospector_0.11.7-6_all.deb
│   ├── file list
│   │ @@ -1,3 +1,3 @@
│   │  -rw-r--r--   0        0        0        4 2016-04-01 20:01:56.000000 debian-binary
│   │ --rw-r--r--   0        0        0     4343 2016-04-01 20:01:56.000000 control.tar.gz
│   │ +-rw-r--r--   0        0        0     4344 2016-04-01 20:01:56.000000 control.tar.gz
│   │  -rw-r--r--   0        0        0    74512 2016-04-01 20:01:56.000000 data.tar.xz
│   ├── control.tar.gz
│   │   ├── control.tar
│   │   │   ├── ./md5sums
│   │   │   │   ├── md5sums
│   │   │   │   │┄ Files in package differ
│   ├── data.tar.xz
│   │   ├── data.tar
│   │   │   ├── ./usr/share/prospector/prospector-0.11.7.egg-info/requires.txt
│   │   │   │ @@ -1,12 +1,12 @@
│   │   │   │  
│   │   │   │  [with_everything]
│   │   │   │ +pyroma>=1.6,<2.0
│   │   │   │  frosted>=1.4.1
│   │   │   │  vulture>=0.6
│   │   │   │ -pyroma>=1.6,<2.0

Reproducible builds folks recognized this to be a toolchain problem and set up the issue randonmness_in_python_setuptools_requires.txt to cover this. Plus, a wishlist bug against python-setuptools was filed to fix this. The patch which was provided by Chris Lamb adds sorting of dependencies in requires.txt for Setuptools by adding sorted() (stdlib) to _write_requirements() in command/egg_info.py:

--- a/setuptools/command/egg_info.py
+++ b/setuptools/command/egg_info.py
@@ -406,7 +406,7 @@ def warn_depends_obsolete(cmd, basename, filename):
 def _write_requirements(stream, reqs):
     lines = yield_lines(reqs or ())
     append_cr = lambda line: line + '\n'
-    lines = map(append_cr, lines)
+    lines = map(append_cr, sorted(lines))
     stream.writelines(lines)

O.k. In the toolchain, no instance sorts these requirements properly if differences appear, but what is the reason for these differences in the Prospector packages, though? The problem is somewhat subtle. In setup.py, [with_everything] is a dictionary entry of _OPTIONAL (which is used for extras_require) that is created by a list comprehension out of the other values in that dictionary. The code goes like this:

_OPTIONAL = {
    'with_frosted': ('frosted>=1.4.1',),
    'with_vulture': ('vulture>=0.6',),
    'with_pyroma': ('pyroma>=1.6,<2.0',),
    'with_pep257': (),  # note: this is no longer optional, so this option will be removed in a future release
}
_OPTIONAL['with_everything'] = [req for req_list in _OPTIONAL.values() for req in req_list]

The result, the new _OPTIONAL dictionary including the key with_everything (which w/o further sorting is the source of the list of requirements requires.txt) e.g. looks like this (code snipped run through IPython):

In [3]: _OPTIONAL
Out[3]: 
{'with_everything': ['vulture>=0.6', 'pyroma>=1.6,<2.0', 'frosted>=1.4.1'],
 'with_vulture': ('vulture>=0.6',),
 'with_pyroma': ('pyroma>=1.6,<2.0',),
 'with_frosted': ('frosted>=1.4.1',),
 'with_pep257': ()}

That list comprehension iterates over the other dictionary entries to gather the value of with_everything, and – Python programmers know that of course – dictionaries are not indexed and therefore the order of the key-value pairs isn't fixed, but is determined by certain conditions from outside the interpreter. That's the source for the non-determinism of this Debian package revision of Prospector.

This problem has been fixed by a patch, which just presorts the list of requirements before it gets added to _OPTIONAL:

@@ -76,8 +76,8 @@
     'with_pyroma': ('pyroma>=1.6,<2.0',),
     'with_pep257': (),  # note: this is no longer optional, so this option will be removed in a future release
 }
-_OPTIONAL['with_everything'] = [req for req_list in _OPTIONAL.values() for req in req_list]
-
+with_everything = [req for req_list in _OPTIONAL.values() for req in req_list]
+_OPTIONAL['with_everything'] = sorted(with_everything)

In comparison to the list method sort(), the function sorted() does not change iterables in-place, but returns a new object, both could be used. As a side note, egg-info/requires.txt isn't even needed, but that's another issue.


  1. As an alternative to prefixing LC_ALL, env -i could be used to get an empty environment. 

  2. 0.11.7-4 already but this package revision was in experimental due to the switch to Python 3 and therefore not tested by reproducible builds continuous integration. 

TEDHow Jane Chen built a better baby warmer — and a thriving business

Jane_Chen_embrace

In her 2013 TEDWomen Talk, entrepreneur (and TED Fellow) Jane Chen noted that “there are 15 million pre-term and underweight babies born every year around the world, and one of the biggest problems they face is staying warm.”

Premature babies can’t properly regulate their body temperatures and need an incubator in order for their organs to develop properly. If a baby is wasting energy on trying to stay warm, a range of problems can result: diabetes, heart disease, low IQ, and sometimes death. Four million of these babies die annually.

Shortly after receiving her MBA from Stanford University, Chen moved to India and set up her company, Embrace Innovations, in order to develop a low-cost, portable, reusable incubator to help women in remote areas of the world where a lack of reliable electricity and the high cost of medical equipment made the traditional incubators we have in hospitals impossible.

After two years of clinical testing, setting up manufacturing and distribution, Chen’s company launched the Embrace. The comfortable infant wrap uses phase-change material to melt at human body temperature and stay the proper temperature for eight hours. After that, the heat source can be replaced with a new one, to continuously supply a nurturing environment for babies who need it.

In a new post for Forbes magazine, Chen talks about what happened next.

“After five years as CEO, I returned to San Francisco and was on the verge of closing a deal with a major medical device company that was taking the full round of our next investment and would become our global distributor. I was ecstatic. This was exactly where I had hoped to take the company — this would make us scalable, and would significantly increase the impact we could make.”

But then, as she describes it, in a “cruel twist of fate,” the company she had signed on with fired its CEO and the deal she had worked so hard on disappeared overnight. Her company had seven days of cash left.

She went on to describe the whirlwind that many start-ups go through: she took out two bridge loans and asked everyone she knew for small investments to keep her company going until she could arrange another deal. She finally found an angel investor in Marc Benioff, the CEO and founder of Salesforce.com, who had personal experience with his own child needing an incubator. He gave her company the lifeline it needed to stay afloat and give Chen the time she needed to look for a new way forward.

Later that year, she started surfing in Hawaii, another lifelong dream. She likens her experiences with her start-up to the profound lessons she has learned as a beginner surfer: “Everything is impermanent. When the waves knock you down, try again. Take the lessons you can from it, and move on to the next wave. Don’t be afraid to catch bigger waves. Accept what cannot be changed. And always have fun.”

As “someone who has failed many times,” she urges people to “try, try, and try again.” Her biggest lesson? “Don’t waste energy fighting the things that cannot be changed. Instead, adapt to the situation and learn to ride with it.”

It worked for her. Today, Chen’s company is flourishing and she has turned an idea into a product that has helped save thousands of lives. To date, the Embrace has helped over 200,000 children in 15 countries. She hopes to grow to the point where the Embrace will save 1 million babies globally, and recently launched Little Lotus, a line of baby swaddles, sleeping bags and blankets for the US market with a temperature control function to help babies sleep better, and a 1:1 model: every purchase helps to save a baby in a developing country with the Embrace warmer.

Jane Chen will be attending this year’s TEDWomen conference, Oct. 26–28, 2016 in San Francisco. Tickets are now available, so register to attend today at the TEDWomen website. Follow Chen on Twitter at @janemariechen

Cross-posted from TEDWomen host Pat Mitchell’s blog.


Planet DebianFrancois Marier: Remplacer un disque RAID défectueux

Traduction de l'article original anglais à https://feeding.cloud.geek.nz/posts/replacing-a-failed-raid-drive/.

Voici la procédure que j'ai suivi pour remplacer un disque RAID défectueux sur une machine Debian.

Remplacer le disque

Après avoir remarqué que /dev/sdb a été expulsé de mon RAID, j'ai utilisé smartmontools pour identifier le numéro de série du disque à retirer :

smartctl -a /dev/sdb

Cette information en main, j'ai fermé l'ordinateur, retiré le disque défectueux et mis un nouveau disque vide à la place.

Initialiser le nouveau disque

Après avoir démarré avec le nouveau disque vide, j'ai copié la table de partitions avec parted.

Premièrement, j'ai examiné la table de partitions sur le disque dur non-défectueux :

$ parted /dev/sda
unit s
print

et créé une nouvelle table de partitions sur le disque de remplacement :

$ parted /dev/sdb
unit s
mktable gpt

Ensuite j'ai utilisé la commande mkpart pour mes 4 partitions et je leur ai toutes donné la même taille que les partitions équivalentes sur /dev/sda.

Finalement, j'ai utilisé les commandes toggle 1 bios_grub (partition d'amorce) et toggle X raid (où X est le numéro de la partition) pour toutes les partitions RAID, avant de vérifier avec la commande print que les deux tables de partitions sont maintenant identiques.

Resynchroniser/recréer les RAID

Pour synchroniser les données du bon disque (/dev/sda) vers celui de remplacement (/dev/sdb), j'ai exécuté les commandes suivantes sur mes partitions RAID1 :

mdadm /dev/md0 -a /dev/sdb2
mdadm /dev/md2 -a /dev/sdb4

et j'ai gardé un oeil sur le statut de la synchronisation avec :

watch -n 2 cat /proc/mdstat

Pour accélérer le processus, j'ai utilisé le truc suivant :

blockdev --setra 65536 "/dev/md0"
blockdev --setra 65536 "/dev/md2"
echo 300000 > /proc/sys/dev/raid/speed_limit_min
echo 1000000 > /proc/sys/dev/raid/speed_limit_max

Ensuite, j'ai recréé ma partition swap RAID0 comme suit :

mdadm /dev/md1 --create --level=0 --raid-devices=2 /dev/sda3 /dev/sdb3
mkswap /dev/md1

Par que la partition swap est toute neuve (il n'est pas possible de restorer une partition RAID0, il faut la re-créer complètement), j'ai dû faire deux choses:

  • remplacer le UUID pour swap dans /etc/fstab, avec le UUID donné par la commande mkswap (ou bien en utilisant la command blkid et en prenant le UUID pour /dev/md1)
  • remplacer le UUID de /dev/md1 dans /etc/mdadm/mdadm.conf avec celui retourné pour /dev/md1 par la commande mdadm --detail --scan

S'assurer que l'on peut démarrer avec le disque de remplacement

Pour être certain de bien pouvoir démarrer la machine avec n'importe quel des deux disques, j'ai réinstallé le boot loader grub sur le nouveau disque :

grub-install /dev/sdb

avant de redémarrer avec les deux disques connectés. Ceci confirme que ma configuration fonctionne bien.

Ensuite, j'ai démarré sans le disque /dev/sda pour m'assurer que tout fonctionnerait bien si ce disque décidait de mourir et de me laisser seulement avec le nouveau (/dev/sdb).

Ce test brise évidemment la synchronisation entre les deux disques, donc j'ai dû redémarrer avec les deux disques connectés et puis ré-ajouter /dev/sda à tous les RAID1 :

mdadm /dev/md0 -a /dev/sda2
mdadm /dev/md2 -a /dev/sda4

Une fois le tout fini, j'ai redémarrer à nouveau avec les deux disques pour confirmer que tout fonctionne bien :

cat /proc/mdstat

et j'ai ensuite exécuter un test SMART complet sur le nouveau disque :

smartctl -t long /dev/sdb

Planet DebianFrancois Marier: Remplacer un disque RAID défectueux

Traduction de l'article original anglais à https://feeding.cloud.geek.nz/posts/replacing-a-failed-raid-drive/.

Voici la procédure que j'ai suivi pour remplacer un disque RAID défectueux sur une machine Debian.

Remplacer le disque

Après avoir remarqué que /dev/sdb a été expulsé de mon RAID, j'ai utilisé smartmontools pour identifier le numéro de série du disque à retirer :

smartctl -a /dev/sdb

Cette information en main, j'ai fermé l'ordinateur, retiré le disque défectueux et mis un nouveau disque vide à la place.

Initialiser le nouveau disque

Après avoir démarré avec le nouveau disque vide, j'ai copié la table de partitions avec parted.

Premièrement, j'ai examiné la table de partitions sur le disque dur non-défectueux :

$ parted /dev/sda
unit s
print

et créé une nouvelle table de partitions sur le disque de remplacement :

$ parted /dev/sdb
unit s
mktable gpt

Ensuite j'ai utilisé la commande mkpart pour mes 4 partitions et je leur ai toutes donné la même taille que les partitions équivalentes sur /dev/sda.

Finalement, j'ai utilisé les commandes toggle 1 bios_grub (partition d'amorce) et toggle X raid (où X est le numéro de la partition) pour toutes les partitions RAID, avant de vérifier avec la commande print que les deux tables de partitions sont maintenant identiques.

Resynchroniser/recréer les RAID

Pour synchroniser les données du bon disque (/dev/sda) vers celui de remplacement (/dev/sdb), j'ai exécuté les commandes suivantes sur mes partitions RAID1 :

mdadm /dev/md0 -a /dev/sdb2
mdadm /dev/md2 -a /dev/sdb4

et j'ai gardé un oeil sur le statut de la synchronisation avec :

watch -n 2 cat /proc/mdstat

Pour accélérer le processus, j'ai utilisé le truc suivant :

blockdev --setra 65536 "/dev/md0"
blockdev --setra 65536 "/dev/md2"
echo 300000 > /proc/sys/dev/raid/speed_limit_min
echo 1000000 > /proc/sys/dev/raid/speed_limit_max

Ensuite, j'ai recréé ma partition swap RAID0 comme suit :

mdadm /dev/md1 --create --level=0 --raid-devices=2 /dev/sda3 /dev/sdb3
mkswap /dev/md1

Par que la partition swap est toute neuve (il n'est pas possible de restorer une partition RAID0, il faut la re-créer complètement), j'ai dû faire deux choses:

  • remplacer le UUID pour swap dans /etc/fstab, avec le UUID donné par la commande mkswap (ou bien en utilisant la command blkid et en prenant le UUID pour /dev/md1)
  • remplacer le UUID de /dev/md1 dans /etc/mdadm/mdadm.conf avec celui retourné pour /dev/md1 par la commande mdadm --detail --scan

S'assurer que l'on peut démarrer avec le disque de remplacement

Pour être certain de bien pouvoir démarrer la machine avec n'importe quel des deux disques, j'ai réinstallé le boot loader grub sur le nouveau disque :

grub-install /dev/sdb

avant de redémarrer avec les deux disques connectés. Ceci confirme que ma configuration fonctionne bien.

Ensuite, j'ai démarré sans le disque /dev/sda pour m'assurer que tout fonctionnerait bien si ce disque décidait de mourir et de me laisser seulement avec le nouveau (/dev/sdb).

Ce test brise évidemment la synchronisation entre les deux disques, donc j'ai dû redémarrer avec les deux disques connectés et puis ré-ajouter /dev/sda à tous les RAID1 :

mdadm /dev/md0 -a /dev/sda2
mdadm /dev/md2 -a /dev/sda4

Une fois le tout fini, j'ai redémarrer à nouveau avec les deux disques pour confirmer que tout fonctionne bien :

cat /proc/mdstat

et j'ai ensuite exécuter un test SMART complet sur le nouveau disque :

smartctl -t long /dev/sdb

CryptogramMajor NSA/Equation Group Leak

The NSA was badly hacked in 2013, and we're just now learning about it.

A group of hackers called "The Shadow Brokers" claim to have hacked the NSA, and are posting data to prove it. The data is source code from "The Equation Group," which is a sophisticated piece of malware exposed last year and attributed to the NSA. Some details:

The Shadow Brokers claimed to have hacked the Equation Group and stolen some of its hacking tools. They publicized the dump on Saturday, tweeting a link to the manifesto to a series of media companies.

The dumped files mostly contain installation scripts, configurations for command and control servers, and exploits targeted to specific routers and firewalls. The names of some of the tools correspond with names used in Snowden documents, such as "BANANAGLEE" or "EPICBANANA."

Nicholas Weaver has analyzed the data and believes it real:

But the proof itself, appear to be very real. The proof file is 134 MB of data compressed, expanding out to a 301 MB archive. This archive appears to contain a large fraction of the NSA's implant framework for firewalls, including what appears to be several versions of different implants, server side utility scripts, and eight apparent exploits for a variety of targets.

The exploits themselves appear to target Fortinet, Cisco, Shaanxi Networkcloud Information Technology (sxnc.com.cn) Firewalls, and similar network security systems. I will leave it to others to analyze the reliability, versions supported, and other details. But nothing I've found in either the exploits or elsewhere is newer than 2013.

Because of the sheer volume and quality, it is overwhelmingly likely this data is authentic. And it does not appear to be information taken from comprised systems. Instead the exploits, binaries with help strings, server configuration scripts, 5 separate versions of one implant framework, and all sort of other features indicate that this is analyst-side code­ -- the kind that probably never leaves the NSA.

I agree with him. This just isn't something that can be faked in this way. (Good proof would be for The Intercept to run the code names in the new leak against their database, and confirm that some of the previously unpublished ones are legitimate.)

This is definitely not Snowden stuff. This isn't the sort of data he took, and the release mechanism is not one that any of the reporters with access to the material would use. This is someone else, probably an outsider...probably a government.

Weaver again:

But the big picture is a far scarier one. Somebody managed to steal 301 MB of data from a TS//SCI system at some point between 2013 and today. Possibly, even probably, it occurred in 2013. But the theft also could have occurred yesterday with a simple utility run to scrub all newer documents. Relying on the file timestamps­ -- which are easy to modify­ -- the most likely date of acquisition was June 11, 2013. That is two weeks after Snowden fled to Hong Kong and six days after the first Guardian publication. That would make sense, since in the immediate response to the leaks as the NSA furiously ran down possibly sources, it may have accidentally or deliberately eliminated this adversary's access.

Okay, so let's think about the game theory here. Some group stole all of this data in 2013 and kept it secret for three years. Now they want the world to know it was stolen. Which governments might behave this way? The obvious list is short: China and Russia. Were I betting, I would bet Russia, and that it's a signal to the Obama Administration: "Before you even think of sanctioning us for the DNC hack, know where we've been and what we can do to you."

They claim to be auctioning off the rest of the data to the highest bidder. I think that's PR nonsense. More likely, that second file is random nonsense, and this is all we're going to get. It's a lot, though. Yesterday was a very bad day for the NSA.

EDITED TO ADD: Snowden's comments. He thinks it's an "NSA malware staging server" that was hacked.

EDITED TO ADD (8/18): Dave Aitel also thinks it's Russia.

EDITED TO ADD (8/19): Two news articles.

Cisco has analyzed the vulnerabilities for their products found in the data. They found several that they patched years ago, and one new one they didn't know about yet. See also this about the vulnerabilities.

EDITED TO ADD (8/20): More about the vulnerabilities found in the data.

Previously unreleased material from the Snowden archive proves that this data dump is real, and that the Equation Group is the NSA.

Planet DebianJose M. Calhariz: Availabilty of at at the Major Linux Distributions

In this blog post I will cover what versions of software at is used by the leading Linux Distributions as reported by LWN.

Also

Currently some distributions are lagging on the use of the latest at software.

Cory DoctorowPodcast: Live from HOPE on Radio Statler

While I was in NYC to keynote the 11th Hackers on Planet Earth convention, I sat down with the Radio Statler folks and explained what I was going to talk about, as well as bantering with the hosts about the relative merits of DEFCON and HOPE and the secret to managing cons and marriages (MP3).

Planet DebianRussell Coker: Basics of Backups

I’ve recently had some discussions about backups with people who aren’t computer experts, so I decided to blog about this for the benefit of everyone. Note that this post will deliberately avoid issues that require great knowledge of computers. I have written other posts that will benefit experts.

Essential Requirements

Everything that matters must be stored in at least 3 places. Every storage device will die eventually. Every backup will die eventually. If you have 2 backups then you are covered for the primary storage failing and the first backup failing. Note that I’m not saying “only have 2 backups” (I have many more) but 2 is the bare minimum.

Backups must be in multiple places. One way of losing data is if your house burns down, if that happens all backup devices stored there will be destroyed. You must have backups off-site. A good option is to have backup devices stored by trusted people (friends and relatives are often good options).

It must not be possible for one event to wipe out all backups. Some people use “cloud” backups, there are many ways of doing this with Dropbox, Google Drive, etc. Some of these even have free options for small amounts of storage, for example Google Drive appears to have 15G of free storage which is more than enough for all your best photos and all your financial records. The downside to cloud backups is that a computer criminal who gets access to your PC can wipe it and the backups. Cloud backup can be a part of a sensible backup strategy but it can’t be relied on (also see the paragraph about having at least 2 backups).

Backup Devices

USB flash “sticks” are cheap and easy to use. The quality of some of those devices isn’t too good, but the low price and small size means that you can buy more of them. It would be quite easy to buy 10 USB sticks for multiple copies of data.

Stores that sell office-supplies sell USB attached hard drives which are quite affordable now. It’s easy to buy a couple of those for backup use.

The cheapest option for backing up moderate amounts of data is to get a USB-SATA device. This connects to the PC by USB and has a cradle to accept a SATA hard drive. That allows you to buy cheap SATA disks for backups and even use older disks as backups.

With choosing backup devices consider the environment that they will be stored in. If you want to store a backup in the glove box of your car (which could be good when travelling) then a SD card or USB flash device would be a good choice because they are resistant to physical damage. Note that if you have no other options for off-site storage then the glove box of your car will probably survive if your house burns down.

Multiple Backups

It’s not uncommon for data corruption or mistakes to be discovered some time after it happens. Also in recent times there is a variety of malware that encrypts files and then demands a ransom payment for the decryption key.

To address these problems you should have older backups stored. It’s not uncommon in a corporate environment to have backups every day stored for a week, backups every week stored for a month, and monthly backups stored for some years.

For a home use scenario it’s more common to make backups every week or so and take backups to store off-site when it’s convenient.

Offsite Backups

One common form of off-site backup is to store backup devices at work. If you work in an office then you will probably have some space in a desk drawer for personal items. If you don’t work in an office but have a locker at work then that’s good for storage too, if there is high humidity then SD cards will survive better than hard drives. Make sure that you encrypt all data you store in such places or make sure that it’s not the secret data!

Banks have a variety of ways of storing items. Bank safe deposit boxes can be used for anything that fits and can fit hard drives. If you have a mortgage your bank might give you free storage of “papers” as part of the service (Commonwealth Bank of Australia used to offer that). A few USB sticks or SD cards in an envelope could fit the “papers” criteria. An accounting firm may also store documents for free for you.

If you put a backup on USB or SD storage in your waller then that can also be a good offsite backup. For most people losing data from disk is more common than losing their wallet.

A modern mobile phone can also be used for backing up data while travelling. For a few years I’ve been doing that. But note that you have to encrypt all data stored on a phone so an attacker who compromises your phone can’t steal it. In a typical phone configuration the mass storage area is much less protected than application data. Also note that customs and border control agents for some countries can compel you to provide the keys for encrypted data.

A friend suggested burying a backup device in a sealed plastic container filled with dessicant. That would survive your house burning down and in theory should work. I don’t know of anyone who’s tried it.

Testing

On occasion you should try to read the data from your backups and compare it to the original data. It sometimes happens that backups are discovered to be useless after years of operation.

Secret Data

Before starting a backup it’s worth considering which of the data is secret and which isn’t. Data that is secret needs to be treated differently and a mixture of secret and less secret data needs to be treated as if it’s all secret.

One category of secret data is financial data. If your accountant provides document storage then they can store that, generally your accountant will have all of your secret financial data anyway.

Passwords need to be kept secret but they are also very small. So making a written or printed copy of the passwords is part of a good backup strategy. There are options for backing up paper that don’t apply to data.

One category of data that is not secret is photos. Photos of holidays, friends, etc are generally not that secret and they can also comprise a large portion of the data volume that needs to be backed up. Apparently some people have a backup strategy for such photos that involves downloading from Facebook to restore, that will help with some problems but it’s not adequate overall. But any data that is on Facebook isn’t that secret and can be stored off-site without encryption.

Backup Corruption

With the amounts of data that are used nowadays the probability of data corruption is increasing. If you use any compression program with the data that is backed up (even data that can’t be compressed such as JPEGs) then errors will be detected when you extract the data. So if you have backup ZIP files on 2 hard drives and one of them gets corrupt you will easily be able to determine which one has the correct data.

Failing Systems – update 2016-08-22

When a system starts to fail it may limp along for years and work reasonably well, or it may totally fail soon. At the first sign of trouble you should immediately make a full backup to separate media. Use different media to your regular backups in case the data is corrupt so you don’t overwrite good backups with bad ones.

One traditional sign of problems has been hard drives that make unusual sounds. Modern drives are fairly quiet so this might not be loud enough to notice. Another sign is hard drives that take unusually large amounts of time to read data. If a drive has some problems it might read a sector hundreds or even thousands of times until it gets the data which dramatically reduces system performance. There are lots of other performance problems that can occur (system overheating, software misconfiguration, and others), most of which are correlated with potential data loss.

A modern SSD storage device (as used in a lot of the recent laptops) doesn’t tend to go slow when it nears the end of it’s life. It is more likely to just randomly fail entirely and then work again after a reboot. There are many causes of systems randomly hanging or crashing (of which overheating is common), but they are all correlated with data loss so a good backup is a good idea.

When in doubt make a backup.

Any Suggestions?

If you have any other ideas for backups by typical home users then please leave a comment. Don’t comment on expert issues though, I have other posts for that.

,

Planet DebianJoey Hess: keysafe alpha release

Keysafe securely backs up a gpg secret key or other short secret to the cloud. But not yet. Today's alpha release only supports storing the data locally, and I still need to finish tuning the argon2 hash difficulties with modern hardware. Other than that, I'm fairly happy with how it's turned out.

Keysafe is written in Haskell, and many of the data types in it keep track of the estimated CPU time needed to create, decrypt, and brute-force them. Running that through a AWS SPOT pricing cost model lets keysafe estimate how much an attacker would need to spend to crack your password.

4.png
(Above is for the password "makesad spindle stick")

If you'd like to be an early adopter, install it like this:

sudo apt-get install haskell-stack libreadline-dev libargon2-0-dev zenity
stack install keysafe

Run ~/.local/bin/keysafe --backup --store-local to back up a gpg key to ~/.keysafe/objects/local/

I still need to tune the argon2 hash difficulty, and I need benchmark data to do so. If you have a top of the line laptop or server class machine that's less than a year old, send me a benchmark:

~/.local/bin/keysafe --benchmark | mail keysafe@joeyh.name -s benchmark

Bonus announcement: http://hackage.haskell.org/package/zxcvbn-c/ is my quick Haskell interface to the C version of the zxcvbn password strength estimation library.

PS: Past 50% of my goal on Patreon!

Planet DebianDirk Eddelbuettel: RQuantLib 0.4.3: Lots of new Fixed Income functions

A release of RQuantLib is now on CRAN and in Debian. It contains a lot of new code contributed by Terry Leitch over a number of pull requests. See below for full details but the changes focus on Fixed Income and Fixed Income Derivatives, and cover swap, discount curves, swaptions and more.

In the blog post for the previous release 0.4.2, we noted that a volunteer was needed for a new Windows library build of QuantLib for Windows to replace the outdated version 1.6 used there. Josh Ulrich stepped up, and built them. Josh and I tried for several month to get the win-builder to install these, but sadly other things took priority and we were unsuccessful. So this release will not have Windows binaries on CRAN as QuantLib 1.8 is not available there. Instead, you can use the ghrr drat and do

if (!require("drat")) install.packages("drat")
drat::addRepo("ghrr")
install.packages("RQuantLib")

to fetch prebuilt Windows binaries from the ghrr drat. Everybody else gets sources from CRAN.

The full changes are detailed below.

Changes in RQuantLib version 0.4.3 (2016-08-19)

  • Changes in RQuantLib code:

    • Discount curve creation has been made more general by allowing additional arguments for day counter and fixed and floating frequency (contributed by Terry Leitch in #31, plus some work by Dirk in #32).

    • Swap leg parameters are now in combined variable and allow textual description (Terry Leitch in #34 and #35)

    • BermudanSwaption has been modfied to take option expiration and swap tenors in order to enable more general swaption structure pricing; a more general search for the swaptions was developed to accomodate this. Also, a DiscountCurve is allowed as an alternative to market quotes to reduce computation time for a portfolio on a given valuation date (Terry Leitch in #42 closing issue #41).

    • A new AffineSwaption model was added with similar interface to BermudanSwaption but allowing for valuation of a European exercise swaption utlizing the same affine methods available in BermudanSwaption. AffineSwaption will also value a Bermudan swaption, but does not take rate market quotes to build a term structure and a DiscountCurve object is required (Terry Leitch in #43).

    • Swap tenors can now be defined up to 100 years (Terry Leitch in #48 fising issue #46).

    • Additional (shorter term) swap tenors are now defined (Guillaume Horel in #49, #54, #55).

    • New SABR swaption pricer (Terry Leitch in #60 and #64, small follow-up by Dirk in #65).

    • Use of Travis CI has been updated and switch to maintained fork of deprecated mainline.

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the rquantlib-devel mailing list off the R-Forge page. Issue tickets can be filed at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

CryptogramFriday Squid Blogging: Stubby Squid

Photo of the cutest squid ever.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

TEDForecasting crime in Rio de Janeiro, a new Marvel comic, and an Airbnb to rejuvenate a rural community

Robert_Muggah_CTA

The TED community has been very busy over the past few weeks. Below, some newsy highlights.

Crime forecasting in Rio. Before the 2016 Olympic Games, worries ran high that crime in Rio might affect the mega-event; one reported attack at the Games (which actually might not have happened) grabbed headlines around the world during the Games. But the longer-running news story is the way crime affects Rio’s locals every single day. How can residents stay safe? Together with Via Science and Mosaico Internet, Robert Muggah’s Igarapé Institute just launched CrimeRadar, a publicly available crime-prediction platform. CrimeRadar uses advanced machine learning to forecast future crime risk and track historical crime tends. The launch is focused on Rio de Janeiro, with plans to take the platform global. (Watch Robert’s TED Talk)

future_crimeradar_overview

CrimeRadar, developed by Robert Muggah’s Igarapé Institute along with Via Science and Mosaico Internet, uses machine learning to forecast crime in Rio de Janeiro. The software runs on both mobile phones and desktops. Above, an example of the desktop version. Photo: courtesy of Robert Muggah

World of microbes. We’ve all heard some of the implications that microbes have for our health –from pandemic-level bad to the life-changing magic they perform in our guts– but Ed Yong is determined to show us how they influence everything in the world around us. Released August 9, his debut book I Contain Multitudes takes a “microbe’s-eye view of the world” to reveal their role in everything from deep oceans to forests, squid to worms. (Watch Ed’s TED Talk)

Breaking the silence. “We have in this country this dynamic where we really don’t like to talk about our problems. We don’t like to talk about our history. And because of that, we really haven’t understood what it’s meant to do the things we’ve done historically,” Bryan Stevenson said at TED2012. A desire to change that dynamic is behind his passionate and tireless work to create the first national memorial to victims of lynching. Designed by fellow TED speaker Michael Murphy of MASS Design Group, the memorial was officially announced on August 16. The memorial will be accompanied by a museum at Equal Justice Initiative’s headquarters in Montgomery, Alabama, and both plan to open in 2017. (Watch Bryan’s TED Talk)

A global warning. Close to 3.3 billion people tuned in to watch the Opening Ceremony of the 2016 Olympic Games, but along with the usual celebration and dazzle, viewers were warned about the dangers of climate change. Many performances in the multi-hour spectacle highlighted the crucial role forests have in absorbing greenhouse gases — along with a video describing how rising CO2 levels lead to climate change.  TED speaker, forester and sustainability activist Tasso Azevedo served as a consultant during development of the film, joining the elite club of TED speakers who’ve also appeared in Olympics opening ceremonies. (Watch Tasso’s TED Talk)

VR tech for paraplegics. Miguel Nicolelis is one of twenty scientists who published a paper in Scientific Reports detailing a new brain training approach that can induce partial neurological recovery in paraplegic patients. The sample size is small, eight patients, but all of them report being able to use their legs and feel sensation after sessions using an artificial exoskeleton, VR technology, and a brain-machine interface. Originally hoping to use the technique to help the patients regain a sense of control in their lives, the researchers stumbled upon its potential as a recovery tool. (Watch Miguel’s TED Talk)

Design for shared spaces. On August 2, Joe Gebbia announced the official launch of Samara, Airbnb’s own internal design studio, but the startup’s newest branch had already been hard at work designing a prototype home for the Japanese exhibition House Vision. The result, Yoshino Cedar House, houses a community center on its ground floor and accommodations beneath a gabled roof, exploring “how architectural features can engender a deeper relationship between hosts and guests.” But the idea doesn’t end there. Once the exhibition is over, the house will be moved to the rural town of Yoshino and become a bookable Airbnb rental. It will be maintained by the Yoshino community and proceeds will be used to benefit the area, which has been struggling since younger residents moved away. If Yoshino Cedar House is successful, the model may be used to rejuvenate rural communities elsewhere. However, Samara won’t just be involved in architecture; the design studio will work on service design and software engineering projects as well. (Watch Joe Gebbia’s TED Talk)

Women in the World of Wakanda. TED speaker and writer Roxane Gay and poet Yona Harvey, both first time comic writers, will pen a spinoff of Ta-Nehisi Coates’ popular Marvel Comic series Black Panther. The comic will also be set in the fictional African country of Wakanda and will follow Ayo and Aneka, two lovers and former members of the Dora Milaje, the Black Panther’s female security force. In an industry historically dominated by white male voices and characters, “the opportunity to write black women and queer black women into the Marvel universe, there’s no saying no to that,” Gay told The New York Times. (Watch Roxane’s TED Talk)

Advance prep. Jennifer Granholm, the former two-term Governor of Michigan, has been appointed to Hillary Clinton’s White House transition team. Both candidates are allotted offices in Washington and other resources to prepare for their potential administrations. (Watch Jennifer’s TED Talk)

Have a news item to share? Write us at contact@ted.com and you may see it included in this weekly round-up.


CryptogramUnintentional DOS Attack against Car-Door Openers

Radio noise from a nearby neon-sign transformer made it impossible for people to unlock their cars remotely.

Planet Linux AustraliaColin Charles: Speaking in August 2016

I know this is a tad late, but there have been some changes, etc. recently, so apologies for the delay of this post. I still hope to meet many of you to chat about MySQL/Percona Server/MariaDB Server, MongoDB, open source databases, and open source in general in the remainder of August 2016.

  • LinuxCon+ContainerCon North America – August 22-24 2016 – Westin Harbour Castle, Toronto, Canada – I’ll be speaking about lessons one can learn from database failures and enjoying the spectacle that is the 25th anniversary of Linux!
  • Chicago MySQL Meetup Group – August 29 2016 – Vivid Seats, Chicago, IL – more lessons from database failures here, and I’m looking forward to meeting users, etc. in the Chicago area

While not speaking, Vadim Tkachenko and I will be present at the @scale conference. I really enjoyed my time there previously, and if you get an invite, its truly a great place to learn and network.

Planet DebianSimon Désaulniers: [GSOC] Final report




The Google Summer of Code is now over. It has been a great experience and I’m very glad I’ve been able to make it. I’ve had the pleasure to contribute to a project showing very good promise for the future of communication: Ring. The words “privacy” and “freedom” in terms of technologies are being more and more present in the mind of people. All sorts of projects wanting to achieve these goals are coming to life each days like decentralized web networks (ZeroNet for e.g.), blockchain based applications, etc.

Debian

I’ve had the great opportunity to go to the Debian Conference 2016. I’ve been introduced to the debian community and debian developpers (“dd” in short :p). I was lucky to meet with great people like the president of the FSF, John Sullivan. You can have a look at my Debian conference report here.

If you want to read my debian reports, you can do so by browsing the “Google Summer Of Code” category on this blog.

What I have done

Ring is now in official debian repositories since June 30th. This is a good news for the GNU/Linux community. I’m proud to say that I’ve been able to contribute to debian by working on OpenDHT and developing new functionalities to reduce network traffic. The goal behind this was to finally optimize the data persistence traffic consumption on the DHT.

Github repository: https://github.com/savoirfairelinux/opendht

Queries

Issues:

  • #43: DHT queries

Pull requests:

  • #79: [DHT] Queries: remote values filtering
  • 93: dht: return consistent query from local storage
  • #106: [dht] rework get timings after queries in master

Value pagination

Issues:

  • #71: [DHT] value pagination

Pull requests:

  • #110: dht: Value pagination using queries
  • #113: dht: value pagination fix

Indexation (feat. Nicolas Reynaud)

Pull requests:

  • #77: pht: fix invalid comparison, inexact match lookup
  • #78: [PHT] Key consistency

General maintenance of OpenDHT

Issues:

  • #72: Packaging issue for Python bindings with CMake: $DESTDIR not honored
  • #75: Different libraries built with Autotools and CMake
  • #87: OpenDHT does not build on armel
  • #92: [DhtScanner] doesn’t compile on LLVM 7.0.2
  • #99: 0.6.2 filenames in 0.6.3

Pull requests:

  • #73: dht: consider IPv4 or IPv6 disconnected on operation done
  • #74: [packaging] support python installation with make DESTDIR=$DIR
  • #84: [dhtnode] user experience
  • #94: dht: make main store a vector>
  • #94: autotools: versionning consistent with CMake
  • #103: dht: fix sendListen loop bug
  • #106: dht: more accurate name for requested nodes count
  • #108: dht: unify bootstrapSearch and refill method using node cache

View by commits

You can have a look at my work by commits just by clicking this link: https://github.com/savoirfairelinux/opendht/commits/master?author=sim590

What’s left to be done

Data persistence

The only thing left before achieving the totality of my work is to rigorously test the data persistence behavior to demonstrate the network traffic reduction. To do so we use our benchmark python module. We are able to analyse traffic and produce plots like this one:


Plot: 32 nodes, 1600 values with normal condition test.

This particular plot was drawn before the enhancements. We are confident to improve the results using my work produced during the GSOC.

TCP

In the middle of the GSOC, we soon realized that passing from UDP to TCP would ask too much efforts in too short lapse of time. Also, it is not yet clear if we should really do that.

Sociological ImagesEnglish Acquisition Among Immigrants to the U.S.

Flashback Friday.

Is it true that Spanish-speaking immigrants to the United States resist assimilation?

Not if you judge by language acquisition and compare them to earlier European immigrants. The sociologist Claude S. Fischer, at Made in America, offers this data:

The bottom line represents the percentage of English-speakers among the wave of immigrants counted in the 1900, 1910, and 1920 census. It shows that less than half of those who had been in the country five years or less could speak English. This jumped to almost 75% by the time they were here six to ten years and the numbers keep rising slowly after that.

Fast forward 80 years. Immigrants counted in the 1980, 1990, and 2000 Census (the top line) outpaced earlier immigrants by more than 25 percentage points. Among those who have just arrived, almost as many can speak English as earlier immigrants who’d been here between 11 and 15 years.

If you look just at Spanish speakers (the middle line), you’ll see that the numbers are slightly lower than all recent immigrants, but still significantly better than the previous wave. Remember that some of the other immigrants are coming from English-speaking countries.

Fischer suggests that the ethnic enclave is one of the reasons that the wave of immigrants at the turn of the 20th century learned English more slowly:

When we think back to that earlier wave of immigration, we picture neighborhoods like Little Italy, Greektown, the Lower East Side, and Little Warsaw – neighborhoods where as late as 1940, immigrants could lead their lives speaking only the language of the old country.

Today, however, immigrants learn to speak with those outside of their own group more quickly, suggesting that all of the flag waving to the contrary is missing the big picture.

Originally posted in 2010.

Lisa Wade, PhD is a professor at Occidental College. She is the author of American Hookup, a book about college sexual culture, and a textbook about gender. You can follow her on Twitter, Facebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

Planet DebianOlivier Grégoire: Conclusion Google Summer of Code 2016

SmartInfo project with Debian alt text

1. Me

Before getting into the thick of my project, let me present myself:
I am Olivier Grégoire (Gasuleg), and I study IT engineering at École de Technologie supérieure in Montreal.
I am a technician in electronics, and I began object-oriented programming just last year.
I applied to GSoC because I loved the concept of the project that I would work on and I really wanted to be part of it. I also wanted to discover the word of the free software.

2. My Project

During this GSoC, I worked on the Ring project.

“Ring is a free software for communication that allows its users to make audio or video calls, in pairs or groups, and to send messages, safely and freely, in confidence.

Savoir-faire Linux and a community of contributors worldwide develop Ring. It is available on GNU/Linux, Windows, Mac OSX and Android. It can be associated with a conventional phone service or integrated with any connected object.

Under this very easy to use software, there is a combination of technologies and innovations opening all kinds of perspectives to its users and developers.

Ring is a free software whose code is open. Therefore, it is not the software that controls you.

With Ring, you take control of your communication!

Ring is an open source software under the GPL v3 license. Everyone can verify the codes and propose new ones to improve the software’s performace. It is a guarantee of transparency and freedom for everyone!”
Source: ring.cx

The problem is about the typical user of Ring, the one who don’t use the terminal to launch Ring. He has no information about what has happened in the system. My goal is to create a tool that display statistics of Ring.

3. Quick Explanation of What My Program Can Do

The Code

Here are the links to the code I was working on all throughout the Google Summer of Code (You can see what I have done after the GSoC by clicking on the newest patchs):

Patch Status
Daemon Merged
Lib Ring Client (LRC) On Review
Gnome client On Review
Remove unused code   Merged


What Can Be Displayed?
This is the final list of information I can display and some ideas on what information we could display in the future:

Information     Details Done?
Call ID The identification number of the call Yes
Resolution Local and remote Yes
Framerate Local and remote Yes
Codec Audio and video in local and remote     Yes
Bandwidth Download and upload No
Performance use CPU, GPU, RAM No
Security level In SIP call No
Connection time   No
Packets lost   No


To launch it you need to right click on the call and click on “Show advanced information”.
alt text
To stop it, same thing: right click on the call and click on “Hide advanced information”.

4. More Details About My Project

My program needs to retrieve information from the daemon (LibRing) and then display it in gnome client. So, I needed to create a patch for the daemon, the D-Bus layer (in the daemon patch), LibRingClient and the GNU/Linux (Gnome) client.

This is what the architecture of the project looks like.
alt text
source: ring.cx

And this is how I implemented my project.
alt text

5. Future of the Project

  • Add background on the gnome client
  • Implement the API smartInfoHub in all the other clients
  • Gather more information, such as bandwidth, resource consumption, security level, connection time, number of packets lost and anything else that could be deemed interesting
  • Display information for every participant in a conference call. I began to implement it for the daemon on patch set 25 .

Weekly report link

Thanks

I would like to thank the following:
- The Google Summer of Code organisation, for this wonderful experience.
- Debian, for accepting my project proposal and letting me embark on this fantastic adventure.
- My mentor, Mr Guillaume Roguez, and all his team, for being there to help me.

Planet DebianOlivier Grégoire: Conclusion Google Summer of Code 2016

SmartInfo project with Debian alt text

1. Me

Before getting into the thick of my project, let me present myself:
I am Olivier Grégoire (Gasuleg), and I study IT engineering at École de Technologie supérieure in Montreal.
I am an technician in electronics, and I began object-oriented programming just last year.
I applied to GSoC because I loved the concept of the project that I would work on and I really wanted to be part of it. I also wanted to discover the word of the free software.

2. My Project

During this GSoC, I worked on the Ring project.

“Ring is a free software for communication that allows its users to make audio or video calls, in pairs or groups, and to send messages, safely and freely, in confidence.

Savoir-faire Linux and a community of contributors worldwide develop Ring. It is available on GNU/Linux, Windows, Mac OSX and Android. It can be associated with a conventional phone service or integrated with any connected object.

Under this very easy to use software, there is a combination of technologies and innovations opening all kinds of perspectives to its users and developers.

Ring is a free software whose code is open. Therefore, it is not the software that controls you.

With Ring, you take control of your communication!

Ring is an open source software under the GPL v3 license. Everyone can verify the codes and propose new ones to improve the software’s performace. It is a guarantee of transparency and freedom for everyone!”
Source: ring.cx

The problem is about the typical user of Ring, the one who don’t use the terminal to launch Ring. He has no information about what has happened in the system. My goal is to create a tool that display statistics of Ring.

3. Quick Explanation of What My Program Can Do

The Code

Here are the links to the code I was working on all throughout the Google Summer of Code (You can see what I have done after the GSoC by clicking on the newest patchs):

Patch Status
Daemon On Review
Lib Ring Client (LRC) On Review
Gnome client On review
Remove unused code   Merged

!!!!!CHANGE LINK TO PUT THE LATEST PATCHES BEFORE THE END OF GSOC!!!!!

What Can Be Displayed?
This is the final list of information I can display and some ideas on what information we could display in the future:

Information     Details Done?
Call ID The identification number of the call Yes
Resolution Local and remote Yes
Framerate Local and remote Yes
Codec Audio and video in local and remote     Yes
Bandwidth Download and upload No
Performance use CPU, GPU, RAM No
Security level In SIP call No
Connection time   No
Packets lost   No


To launch it you need to right click on the call and click on “Show advanced information”.
alt text
To stop it, same thing: right click on the call and click on “Hide advanced information”.

4. More Details About My Project

My program needs to retrieve information from the daemon (LibRing) and then display it in gnome client. So, I needed to create a patch for the daemon, the D-Bus layer (in the daemon patch), LibRingClient and the GNU/Linux (Gnome) client.

This is what the architecture of the project looks like.
alt text
source: ring.cx

And this is how I implemented my project.
alt text

5. Future of the Project

  • Add background on the gnome client
  • Implement the API smartInfoHub in all the other clients
  • Gather more information, such as bandwidth, resource consumption, security level, connection time, number of packets lost and anything else that could be deemed interesting
  • Display information for every participant in a conference call. I began to implement it for the daemon on patch set 25 .

6. Thanks

I would like to thank the following:
- The Google Summer of Code organisation, for this wonderful experience.
- Debian, for accepting my project proposal and letting me embark on this fantastic adventure.
- My mentor, Mr Guillaume Roguez, and all his team, for being there to help me.

Planet DebianNorbert Preining: Debian/TeX Live 2016.20160819-1

A new – and unplanned – release in quick succession. I have uploaded testing packages to experimental which incorporate tex4ht into the TeX Live packages, but somehow the tex4ht transitional updated slipped into sid, and made many packages uninstallable. Well, so after a bit more testing let’s ship the beast to sid, meaning that tex4ht will finally updated from the last 2009 version to what is the current status in TeX Live.

texlive2016-debian

From the list of new packages I want to pick out the group of phf* packages that seem from a quick reading over the package documentations as very interesting.

But most important is the incorporation of tex4ht into the TeX Live packages, so please report bugs and shortcomings to the BTS. Thanks.

New packages

aurl, bxjalipsum, cormorantgaramond, notespages, phffullpagefigure, phfnote, phfparen, phfqit, phfquotetext, phfsvnwatermark, phfthm, table-fct, tocdata.

Updated packages

acmart, acro, biblatex-abnt, biblatex-publist, bxdpx-beamer, bxjscls, bxnewfont, bxpdfver, dccpaper, etex-pkg, europasscv, exsheets, glossaries-extra, graphics-def, graphics-pln, guitarchordschemes, ijsra, kpathsea, latexpand, latex-veryshortguide, ledmac, libertinust1math, markdown, mcf2graph, menukeys, mfirstuc, mhchem, mweights, newpx, newtx, optidef, paralist, parnotes, pdflatexpicscale, pgfplots, philosophersimprint, pstricks-add, showexpl, tasks, tetex, tex4ht, texlive-docindex, udesoftec, xcolor-solarized.

CryptogramMore on Election Security

Andrew Appel has a good two-part essay on securing elections.

And three organizations -- Verified Voting, EPIC, and Common Cause -- have published a report on the risks of Internet voting. The report is primarily concerned with privacy, and the threats to a secret ballot.

Worse Than FailureError'd: A Birthday You'll Never Forget

"Look like Microsoft really, really wants me to celebrate Windows 10's birthday," wrote Andrew.

 

Brandon R. writes, "Think you'll need to cancel? No problem! Just make sure you booked 999 days in advance."

 

"I imagine someone in an office far away saying 'We can't waste precious computer cycles on a string equality check! Utter prodigality!! We'll just ask the user to confirm'," writes Al.

 

"Here's a poliwag that I caught in Pokemon Go. Or, what's left of him," writes Mark B.

 

"I guess it makes sense that Microsoft doesn't really 'do' CSS, but at least they know someplace that does," Jos wrote.

 

"I wasn't sure if $50 from $79.95 was 62% off," wrote Ben R., "But I was fairly sure that 50$ from 89.95$ wasn't 77% off at the same time."

 

"So, if I don't have an SSH key, click on 'No, you must load an SSH key'?" write Shahim M.

 

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!

Planet DebianGuido Günther: Foreman's Ansible integration

Gathering from some recent discussions it seems to be not that well known that Foreman (a lifecycle tool for your virtual machines) does not only integrate well with Puppet but also with ansible. This is a list of tools I find useful in this regard:

  • The ansible-module-foreman ansible module allows you to setup all kinds of resources like images, compute resources, hostgroups, subnets, domains within Foreman itself via ansible using Foreman's REST API. E.g. creating a hostgroup looks like:

    - foreman_hostgroup:
        name: AHostGroup
        architecture: x86_64
        domain: a.domain.example.com
        foreman_host: "{{ foreman_host }}"
        foreman_user: "{{ foreman_user }}"
        foreman_pass: "{{ foreman_pw }}"
    
  • The foreman_ansible plugin for Foreman allows you to collect reports and facts from ansible provisioned hosts. This requires an additional hook in your ansible config like:

    [defaults]
    callback_plugins = path/to/foreman_ansible/extras/
    

    The hook will report to Foreman back after a playbook finished.

  • There are several options for creating hosts in Foreman via the ansible API. I'm currently using ansible_foreman_module tailored for image based installs. This looks in a playbook like:

    - name: Build 10 hosts
      foremanhost:
        name: "{{ item }}"
        hostgroup: "a/host/group"
        compute_resource: "hopefully_not_esx"
        subnet: "webservernet"
        environment: "{{ env|default(omit) }}"
        ipv4addr: {{ from_ipam|default(omit) }}"
        # Additional params to tag on the host
        params:
            app: varnish
            tier: web
            color: green
        api_user: "{{ foreman_user }}"
        api_password: "{{ foreman_pw }}"
        api_url: "{{ foreman_url }}"
      with_sequence:  start=1 end=10 format="newhost%02d"
    
  • The foreman_ansible_inventory is a dynamic inventory script for ansible that fetches all your hosts and groups via the Foreman REST APIs. It automatically groups hosts in ansible from Foreman's hostgroups, environments, organizations and locations and allows you to build additional groups based on any available host parameter (and combinations thereof). So using the above example and this configuration:

    [ansible]
    group_patterns = ["{app}-{tier}",
                      "{color}"]
    

    it would build the additional ansible groups varnish-web, green and put the above hosts into them. This way you can easily select the hosts for e.g. blue green deployments. You don't have to pass the parameters during host creation, if you have parameters on e.g. domains or hostgroups these are available too for grouping via group_patterns.

  • If you're grouping your hosts via the above inventory script and you use lots of parameters than having these displayed in the detail page can be useful. You can use the foreman_params_tab plugin for that.

There's also support for triggering ansible runs from within Foreman itself but I've not used that so far.

Planet DebianMichal Čihař: Wammu 0.42

Yesterday, I've released Wammu 0.42. There are no major updates, more likely it's usual localization and minor bugfixes release.

As usual up to date packages are now available in Debian sid, Gammu PPA for Ubuntu or openSUSE buildservice for various RPM based distros.

Want to support further Wammu development? Check our donation options or support Gammu team on BountySource Salt.

Filed under: Debian English Gammu | 0 comments

Planet DebianEriberto Mota: Debian: GnuPG 2, chroot and debsign

Since GPG 2 was set as default for Debian (Sid, August 2016), an error message appeared inside jails triggered by chroot, when using debuild/debsign commands:

clearsign failed: Inappropriate ioctl for device

The problem is that GPG 2 uses a dialog window to ask for a passphrase. This dialog window needs a tty (from /dev/pts/ directory). To solve the problem, you can use the following command (inside the jail):

# mount devpts -t devpts /dev/pts

Alternatively, you can add to /etc/fstab file in jail:

devpts /dev/pts devpts defaults 0 0

and use the command:

# mount /dev/pts

Enjoy!

Planet DebianZlatan Todorić: Defcon24

I went to Defcon24 as Purism representative. It was (as usual) held in Las Vegas, the city of sin. In the same module as with DebConf, here we go with good, bad and ugly.

Good

Badges are really cool. You can find good hackers here and there (but very small number compared to total number). Some talks are good and workshop + village idea looks good (although I didn't manage to attend any workshop as there was place for 1100 and there were 22000 attendees). The movie night idea is cool and Arcade space (where you can play old arcade games, relax and hack and also listen to some cool music) is really lovely. Also you have a camp/village for kids learning things such as electronics, soldering etc but you need to pay attention that they don't see too much of twisted folks that also gather on this con. And that's it. Oh, yea, Dark Tangent appears actually to be cool dude.

Bad

One does not simply hold a so-called hacker conference in Las Vegas. Having a conference inside hotel/casino where you mix with gamblers and casino workes (for good or for bad) is simply not in hacker spirit and certainly brings all kind of people to the same place. Also, there were simply not enough space for 22000 Defcon attendees, and you don't get proud of having on average ONLY 40min lines. You get proud if you don't have lines! Organization is not the strongest part of Defcon.

Huge majority of attendees are not hackers. They are script kiddies, hacker wannabes, comic con people, few totally lost souls etc etc. That simply brings the quality of a conference down. Yes it is cool to have mix of many diverse people but not for the sake of just having people.

Ugly

They lack Code of Conduct (everyone knows I am not in favor of any writens rules how people should behave but after Defcon I clearly see need for it). Actually, tbh, they do have it but no one gives a damn about it. And you should report to Goons, more about them below. Sexism is huge here. I remember and hear about stories of sexual harassment in IT industry, but Debian somehow mitigated that before me entering its domains, so I never experienced it. The sheer number of sexist behavior on Defcon is tremendous. It appears to me that those people had lonely childhood and now they act as a spoiled 6 year old: they're spoiled, they need to yell to show their point, they have low and stupid sexist jokes and they simply think that is cool.

Majority of Goons (their coordinators or whatever) are simply idiots. I don't know do they feel they have some superpowers, or are drunk or just stupid but yelling on people, throwing low jokes on people, more yelling, cursing all the time, more yelling - simply doesn't work for me. So now you can see the irony of CoC on Defcon. They even like to say, hey we are old farts, let us our con be as we want it to be. So no real diversity there. Either it is their way, and god forsaken if you try to change something for better and make them stop cursing or throwing sexist jokes ("squeeze, people. together, touch each other, trust me it will feel good"), or highway.

Also it appears that to huge number of vocal people, word "fuck" has some fetish meaning. Either it needs to show how "fucking awesome this con or they are" or to "fucking tell few things about random fucking stuff". Thank you, but no thank you.

So what did I do during con. I attended few talks, had some discussion with people, went to one party (great DJs, again people doing stupid things, like breaking invertory to name just one of them) and had so much time (read "I was bored") that I bought domain, brough up server on which I configured nginx and cp'ed this blog to blog.zlatan.tech (yes, recently I added letsencrypt because it is, let me be in Defcon mood, FUCKING AWESOME GRRR UGH) and now I even made .onion domain for it. What can boredom do to people, right?

So the ultimate question is - would I go again to Defcon. I am strongly leaning to no, but in my nature is to give second chance and now I have more experience (and I also have thick skin so I guess I can play calm for one more round).

,

Krebs on SecurityMalware Infected All Eddie Bauer Stores in U.S., Canada

Clothing store chain Eddie Bauer said today it has detected and removed malicious software from point-of-sale systems at all of its 350+ stores in North America, and that credit and debit cards used at those stores during the first six months of 2016 may have been compromised in the breach. The acknowledgement comes nearly six weeks after KrebsOnSecurity first notified the clothier about a possible intrusion at stores nationwide.

ebstoreOn July 5, 2016, KrebsOnSecurity reached out to Bellevue, Wash., based Eddie Bauer after hearing from several sources who work in fighting fraud at U.S. financial institutions. All of those sources said they’d identified a pattern of fraud on customer cards that had just one thing in common: They were all recently used at some of Eddie Bauer’s 350+ locations in the U.S. The sources said the fraud appeared to stretch back to at least January 2016.

A spokesperson for Eddie Bauer at the time said the company was grateful for the outreach but that it hadn’t heard any fraud complaints from banks or from the credit card associations.

Earlier today, however, an outside public relations firm circled back on behalf of Eddie Bauer. That person told me Eddie Bauer — working with the FBI and an outside computer forensics firm — had detected and removed card-stealing malware from cash registers at all of its locations in the United States and Canada.

The retailer says it believes the malware was capable of capturing credit and debit card numbers from customer transactions made at all 350 Eddie Bauer stores in the United States and Canada between January 2, 2016 to July 17, 2016. The company emphasized that this breach did not impact purchases made at the company’s online store eddiebauer.com.

“While not all transactions during this period were affected, out of an abundance of caution, Eddie Bauer is offering identity protection services to all customers who made purchases or returns during this period,” the company said in a press release issued directly after the markets closed in the U.S. today.

Given the volume of point-0f-sale malware attacks on retailers and hospitality firms in recent months, it would be nice if each one of these breach disclosures didn’t look and sound exactly the same. For example, in addition to offering customers the predictable and irrelevant credit monitoring services topped with bland assurances that the “security of our customers’ information is a top priority,” breached entities could offer the cyber defenders of the world just a few details about the attack tools and online staging grounds the intruders used.

That way, other companies could use the information to find out if they are similarly victimized and to stop the bleeding of customer card data as quickly as possible. Eddie Bauer’s spokespeople say the company has no intention of publishing these so-called “indicators of compromise,” but emphasized that Eddie Bauer worked closely with the FBI and outside security experts.

For more on the importance of IOCs in helping to detect and ultimately stymie cybercrime, check out last Saturday’s story about IOCs released by Visa in connection with the recent intrusion at Oracle’s MICROS point-of-sale unit. And for the record, I have no information connecting this breach or any other recent POS malware attack with the breach at Oracle’s MICROS unit. If that changes, hopefully you’ll read about it here first.

Krebs on SecurityMassive Email Bombs Target .Gov Addresses

Over the weekend, unknown assailants launched a massive cyber attack aimed at flooding targeted dot-gov (.gov) email inboxes with subscription requests to thousands of email lists. According to experts, the attack — designed to render the targeted inboxes useless for a period of time — was successful largely thanks to the staggering number of email newsletters that don’t take the basic step of validating new signup requests.

These attacks apparently have been going on at a low level for weeks, but they intensified tremendously over this past weekend. This most recent assault reportedly involved more than 100 government email addresses belonging to various countries that were subscribed to large numbers of lists in a short space of time by the attacker(s). That’s according to Spamhaus, an entity that keeps a running list of known spamming operations to which many of the world’s largest Internet service providers (ISPs) subscribe.

What my inbox looked like on Saturday, Aug. 13. Yours Truly and apparently at least 100 .gov email addresses got hit with an email bombing attack.

What my inbox looked like on Saturday, Aug. 13. Yours Truly and apparently at least 100 .gov email addresses got hit with an email bombing attack.

When Spamhaus lists a swath of Internet address space as a source of junk email, ISPs usually stop routing email for organizations within those chunks of addresses. On Sunday, Spamhaus started telling ISPs to block email coming from some of the largest email service providers (ESPs) — companies that help some of the world’s biggest brands reach customers via email. On Monday, those ESPs soon began hearing from their clients who were having trouble getting their marketing emails delivered.

In two different posts published at wordtothewise.com, Spamhaus explained its reasoning for the listings, noting that a great many of the organizations operating the lists that were spammed in the attack did not bother to validate new signups by asking recipients to click a confirmation link in an email. In effect, Spamhaus reasoned, their lack of email validation caused them to behave in a spammy fashion.

“The issue is the badly-run ‘open’ lists which happily subscribed every address without any consent verification and which now continue as participants in the list-bombing of government addresses,” wrote Spamhaus CEO Steve Linford. It remains unclear whether hacked accounts at ESPs also played a role.

Also writing for wordtothewise.com, Laura Atkins likened email subscription bombs like this to “distributed denial of service” (DDoS) attacks on individuals.

“They get so much mail from different places they are unable to use their mailbox for real mail,” she wrote. “The hostile traffic can’t be blocked because the mail is coming from so many different sources.”

Atkins said over 100 addresses were added to mailing lists, many from Internet addresses outside the United States.

“The volumes I’m hearing here are significantly high that people cannot use their mailboxes. One sender identified fewer than 10 addresses each signed up to almost 10,000 of their customer lists during a 2 week period,” Atkins wrote. “Other senders have identified addresses that look to be part of the harassment campaign and are working to block mail to those addresses and get them off their lists.”

I WAS ON THE LIST, TOO!

Make that 101 targets, apparently. At approximately 9:00 a.m. ET on Saturday, KrebsOnSecurity’s inbox began filling up with new newsletter subscriptions. The emails came in at a rate of about one new message every 2-3 seconds. By the time I’d finished deleting and unsubscribing from the first page of requests, there would be another page or two of new newsletter-related emails. For most of the weekend until I got things under semi-control, my Gmail account was basically useless.

Some of the lists I was signed up for did require confirmation, but the trouble is if you don’t validate the request within a certain time they still send you additional emails reminding you to complete the signup process.

But those that required validation were in the minority, at least in the emails that I saw. I was aghast at how many of these email lists and newsletters did not require me to click a link to verify my subscription. I used Gmail’s “mark as spam and unsubscribe” option to report all of those subscriptions. It’s taken me almost a day’s worth of effort so far to clean up, and I’m still getting one or two new junk newsletters per minute.

Atkins said many ESPs are now asking their customers to tighten signup requirements to include verification, and to comb through their lists for any recent signups that match certain fingerprints associated with this attack.

I have no idea why I’d be on a list of targets, and no one has contacted me about the attack thus far. But this isn’t the first time that KrebsOnSecurity has been the target of an email bombing attack. A very similar deluge was launched specifically at my inbox in July 2012. I later traced that inbox flooding service back to a guy in Ukraine who was intimately involved in selling credit and debit cards stolen in the 2013 breach at Target.

I don’t know who’s responsible for this latest attack, and I’m not suggesting a connection between it and the 2012 attacks I just mentioned. But I do marvel at how little seems to have changed since 2012 in terms of how organizations run their newsletters.  It’s also mind-boggling to ponder how many of these time-wasting attacks are the result of organizations that fail to secure or properly configure their software, technology and services.

In the past week alone, for example, KrebsOnSecurity.com has been the target of more than a half-dozen DDoS attacks aimed at knocking this site offline. These attacks are increasing in both frequency and intensity because the criminals behind them have access to virtually limitless firepower — millions of poorly-configured systems that can be leveraged to flood the target with so much junk traffic that it is rendered unreachable to legitimate visitors.

Let’s hope the ESPs of the world step up and insist that customers using their email infrastructure take a bit more care to ensure they’re part of the solution and not part of the problem. Atkins captures my thoughts on this subject precisely in the conclusion of her writeup on the attacks.

“Internet harassment seems to be a bigger and bigger issue,” she wrote. “I don’t know if it’s because people are being more open about harassment or if it’s actually more common. In either case, it is the responsibility of networks to minimize the harassment. If your network is a conduit for harassment, you need to do something to stop it.”

Planet DebianSimon Désaulniers: [GSOC] Week 10&11&12 Report

Week 10 & 11

During these two weeks, I’ve worked hard on paginating values on the DHT.

Value pagination

As explained on my post on data persistence, we’ve had network traffic issues. The solution we have found for this is to use the queries (see also this) to filter data on the remote peer we’re communicating with. The queries let us select fields of a value instead of fetching whole values. This way, we can fetch values with unique ids. The pagination is the process of first selecting all value ids for a given hash, then making a separate “get” request packet for each of the values.

This feature makes the DHT more friendly with UDP. In fact, UDP packets can be dropped when of size greater than the UDP MTU. Paginating values will help this as all UDP packets will now contain only one value.

Week 12

I’ve been working on making the “put” request lighter, again using queries. This is a key feature which will make it possible to enable data persistence. In fact, it enables us to send values to a peer only if it doesn’t already have the value we’re announcing. This will substantially reduce the overall traffic. This feature is still being tested. The last thing I have to do is to demonstrate the reduction of network traffic.

Google AdsenseOptimize your content for mobile to #drawthecrowds


Throughout the summer, fans are looking for great content, regardless of whether they’re viewing it from a desktop, tablet or mobile device. Did you know that 1/3 of followers used more than one screen to follow the 2012 Olympic games?[1]

With this in mind, think about whether your content is mobile-optimized and easily accessible for your users in the moments that matter. If they receive a link on their smartphone but are unable to view the content because it’s not mobile-optimized, it’s easy for them to feel frustrated with the poor user experience. On the other hand, If the users are delighted with the content and their first visit to your site is a positive one, they’re more likely to share your content and come back again.

If you already have a mobile optimized site, don’t forget that you can grow your earnings potential by understanding which mobile ad sizes are most effective for you. Finally don't forget these 4 important tips to maximize your earnings with AdSense:

  1. Swap out the 320x50 ad units for 320x100 for a potential RPM increase.
  2. Place a 320x100 ad unit just above the fold or peek the 300x250 -- that is, place a small portion of the ad unit just above the fold (ATF).
  3. Use the 300x250 ad unit below the fold (BTF) mixed in with your content.
  4. Prevent accidental clicks on enhanced features in text ads by moving ad units 150 pixels away from your content.


Is your site mobile ready for the summer? 

Download the #AdSenseGuide to Mobile Web Success to learn more. Are you new to AdSense? If so, sign up now and turn your passion into profit.


Posted by Barbara Sarti, AdSense Account Associate

[1] Google internal data

CryptogramPrisoner's Dilemma Experiment Illustrates Four Basic Phenotypes

If you've read my book Liars and Outliers, you know I like the prisoner's dilemma as a way to think about trust and security. There is an enormous amount of research -- both theoretical and experimental -- about the dilemma, which is why I found this new research so interesting. Here's a decent summary:

The question is not just how people play these games­ -- there are hundreds of research papers on that­ -- but instead whether people fall into behavioral types that explain their behavior across different games. Using standard statistical methods, the researchers identified four such player types: optimists (20 percent), who always go for the highest payoff, hoping the other player will coordinate to achieve that goal; pessimists (30 percent), who act according to the opposite assumption; the envious (21 percent), who try to score more points than their partners; and the trustful (17 percent), who always cooperate. The remaining 12 percent appeared to make their choices completely at random.

Worse Than FailureInjection By Design

Computersicherheit.jpg

As web developers, we spend a fair amount of time protecting our valuable server resources from the grasping tentacles of Internet ruffians and malfeasants. Occasionally, we're tasked with exposing data endpoints to the public Internet. This is generally a carefully crafted solution of whitelists, authentication, authorization, escaping input, limiting access and other protective measures.

But we are not, and cannot be, the domain experts for the system. There is always an inherit tension between our area of expertise, namely software development, and the needs of our business users for their own purposes. Never is this more true than when the problem domain is something that borders on our own area of expertise.

Kit was a quantitative analyst; he knew enough Ruby to be dangerous, but nothing of software engineering as a discipline. Nevertheless, his understanding of the problem domain was deep and thorough. He knew what he needed to accomplish, and enough of how to do so that all he required from Karla’s company was server maintenance for his analysis and the accompanying “big data” SQL database. He was spending more time than he wanted to be administrating the machine sitting in his closet. Since his group had an existing relationship with Karla’s company, it made sense to delegate to her team.

Karla was tasked with examining the code to ensure it would pose no threat to their other existing clients. The code was typical of new programmers who understand the problem better than the solution: inconsistently spaced, with a coding style that matched no known format, and weak in places—but it would work, and it wasn’t just a rootkit disguised as an app. Karla was about to advise accepting the code when something caught her eye: it was designed to take in POST requests and execute them as dynamic SQL queries.

post "/" do
 content_type :json
  payload=request.body.read
connection=Vertica.connect({
<redacted>
})
    query=payload
output = Array.new
connection.query( query) do |row|
        output.push row
        end
connection.close

“We can hardly say no,” Karla’s boss responded when advised of the risk.

“What? Why not?” Karla demanded.

“Every customer is important to us. Kit’s group does millions of dollars of business with us annually."

Ah, yes. Every customer was equal, but the spendiest ones were the most equal.

"We need to make this work," the boss continued. "Why don’t we just move his machine into our datacenter?”

“Absolutely not! Are you insane?” Sirel, the sysadmin, was just as adamant as Karla’s boss. “You want to install some random piece of kit into my datacenter? For all we know, this has been sitting on his desk, talking to God-knows-what. It’s probably infested with trojans and keyloggers. I won’t have it in my nice clean network!”

“Okay, but the boss wants us to help this guy,” Karla pointed out, feeling a bit desperate.

“So take it to Dave," Sirel said. "He can figure something out, I’m sure.”

Dave was in charge of Devops. It was his job to ensure that the code Karla’s team developed was deployed and maintained in their environment. If anyone could make this walking time-bomb play nicely with their existing environment, Dave could.

He, of course, was having nothing of this either. “It’s an obvious security risk, Karla. No, more than that: the entire design is inherently unsecurable. It’d compromise everything else in our database cluster. Can’t be done.”

“Come on, Dave. I know you’re good at this stuff,” Karla pleaded.

"Just last week, I promoted a SQL injection risk your team let past code review," Dave countered. "It took me three days to track down. Do you even understand how many alarm bells were going off? I'd be skinned alive if I let you put this into prod."

Karla trudged back to her boss. “There’s just no way. Nobody wants this thing in our environment. We’ll have to refuse until Kit can find a more secure way to implement it.”

“Nonsense!" the boss cried. "The customer wants it, and the customer is always right!”

Dave fought long and hard, but in the end, Karla’s boss outranked him; the code was put into production unchanged. Karla’s boss didn’t even have the decency to buy Dave a drink first—though he did manage to look surprised when Dave left six months later to pursue a career in finance instead. You see, Dave had learned a valuable lesson: it’s always better to be the client than the hapless souls supporting him.

[Advertisement] Easily create complex server configurations and orchestrations using both the intuitive, drag-and-drop editor and the text/script editor.  Find out more and download today!

CryptogramFriday Squid Blogging: Squid Not Killing New Zealand Sea Lions

Experts are blaming bacteria, not squid nets.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Planet DebianZlatan Todorić: DebConf16 - new age in Debian community gathering

DebConf16

Finally got some time to write this blog post. DebConf for me is always something special, a family gathering of weird combination of geeks (or is weird a default geek state?). To be honest, I finally can compare Debian as hacker conference to other so-called hacker conferences. With that hat on, I can say that Debian is by far the most organized and highest quality conference. Maybe I am biased, but I don't care too much about that. I simply love Debian and that is no secret. So lets dive into my view on DebConf16 which was held in Cape Town, South Africa.

Cape Town

This was the first time we had conference on African continent (and I now see for the first time DebConf bid for Asia, which leaves only Australia and beautiful Pacific islands to start a bid). Cape Town by itself, is pretty much Europe-like city. That was kinda a bum for me on first day, especially as we were hosted at University of Cape Town (which is quite beautiful uni) and the surrounding neighborhood was very European. Almost right after the first day I was fine because I started exploring the huge city. Cape Town is really huge, it has by stats ~4mil people, and unofficially it has ~6mil. Certainly a lot to explore and I hope one day to be back there (I actually hope as soon as possible).

The good, bad and ugly

I will start with bad and ugly as I want to finish with good notes.

Racism down there is still HUGE. You don't have signs on the road saying that, but there is clearly separation between white and black people. The houses near uni all had fences on walls (most of them even electrical ones with sharp blades on it) with bars on windows. That just bring tensions and certainly doesn't improve anything. To be honest, if someone wants to break in they still can do easily so the fences maybe need to bring intimidation but they actually only bring tension (my personal view). Also many houses have sign of Armed Force Response (something in those lines) where in case someone would start breaking in, armed forces would come to protect the home.

Also compared to workforce, white appear to hold most of profit/big business positions and fields, while black are street workers, bar workers etc etc. On the street you can feel from time to time the tension between people. Going out to bars also showed the separation - they were either almost exclusively white or exclusively black. Very sad state to see. Sharing love and mixing is something that pushes us forward and here I saw clear blockades for such things.

The bad part of Cape Town is, and this is not only special to Cape Town but to almost all major cities, is that small crime is on wide scale. Pickpocketing here is something you must pay attention to it. To me, personally, nothing happened but I heard a lot of stories from my friends on whom were such activities attempted (although I am not sure did the criminals succeed).

Enough of bad as my blog post will not change this and it is a topic for debate and active involvement which I can't unfortunately do at this moment.

THE GOOD!

There are so many great local people I met! As I mentioned, I want to visit that city again and again and again. If you don't fear of those bad things, this city has great local cuisine, a lot of great people, awesome art soul and they dance with heart (I guess when you live in rough times, you try to use free time at your best). There were difference between white and black bars/clubs - white were almost like standard European, a lot of drinking and not much dancing, and black were a lot of dancing and not much drinking (maybe the economical power has something to do with it but I certainly felt more love in black bars).

Cape Town has awesome mountain, the Table Mountain. I went on hiking with my friends, and I must say (again to myself) - do the damn hiking as much as possible. After every hike I feel so inspired, that I will start thinking that I hate myself for not doing it more often! The view from Table mountain is just majestic (you can even see the Cape of Good Hope). The WOW moments are just firing up in you.

Now lets transfer to DebConf itself. As always, organization was on quite high level. I loved the badge design, it had a map and nice amount of information on it. The place we stayed was kinda not that good but if you take it into account that those a old student dorms (in we all were in female student dorm :D ) it is pretty fancy by its own account. Talks were near which is always good. The general layout of talks and front desk position was perfect in my opinion. All in one place basically.

Wine and Cheese this year was kinda funny story because of the cheese restrictions but Cheese cabal managed to pull out things. It was actually very well organized. Met some new people during the party/ceremony which always makes me grow as a person. Cultural mix on DebConf is just fantastic. Not only you learn a lot about Debian, hacking on it, but sheer cultural diversity makes this small con such a vibrant place and home to a lot.

Debian Dinner happened in Aquarium were I had nice dinner and chat with my old friends. Aquarium by itself is a thing where you can visit and see a lot of strange creatures that live on this third rock from Sun.

Speaking of old friends - I love that I Apollo again rejoined us (by missing the DebConf15), seeing Joel again (and he finally visited Banja Luka as aftermath!), mbiebl, ah, moray, Milan, santiago and tons of others. Of course we always miss a few such as zack and vorlon this year (but they had pretty okay-ish reasons I would say).

Speaking of new friends, I made few local friends which makes me happy and at least one Indian/Hindu friend. Why did I mention this separately - well we had an accident during Group Photo (btw, where is our Lithuanian, German based nowdays, photographer?!) where 3 laptops of our GSoC students were stolen :( . I was luckily enough to, on behalf of Purism, donate Librem11 prototype to one of them, which ended up being the Indian friend. She is working on real time communications which is of interest also to Purism for our future projects.

Regarding Debian Day Trip, Joel and me opted out and we went on our own adventure through Cape Town in pursue of meeting and talking to local people, finding out interesting things which proved to be a great decision. We found about their first Thursday of month festival and we found about Mama Africa restaurant. That restaurant is going into special memories (me playing drums with local band must always be a special memory, right?!).

Huh, to be honest writing about DebConf would probably need a book by itself and I always try to keep my posts as short as possible so I will try to stop here (maybe I write few bits in future more about it but hardly).

Now the notes. Although I saw the racial segregation, I also saw the hope. These things need time. I come from country that is torn apart in nationalism and religious hate so I understand this issues is hard and deep on so many levels. While the tensions are high, I see people try to talk about it, try to find solution and I feel it is slowly transforming into open society, where we will realize that there is only one race on this planet and it is called - HUMAN RACE. We are all earthlings, and as sooner we realize that, sooner we will be on path to really build society up and not fake things that actually are enslaving our minds.

I just want in the end to say thank you DebConf, thank you Debian and everyone could learn from this community as a model (which can be improved!) for future societies.

Planet DebianNorbert Tretkowski: No MariaDB MaxScale in Debian

Last weekend I started working on a MariaDB MaxScale package for Debian, of course with the intention to upload it into the official Debian repository.

Today I got pointed to an article by Michael "Monty" Widenius he published two days ago. It explains the recent license change of MaxScale from GPL so BSL with the release of MaxScale 2.0 beta. Justin Swanhart summarized the situation, and I could not agree more.

Looks like we will not see MaxScale 2.0 in Debian any time soon...

,

TEDA TEDWomen update: Hanna Rosin on the ‘End of Men’

Cross-posted from TEDWomen curator Pat Mitchell’s blog on the Huffington Post.

When Hanna Rosin, the first speaker at the very first TEDWomen conference in 2010, delivered her talk she had titled “The End of Men,” she had only just begun the research for what became her bestselling 2012 book by the same name.

And as the editors at The Guardian pointed out in a recent editorial, even though women are rising to the top in the US (Hillary Clinton), UK (Theresa May or Andrea Leadsom) and the UN (Helen Clark), “women’s leadership in politics, as well as in business, is not yet normal. But it is becoming normal.” Indeed, if Hillary Clinton is elected president and puts her 50% female cabinet in place, American women will see true representation proportional to population for the first time ever in a White House cabinet.

But as The Guardian editors cautioned in that same editorial, we shouldn’t make the mistake of allowing the “representation to be taken not as a victory, but as the victory.” We still have a long way to go.

Indeed, when Rosin’s book came out in 2012, it received much praise, but also some reductive criticism that mostly revolved around its title. NPR’s Annalee Newitz wrote in her review of the book, “fundamentally, The End of Men isn’t about men at all; it’s about the rise of economically powerful women.”

As Rosin explained in both her TEDdWomen talk and in her book, the big story for women in the 21st century is that more education (for every two men who graduate from college, three women graduate), more leadership opportunities (women make up more than 50% of managers in the workplace) and more economic security (younger women are out-earning their male peers) mean that women can make choices in their lives with more freedom than ever before.

Since her TEDWomen presentation, Rosin has appeared on The Colbert Report, become the co-host of Slate‘s excellent DoubleX podcast in which Rosin and her cohosts, writers June Thomas and Norene Malone, “discuss things women want to talk about and men want to eavesdrop on,” and continued writing award-winning articles on a variety of topics, including the secret lives of teenagers and challenging modern parenting practices.

This summer, Rosin herself did something rather daring that may be the best example yet of the opportunities that economically powerful women have these days. After 20-plus years of writing for a living, she decided to make a leap and accepted a job offer in a field in which she had no previous experience. Rosin is now one of the hosts of NPR’s popular and award-winning Invisibilia podcast series. Having never been a radio journalist before, the move presented a number of challenges for Rosin. She wrote about them earlier this summer in Lena Dunham’s Lenny newsletter in a piece titled “Screw Mastery.”

While getting outside her comfort zone was hard, Rosin noted that the payoff of, as she put it, “dropping back to zero” and forcing yourself to learn new things, even in your 40s, made the successes that much sweeter. She wrote,

I learned a ton of new things about myself, in the way you can only do if you are f—king up daily. I learned that I am defensive but trainable. That I have capacity for patience but that my immediate default is speed, bluntness and ironic distance. That although I am used to working alone, I will happily collaborate. And that I really like working with women, even if they cry more during the day.”

The support system of having women in leadership positions mentoring her at Invisibilia has made all the difference for Rosin. She writes, “I got a lot of help. The people I work with taught me things the way you teach a kid to ride a bicycle — they were right on top of me, day after day. Still, nine months later I listen to the shows we produced and I can completely recognize them as my own.”

Watch her 2010 TEDWomen talk.


TEDWomen 2016 continues a journey that began in Washington, D.C., in 2010, the first major TED conference to focus on the ideas, stories and global narrative of women and girls. The response was overwhelming! Our speakers, both women and men, have come from many different backgrounds and experiences to share ideas and stories in TEDTalks that have been viewed more than 60 million times on TED.com.

Our theme for TEDWomen 2016 is “It’s about time.” We will be exploring how time and attention shape our very lives. In theory, we’ve all got an equal amount of time — 24 precious hours in a day — and yet, our capacity to harness the most out of it is wildly different depending on our circumstances and state of mind. Together, we’ll plot how to push the tipping points even faster and move even slower when it comes to the things we most want to savor and care about.

If you’re interested in being present for these TEDTalks and many more on this year’s theme, there are still a few spaces left to attend TEDWomen 2016. (After the theater is completely sold out, we will also offer some discounted tickets for simulcast viewing.) We will announce our speaker lineup next week — find out more and register at TED.com.


Planet DebianGunnar Wolf: Talking about the Debian keyring in Investigaciones Nucleares, UNAM

For the readers of my blog that happen to be in Mexico City, I was invited to give a talk at Instituto de Ciencias Nucleares, Ciudad Universitaria, UNAM.

I will be at Auditorio Marcos Moshinsky, on August 26 starting at 13:00. Auditorio Marcos Moshinsky is where we met for the early (~1996-1997) Mexico Linux User Group meetings. And... Wow. I'm amazed to realize it's been twenty years that I arrived there, young and innocent, the newest of what looked like a sect obsessed with world domination and a penguin fetish.

AttachmentSize
llavero_chico.png220.84 KB
llavero_orig.png1.64 MB

Cory DoctorowPodcast: How we’ll kill all the DRM in the world, forever

I’m keynoting the O’Reilly Security Conference in New York in Oct/Nov, so I stopped by the O’Reilly Security Podcast (MP3) to explain EFF’s Apollo 1201 project, which aims to kill all the DRM in the world within a decade.


A couple things changed in the last decade. The first is that the kinds of technologies that have access controls for copyrighted works have gone from these narrow slices (consoles and DVD players) to everything (the car in your driveway). If it has an operating system or a networking stack, it has a copyrighted work in it. Software is copyrightable, and everything has software. Therefore, manufacturers can invoke the DMCA to defend anything they’ve stuck a thin scrim of DRM around, and that defense includes the ability to prevent people from making parts. All they need to do is add a little integrity check, like the ones that have been in printers for forever, that asks, “Is this part an original manufacturer’s part, or is it a third-party part?” Original manufacturer’s parts get used; third-party parts get refused. Because that check restricts access to a copyrighted work, bypassing it is potentially a felony. Car manufacturers use it to lock you into buying original parts.

This is a live issue in a lot of domains. It’s in insulin pumps, it’s in voting machines, it’s in tractors. John Deere locks up the farm data that you generate when you drive your tractor around. If you want to use that data to find out about your soil density and automate your seed broadcasting, you have to buy that data back from John Deere in a bundle with seed from big agribusiness consortia like Monsanto, who license the data from Deere. This metastatic growth is another big change. It’s become really urgent to act now because, in addition to this consumer rights dimension, your ability to add things to your device, take it for independent service, add features, and reconfigure it are all subject to approval from manufacturers.

All of this has become a no-go zone for security researchers. In the last summer, the Copyright Office entertained petitions for people who have been impacted by Section 1201 of the DMCA. Several security researchers filed a brief saying they had discovered grave defects in products as varied as voting machines, insulin pumps and cars, and they were told by their counsel that they couldn’t disclose because, in so doing, they would reveal information that might help someone bypass DRM, and thus would face felony prosecution and civil lawsuits.


Cory Doctorow on legally disabling DRM (for good)
[Courtney Nash/O’Reilly]

Planet DebianRaphaël Hertzog: Freexian’s report about Debian Long Term Support, July 2016

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS.

Individual reports

In July, 136.6 work hours have been dispatched among 11 paid contributors. Their reports are available:

  • Antoine Beaupré has been allocated 4 hours again but in the end he put back his 8 pending hours in the pool for the next months.
  • Balint Reczey did 18 hours (out of 7 hours allocated + 2 remaining, thus keeping 2 extra hours for August).
  • Ben Hutchings did 15 hours (out of 14.7 hours allocated + 1 remaining, keeping 0.7 extra hour for August).
  • Brian May did 14.7 hours.
  • Chris Lamb did 14 hours (out of 14.7 hours, thus keeping 0.7 hours for next month).
  • Emilio Pozuelo Monfort did 13 hours (out of 14.7 hours allocated, thus keeping 1.7 hours extra hours for August).
  • Guido Günther did 8 hours.
  • Markus Koschany did 14.7 hours.
  • Ola Lundqvist did 14 hours (out of 14.7 hours assigned, thus keeping 0.7 extra hours for August).
  • Santiago Ruano Rincón did 14 hours (out of 14.7h allocated + 11.25 remaining, the 11.95 extra hours will be put back in the global pool as Santiago is stepping down).
  • Thorsten Alteholz did 14.7 hours.

Evolution of the situation

The number of sponsored hours jumped to 159 hours per month thanks to GitHub joining as our second platinum sponsor (funding 3 days of work per month)! Our funding goal is getting closer but it’s not there yet.

The security tracker currently lists 22 packages with a known CVE and the dla-needed.txt file likewise. That’s a sharp decline compared to last month.

Thanks to our sponsors

New sponsors are in bold.

2 comments | Liked this article? Click here. | My blog is Flattr-enabled.

Sociological ImagesThe “Glass Wall”: Gender, Inequality, and Coaching

Originally posted at Scatterplot.

Olympic fever has hit! As we all marvel at the power, precision, and grace of the athletes, a more disturbing commentary has also emerged, one that diminishes women athletes’ accomplishments, defines them by the men around them, places them in tired tropes of sex objects, or infantilizes them as “girls.” Some journalists, in combination with a robust social media discussion, are calling this bad behavior out. But should we be so surprised?

According to past research, no. In our work, we see this as a more pervasive issue, and women’s collegiate coaching is a prime example. When Title IX was enacted in 1972 approximately 90% of women’s teams were coached by women; in 2014 that number dropped to 43%. Women comprise only 23% of head coaching positions. Why are women coaches – especially of women’s teams – being left out? We talked to 9 female and 12 male coaches of women’s and men’s teams and many of their own explanations suggest a view of fundamental and “natural” differences between men and women.

.

Talking to Coaches… Gender Matters

In general, the qualities of sport – competition, confidence, physical strength, aggression – are seen as masculine, while characteristics of cooperation, passivity, and dependency are coded feminine, raising suspicions about women’s capacity to excel. Masculine dominance has helped to define the parameters of what it means to be a coach.

Interestingly, coaching may be seen as an example of conflicting masculine roles. Given the low pay and high time commitment, coaching undermines the traditional male family role as breadwinner. As this male head women’s tennis coach explains,

I’ve been kind of lucky… I didn’t feel like I had to make a certain amount of money, X amount of dollars to be happy. So I was ok with where I was at salary wise… I think that the key to that is having a wife that also works, and that we can still make it happen, and sort of live the way we want to live and be happy.

Many of the men echoed the idea that without a spouse’s support, a coaching career would be difficult. Although respondents all felt women opt out of coaching due to family pressures, none felt that men needed to opt out to support their families. Arguably, the relationship between masculinity and athletics provides men with the social compensation necessary to remain in coaching in a way that does not operate for women.

Especially when asked why women don’t coach men, many of the respondents did not think women would have the strength, athleticism, authority, and leadership abilities to be effective men’s coaches. As a male head men’s soccer coach expresses:

I think the game is slightly different. The understanding of the nuances of the men’s game versus the women’s game… for a female to go into a men’s athletic team and command respect from those guys, it’s difficult. A female wouldn’t be able to step in and play seven versus seven and be able to play at the same level. Not technically, not tactically, I mean simply physically…just the strength factor.

Other arguments highlight the assumed biological connection between men and leadership. A female assistant women’s soccer coach argued that “the leadership gene is much more apparent in guys, it’s much more inherent in them.” Additionally challenging is the perception that taking orders and guidance from a female threatens masculinity and calls into question male superiority in a male dominated field. A former male head golf coach notes,

A woman coach is going to have to work harder to gain respect from a guy player than a male coach will have to work from a female player. … [Individuals are] raised to say if a guy’s leading, you give them a little benefit of the doubt. A woman has to prove herself, and until she does there’s going to be doubt.

By internalizing and enforcing stereotypes a gender pecking-order can be preserved. As this woman, an assistant women’s soccer coach, suggests, socialization improves men’s leadership ability:

When girls are socialized… it’s share, everyone in groups, be nice to everyone; guys are taught much more of competitiveness… a guy leader comes out in a group much easier… because in a girl’s environment it’s no one should be above anyone else… guys and girls are just different. They’re socialized different.

Stereotypes about men’s competitiveness and women’s need for emotional bonding were prevalent, and if these are carried into hiring decisions it is easy to see why male coaches are favored. Yet, if gender differences are so stark, we would expect to see same-sex coaching across the board, instead of the current disparity. Instead, this difference only legitimated women’s absence and was not used to question men’s presence as coaches of women’s teams. None of the women said they wanted to coach men’s teams and nor were they upset at being denied access to these positions. Respondents were more in favor of increasing women coaching women, but did not question or challenge any of the main gender stereotypes. This man, a former head men’s golf coach said,

I’m a fan of a woman coaching women’s sports, if skill levels are equal, because there are certain intangibles – I don’t understand the woman animal as well on certain things.

Shattering the “Glass Wall”?

Coaches we interviewed recognized the role that resources and opportunities played in incentivizing men into coaching women, but none challenged any aspect of the system. Respondents automatically buy into the “glass wall” such that 50 percent of jobs (those coaching men) are off-limits, thus if women coach approximately 50 percent of women’s teams, it’s “fair.” We see that unquestioned assumptions of gender difference supported perceptions that masculinity and men were superior to femininity and women. Twenty years ago scholars on this topic said it is beliefs in male athletic superiority that justify gender disparities in coaching, and according to these interviews little has changed. So, yes, observers should continue to call out the failures of Olympic commentators to treat women athletes equally, but as we say goodbye to Rio, let’s not forget how these issues are shaping coaches’ and athletes’ experiences every day.

Catherine Bolzendahl is a professor of sociology and the co-author of Counted Out: Same Sex Relations and Americans’ Definitions of Family. Vanessa Kauffman is a PhD student.  Both are at the University of California, Irvine. Jessica Broadfoot-(Lee) is an alum and was a member of the women’s tennis team and a two-time Big West Scholar-Athlete..

(View original at https://thesocietypages.org/socimages)

Planet DebianJamie McClelland: Nice Work Apertium

For the last few years I have been periodically testing out apertium and today I did again and was pleasantly surprised with the quality of the english-spanish and spanish-english translations (and also their nifty web site translator).

So, I dusted off some of my geeky code to make it easier to use and continue testing.

For starters...

    sudo apt-get install apertium-en-es xclip coreutils

Then, I added the following to my .muttrc file:

    macro pager <F2> "<enter-command>set pipe_decode<enter><pipe-entry> sed '1,/^$/d' | apertium es-en | less<enter><enter-command>unset pipe_decode<enter>" "translate from spanish"

If you press F2 while reading a message in spanish it will print out the English translation.

If you use vim, you can create ~/.vim/plugins/apertium.vim with:

    function s:Translate()
        silent !clear
        execute "! apertium en-es " . bufname("%") . " | tee >(xclip)"
    endfunction
    command Translate :call <SID>Translate()

Then, you can type the command:

:Translate

And it will display the English to Spanish translation of the file you are editing and copy the translation into your clip board so you can paste it into your document.

Google AdsenseHow to make the most of Matched content

Matched content is a recommendation tool that allows you to promote relevant content from your site and sponsored content to your visitors. Matched content can help capture visitor attention and loyalty by showing relevant content which could increase pageviews on your site. Here’s a few tips to help you get started with Matched content.

Strategically place your ads to improve viewability.

According to a Think With Google study, “56.1% of all impressions are not seen.” So when determining the placement of your Matched content units, think about which spot(s) would improve viewability and engagement. To increase these metrics we recommend placing this unit directly below your article and either above or below your ad unit. This way readers can easily click on the next piece of content that is interesting to them. Placing it directly below the article often drives higher click-through rates (CTR) than other placements.



Since Matched content units help your users learn more about similar content, you should think about placing it at points in your website where your user will engage more with the content.

Track performance.

To determine if your placement is effective for your site, be sure to track the performance of your Matched content units. Understanding performance is key to maximizing your ad revenue and satisfying your readers. On your Performance reports tab, you’ll see an updated way to view two metric families -- the Overview metric family and the Matched content metric family:
Overview metric family

Selecting this metric family will allow you to understand how much you’re earning from your Matched content units and how these units compare to other ad units on your site. You can also review metrics such as impressions, clicks, and estimated earnings.

Matched content metric family

To get a more detailed performance of your ads and recommendations, use this metric family to view metrics like total impressions, ad impressions, ad clicks, ad revenue per thousand impressions (RPM), and recommendation clicks. Since this metric family is more specific, you should be able to use this data to optimize the ad unit itself.

To see all metric offerings and how they work, take a look at the AdSense Help Center.

By optimizing your Matched content units, you will allow your readers to learn more about topics they are interested in and potentially increase your site’s engagement rate and revenue. If you do not currently have Matched content but are looking to add the feature, determine if you’re eligible by viewing our Help Center article.


Posted by Bserat Ghebremicael
From the AdSense team




Planet DebianRaphaël Hertzog: My Free Software Activities in July 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it’s one of the best ways to find volunteers to work with me on projects that matter to me.

DebConf 16

I was in South Africa for the whole week of DebConf 16 and gave 3 talks/BoF. You can find the slides and the videos in the links of their corresponding page:

I was a bit nervous about the third BoF (on using Debian money to fund Debian projects) but discussed with many persons during the week and it looks like the project evolved quite a bit in the last 10 years and while it’s still a sensitive topic (and rightfully so given the possible impacts) people are willing to discuss the issues and to experiment. You can have a look at the gobby notes that resulted from the live discussion.

I spent most of the time discussing with people and I did not do much technical work besides trying (and failing) to fix accessibility issues with tracker.debian.org (help from knowledgeable people is welcome, see #830213).

Debian Packaging

I uploaded a new version of zim to fix a reproducibility issue (and forwarded the patch upstream).

I uploaded Django 1.8.14 to jessie-backports and had to fix a failing test (pull request).

I uploaded python-django-jsonfield 1.0.1 a new upstream version integrating the patches I prepared in June.

I managed the (small) ftplib library transition. I prepared the new version in experimental, ensured reverse build dependencies do still build and coordinated the transition with the release team. This was all triggered by a reproducible build bug that I got and that made me look at the package… last time upstream had disappeared (upstream URL was even gone) but it looks like he became active again and he pushed a new release.

I filed wishlist bug #832053 to request a new deblog command in devscripts. It should make it easier to display current and former build logs.

Kali related Debian work

I worked on many issues that were affecting Kali (and Debian Testing) users:

  • I made an open-vm-tools NMU to get the package back into testing.
  • I filed #830795 on nautilus and #831737 on pbnj to forward Kali bugs to Debian.
  • I wrote a fontconfig patch to make it ignore .dpkg-tmp files. I also forwarded that patch upstream and filed a related bug in gnome-settings-daemon which is actually causing the problem by running fc-cache at the wrong times.
  • I started a discussion to see how we could fix the synaptics touchpad problem in GNOME 3.20. In the end, we have a new version of xserver-xorg-input-all which only depends on xserver-xorg-input-libinput and not on xserver-xorg-input-synaptics (no longer supported by GNOME). This is after upstream refused to reintroduce synaptics support.
  • I filed #831730 on desktop-base because KDE’s plasma-desktop is no longer using the Debian background by default. I had to seek upstream help to find out a possible solution (deployed in Kali only for now).
  • I filed #832503 because the way dpkg and APT manages foo:any dependencies when foo is not marked “Multi-Arch: allowed” is counter-productive… I discovered this while trying to use a firefox-esr:any dependency. And I filed #832501 to get the desired “Multi-Arch: allowed” marker on firefox-esr.

Thanks

See you next month for a new summary of my activities.

3 comments | Liked this article? Click here. | My blog is Flattr-enabled.

Worse Than FailureCodeSOD: Location Not Found

Let’s say you have a collection of objects which contain geographic points. You want to find a specific item in that collection, and then extract the lat/lon of that item. You might write code like:

    var point = userGeoPositions.Where(x => x.userId = userId);
    decimal lat = point.Latitude;
    decimal lon = point.Longitude;

Of course, this means writing getters and setters for the Latitude and Longitude properties; getters/setters are somewhat repetitive, and repetitive code is a code smell, so obviously, this can’t be the correct solution.

Cody’s company outsourced this problem, and they got back a solution that is obviously much better.

    public static decimal? GetPosition(string endpointId, ICollection<UserGeoPositionModel> userGeoPositions, bool isLatitude)
    {
        var position = userGeoPositions.FirstOrDefault(x => endpointId.Contains(x.UserID));
        return position != null
            ? (decimal?)Convert.ToDecimal(isLatitude ? position.GeoLatitude : position.GeoLongitude)
            : null;
    }

Instead of writing separate getters for each property, this is one function that can get either property. That’s reusability! And you don’t even have to filter the collection before you call this function! Now, when you want the lat/lon of a point, you simply write:

    decimal lat = GetPosition(endpointId, geoPositions, true);
    decimal lon = GetPosition(endpointId, geoPositions, false);

That’s one fewer lines of code than my initial solution. Now, since this function filters on each call, getting the latitude and longitude requires two searches through the whole list, but hey- CPU time is cheap. Programmer time is expensive.

[Advertisement] Infrastructure as Code built from the start with first-class Windows functionality and an intuitive, visual user interface. Download Otter today!

Planet DebianMichal Čihař: Weekly phpMyAdmin contributions 2016-W32

Tonight phpMyAdmin 4.0.10.17, 4.4.15.8, and 4.6.4 were released and you can probably see that there are quite some security issues fixed. Most of them are not really exploitable unless your PHP and webserver are poorly configured, but still it's good idea to upgrade.

If you are running Debian unstable, use our phpMyAdmin PPA for Ubuntu or use phpMyAdmin Docker image upgrading should be as simple as pulling new version.

Besides fixing security issues, we're generally hardening our infrastructure. I'm really grateful that Emanuel Bronshtein (@e3amn2l) is doing great review of all of our code and helps us in this area. This will really make our code and infrastructure much better.

Handled issues:

Filed under: Debian English phpMyAdmin | 0 comments

Planet DebianMichal Čihař: Revoking old PGP key

It has been already six years since I've moved to using RSA4096 PGP key. For various reasons, the old DSA key was still kept valid till today. This is no longer true and it has been revoked now.

The revoked key is DC3552E836E75604 and new one is 9C27B31342B7511D. In case you've signed the old one and not the new one (quite unlikely if you did not sign it more than six years ago), there has been migration document, where you can verify my new key being signed by the old one.

Filed under: Debian English | 0 comments

Planet Linux Australiasthbrx - a POWER technical blog: Getting In Sync

Since at least v1.0.0 Petitboot has used device-mapper snapshots to avoid mounting block devices directly. Primarily this is so Petitboot can mount disks and potentially perform filesystem recovery without worrying about messing it up and corrupting a host's boot partition - all changes happen to the snapshot in memory without affecting the actual device.

This of course gets in the way if you actually do want to make changes to a block device. Petitboot will allow certain bootloader scripts to make changes to disks if configured (eg, grubenv updates), but if you manually make changes you would need to know the special sequence of dmsetup commands to merge the snapshots back to disk. This is particulary annoying if you're trying to copy logs to a USB device!

Depending on how recent a version of Petitboot you're running, there are two ways of making sure your changes persist:

Before v1.2.2

If you really need to save changes from within Petitboot, the most straightforward way is to disable snapshots. Drop to the shell and enter

nvram --update-config petitboot,snapshots?=false
reboot

Once you have rebooted you can remount the device as read-write and modify it as normal.

After v1.2.2

To make this easier while keeping the benefit of snapshots, v1.2.2 introduces a new user-event that will merge snapshots on demand. For example:

mount -o remount,rw /var/petitboot/mnt/dev/sda2
cp /var/log/messages /var/petitboot/mnt/dev/sda2/
pb-event sync@sda2

After calling pb-event sync@yourdevice, Petitboot will remount the device back to read-only and merge the current snapshot differences back to disk. You can also run pb-event sync@all to sync all existing snapshots if desired.

Planet Linux AustraliaColin Charles: What’s next

I received an overwhelming number of comments when I said I was leaving MariaDB Corporation. Thank you – it is really nice to be appreciated.

I haven’t left the MySQL ecosystem. In fact, I’ve joined Percona as their Chief Evangelist in the CTO Office, and I’m going to focus on the MySQL/Percona Server/MariaDB Server ecosystem, while also looking at MongoDB and other solutions that are good for Percona customers. Thanks again for the overwhelming response on the various social media channels, and via emails, calls, etc.

Here’s to a great time at Percona to focus on open source databases and solutions around them!

My first blog post on the Percona blog – I’m Colin Charles, and I’m here to evangelize open source databases!, the press release.

Planet DebianCharles Plessy: Who finished DEP 5?

Many people worked on finishing DEP 5. I think that the blog of Lars does not show enough how collective the effort was.

Looking in the specification's text, one finds:

The following alphabetical list is incomplete; please suggest missing people:
Russ Allbery, Ben Finney, Sam Hocevar, Steve Langasek, Charles Plessy, Noah
Slater, Jonas Smedegaard, Lars Wirzenius.

The Policy's changelog mentions:

  * Include the new (optional) copyright format that was drafted as
    DEP-5.  This is not yet a final version; that's expected to come in
    the 3.9.3.0 release.  Thanks to all the DEP-5 contributors and to
    Lars Wirzenius and Charles Plessy for the integration into the
    Policy package.  (Closes: #609160)

 -- Russ Allbery <rra@debian.org>  Wed, 06 Apr 2011 22:48:55 -0700

and

debian-policy (3.9.3.0) unstable; urgency=low

  [ Russ Allbery ]
  * Update the copyright format document to the version of DEP-5 from the
    DEP web site and apply additional changes from subsequent discussion
    in debian-devel and debian-project.  Revise for clarity, to add more
    examples, and to update the GFDL license versions.  Thanks, Steve
    Langasek, Charles Plessy, Justin B Rye, and Jonathan Nieder.
    (Closes: #658209, #648387)

On my side, I am very grateful to Bill Alombert for having committed the document in the Git repository, which ended the debates.

Planet DebianSean Whitton: Tucson monsoon rains

When it rains in Tucson, people are able to take an unusually carefree attitude towards it. Although the storm is dramatic, and the amount of water means that the streets turn to rivers, everyone knows that it will be over in a few hours and the heat will return (and indeed, that’s why drain provision is so paltry).

In other words, despite the arresting thunderclaps, the weather is not threatening. By contrast, when there is a storm in Britain, one feels a faint primordial fear that one won’t be able to find shelter after the storm, in the cold and sodden woods and fields. Here, that threat just isn’t present. I think that’s what makes us feel so free to move around in the rain.

I rode my bike back from the gym in my $5 plastic shoes. The rain hitting my body was cold, but the water splashing up my legs and feet was warm thanks of the surface of the road—except for one area where the road was steep enough that the running water had already taken away all lingering heat.

Planet DebianBen Hutchings: Debian LTS work, July 2016

I was assigned another 14.7 hours of work by Freexian's Debian LTS initiative and carried over 1 from last month. I worked a total of 15 hours, carrying over a fraction of an hour.

I spent another week in the Front Desk role and triaged various new CVEs for wheezy.

I spent the remainder of the time working on the next Linux stable updates (3.2.82 and Debian 3.2.81-2), but didn't release them - that will be done in the next few days.

,

Google AdsenseLearn how Google’s research tools can enhance your content




When you know what the world’s talking about, you can participate in the conversation. But the online world moves quickly, so if you want to keep the crowds coming back to your site, your content needs to move with it.

Google’s News Lab is Google’s effort to empower innovation at the intersection of technology and media. Its mission is to collaborate with journalists and entrepreneurs to build the future of media. An important part of that is ensuring that Google tools are available and easy-to-use for journalists around the world.


Google News Lab offers lessons on how to use Google tools relevant to publishers’ needs. Say something newsworthy at a sports event is grabbing headlines, Google tools can ensure that you’re informed. It’s then over to you to draw on this story and incorporate it into your content.

A great way to keep your finger on the pulse is Google Alerts, a tool that allows you to follow developing stories from your inbox. Simply select the topics you want to follow and have emails sent directly to your inbox any time that Google finds new results for this topic.

Google Alerts removes the need for you to keep checking back on a topic and simplifies the journalistic process by having all your information come from one reliable source. Once you’re using Google Alerts to stay informed about an event, you can ensure that the content on your site stays current and keeps users coming back for more.

If you want to take a more proactive approach, Google Trends gives you access to global data, to power insightful storytelling. One way to use this data could be to look at what users are searching globally. You can select topic areas and drill down into regions for those topics, ensuring that you can take advantage of the data relevant to you and your users to create the most timely and engaging content.


Google tools are designed to help you create great content. The more topical your content, the more likely you are to keep drawing the crowds.

Get started with Google News Lab today.

New to AdSense? Sign up now and turn your passion into profit


Posted by Jay Castro,
From the AdSense team
@jayciro

Google AdsenseAmplify your content with social

Social media is more than just vacation photos and stories about babies, it’s about sharing experiences and creating interactions with people.

Content on social media can instantly reach millions of users and create thousands of online stories. Do you remember the social media storm that Ellen DeGeneres created with a single photo? Her selfie from the 2014 Oscars started a global conversation and has since been added to Google’s 2014 web culture guide. Her content holds the win for the most retweeted tweet with more than 3M retweets, 6.8M site embeds, and even temporarily led to Twitter going down.

Big events like the Oscars create opportunities for publishers to interact with users through both original content and viral content. This summer, users will be searching, sharing, and consuming content like never before. Be sure to examine your content strategy, and find ways to spark conversations on social channels or, contribute to an existing digital conversation.



How can AdSense Publishers participate in big events? If you're new to AdSense, you can sign up now and turn your passion into profit.

  1. Include editorial coverage around topical events.
  2. Make sure your content is easy to consume and share.
  3. Create a social media strategy around a big event.


Here are four tips to help jumpstart your social media strategy:

  1. Know your audience. Trigger an emotional response from your viewers by humanizing your brand.
  2. Be social to win on social by creating relationships with your users.
  3. Don’t be afraid to follow the leaders and the trends. If there’s social proof behind it, do it.
  4. Use tools to help you create great content.


More on the fourth tip, here are a few tools to help you #drawthecrowds through social channels:

  • Buffer is a well-known tool used by social media marketers. It helps streamline your social media management.
  • Pablo is a tool created by Buffer that is “the simplest way to create beautiful images that fit every social network perfectly”.
  • Canva is another image creating tool that can help you create visual content on a budget.
  • Feedly can help you organize and read relevant content in your industry to help fuel your content creation.
  • Google Trends is a widely used tool to help you identify key trends happening around the world.
  • IFTTT is a popular tool used to streamline many tasks. For example, here’s a pre-built recipe that will automate the process of tweeting each new wordpress blog post.


According to the New York Times, 68% of users share [online content] to give people a better sense of who they are and what they care about. To amplify your content this summer, be sure to create content that resonates with your audience.


New to AdSense? Sign up now and turn your passion into profit. Let us know your thoughts on Twitter @AdSense.


Posted by Barbara Sarti and Jay Castro from the AdSense team

Google AdsenseDeliver more viewable impressions to get your ads seen



Viewability is a trendy word in the advertising world. This popular metric has become key for advertisers to measure the success of an ad campaign. But what is Viewability?

Viewability is the measurement of a viewable ad impression, meaning that it has appeared within a user’s browser and had the opportunity to be seen. The Interactive Advertising Bureau (IAB) defines a viewable impression as an ad that meets the following criteria:

50% of the ad’s pixels are visible in the browser window for a continuous one second.

Think with Google says “If an ad isn't seen, it can't have an impact, change perception, or build brand trust. That's why measuring viewability matters.” For AdSense publishers, this means that increased viewability may encourage greater investments from advertisers.

Global events are opportunities for advertisers to connect with larger audiences and create brand awareness. For advertisers with these goals in mind, viewability matters. Here are a few resources to help get your ads seen this summer.


See Google’s Active View in action and move towards delivering more viewable impressions this summer.

New to AdSense? Sign up now and start drawing the crowd.


Posted by Barbara Sarti, from the AdSense team

Google Adsense4 steps to build a strong brand experience

Exposing your audience to a rock solid brand leaves a lasting impression on your site’s visitors, and helps separate you from your competitors. To establish brand consistency across multiple touch points, it’s important to create and stick to guidelines unique to your brand.

Building a strong brand experience comes down to four things:



1. Find your voice



A brand’s voice means more than just the tone you use in your content and communications. It also applies to style, colors, and graphics. Is your brand bubbly, bright, and fun? Or is it straight to the point with clean lines and a matter-of-fact tone? Often times, the type of product or services you're selling as well as your company philosophy can help you determine an appropriate tone. There’s no secret for determining what an audience will respond best to, as all styles can be effective in their own way. So choose what works for you and your creative vision.



2. Be consistent



Once you’ve laid the groundwork for what defines your brand, it’s important to stick to these principles. This applies to your website, emails, social media posts, and any other place users come into contact with your brand. Taking the time to stick to an easy to read font, finding a color scheme that draws the eye and guides your readers, or having consistent verbiage can do wonders to further cement your brand’s presence and make it memorable.




3. Know your audience



While it’s important to decide what your brand is, it’s also important to know your audience, their interests, and how they prefer to communicate. For example, if you’re targeting busy, high-level decision makers, they may prefer something short and sweet—perhaps bullet points are the way to go. If you’re targeting creative individuals, it may be worth investing in a personalized logo and site. Highly visual assets such as videos would also be a great way to go. The more you know and cater to your intended audience, the more successful your brand will be.


To understand your users’ interests, use Google Analytics to view your bounce rates, time on pages, and pageviews—three indicators of user engagement. Understand where you stand in comparison to other sites and, if needed, improve on these rates by creating a stronger connection between your site and your audience, i.e. creating content relevant to your audience’s interests.



4. Prove your Worth



Having a particular value that you provide to your customers (not to be mistaken for price) can help separate your brand from competitors. For instance, what do you provide to your customers that is different or special? This can include everything from innovative products to great customer service and can also be an emotional value (think Kleenex being associated with comfort and support). Just make sure to deliver on any and all promises made on your site.

To learn more about how to develop your user experience, check out the AdSense Guide to Audience Engagement.




Posted by Jay Castro
From the AdSense team
@jayciro

Krebs on SecuritySSA: Ixnay on txt msg reqmnt 4 e-acct, sry

The U.S. Social Security Administration says it is reversing a newly enacted policy that required a cell phone number from all Americans who wished to manage their retirement benefits at ssa.gov. The move comes after a policy rollout marred by technical difficulties and criticism that the new requirement did little to prevent identity thieves from siphoning benefits from Americans who hadn’t yet created accounts at ssa.gov for themselves.

In an announcement last month, the SSA said all new and existing ‘my Social Security’ account holders would need to provide a cell phone number. The SSA said the numbers would be used to send recipients an 8-digit code via text message that needs to be entered along with a username and password to log in to the site.

But sometime in the past few days, apparently, the SSA decided to rescind the cell phone rule.

“We removed the requirement to use a cell phone to access your account,” the agency noted in a message posted to its mySocial Security portal. “While it’s not mandatory, we encourage those of you who have a text capable cell phone to take advantage of this optional extra security. We continue to pursue more options beyond cell phone texting.”

Hopefully, those options will include using the U.S. Mail to send Americans a one-time code that needs to be entered at the SSA’s Web site to complete the sign-up process. I should note that the SSA is already mailing out paper letters via snail mail to Americans who’ve signed up for an SSA account online; they’re just not using that mailing to securely complete the signup and authentication process.

Here’s a redacted letter that a friend of mine received and shared the other day after signing up for an account online. It merely explains what the agency already explained about the texting policy via its Web site.

A letter that the Social Security Administration sends out via the U.S. Mail for every American who signs up to manage their benefits at ssa.gov.

A letter that the Social Security Administration sends out via the U.S. Mail for every American who signs up to manage their benefits at ssa.gov.

The SSA does still offer the text message feature as part of what it calls “extra security” options. These extra options by the way do include the sending users a special code via the U.S. Mail that has to be entered on the agency’s site to complete the signup process. If you choose to enable extra security, the SSA will then ask you for:

  • The last eight digits of your Visa, MasterCard, or Discover credit card;
  • Information from your W2 tax form;
  • Information from a 1040 Schedule SE (self-employment) tax form; or
  • Your direct deposit amount, if you receive Social Security benefits.

Sadly, crooks won’t go through the more rigorous signup process — they’ll choose the option that requires less information. That means it is still relatively easy for thieves to create an account in the name of Americans who have not already created one for themselves. All one would need is the target’s name, date of birth, Social Security number, residential address, and phone number. This personal data can be bought for roughly $3-$4 from a variety of cybercrime shops online.

What else does the SSA require to prove you’re you? Assuming you can buy or supply the above personal data, the agency relays four multiple-guess, so-called “knowledge-based authentication” or KBA questions from credit bureau Equifax. In practice, many of these KBA questions — such as previous address, loan amounts and dates — can be successfully enumerated with random guessing.  What’s more, very often the answers to these questions can be found by consulting free online services, such as Zillow and Facebook.

In September 2013, I warned that SSA and financial institutions were tracking a rise in cases wherein identity thieves register an account at the SSA’s portal using a retiree’s personal information and have the victim’s benefits diverted to prepaid debit cards that the crooks control. Unfortunately, because the SSA’s new security features are optional, they do little to block crooks from hijacking SSA benefit payments from retirees.

Because it’s possible to create just one my Social Security account per Social Security number, registering an account on the portal is one basic way that Americans can avoid becoming victims of this scam.

In addition to the SSA’s optional security measures, Americans can further block ID thieves by placing a security freeze on their credit files with the major credit bureaus. Readers who have taken my ceaseless advice to freeze their credit will need to temporarily thaw the freeze in order to complete the process of creating an account at ssa.gov. Looked at another way, having a freeze in place blocks ID thieves from fraudulently creating an account in your name and potentially diverting your government benefits.

Alternatively, citizens can block online access to their Social Security account. Instructions for doing that are here.

CryptogramPowerful Bit-Flipping Attack

New research: "Flip Feng Shui: Hammering a Needle in the Software Stack," by Kaveh Razavi, Ben Gras, Erik Bosman Bart Preneel, Cristiano Giuffrida, and Herbert Bos.

Abstract: We introduce Flip Feng Shui (FFS), a new exploitation vector which allows an attacker to induce bit flips over arbitrary physical memory in a fully controlled way. FFS relies on hardware bugs to induce bit flips over memory and on the ability to surgically control the physical memory layout to corrupt attacker-targeted data anywhere in the software stack. We show FFS is possible today with very few constraints on the target data, by implementing an instance using the Rowhammer bug and memory deduplication (an OS feature widely deployed in production). Memory deduplication allows an attacker to reverse-map any physical page into a virtual page she owns as long as the page's contents are known. Rowhammer, in turn, allows an attacker to flip bits in controlled (initially unknown) locations in the target page.

We show FFS is extremely powerful: a malicious VM in a practical cloud setting can gain unauthorized access to a co-hosted victim VM running OpenSSH. Using FFS, we exemplify end-to-end attacks breaking OpenSSH public-key authentication, and forging GPG signatures from trusted keys, thereby compromising the Ubuntu/Debian update mechanism. We conclude by discussing mitigations and future directions for FFS attacks.

Worse Than FailureTechnical Debt

If you get the rare luxury to start a new project from scratch, there's something deep down inside you that makes you want to do it right. You pick the right people, equipment and tools so that you have the best chance of success. Unfortunately, sometimes incorrect decisions are innocently made and a technical time bomb is placed in the code.

About 20 years ago at Big Money Inc., such a project was started and such a mistake was made. In this case, the mistake revolved around choosing a messaging platform that failed miserably when asked to pump more messages than was intended. The original developers knew not of this otherwise widely-known limitation.

Compounding things was the choice of implementation. Usually, when you build a transport layer, you do something like this:

  public interface ICallback {
    void callback(Record rec);
  }

  public class Handler implements ICallback {
    public void callback(Record rec) {
      // do something with it
    }
  }

  public interface ITransport {
    void connect();
    void disconnect();
    void retryConnectionWithDegradingFrequencyAndAlerts();
    void publish(Record rec);
    void registerCallback(ICallback callback, String subject);
    // ...
  }

  public class ConcreteTransport implements ITransport {
    // Implementation here
  }

You instantiate it with relevant parameters for each transport instance you need, and inject or pass the concrete object - by the interface - wherever it's needed. This way, when a new transport comes along tomorrow, you only need to implement a new concrete interface, once, instantiate the new wrapper and inject or pass it in lieu of the previous one.

This is not a new way of doing things. Encapsulation is not a new concept. Many souls far wiser than us documented this in great detail a very long time ago.

But the original developers were unaware of such sorcery as encapsulation, interfaces or (Spring) injection, so the system was built with all of the code to connect, publish and subscribe hard wired into every single class that needed to send or receive messages. Literally more than 800 of them. Decapsulation at its finest!

Fast forward more than a decade and there's an economic event that caused the volume being pushed through the messaging system to spike; the messaging system collapsed and brought everything that depended on it to a screeching halt. In the middle of business. World wide. Naturally, high level managers suddenly pushed this issue to the top of their priorities: This MUST be fixed IMMEDIATELY!

Of course, when the current developers on the team looked into it, they had to report unpleasant news regarding 800+ places where code needed to be changed. Rather than attempt to factor it out into encapsulated classes and merely use it where required, management decided that they could not afford to risk introducing bugs by rearranging code, and that they should keep things as they are and fix it everywhere it's used. The available staff of two people reported that this would take a while. Management roared: Unacceptable! Start coding post-haste! Type, young minions - TYPE!

And type they did. Until one of them gave up on IT and left to pursue a career as a florist. The lone remaining developer assigned to the task slogged forward, but at the pace of a single developer. When management queried why the pace of progress was now 50% of what it was, he reported: You KNOW that the other guy is no longer here, right?

Management realized they had to do something, so they did what they do best; they ignored reality and barked orders, leading to many conversations along the lines of...

  Manager: We MUST maintain the planned rate of fixes or we won't meet the schedule!

  Developer:: The schedule assumes two folks, already up to speed, working on it full
  time - if you maintain the scheduled level of resources, we can maintain
  the expected throughput

  M: But we don't have time to find and train someone new

  D:...but you have time to miss your deliverable?
...and downhill it went from there.

To complicate matters, there was a new corporate-wide directive to move off of the messaging system (to which they were moving) onto yet another corporate-mandated tool, implying that all of the in-line rework would need to be re-reworked.

Management insisted that development had no choice but to proceed as they had been instructed.

The lone developer realized that he was being assigned full responsibility for numerous earlier management failures, that he was in a corner and had no choice but to go for broke. He pointed out that this was an unreasonable approach that was guaranteed to fail. If he had been allowed to refactor the code into an encapsulated/injected model, it would be relatively trivial to change messaging systems. Until a sane direction was to be adopted, he was through being set up to pay for technical debt that the project had amassed while he was still in high school.

They ordered him to do as they said, and that he needed to own this problem and do whatever it took to get it fixed.

He pointed out that before he would take ownership of a problem, it had to be reasonably fixable, and that he needed to get some say in how the solution would be implemented. While there are many ways to succeed, some paths are guaranteed to fail, and he would not accept responsibility for 12 years of someone else's bad decisions and accumulated technical debt.

M: You don't have a choice!

D: Yes I do. I resign effective 2 weeks from today!

They really couldn't argue with that, so over the next 2 weeks, he wrote down everything he knew about it, mostly in an attempt to warn the next poor soul inflicted with this mess. Then he halted the remaining 50% of the development effort on the project by saying goodbye and walking out.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

,

Planet Linux AustraliaBinh Nguyen: Neo-Colonialism and Neo-Liberalism, Intelligence Analysis, and More

Watch a lot of media outlets and over and over again and you hear the terms 'Neocolonialism' and 'Free Trade' from time to time. Until fairly recently, I wasn't entirely aware of what exactly this meant and how it came to be. As indicated in my last post, up until a certain point wealth was distributed rather evenly throughout the world. Then 'colonialism' happened and the wealth gap between

CryptogramYet Another Government-Sponsored Malware

Both Kaspersky and Symantec have uncovered another piece of malware that seems to be a government design:

The malware -- known alternatively as "ProjectSauron" by researchers from Kaspersky Lab and "Remsec" by their counterparts from Symantec -- has been active since at least 2011 and has been discovered on 30 or so targets. Its ability to operate undetected for five years is a testament to its creators, who clearly studied other state-sponsored hacking groups in an attempt to replicate their advances and avoid their mistakes.

[...]

Part of what makes ProjectSauron so impressive is its ability to collect data from computers considered so sensitive by their operators that they have no Internet connection. To do this, the malware uses specially prepared USB storage drives that have a virtual file system that isn't viewable by the Windows operating system. To infected computers, the removable drives appear to be approved devices, but behind the scenes are several hundred megabytes reserved for storing data that is kept on the "air-gapped" machines. The arrangement works even against computers in which data-loss prevention software blocks the use of unknown USB drives.

Kaspersky researchers still aren't sure precisely how the USB-enabled exfiltration works. The presence of the invisible storage area doesn't in itself allow attackers to seize control of air-gapped computers. The researchers suspect the capability is used only in rare cases and requires use of a zero-day exploit that has yet to be discovered. In all, Project Sauron is made up of at least 50 modules that can be mixed and matched to suit the objectives of each individual infection.

"Once installed, the main Project Sauron modules start working as 'sleeper cells,' displaying no activity of their own and waiting for 'wake-up' commands in the incoming network traffic," Kaspersky researchers wrote in a separate blog post. "This method of operation ensures Project Sauron's extended persistence on the servers of targeted organizations."

We don't know who designed this, but it certainly seems likely to be a country with a serious cyberespionage budget.

EDITED TO ADD (8/15): Nicholas Weaver comment on the malware and what it means.

Planet Linux AustraliaColin Charles: Changing of the guard

I posted a message to the internal mailing lists at MariaDB Corporation. I have departed (I resigned) the company, but definitely not the community. Thank you all for the privilege of serving the large MariaDB Server community of users, all 12 million+ of you. See you on the mailing lists, IRC, and the developer meetings.

The Japanese have a saying, “leave when the cherry blossoms are full”.

I’ve been one of the earliest employees of this post-merge company, and was on the founding team of the MariaDB Server having been around since 2009. I didn’t make the first company meeting in Mallorca (August 2009) due to the chickenpox, but I’ve been to every one since.

We made the first stable MariaDB Server 5.1 release in February 2010. Our first Linux distribution release was in openSUSE. Our then tagline: MariaDB: Community Developed. Feature Enhanced. Backward Compatible.

In 2013, we had to make a decision: merge with our sister company SkySQL or take on investment of equal value to compete; majority of us chose to work with our family.

Our big deal was releasing MariaDB Server 5.5 – Wikipedia migrated, Google wanted in, and Red Hat pushed us into the enterprise space.

Besides managing distributions and other community related activities (and in the pre-SkySQL days Rasmus and I did everything from marketing to NRE contract management, down to even doing press releases – you wear many hats when you’re in a startup of less than 20 people), in this time, I’ve written over 220 blog posts, spoken at over 130 events (an average of 18 per year), and given generally over 250 talks, tutorials and keynotes. I’ve had numerous face-to-face meetings with customers, figuring out what NRE they may need and providing them solutions. I’ve done numerous internal presentations, audience varying from the professional services & support teams, as well as the management team. I’ve even technically reviewed many books, including one of the best introductions by our colleague, Learning MySQL & MariaDB.

Its been a good run. Seven years. Uncountable amount of flights. Too many weekends away working for the cause. A whole bunch of great meetings with many of you. Seen the company go from bootstrap, merger, Series A, and Series B.

It’s been a true privilege to work with many of you. I have the utmost respect for Team MariaDB (and of course my SkySQL brethren!). I’m going to miss many of you. The good thing is that MariaDB Server is an open source project, and I’m not going to leave the project or #maria. I in fact hope to continue speaking and working on MariaDB Server.

I hope to remain connected to many of you.

Thank you for this great privilege.

Kind Regards,
Colin Charles

Sociological ImagesThe Anthropology of Optical Illusions

Which line is longer?

20160806_223632

Most people who grow up in industrialized environments will be at least a little bit tricked by this optical illusion, called the Müller-Lyer illusion. At first look, it may seem as if the line on the left is shorter than the line on the right. In fact, if you look closely and carefully, you can probably see that both lines are the same length.

Some psychologists theorize that susceptibility to this illusion is due to a strongly “carpentered” environment, one built by humans with the help of machines. Such environments are made mostly of straight lines and right angles. If this geometry is all around us all the time, our brains get very good at interpreting these environments.

That advantage, though, is a disadvantage when looking at the Müller-Lyer lines because our brain learns to associate angles like the one on the right with distance and ones like the one on the left with closeness. Then, it alters our perception of their height to adjust for perceived space.

Bear with me.

Consider my drawing of a room and hallway below. You can see that the corner closest to us (A) has lines like the point of an arrow on both ends (like the line on the left above), while the one further away (B) has lines like the rear of an arrow on both sides (like the line on the right). Our brain gets so used to inferring distance when it sees these angles, it assumes that any line with angles like B appears inaccurately short because it’s far away. That’s how the illusion tricks our brain.

20160806_221608

People who don’t grow up in a carpentered environments, though—hunter gatherers and other groups who spend most of their time in nature and other uncarpentered environments—don’t have brains adjusted to understanding straight lines and angles, so the illusion doesn’t work on them.

The Müller-Lyer illusion, then, is a great example of how our brains get acculturated in ways that shape even simple and straightforward perception tasks.

Lisa Wade, PhD is a professor at Occidental College. She is the author of American Hookup, a book about college sexual culture, and a textbook about gender. You can follow her on Twitter, Facebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

TEDOrganizing principles: Notes from Session 5 of TEDSummit

Do we have the vision and the energy to confront seemingly impossible problems — like predatory corporations, political deadlock, the wasted potential of millions of refugees? Session 5 rounded up people who are jumping right in.

A call to action on fossil fuels. Costa Rica, climate advocate Monica Araya’s native country, gets almost 100 percent of its electricity from renewable sources, including hydropower, geothermal and solar. It started with the country’s bold decision to abolish its military in 1948. Investing that money in social spending created stability, which gave Costa Rica the freedom to explore alternative energy options. But it’s no utopia, Araya explains, because fossil fuels are still used for the country’s transportation systems — systems that are gridlocked and crumbling. Going forward, she urges the next generation to form coalitions of citizens, corporations and clean energy champions to get Costa Rica off fossil fuels completely and commit clean energy in all sectors.

Photo by Ryan Lash/TED.

Monica Araya suggests that the future of alternative energy is in places like her home, Costa Rica. Photo by Ryan Lash/TED.

There are reasons to hope. Across the world, there are true signs of progress, despite the media’s constant drone of doom and gloom in their headlines. Global affairs thinker Jonathan Tepperman has seen it with his own eyes in three countries: Canada, Indonesia and Mexico. In each country, Tepperman examines their historical trajectory and transformation into places of societal advancement and inclusivity — drawing a common thread that connects them all. Within their borders, these nations have embraced the extreme in times of existential peril, found power in promiscuous, open-minded thinking and exercised compromise to its fullest extent. “The real obstacle is not ability and it’s not circumstances,” says Tepperman. “It’s much simpler: Making big changes involves taking big risks, and taking big risks is scary. Overcoming that fear requires guts.”

Online education for all. Imagine a world where every refugee has access to a free higher education, anywhere, at anytime. Although this may seem unbelievable, this is Shai Reshefs dream, and so far he has already made progress towards achieving it. Soon, the University of the People, founded by Reshef, will admit 500 Syrian refugees at no cost to them. University of the People is an online education platform that he believes will make this goal not only accessible and affordable but also replicable and scalable across the world. Despite the return on investment for education being incredibly high, currently refugees are 10% likely to receive higher education in their host countries. Beyond increasing this dismal statistic, Reshef hopes his institution will be able to help refugees with the lack of legal identification often holding them back, and eventually facilitate their transfer into local universities. Right now, 250 additional students are slated to be enrolled in the coming months and eventually they hope to sustain 12,000. Reshef wants to create an entire program ran by refugees for other refugees, proving that higher education need not exclude anyone, because as Reshef says “online, everyone gets a front row seat.”

Photo by Ryan Lash/TED.

Pavan Sukhdev says: While the backbone of our global economy is the corporation, we’ve evolved corporate systems that ruthlessly drain public benefits for private gain.  Photo by Ryan Lash/TED.

A new company for a new economy. “The last two and a half decades have seen scientists, economists, and politicians say again and again and more and more often that we need to change economic direction. we need a green economy, a circular economy. Despite all that agreement, we are still hurtling towards planetary boundaries.” To understand why, we need to ask an important question: can the corporations of today deliver the economy of tomorrow? According to environmental economist Pavan Sukhdev, the answer is no. That’s because today’s business as usual creates huge public costs to generate private profits — “this is the biggest free lunch in the history of mankind.” The good news? There are micro-solutions and if we follow them, we can evolve a new type of corporation whose goals are aligned with society rather than at its expense.

Who is making the decisions that increasingly govern our lives? What we see and then think? What we think and then do? The questions isn’t who — it’s what. And the answer is the increasingly powerful algorithms employed by entities  from Facebook to human resources departments to prison sentencing boards. It’s a problem that troubles sociologist Zeynep Tufekci, who explains that the complex way that algorithms grow and improve — through  a semi-autonomous form of computing called machine learning, which evolved from pattern recognition and prediction software — makes them hard to see through and hard to steer effectively. “What safeguards do you have that your black box isn’t doing something shady?” wonders Tufekci. Making things worse, companies are very protective of their secret recipes for algorithms, so it’s almost impossible to gauge how objective they really are — but given that they’re only as unbiased as the data they are fed, that doesn’t sound like a recipe for fairness.

Photo by Bret Hartman/TED.

As AIs learn to learn, there’s a point where, says Sam Harris, they might outstrip our own intelligence. Photo by Bret Hartman/TED.

Scared of AI? You should be. Regardless of whether or not you’re afraid of Artificial Intelligence, Sam Harris wants you to be more afraid. He believes that we are culturally “unable to marshall an appropriate emotional response to the dangers that lay ahead.” Although it may seem alarming, Harris is not imagining a dystopian terminator future straight out of science fiction. Rather, his fear is based on three rational assumptions: 1. Intelligence is a matter of information processing information through physical systems, 2. We will continue to improve our intelligent machines, and 3. We as humans do not rank anywhere close to the possible apex of intelligent life. The eventual existence of a hyper intelligent machine is undeniable and when our goals and the machine’s inevitably differ, these superior machines will waste no time disposing of any thing standing between them and their objective. Due to the immense havoc these innovations are capable of wreaking, Harris urges that the time to begin tackling the ethics of AI is now, regardless of how far away it may seem. Because we only have one shot at getting the initial conditions right and we better make sure they’re conditions we can live with.  

Humility in the face of fear. In a vulnerable, striking and meditative move, author Anand Giridharadas read “A letter to the other half” to the TEDSummit audience. Penned just days before the conference, it reflected Giridharadas’ regret over ignoring the legitimate struggles and instability of a people enraged over a changing globalized world — echoing events such as Brexit and the rise of Donald Trump.

Unsubscribe. Comedian James Veitch wrapped up session 5, turning his frustrations into whimsy and amusement when his local supermarket refused to take him off their email list, despite numerous attempts on his end. The hijinks that ensues is an entertaining and priceless venture into the world of online customer care.


CryptogramMicrosoft Accidentally Leaks Key to Windows Backdoor

In a cautionary tale to those who favor government-mandated backdoors to security systems, Microsoft accidentally leaked the key protecting its UEFI Secure boot feature.

As we all know, the problems with backdoors are less the cryptography and more the systems surrounding the cryptography.

Worse Than FailureCodeSOD: Drugsort

I did a brief contract with Hershey, the candy manufacturer, once. The biggest thing I recall from the experience was that they had bowls full of candy all over the place. You could just grab them by the handful.

I bring this up, because Brenda worked for a pharmaceutical company, and I can only assume that there are bowls full of random drugs scattered around, and someone has been chowing down on them by the handful. That’s the most logical explanation for the following code:

'From ...(7/21/2008 3:18:30 PM): Sort the additional additives in alphabetical order

'Albumin
'Ascorbic Acid
'Chromium
'Cysteine + Acetate
'Cysteine - NoAcetate
'Epogen
'Famotidine
'Fervens Canis (aquaeous)
'FolicAcid
'Heparin
'Insulin *R* Human
'IronDextran
'Levocarnitine
'MVI - Adult Without Vitamin K
'MVI Pediatric
'Metoclopramide
'Ped-Trace 4
'Phytonadione-K1
'Ranitidine
'Selenium
'Selenium > 5years
'Zinc

Dim additivescount As Integer = Me.treeIngredients.Nodes(4).GetNodeCount(True)

Dim node_albumin As TreeNode = Nothing
Dim node_ascorbic As TreeNode = Nothing
Dim node_chromium As TreeNode = Nothing
Dim node_cysteineplus As TreeNode = Nothing
Dim node_cysteineminus As TreeNode = Nothing
Dim node_epogen As TreeNode = Nothing
Dim node_famotidine As TreeNode = Nothing
Dim node_folic As TreeNode = Nothing
Dim node_heparin As TreeNode = Nothing
Dim node_insulin As TreeNode = Nothing
Dim node_iron As TreeNode = Nothing
Dim node_levocarnitine As TreeNode = Nothing
Dim node_lipids3in1 As TreeNode = Nothing
Dim node_mviadult As TreeNode = Nothing
Dim node_mviped As TreeNode = Nothing
Dim node_metoclopramide As TreeNode = Nothing
Dim node_pedtrace4 As TreeNode = Nothing
Dim node_phytonadione As TreeNode = Nothing
Dim node_ranitidine As TreeNode = Nothing
Dim node_selenium As TreeNode = Nothing
Dim node_selenium5 As TreeNode = Nothing
Dim node_zinc As TreeNode = Nothing
Dim node_hotdogwater As TreeNode = Nothing

For cnt = 0 To additivescount - 1

    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Albumin") Then
        node_albumin = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Ascorbic") Then
        node_ascorbic = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Chromium") Then
        node_chromium = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Cysteine + Acetate") Then
        node_cysteineplus = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Cysteine - No Acetate") Then
        node_cysteineminus = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Epogen") Then
        node_epogen = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Famotidine") Then
        node_famotidine = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Fervens Canis (aquaeous)") Then
        node_hotdogwater = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Folic") Then
        node_folic = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Heparin") Then
        node_heparin = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Insulin") Then
        node_insulin = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Iron") Then
        node_iron = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Levocarnitine") Then
        node_levocarnitine = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Lipids (3-In-1)") Then
        node_lipids3in1 = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("MVI - Adult") Then
        node_mviadult = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("MVI Pediatric") Then
        node_mviped = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Metoclopramide") Then
        node_metoclopramide = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Ped-Trace") Then
        node_pedtrace4 = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Phytonadione-K1") Then
        node_phytonadione = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Ranitidine") Then
        node_ranitidine = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Selenium") Then
        If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.IndexOf(">") > 0 Then
            node_selenium5 = Me.treeIngredients.Nodes(4).Nodes(cnt)
        Else
            node_selenium = Me.treeIngredients.Nodes(4).Nodes(cnt)
        End If
    End If
    If Me.treeIngredients.Nodes(4).Nodes(cnt).Text.StartsWith("Zinc") Then
        node_zinc = Me.treeIngredients.Nodes(4).Nodes(cnt)
    End If

Next
    For cnt = 0 To additivescount - 1
        Me.treeIngredients.Nodes(4).Nodes(0).Remove()
    Next

If Not node_zinc Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_zinc)
If Not node_selenium5 Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_selenium5)
If Not node_selenium Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_selenium)
If Not node_ranitidine Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_ranitidine)
If Not node_phytonadione Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_phytonadione)
If Not node_pedtrace4 Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_pedtrace4)
If Not node_metoclopramide Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_metoclopramide)
If Not node_mviped Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_mviped)
If Not node_mviadult Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_mviadult)
If Not node_lipids3in1 Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_lipids3in1)
If Not node_levocarnitine Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_levocarnitine)
If Not node_iron Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_iron)
If Not node_insulin Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_insulin)
If Not node_heparin Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_heparin)
If Not node_folic Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_folic)
If Not node_hotdogwater Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_hotdogwater)
If Not node_famotidine Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_famotidine)
If Not node_epogen Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_epogen)
If Not node_cysteineminus Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_cysteineminus)
If Not node_cysteineplus Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_cysteineplus)
If Not node_chromium Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_chromium)
If Not node_ascorbic Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_ascorbic)
If Not node_albumin Is Nothing Then Me.treeIngredients.Nodes(4).Nodes.Insert(0, node_albumin)

The comment does us the courtesy of telling us what the goal of this code is: sort the list of possible additives.

Understanding how it accomplishes that goal is a bit trickier, and to do that, we need to understand something about old-timey VisualBasic. Prior to .NET, VisualBasic was a terrible language, and did not have a particularly rich set of collection primitives, and the API wasn’t that clear. Many VB programmers didn’t even bother to learn about them, since VisualBasic was all about slapping form controls together on a designer and then wiring code up to them. This meant that, instead of fighting their way through collection types, many VB developers used the hammer they were already familiar with to solve problems. Developers that learned this habit when it was common in, say, 1998, often continued in this habit until, say, 2008, when this code was written.

With that in mind, Me.treeIngredients is a TreeView control, meant to be used like well, a tree-view. Each variable declaration (the Dim statements) is creating a new TreeNode, which are the individual branches and leaves we want to see in the tree. This particular set of nodes is never actually used in a display (there’s a separate TreeView for that).

So, section by section, let’s look at what this does. First, it creates a pile of TreeNode objects. Then, for each element currently in the TreeView, we run a loop and check: if it starts with, say, “Metoclopramide”, grab that node and put it in the variable node_metoclopramide. Then loop across the TreeView again, but this time, remove all the nodes. Finally, look at that set of variables you just created, and if each one of them has a value, insert it into the first position in the list. So long as we do those inserts in reverse alphabetical order, the end result will be that items are added to the list in alphabetical order.

As you can imagine, this wasn’t written as a reusable function, but instead, was copied and pasted everywhere additives needed to be sorted. This kept Brenda very busy when she needed to add a few dozen new additives to the application.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

Planet Linux AustraliaSteven Hanley: [various] Safety Sewing


No reflections (fullsize)

None outside either (fullsize)

Better when full/open (fullsize)

Also better when closed, much brightness (fullsize)
For over a year I have been planning to do this, my crumpler bag (the complete seed) which I bought in 2008 has been my primary commuting and daily use bag for stuff since that time and as much as I love the bag there is one major problem. No reflective marking anywhere on the bag.

Some newer crumplers have reflective strips and other such features and if I really wanted to spend big I could get them to do a custom bag with whatever colours and reflective bits I can dream up. There are also a number of other brands that do a courier bag with reflective bits or even entire panels or similar that are reflective. However this is the bag I own and it is still perfectly good for daily use so no need to go buy something new.

So I got a $4 sewing kit I had sitting around in the house, some great 3M reflective tape material and finally spent the time to rectify this feature missing from the bag. After breaking 3 needles and spending a while getting it done I now have a much safer bag especially commuting home on these dark winter nights. The sewing work is a bit messy however it is functional which is all that matters to me.

Planet Linux AustraliaSteven Hanley: [various] Vote Greens Maybe

I have had the parodies of the Call me maybe song in my head again today (the Orica Green edge one was brilliant and there are some inspired versions out there). This had me thinking of different lyrics, maybe something to suggest people vote Green this Saturday for a better and fairer Australia.

Vote Green Maybe
I threw a wish in the well
For a better Australia today
I looked at our leaders today
And now they're in our way

I'll not trade my freedom for them
All our dollars and cents to the rich
I wasn't looking for this
But now they're in our way

Our democracy is squandered
Broken promises
Lies everywhere
Hot nights
Winds are blowing
Freak weather events, climate change

Hey I get to vote soon
And this isn't crazy
But here's my idea
So vote Greens maybe
It's hard to look at our future 
But here's my idea
So vote Greens maybe

Hey I get to vote soon
And this isn't crazy
But here's my idea
So vote Greens maybe
And all the major parties
Try to shut us up
But here's my idea
So vote Greens maybe

Liberal and Labor think they should rule
I take no time saying they fail
They gave us nothing at all
And now they're in our way

I beg for a fairer Australia
At first sight our policies are real
I didn't know if you read them
But it's the Greens way  

Your vote can fix things
Healthier people
Childrens education
Fairer policies
A change is coming
Where you think you're voting, Greens?

Hey I get to vote soon
And this isn't crazy
But here's my idea
So vote Greens maybe
It's worth a look to a brighter future
But here's my idea
So vote Greens maybe

Before this change in our lives
I see children in detention
I see humans fleeing horrors
I see them locked up and mistreated
Before this change in our lives
I see a way to fix this
And you should know that
Voting Green can help fix this, Green, Green, Green...

It's bright to look at our future 
But here's my idea
So vote Greens maybe

Hey I get to vote soon
And this isn't crazy
But here's my idea
So vote Greens maybe
And all the major parties
Try to shut us up
But here's my idea
So vote Greens maybe

Before this change in our lives
I see children in detention
I see humans fleeing horrors
I see them locked up and mistreated
Before this change in our lives
I see a way to fix this
And you should know that
So vote Green Saturday
Call Me Maybe (Carly Rae Jepsen)
I threw a wish in the well
Don't ask me I'll never tell
I looked at you as it fell
And now you're in my way

I trade my soul for a wish
Pennies and dimes for a kiss
I wasn't looking for this
But now you're in my way

Your stare was holding
Ripped jeans
Skin was showing
Hot night
Wind was blowing
Where you think you're going baby?

Hey I just met you
And this is crazy
But here's my number
So call me maybe
It's hard to look right at you baby
But here's my number
So call me maybe

Hey I just met you
And this is crazy
But here's my number
So call me maybe
And all the other boys
Try to chase me
But here's my number
So call me maybe

You took your time with the call
I took no time with the fall
You gave me nothing at all
But still you're in my way

I beg and borrow and steal
At first sight and it's real
I didn't know I would feel it
But it's in my way

Your stare was holding
Ripped jeans
Skin was showing
Hot night
Wind was blowing
Where you think you're going baby?

Hey I just met you
And this is crazy
But here's my number
So call me maybe
It's hard to look right at you baby
But here's my number
So call me maybe

Before you came into my life
I missed you so bad
I missed you so bad
I missed you so so bad
Before you came into my life
I missed you so bad
And you should know that
I missed you so so bad, bad, bad, bad....

It's hard to look right at you baby
But here's my number
So call me maybe

Hey I just met you
And this is crazy
But here's my number
So call me maybe
And all the other boys
Try to chase me
But here's my number
So call me maybe

Before you came into my life
I missed you so bad
I missed you so bad
I missed you so so bad
Before you came into my life
I missed you so bad
And you should know that
So call me, maybe

Planet Linux AustraliaSteven Hanley: [mtb] The lots of vert lunch run, reasons to live in Canberra


Great view of the lake from the single track on the steep side of BM (fullsize)
This run that is so easy to get out for at lunch is a great quality climbing session and shows off canberra beautifully. What fun.

Photos and some words are online on my Lots of vert lunch run page.

Planet Linux AustraliaSteven Hanley: [mtb/events] Geoquest 2016 - Port Mac again with Resultz


My Mirage 730 - Matilda, having a rest while we ran around (fullsize)
I have fun at Goequest and love doing the event however have been a bit iffy about trying to organise a team for a few years. As many say one of the hardest things in the event is getting 4 people to the start line ready to go.

This year my attitude was similar to last, if I was asked to join a team I would probably say yes. I was asked and thus ended up racing with a bunch of fun guys under the banner of Michael's company Resultz Racing. Another great weekend on the mid north NSW coast with some amazing scenery (the two rogaines were highlights, especially the punchbowl waterfall on the second one).

My words and photos are online in my Geoquest 2016 gallery. Always good fun and a nice escape from winter.

Planet Linux AustraliaSteven Hanley: [mtb/events] Razorback Ultra - Spectacular run in the Victorian Alps


Alex and another Canberran on the Razorback (fullsize)
Alex and I signed up for the Razorback Ultra because it is in an amazing part of the country and sounded like a fun event to go do. I was heading into it a week after Six Foot, however this is all just training for UTA100 so why not. All I can say is every trail runner should do this event, it is amazing.

The atmosphere at the race is laid back and it is all about heading up into the mountains and enjoying yourself. I will be back for sure.

My words and photos are online in my Razorback Ultra 2016 gallery. This is truly one of the best runs in Australia.

,

Planet Linux AustraliaOpenSTEM: The rise and fall of the Gopher protocol | MinnPost

Twenty-five years ago, a small band of programmers from the University of Minnesota ruled the internet. And then they didn’t.

The committee meeting where the team first presented the Gopher protocol was a disaster, “literally the worst meeting I’ve ever seen,” says Alberti. “I still remember a woman in pumps jumping up and down and shouting, ‘You can’t do that!’ ”

Among the team’s offenses: Gopher didn’t use a mainframe computer and its server-client setup empowered anyone with a PC, not a central authority. While it did everything the U (University of Minnesota) required and then some, to the committee it felt like a middle finger. “You’re not supposed to have written this!” Alberti says of the group’s reaction. “This is some lark, never do this again!” The Gopher team was forbidden from further work on the protocol.

Read the full article (a good story of Gopher and WWW history!) at https://www.minnpost.com/business/2016/08/rise-and-fall-gopher-protocol

Planet Linux AustraliaCraige McWhirter: Have It Your Way: Maximizing Drive-Thru Contributions - PyConAu 2016

by VM (Vicky) Brasseur.

Slides.

Vicky talked about the importance non-committing contributors but the primary focus is on committing contributors due to time limits.

Covered the different types of drive-thru contributors and why they show up.

  • Scratching an itch.
  • Unwilling / Unable to find an alternative to this project
  • They like you.

Why do they leave?

  • Itch has been sratched.
  • Not enough time.
  • No longer using the project.
  • Often a high barrier to contribution.
  • Absence of appreciation.
  • Unpleasant people.
  • Inappropriate attribution.

Disadvantages

  • It takes more time to help them land patches
    • Reluctance to help them "as they're not community".

It appears to be that many project see community as the foundation but Vicky contended it is contributors.

More drive-thru contributors are a sign of a healthy project and can lead to a larger community.

Advantages:

  • Have better processes in place.
  • Faster patch and release times.
  • More eyes and shallower bugs
  • Better community, code and project reputation.

Leads to a healthier overall project.

Methods for Maxmising drive-thru contributions:

Documentation!

  • give your project super powers.
  • Scales!
  • Ensures efficient and successful contributions.
  • Minimises questions.
  • Standardises processes.
  • Vicky provided a documentation quick start guide.

Mentoring!

  • Code review.
  • "Office hours" for communication.
  • Hackfests.
  • New contributor events.

Process improvements!

  • Tag starter bugs
  • Contributor SLA
  • Use containers / VM of dev environment

Culture!

  • Value contributions and contributors
  • Culture of documentation
  • Default to assistance

Outreach! * Gratitude * Recognition * Follow-up!

Institute the "No Asshole" rule.

PyConAu 2016

Planet Linux AustraliaCraige McWhirter: Keynote - Python All the Things - PyConAu 2016

by Russell Keith-Magee.

Keith spoke about porting Python to mobile devices. CPython being written in C enables it to leverage the supported platforms of the C language and be compiled a wide range of platforms.

There was a deep dive in the options and pitfalls when selecting a method to and implementing Python on Android phones.

Ouroboros is a pure Python implementation of the Python standard library.

Most of the tools discussed are at an early stage of development.

Why?

  • Being able to run on new or mobile platforms addresses an existential threat.
  • The threat also presents an opportunity to grown, broaden and improve Python.
  • Wants Python to be a "first contact" language, like (Visual) Basic once was.
  • Unlike Basic, Python also support very complex concepts and operations.
  • Presents an opportunity to encourage broader usage by otherwise passive users.
  • Technical superiority is rarely enough to guarantee success.
  • A breadth of technical domains is required for Python to become this choice.
  • Technical problems are the easiest to solve.
  • Te most difficult problems are social and community and require more attention.

Keith's will be putting his focus into BeeWare and related projects.

Fortune favours the prepared mind

(Louis Pasteur)

PyConAu 2016

,

Krebs on SecurityVisa Alert and Update on the Oracle Breach

Credit card industry giant Visa on Friday issued a security alert warning companies using point-of-sale devices made by Oracle‘s MICROS retail unit to double-check the machines for malicious software or unusual network activity, and to change passwords on the devices. Visa also published a list of Internet addresses that may have been involved in the Oracle breach and are thought to be closely tied to an Eastern European organized cybercrime gang.

VSA-oracle

The Visa alert is the first substantive document that tries to help explain what malware and which malefactors might have hit Oracle — and by extension many of Oracle’s customers — since KrebsOnSecurity broke news of the breach on Aug. 8. That story cited sources close to the investigation saying hackers had broken into hundreds of servers at Oracle’s retail division, and had completely compromised Oracle’s main online support portal for MICROS customers.

MICROS is among the top three point-of-sale vendors globally. Oracle’s MICROS division sells point-of-sale systems used at more than 330,000 cash registers worldwide. When Oracle bought MICROS in 2014, the company said MICROS’s systems were deployed at some 200,000+ food and beverage outlets, 100,000+ retail sites, and more than 30,000 hotels.

In short, tens of millions of credit cards are swiped at MICROS terminals monthly, and a breach involving the theft of credentials that might have granted remote access to even just a small percentage of those systems is potentially a big and costly problem for all involved.

So far, however, most MICROS customers are left scratching their heads for answers. A frequently asked questions bulletin (PDF) Oracle also released last Monday held little useful information. Oracle issued the same cryptic response to everyone who asked for particulars about how far the breach extended. “Oracle has detected and addressed malicious code in certain legacy MICROS systems.”

Oracle also urged MICROS customers to change their passwords, and said “we also recommend that you change the password for any account that was used by a MICROS representative to access your on-premises systems.”

One of two documents Oracle sent to MICROS customers and the sum total of information the company has released so far about the breach.

One of two documents Oracle sent to MICROS customers and the sum total of information the company has released so far about the breach.

Some technology and fraud experts, including Gartner Analyst Avivah Litan, read that statement highlighted in yellow above as an acknowledgement by Oracle that hackers may have abused credentials gained in the MICROS portal breach to plant malicious code on the point-of-sale devices run by an unknown number of MICROS customers.

“This [incident] could explain a lot about the source of some of these retail and merchant point-of-sale hacks that nobody has been able to definitively tie to any one point-of-sale services provider,” Litan told me last week. “I’d say there’s a big chance that the hackers in this case found a way to get remote access” to MICROS customers’ on-premises point-of-sale devices.”

Clearly, Visa is concerned about this possibility as well.

INDICATORS OF COMPROMISE

In my original story about the breach, I wasn’t able to reveal all the data I’d gathered about the apparent source of the attacks and attackers. A key source in that story asked that I temporarily delay publishing certain details of the investigation, specifically those known as indicators of compromise (IOCs). Basically, IOCs are list of suspect Internet addresses, domain names, filenames and other curious digital clues that are thought to connect the victim with its attacker.

I’ve been inundated all week with calls and emails from security experts asking for that very data, but sharing it wasn’t my call. That is, until yesterday (8/12/16), when Visa published a “merchant communication alert” to some customers. In that alert (PDF), Visa published IOCs that may be connected with the intrusion. These IOCs could be extremely useful to MICROS customers because the presence of Internet traffic to and from these online destinations would strongly suggest the organization’s point-of-sale systems may be similarly compromised.

Some of the addresses on this list from Visa are known to be associated with the Carbanak Gang, a group of Eastern European hackers that Russian security firm Kaspersky Lab estimates has stolen more than $1 billion from banks and retailers. Here’s the IOCs list from the alert Visa pushed out Friday:

VISA warned merchants to check their systems for any communications to and from these Internet addresses and domain names associated with a Russian organized cybercrime gang called "Carbanak."

Visa warned merchants to check their systems for any communications to and from these Internet addresses and domain names associated with a Russian organized cybercrime gang called “Carbanak.”

Thankfully, since at least one of the addresses listed above (192.169.82.86) matched what’s on my source’s list, the source agreed to let me publish the entire thing. Here it is. I checked my source’s list and found at least five Internet addresses that were seen in both the Oracle attack and in a Sept. 2015 writeup about Carbanak by ESET Security, a Slovakian antivirus and security company. [NB: If you are unskilled at safely visiting malicious Web sites and/or handling malware, it’s probably best not to visit the addresses in the above-linked list.]

Visa also mentioned a specific POS-malware threat in its alert called “MalumPOS.” According to researchers at Trend Micro, MalumPOS is malware designed to target point-of-sale systems in hotels and related industries. In fact, Trend found that MalumPOS is set up to collect data specifically from point-of-sale systems running on Oracle’s MICROS platform.

It should come as no surprise then that many of Oracle’s biggest customers in the hospitality industry are starting to make noise, accusing Oracle of holding back key information that could help MICROS-based companies stop and clean up breaches involving malware and stolen customer credit card data.

“Oracle’s silence has been deafening,” said Michael Blake, chief executive officer at HTNG, a trade association for hotels and technology. “They are still grappling and trying to answer questions on the extent of the breach. Oracle has been invited to the last three [industry] calls this week and they are still going about trying to reach each customer individually and in the process of doing so they have done nothing but given the lame advice of changing passwords.”

The hospitality industry has been particularly hard hit by point-of-sale compromises over the past two years. Last month, KrebsOnSecurity broke the news of a breach at Kimpton Hotels (Kimpton appears to run MICROS products, but the company declined to answer questions for this story).

Kimpton joins a long list of hotel brands that have acknowledged card breaches over the last year, including Trump Hotels (twice), Hilton, Mandarin Oriental, and White Lodging (twice), Starwood Hotels and Hyatt. In many of those incidents, thieves had planted malicious software on the point-of-sale devices at restaurants and bars inside of the hotel chains. And, no doubt, many of those cash registers were run on MICROS systems.

If Oracle doesn’t exactly know which — if any — of its MICROS customers had malware on their point-of-sale systems as a result of the breach, it may be because the network intruders didn’t have any reason to interact with Oracle’s customers via the MICROS portal after stealing usernames and passwords that would allow them to remotely access customer on-premises systems. In theory, at that point the fraudsters could have bypassed Oracle altogether from then on.

BREACHED BY MULTIPLE ACTORS?

Another possibly interesting development in the Oracle breach story: There are indications that Oracle may have been breached by more than one cybercrime group. Or at least handed off from one to the other.

Late this week, Thomas Fox-Brewster at Forbes published a story noting that MICROS was just one of at least five point-of-sale companies that were recently hacked by a guy who — from an exhaustive review of his online chats — appears to have just sat himself down one day and decided to hack a bunch of point-of-sale companies.

Forbes quoted my old friend Alex Holden of Hold Security saying he had evidence that hackers had breached at least 10 payment companies, and the story focuses on getting confirmation from the various other providers apparently breached by the same cybercriminal actor.

Holden showed me multiple pages worth of chat logs between two individuals on a cybercrime forum [full disclosure: Holden’s company lists me as an adviser, but I accept no compensation for that role, and he ignores most of my advice].

The discussion between the two hackers begins around July 15, 2016, and goes on for more than a week. In it, the two hackers have been introduced to one another through a mutual, trusted contact. For a while, all they discuss is whether the seller can be trusted to deliver the Oracle MICROS database and control over the Oracle MICROS customer ticketing portal.

In the end, the buyer is convinced by what he sees and agrees to pay the bitcoin equivalent of roughly USD $13,000 for access to Oracle’s MICROS portal, as well as a handful of other point-of-sale Web sites. The buyer’s bitcoin wallet and the associated transactions can be seen here.

A screen shot shared by one of the hackers involved in compromising Oracle's MICROS support portal. This screen shot was taken of a similar Web shell the hackers placed on the Web site of another POS provider (this is not the shell that was on Oracle).

A screen shot shared by one of the hackers involved in compromising Oracle’s MICROS support portal. This screen shot was taken of a similar Web shell the hackers placed on the Web site of another POS provider (this is not the shell that was on Oracle).

According to the chat log, the hacker broke in by exploiting a file-upload function built into the MICROS customer support portal. From there the attackers were able to upload an attack tool known as a “WSO Web Shell.” This is a crude but effective text-based control panel that helps the attacker install additional attack tools to harvest data from the compromised Web server (see screen shot above). The beauty of a Web shell is that the attacker can control the infected site using nothing more than a Web browser, using nothing more than a hidden login page and a password that only he knows.

The two hackers discussed and both viewed more than a half-dozen files that were apparently left behind on the MICROS portal by the WSO shell they uploaded in mid-July (most of the malicious files ended in the file extension “wso.aspx”). The chat logs show the pair of miscreants proceeding to target another 9 online payment providers or point-of-sale vendors.

Some of those companies were quoted in the Forbes piece having acknowledged a breach similar to the Web shell attack at Oracle. But none of them have anywhere near the size of Oracle’s MICROS customer base.

GOOD HOSPITALITY, OR SWEPT UNDER THE RUG?

Oracle maintains in its FAQ (PDF) about the MICROS attack that “Oracle’s Corporate network and Oracle’s other cloud and service offerings were not impacted.” But a confidential source within Oracle’s Hospitality Division told KrebsOnSecurity that the breach first started in one of Oracle’s major point-of-sale data centers — specifically the company’s large data center in Manassas, Va.

According to my source, that particular center helps large Oracle hospitality industry clients manage their fleets of MICROS point-of-sale devices.

“Initially, the customer’s network and the internal Oracle network were on the same network,” said my source, who spoke under condition of anonymity because he did not have permission from his employer to speak on the record. “The networking team did a network segmentation of these two networks — ironically for security purposes. However, it seems as if what they have done actually allowed access from the Russian Cybercrime group.”

My source said that in mid-July 2016 Oracle sent out an email alert to employees of its hospitality division that they had to re-image their laptops without backing anything up.

“All of the files and software that were on an employee’s computer were deleted, which was crippling to business operations,” my source recalled. “Project management lost all their schedules, deployment teams lost all the software that they use to install on customer sites. Oracle did not tell the employees in this email that they got hacked but just to re-image everything with no backups. It seems as if Oracle did a pretty good job sweeping this incident under the rug. Most employees don’t know about the hack and it hasn’t been a huge deal to the customers. However, it is estimated that this cost them billions, so it is a really major breach.”

I sent Oracle a litany of questions based on the above, but a spokesperson for the company said Oracle would comment on none of it.

Planet Linux AustraliaRussell Coker: SSD and M.2

The Need for Speed

One of my clients has an important server running ZFS. They need to have a filesystem that detects corruption, while regular RAID is good for the case where a disk gives read errors it doesn’t cover the case where a disk returns bad data and claims it to be good (which I’ve witnessed in BTRFS and ZFS systems). BTRFS is good for the case of a single disk or a RAID-1 array but I believe that the RAID-5 code for BTRFS is not sufficiently tested for business use. ZFS doesn’t perform very well due to the checksums on data and metadata requiring multiple writes for a single change which also causes more fragmentation. This isn’t a criticism of ZFS, it’s just an engineering trade-off for the data integrity features.

ZFS supports read-caching on a SSD (the L2ARC) and write-back caching (ZIL). To get the best benefit of L2ARC and ZIL you need fast SSD storage. So now with my client investigating 10 gigabit Ethernet I have to investigate SSD.

For some time SSDs have been in the same price range as hard drives, starting at prices well below $100. Now there are some SSDs on sale for as little as $50. One issue with SATA for server use is that SATA 3.0 (which was released in 2009 and is most commonly used nowadays) is limited to 600MB/s. That isn’t nearly adequate if you want to serve files over 10 gigabit Ethernet. SATA 3.2 was released in 2013 and supports 1969MB/s but I doubt that there’s much hardware supporting that. See the SATA Wikipedia page for more information.

Another problem with SATA is getting the devices physically installed. My client has a new Dell server that has plenty of spare PCIe slots but no spare SATA connectors or SATA power connectors. I could have removed the DVD drive (as I did for some tests before deploying the server) but that’s ugly and only gives 1 device while you need 2 devices in a RAID-1 configuration for ZIL.

M.2

M.2 is a new standard for expansion cards, it supports SATA and PCIe interfaces (and USB but that isn’t useful at this time). The wikipedia page for M.2 is interesting to read for background knowledge but isn’t helpful if you are about to buy hardware.

The first M.2 card I bought had a SATA interface, then I was unable to find a local company that could sell a SATA M.2 host adapter. So I bought a M.2 to SATA adapter which made it work like a regular 2.5″ SATA device. That’s working well in one of my home PCs but isn’t what I wanted. Apparently systems that have a M.2 socket on the motherboard will usually take either SATA or NVMe devices.

The most important thing I learned is to buy the SSD storage device and the host adapter from the same place then you are entitled to a refund if they don’t work together.

The alternative to the SATA (AHCI) interface on an M.2 device is known as NVMe (Non-Volatile Memory Express), see the Wikipedia page for NVMe for details. NVMe not only gives a higher throughput but it gives more command queues and more commands per queue which should give significant performance benefits for a device with multiple banks of NVRAM. This is what you want for server use.

Eventually I got a M.2 NVMe device and a PCIe card for it. A quick test showed sustained transfer speeds of around 1500MB/s which should permit saturating a 10 gigabit Ethernet link in some situations.

One annoyance is that the M.2 devices have a different naming convention to regular hard drives. I have devices /dev/nvme0n1 and /dev/nvme1n1, apparently that is to support multiple storage devices on one NVMe interface. Partitions have device names like /dev/nvme0n1p1 and /dev/nvme0n1p2.

Power Use

I recently upgraded my Thinkpad T420 from a 320G hard drive to a 500G SSD which made it faster but also surprisingly quieter – you never realise how noisy hard drives are until they go away. My laptop seemed to feel cooler, but that might be my imagination.

The i5-2520M CPU in my Thinkpad has a TDP of 35W but uses a lot less than that as I almost never have 4 cores in use. The z7k320 320G hard drive is listed as having 0.8W “low power idle” and 1.8W for read-write, maybe Linux wasn’t putting it in the “low power idle” mode. The Samsung 500G 850 EVO SSD is listed as taking 0.4W when idle and up to 3.5W when active (which would not be sustained for long on a laptop). If my CPU is taking an average of 10W then replacing the hard drive with a SSD might have reduced the power use of the non-screen part by 10%, but I doubt that I could notice such a small difference.

I’ve read some articles about power use on the net which can be summarised as “SSDs can draw more power than laptop hard drives but if you do the same amount of work then the SSD will be idle most of the time and not use much power”.

I wonder if the SSD being slightly thicker than the HDD it replaced has affected the airflow inside my Thinkpad.

From reading some of the reviews it seems that there are M.2 storage devices drawing over 7W! That’s going to create some cooling issues on desktop PCs but should be OK in a server. For laptop use they will hopefully release M.2 devices designed for low power consumption.

The Future

M.2 is an ideal format for laptops due to being much smaller and lighter than 2.5″ SSDs. Spinning media doesn’t belong in a modern laptop and using a SATA SSD is an ugly hack when compared to M.2 support on the motherboard.

Intel has released the X99 chipset with M.2 support (see the Wikipedia page for Intel X99) so it should be commonly available on desktops in the near future. For most desktop systems an M.2 device would provide all the storage that is needed (or 2*M.2 in a RAID-1 configuration for a workstation). That would give all the benefits of reduced noise and increased performance that regular SSDs provide, but with better performance and fewer cables inside the PC.

For a corporate desktop PC I think the ideal design would have only M.2 internal storage and no support for 3.5″ disks or a DVD drive. That would allow a design that is much smaller than a current SFF PC.

Planet Linux AustraliaChris Samuel: Playing with Shifter Part 2 – converted Docker containers inside Slurm

This is continuing on from my previous blog about NERSC’s Shifter which lets you safely use Docker containers in an HPC environment.

Getting Shifter to work in Slurm is pretty easy, it includes a plugin that you must install and tell Slurm about. My test config was just:

required /usr/lib64/shifter/shifter_slurm.so shifter_config=/etc/shifter/udiRoot.conf

as I was installing by building RPMs (out preferred method is to install the plugin into our shared filesystem for the cluster so we don’t need to have it in the RAM disk of our diskless nodes). One that is done you can add the shifter programs arguments to your Slurm batch script and then just call shifter inside it to run a process, for instance:

#!/bin/bash

#SBATCH -p debug
#SBATCH --image=debian:wheezy

shifter cat /etc/issue

results in the following on our RHEL compute nodes:

[samuel@bruce Shifter]$ cat slurm-1734069.out 
Debian GNU/Linux 7 \n \l

simply demonstrating that it works. The advantage of using the plugin and this way of specifying the images is that the plugin will prep the container for us at the start of the batch job and keep it around until it ends so you can keep running commands in your script inside the container without the overhead of having to create/destroy it each time. If you need to run something in a different image you just pass the --image option to shifter and then it will need to set up & tear down that container, but the one you specified for your batch job is still there.

That’s great for single CPU jobs, but what about parallel applications? Well turns out that’s easy too – you just request the configuration you need and slap srun in front of the shifter command. You can even run MPI applications this way successfully. I grabbed the dispel4py/docker.openmpi Docker container with shifterimg pull dispel4py/docker.openmpi and tried its Python version of the MPI hello world program:

#!/bin/bash
#SBATCH -p debug
#SBATCH --image=dispel4py/docker.openmpi
#SBATCH --ntasks=3
#SBATCH --tasks-per-node=1

shifter cat /etc/issue

srun shifter python /home/tutorial/mpi4py_benchmarks/helloworld.py

This prints the MPI rank to demonstrate that the MPI wire up was successful and I forced it to run the tasks on separate nodes and print the hostnames to show it’s communicating over a network, not via shared memory on the same node. But the output bemused me a little:

[samuel@bruce Python]$ cat slurm-1734135.out
Ubuntu 14.04.4 LTS \n \l

libibverbs: Warning: couldn't open config directory '/etc/libibverbs.d'.
libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0
--------------------------------------------------------------------------
[[30199,2],0]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
  Host: bruce001

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
libibverbs: Warning: couldn't open config directory '/etc/libibverbs.d'.
libibverbs: Warning: couldn't open config directory '/etc/libibverbs.d'.
Hello, World! I am process 0 of 3 on bruce001.
libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0
--------------------------------------------------------------------------
[[30199,2],1]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
  Host: bruce002

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
Hello, World! I am process 1 of 3 on bruce002.
libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0
--------------------------------------------------------------------------
[[30199,2],2]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
  Host: bruce003

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
Hello, World! I am process 2 of 3 on bruce003.

It successfully demonstrates that it is using an Ubuntu container on 3 nodes, but the warnings are triggered because Open-MPI in Ubuntu is built with Infiniband support and it is detecting the presence of the IB cards on the host nodes. This is because Shifter is (as designed) exposing the systems /sys directory to the container. The problem is that this container doesn’t include the Mellanox user-space library needed to make use of the IB cards and so you get warnings that they aren’t working and that it will fall back to a different mechanism (in this case TCP/IP over gigabit Ethernet).

Open-MPI allows you to specify what transports to use, so adding one line to my batch script:

export OMPI_MCA_btl=tcp,self,sm

cleans up the output a lot:

Ubuntu 14.04.4 LTS \n \l

Hello, World! I am process 0 of 3 on bruce001.
Hello, World! I am process 2 of 3 on bruce003.
Hello, World! I am process 1 of 3 on bruce002.

This also begs the question then – what does this do for latency? The image contains a Python version of the OSU latency testing program which uses different message sizes between 2 MPI ranks to provide a histogram of performance. Running this over TCP/IP is trivial with the dispel4py/docker.openmpi container, but of course it’s lacking the Mellanox library I need and as the whole point of Shifter is security I can’t get root access inside the container to install the package. Fortunately the author of the dispel4py/docker.openmpi has their implementation published on Github and so I forked their repo, signed up for Docker and pushed a version which simply adds the libmlx4-1 package I needed.

Running the test over TCP/IP is simply a matter of submitting this batch script which forces it onto 2 separate nodes:

#!/bin/bash
#SBATCH -p debug
#SBATCH --image=chrissamuel/docker.openmpi:latest
#SBATCH --ntasks=2
#SBATCH --tasks-per-node=1

export OMPI_MCA_btl=tcp,self,sm

srun shifter python /home/tutorial/mpi4py_benchmarks/osu_latency.py

giving these latency results:

[samuel@bruce MPI]$ cat slurm-1734137.out
# MPI Latency Test
# Size [B]        Latency [us]
0                        16.19
1                        16.47
2                        16.48
4                        16.55
8                        16.61
16                       16.65
32                       16.80
64                       17.19
128                      17.90
256                      19.28
512                      22.04
1024                     27.36
2048                     64.47
4096                    117.28
8192                    120.06
16384                   145.21
32768                   215.76
65536                   465.22
131072                  926.08
262144                 1509.51
524288                 2563.54
1048576                5081.11
2097152                9604.10
4194304               18651.98

To run that same test over Infiniband I just modified the export in the batch script to force it to use IB (and thus fail if it couldn’t talk between the two nodes):

#!/bin/bash
#SBATCH -p debug
#SBATCH --image=chrissamuel/docker.openmpi:latest
#SBATCH --ntasks=2
#SBATCH --tasks-per-node=1

export OMPI_MCA_btl=openib,self,sm

srun shifter python /home/tutorial/mpi4py_benchmarks/osu_latency.py

which then gave these latency numbers:

[samuel@bruce MPI]$ cat slurm-1734138.out
# MPI Latency Test
# Size [B]        Latency [us]
0                         2.52
1                         2.71
2                         2.72
4                         2.72
8                         2.74
16                        2.76
32                        2.73
64                        2.90
128                       4.03
256                       4.23
512                       4.53
1024                      5.11
2048                      6.30
4096                      7.29
8192                      9.43
16384                    19.73
32768                    29.15
65536                    49.08
131072                   75.19
262144                  123.94
524288                  218.21
1048576                 565.15
2097152                 811.88
4194304                1619.22

So you can see that’s basically an order of magnitude improvement in latency using Infiniband compared to TCP/IP over gigabit Ethernet (which is what you’d expect).

Because there’s no virtualisation going on here there is no extra penalty to pay when doing this, no need to configure any fancy device pass through, no loss of any CPU MSR access, and so I’d argue that Shifter makes Docker containers way more useful for HPC than virtualisation or even Docker itself for the majority of use cases.

Am I excited about Shifter – yup! The potential to allow users build and application stack themselves right down to the OS libraries and (with a little careful thought) having something that could get native interconnect performance is fantastic. Throw in the complexities of dealing with conflicting dependencies between Python modules, system libraries, bioinformatics tools, etc, etc, and needing to provide simple methods for handling these and the advantages seem clear.

So the plan is to roll this out into production at VLSCI in the near future. Fingers crossed! 🙂

This item originally posted here:

Playing with Shifter Part 2 – converted Docker containers inside Slurm

Planet Linux AustraliaStewart Smith: Microsoft Chicago – retro in qemu!

So, way back when (sometime in the early 1990s) there was Windows 3.11 and times were… for Workgroups. There was this Windows NT thing, this OS/2 thing and something brewing at Microsoft to attempt to make the PC less… well, bloody awful for a user.

Again, thanks to abandonware sites, it’s possible now to try out very early builds of Microsoft Chicago – what would become Windows 95. With the earliest build I could find (build 56), I set to work. The installer worked from an existing Windows 3.11 install.

I ended up using full system emulation rather than normal qemu later on, as things, well, booted in full emulation and didn’t otherwise (I was building from qemu master… so it could have actually been a bug fix).

chicago-launch-setupMmmm… Windows 3.11 File Manager, the fact that I can still use this is a testament to something, possibly too much time with Windows 3.11.

chicago-welcome-setupchicago-setupUnfortunately, I didn’t have the Plus Pack components (remember Microsoft Plus! ?- yes, the exclamation mark was part of the product, it was the 1990s.) and I’m not sure if they even would have existed back then (but the installer did ask).

chicago-install-dirObviously if you were testing Chicago, you probably did not want to upgrade your working Windows install if this was a computer you at all cared about. I installed into C:\CHICAGO because, well – how could I not!

chicago-installingThe installation went fairly quickly – after all, this isn’t a real 386 PC and it doesn’t have of-the-era disks – everything was likely just in the linux page cache.

chicago-install-networkI didn’t really try to get network going, it may not have been fully baked in this build, or maybe just not really baked in this copy of it, but the installer there looks a bit familiar, but not like the Windows 95 one – maybe more like NT 3.1/3.51 ?

But at the end… it installed and it was time to reboot into Chicago:
chicago-bootSo… this is what Windows 95 looked like during development back in July 1993 – nearly exactly two years before release. There’s some Windows logos that appear/disappear around the place, which are arguably much cooler than the eventual Windows 95 boot screen animation. The first boot experience was kind of interesting too:
Screenshot from 2016-08-07 20-57-00Luckily, there was nothing restricting the beta site ID or anything. I just entered the number 1, and was then told it needed to be 6 digits – so beta site ID 123456 it is! The desktop is obviously different both from Windows 3.x and what ended up in Windows 95.

Screenshot from 2016-08-07 20-57-48Those who remember Windows 3.1 may remember Dr Watson as an actual thing you could run, but it was part of the whole diagnostics infrastructure in Windows, and here (as you can see), it runs by default. More odd is the “Switch To Chicago” task (which does nothing if opened) and “Tracker”. My guess is that the “Switch to Chicago” is the product of some internal thing for launching the new UI. I have no ideawhat the “Tracker” is, but I think I found a clue in the “Find File” app:

Screenshot from 2016-08-13 14-10-10Not only can you search with regular expressions, but there’s “Containing text”, could it be indexing? No, it totally isn’t. It’s all about tracking/reporting problems:

Screenshot from 2016-08-13 14-15-19Well, that wasn’t as exciting as I was hoping for (after all, weren’t there interesting database like file systems being researched at Microsoft in the early 1990s?). It’s about here I should show the obligatory About box:
Screenshot from 2016-08-07 20-58-10It’s… not polished, and there’s certainly that feel throughout the OS, it’s not yet polished – and two years from release: that’s likely fair enough. Speaking of not perfect:

Screenshot from 2016-08-07 20-59-03When something does crash, it asks you to describe what went wrong, i.e. provide a Clue for Dr. Watson:

Screenshot from 2016-08-13 12-09-22

But, most importantly, Solitaire is present! You can browse the Programs folder and head into Games and play it! One odd tihng is that applications have two >> at the end, and there’s a “Parent Folder” entry too.

Screenshot from 2016-08-13 12-01-24Solitair itself? Just as I remember.

Screenshot from 2016-08-07 21-21-27Notably, what is missing is anything like the Start menu, which is probably the key UI element introduced in Windows 95 that’s still with us today. Instead, you have this:

Screenshot from 2016-08-13 11-55-15That’s about the least exciting Windows menu possible. There’s the eye menu too, which is this:

Screenshot from 2016-08-13 11-56-12More unfinished things are found in the “File cabinet”, such as properties for anything:
Screenshot from 2016-08-13 12-02-00But let’s jump into Control Panels, which I managed to get to by heading to C:\CHICAGO\Control.sys – which isn’t exactly obvious, but I think you can find it through Programs as well.Screenshot from 2016-08-13 12-02-41Screenshot from 2016-08-13 12-05-40The “Window Metrics” application is really interesting! It’s obvious that the UI was not solidified yet, that there was a lot of experimenting to do. This application lets you change all sorts of things about the UI:

Screenshot from 2016-08-13 12-05-57My guess is that this was used a lot internally to twiddle things to see what worked well.

Another unfinished thing? That familiar Properties for My Computer, which is actually “Advanced System Features” in the control panel, and from the [Sample Information] at the bottom left, it looks like we may not be getting information about the machine it’s running on.

Screenshot from 2016-08-13 12-06-39

You do get some information in the System control panel, but a lot of it is unfinished. It seems as if Microsoft was experimenting with a few ways to express information and modify settings.

Screenshot from 2016-08-13 12-07-13But check out this awesome picture of a hard disk for Virtual Memory:

Screenshot from 2016-08-13 12-07-47The presence of the 386 Enhanced control panel shows how close this build still was to Windows 3.1:

Screenshot from 2016-08-13 12-08-08At the same time, we see hints of things going 32 bit – check out the fact that we have both Clock and Clock32! Notepad, in its transition to 32bit, even dropped the pad and is just Note32!

Screenshot from 2016-08-13 12-11-10Well, that’s enough for today, time to shut down the machine:
Screenshot from 2016-08-13 12-15-45

Planet Linux AustraliaCraige McWhirter: Python for science, side projects and stuff! - PyConAu 2016

By Andrew Lonsdale.

  • Talked about using python-ppt for collaborating on PowerPoint presentations.
  • Covered his journey so far and the lessons he learned.
  • Gave some great examples of re-creating XKCD comics in Python (matplotlib_venn).
  • Claimed the diversion into Python and Matplotlib has helped is actual research.
  • Spoke about how using Python is great for Scientific research.
  • Summarised that side projects are good for Science and Python.
  • Recommended Elegant SciPy
  • Demo's using Emoji to represent bioinformatics using FASTQE (FASTQ as Emoji).

PyConAu 2016

Planet Linux AustraliaCraige McWhirter: MicroPython: a journey from Kickstarter to Space by Damien George - PyConAu 2016

Damien George.

Motivations for MicroPython:

  • To provide a high level language to control sophisticated micro-controllers.
  • Approached it as an intellectually stimulating research problem.
  • Wasn't even sure it was possible.
  • Chose Python because:
    • It was a high level language with powerful features.
    • Large existing community.
    • Naively thought it would be easy.
    • Found Python easy to learn.
    • Shallow but long learning curve of python makes it good for beginners and advanced programmers.
    • Bitwise operaitons make it usefult for micro-controllers.

Why Not Use CPython?

  • CPython pre-allocates memory, resulting in inefficient memory usage which is problematic for low RAM devices like micro controllers.

Usage:

  • If you know Python, you know MicroPython - it's implemented the same

Kickstarter:

Damien covered his experiences with Kickstarter.

Internals of MicroPython:

  • Damien covered the parser, lexer, compiler and runtime.
  • Walked us through the workflows of the internals.
  • Spoke about object represntation and the three machine word object forms:
    • Integers.
    • Strings.
    • Objects.
  • Covered the emitters:
    • Bytecode.
    • Native (machine code).
    • Inline assembler.

Coding Style:

Coding was more based on a physicist trying to make things work, than a computer engineer.

  • There's a code dashboard
  • Hosted on GitHub
  • Noted that he could not have done this without the support of the community.

Hardware:

Listed some of the micro controller boards that it runs on ad larger computers that currently run OpenWRT.

Spoke about the BBC micron:bit project. Demo'd speech synthesis and image display running on it.

MicroPython in Space:

Spoke about the port to LEON / SPARC / RTEMS for the European Space agency for satellite control, particularly the application layer.

Damien closed with an overview of current applications and ongoing software and hardware development.

Links:

micropython.org forum.micropython.org github.com/micropython

PyConAu 2016

,

CryptogramCyberattacks via Submarine

Some minimal information about the NSA's abilities to hack networks via submarine.

Sociological ImagesEvolution, Complexity, and Human Mating Strategies

Flashback Friday.

I heard stories this week about dung beetles and cuttlefish.  Both made me think about the typical stories we hear in the media about evolved human mating strategies.  First, the stories:

—————————-

Story #1 :The Dung Beetle

Photo from flickr by Camilo Hdo.
Photo by Camilo Hdo, retrieved from flickr.

A story on Quirks and Quarks discussed the mating strategies of the dung beetle.  The picture above is of a male beetle; only the males have those giant horns.  He uses it to defend the entrance to a tiny burrow in which he keeps a female.  He’ll violently fight off other dung beetles who try to get access to the burrow.

So far this sounds like the typical story of competitive mating that we hear all the time about all kinds of animals, right?

There’s a twist: while only male dung beetles have horns, not all males have horns.  Some are completely hornless.  But if horns help you win the fight, how is hornlessness being passed down genetically?

Well, it turns out that when a big ol’ horned male is fighting with some other big ol’ horned male, little hornless males sneak into burrows and mate with the females.  They get discovered and booted out, of course, and the horned male will re-mate with the female with the hopes of displacing his sperm.

But.

Those little hornless males have giant testicles, way gianter than the horned males.  While the horned males are putting all of their energy into growing horns, the hornless males are making sperm.  So, even though they have limited access to females, they get as much mileage out of their access as they can.

The result: two distinct types of male dung beetles with two distinct mating strategies.

———————————

Story #2: The Giant Australian Cuttlefish

846693304_2d180b3fff_z
Photo by Paul Oughton, retrieved from Flickr.

The Naked Scientists podcast featured a story about Giant Australian Cuttlefish.  During mating season the male cuttlefish, much larger than the females, collect “harems” and spend their time mating and defending access.  Other males try to “muscle in,” but the bigger cuttlefish “throws his weight around” to scare him off. The biggest cuttlefish wins.

So far this sounds like the typical story of competitive mating that we hear all the time about all kinds of animals, right?

Well, according The Naked Scientists story, researchers have discovered an alternative mating strategy.  Small males, who are far too small to compete with large males, will pretend to be female, sneak into the defended territory, mate, and leave.

How do they do this?  They change their color pattern and rearrange their tentacles in a more typical female arrangement (they didn’t specify what this was) and, well, pass.  The large male thinks he’s another female. In the video below, the cuttlefish uses his ability to change the pattern on his body. He simultaneously displays a male pattern to the female and a female pattern to the large male on the other side.

———————————

So, can the crossdressing cuttlefish and dodge-y dung beetle tell us anything about evolved human mating strategies?

Probably not.

But I do think it tells us something about how we should think about evolution and the reproduction of genes. If you listen to the media cover evolutionary psychological explanations of human mating, you only hear one story about the strategies that males use to try to get sex. That story sounds a lot like the one told about the horned beetle and the large male cuttlefish.

But these species have demonstrated that there need not be only one mating strategy. In these cases, there are at least two. So, why in Darwin’s name would we assume that human beings, in all of their beautiful and incredible complexity, would only have one? Perhaps we see a diversity in types of human males (different body shapes and sizes, different intellectual gifts, etc) because there are many different ways to attract females. Maybe females see something valuable in many different kinds of males! Maybe not all females are the same!

Let’s set aside the stereotypes about men and women that media reporting on evolutionary psychology tends to reproduce and, instead, consider the possibility that human mating is at least as complex as that of dung beetles and cuttlefish.

Originally posted in 2010.

Lisa Wade, PhD is a professor at Occidental College. She is the author of American Hookup, a book about college sexual culture, and a textbook about gender. You can follow her on Twitter, Facebook, and Instagram.

(View original at https://thesocietypages.org/socimages)

TEDA TED Talk from a war zone

A view over the Old Square of Homs, looking towards the Old Souk, taken from the remains of Marwa and Ghassan's destroyed architecture studio. Photo: Marwa Al-Sabouni, 2016.

A view over the Old Square of Homs, looking towards the Old Souk, taken from the remains of Marwa and Ghassan’s destroyed architecture studio. Photo: Marwa Al-Sabouni, 2016.

At the TEDSummit in June, we featured a talk by a young Syrian architect, Marwa Al-Sabouni. In it, she shares an important and original insight about how the roots of conflict can be traced, among other better-studied reasons, to misdirected and divisive urbanism. She offers the example of her own country, where violent conflict has been raging and spreading for more than five years now, destabilizing the whole region and driving millions of refugees into the neighboring countries and, more recently, Europe.

Marwa herself, however, could not travel to our conference to give her talk, because she lives in Homs, a city in the central-western part of Syria. Traveling outwards isn’t easy, to say the least, and there is no guarantee of being able to travel back. And with her family, she’s determined to stay despite the dangers.

Homs is today a half-destroyed city. Reporters have equated it to Berlin after World War II. Before the war, the province’s population was nearly 2 million people; it is down by more than half now. “Almost everyone we knew has left,” says Marwa, who’s 35. With her husband Ghassan, 43, and their daughter Naya and son Ayk (11 and 8, respectively), she’s among those who have stayed. “We were lucky: our house is still standing,” she adds candidly. The small architecture studio she and Ghassan ran in the center of town before the war, however, is a ruin, only rubble surrounding what’s barely recognizable as a whiteboard.

Marwa Al-Sabouni photographed earlier this year in Homs with her husband, Ghassan, their daugther Naya and son Ayk.

Marwa Al-Sabouni photographed earlier this year in Homs with her husband, Ghassan, their daugther Naya and son Ayk.

To bring Marwa’s idea to TED, therefore, we resolved to record the talk over the Internet. Which meant dealing with unstable connectivity, electricity cuts and background noise. More on that below.

I discovered Marwa Al-Sabouni during one of my “reading storms” last Spring. I’ve curated the TEDGlobal conferences for 11 years, as well as many other TED events, and in the process of designing the speaker programs I’ve come to develop the habit of doing research in waves. While I work with a first group of speakers, I collect books and field notes and clippings from newspapers, journals, blogs and social media about other potential speakers. Something would pique my interest online or offline. I would visit a lab or meet a scientist or artist and take notes. A TED community member would point me to something intriguing. All goes into a folder that, after a while, may contain ideas for dozens of potential TED talks. When enough ideas have accumulated, I then go off for a few days and “storm” (for lack of a better word) that folder, reading voraciously through a wide variety of topics and researching them with my assistant Katerina Biliouri, establishing priorities and connections and imagining possible narratives. That’s how the speaker programs come together.

One of those articles was a profile of this young Syrian architect who had just published a book written during the war, while living in the middle of, called The Battle for Home. I picked it up and read it, and found, first, an amazing story of courage and resilience, of insight and hope. But the core of the book is a convincing study on how communities move apart, mixed urban fabrics turn into segregated islands, and living together morphs into sectarian hatred. The Syrian war has many roots — political, social, religious and economic. Marwa’s book highlights another cause that has been overlooked: the role played by decades of mismanaged architecture and divisive urban planning. A sentence from her talk summarizes her main argument: “From my point of view, losing the sense of belonging to a place and the sense of sharing it with someone else has made it a lot easier to destroy.”

Page after page, it became obvious that there was a potential TED Talk here. Not only for the provocative originality of her insights. Her reasoning also maps with similar developments in other parts of the world — reading some chapters of her book, it’s easy to think for instance of the marginalized suburbs, the banlieues, of Paris or Brussels. Furthermore, understanding how divisive urban planning can create divisions in society may be the only way to prevent this kind of bad planning from happening again.

While the book was fascinating, I didn’t know whether Marwa could tell the story in the form of a talk. A friend who’s published by the same house introduced me to her publisher, Thames & Hudson in London, who after a discussion put me in touch with her. We started emailing, then talking over video calls, and I learned to know a brilliant, optimistic, very articulate woman. She told me that life and economic activity have been slowly coming back to the least-destroyed parts of Homs since the beginning of the year. Ghassan, Marwa and their kids live on a second-floor apartment overlooking rows of small shops and workshops. When the war broke out, they were mostly relying on Internet cafés for their electronic communication, but the fighting took connectivity with it for long stretches of time, and power blackouts weren’t kind to the computers. Only a couple of years ago they got a friend to bring them a laptop from the UAE, and managed to set up a capricious connection at home.

While working on successive drafts of the talk, which focuses on architecture, the conversations were often about daily life during the war. “From 2012 to the beginning of 2015, fighting was very intense. Bombing and shelling were relentless. We almost learned to differentiate the weaponry just by their ‘tone’. There were snipers. At times even birds and cats had fled the city,” she told me. While their studio was right where most destruction happened, their home was in another neighborhood, a less-targeted one. “We got a few bullets inside the kitchen, broken windows from the shock of shelling. We spent most of the time indoors and never went out after dark. When the walls shook during the night, we called it a ‘noisy night.’ Garbage filled the streets. The cold winters were colder without heating — there was no fuel, and only a few hours of electricity which, like water and gas, was scarce and expensive. At a certain point, these essentials became the only things people talked about: how to find them, how to cope.”

“There were times when we had to bathe using a pot, and read using candles” until LED batteries arrived. People who were sick got sicker because hospitals weren’t functioning, “and the Red Crescent with its humble resources was the main help.” Ghassan and Marwa had no car, but those who did “struggled to find gas, and often it would be mixed with other substances, thus ruining the engines, for which there were no spare parts.” That statement applies more generally to almost any kind of goods. “If they existed,” she says, “the quality was very bad, the prices very high, and the one who had them generally had very bad manners.”

To pay those prices — to keep going during the worst period of the war — Ghassan scraped together some small income and his brother sent help from abroad, and the family drained their small savings.

In late 2015, things in Homs started getting gradually better, with a ceasefire agreed between the government and the rebel factions. “Until last year, clothes were mostly damaged or second-hand, people would buy a pair of torn pants and fix them. Now there are more goods, those who had stalls in the old city market now sell off shacks on the streets. The quality is still bad though.” There are now makeshift hospitals, schools and other facilities, mostly located in the residential buildings that are still standing.

The obvious question at this point is: have Ghassan and Marwa ever thought of leaving, like so many others? “No. We believe that staying was the right choice for our family,” she said, making me marvel at their bravery.

It is absolutely remarkable that Naya and Ayk managed to never miss a day of school despite the war, thanks to the presence of a small school very close to their home. They now attend a bigger one farther away, and “are both very good”, says their mother. Before the war, they were into art. Naya took violin and drawing lessons, Ayk preferred the piano, “but all had to stop because there were no more good teachers and no instruments”. Marwa has recently started teaching architectural design to second-year students at a private university in Hama, about 40 kilometers from Homs (the main road is now blocked and she has to take a much longer detour). Ghassan juggles several small jobs to support the family, including a tiny bookstore that he and Marwa opened recently, having found a way to procure books every few weeks from a handful of publishers, mostly in English, through a wholesaler in Damascus. “We’re making a little money from it, enough to keep it going, but our main goal is to make a small cultural contribution to restoring some normalcy in the town; people tell us that they see it as a sign of the worst being over,” she told me during one of our conversations.

A few weeks before the TEDSummit, with our video team in New York, across seven time zones, we made several attempts at recording Marwa’s talk, over the Internet, from her home in Homs. We tried at different times of the day, because the quality of the connection would vary greatly. We used different types of videoconferencing software. At moments, Marwa’s voice would disappear for a few seconds, the light would change in her home, and a new noise would signal that the grid had gone off and the generators had kicked in. At other times, connectivity was so bad that it was impossible to distinguish Marwa’s words. Making the recording more difficult were the noises of the town, the honking of cars and the racket of trucks. But accustomed to poor connectivity, she persevered, speaking to us — to the old camera attached to her laptop — over and over, with Ghassan helping out to make sure things were working at their end.

We finally got the full talk onto our hard drives. She appears in her living room, wearing a purple headscarf, against the backdrop of two framed drawings by her children. We had to apply filtering software to improve the quality of the sound. Our video editors added photos and images, some of them aerial shots of the city, courtesy of UNHCR. And the brilliant TED volunteer translators worked fast to get the first subtitles (in Arabic, of course) ready before the talk was screened at the conference (in the meantime, nine other languages have been added).

It is an important talk. Fully worth 10 minutes and a half of your time. Watch it here.

Bruno Giussani is the European director of TED and the curator of TEDGlobal and TEDSummit.


CryptogramHacking Electronic Safes

Nice attack against electronic safes:

Plore used side-channel attacks to pull it off. These are ways of exploiting physical indicators from a cryptographic system to get around its protections. Here, all Plore had to do was monitor power consumption in the case of one safe, and the amount of time operations took in other, and voila, he was able to figure out the keycodes for locks that are designated by independent third-party testing company Underwriter's Laboratory as Type 1 High Security. These aren't the most robust locks on the market by any means, but they are known to be pretty secure. Safes with these locks are the kind of thing you might have in your house.

Worse Than FailureError'd: A Model of a Modern Modal Window

Nick writes, "Well, it just goes to show you - don't under estimate the cultural significance of modal windows."

 

"I was trying to buy a costume, but it seems that I'm about to buy some modal text instead," Medalla writes.

 

Mort writes, "It's the personal touches that make me want to play the lottery."

 

"I thought 2006 was a good year for me, shame it has been erased," wrote Mitch.

 

Cynthia wrote, "Received new XPS 15 laptop from Dell which attempted to mash a US keyboard into a UK chassis."

 

"I'm only going to be in Belgium for three weeks, but I want to make sure my prepaid card doesn't run out," Dug S. wrote, "This should be enough, right?"

 

Andre writes, "Next time, just play it safe and go with Lorem Ipsum."

 

[Advertisement] BuildMaster integrates with an ever-growing list of tools to automate and facilitate everything from continuous integration to database change scripts to production deployments. Interested? Learn more about BuildMaster!

CryptogramScott Atran on Why People Become Terrorists

Scott Atran has done some really interesting research on why ordinary people become terrorists.

Academics who study warfare and terrorism typically don't conduct research just kilometers from the front lines of battle. But taking the laboratory to the fight is crucial for figuring out what impels people to make the ultimate sacrifice to, for example, impose Islamic law on others, says Atran, who is affiliated with the National Center for Scientific Research in Paris.

Atran's war zone research over the last few years, and interviews during the last decade with members of various groups engaged in militant jihad (or holy war in the name of Islamic law), give him a gritty perspective on this issue. He rejects popular assumptions that people frequently join up, fight and die for terrorist groups due to mental problems, poverty, brainwashing or savvy recruitment efforts by jihadist organizations.

Instead, he argues, young people adrift in a globalized world find their own way to ISIS, looking to don a social identity that gives their lives significance. Groups of dissatisfied young adult friends around the world ­ often with little knowledge of Islam but yearning for lives of profound meaning and glory ­ typically choose to become volunteers in the Islamic State army in Syria and Iraq, Atran contends. Many of these individuals connect via the internet and social media to form a global community of alienated youth seeking heroic sacrifice, he proposes.

Preliminary experimental evidence suggests that not only global terrorism, but also festering state and ethnic conflicts, revolutions and even human rights movements -- think of the U.S. civil rights movement in the 1960s -- depend on what Atran refers to as devoted actors. These individuals, he argues, will sacrifice themselves, their families and anyone or anything else when a volatile mix of conditions are in play. First, devoted actors adopt values they regard as sacred and nonnegotiable, to be defended at all costs. Then, when they join a like-minded group of nonkin that feels like a family ­ a band of brothers ­ a collective sense of invincibility and special destiny overwhelms feelings of individuality. As members of a tightly bound group that perceives its sacred values under attack, devoted actors will kill and die for each other.

Paper.

EDITED TO ADD (8/13): Related paper, also by Atran.

Planet Linux AustraliaCraige McWhirter: Doing Math with Python - Amit Saha - PyConAu 2016

Amit Saha.

Slides and demos.

Why Math with Python?

  • Provides an interactive learning experience.
  • Provides a great base for future programming (ie: data science, machine learning).

Tools:

  • Python 3
  • SymPy
  • matplotlib

Amit's book: Doing Math with Python

PyConAu 2016

Planet Linux AustraliaCraige McWhirter: The Internet of Not Great Things - Nick Moore - PyConAu 2016

Nick Moore.

aka "The Internet of (Better) Things".

  • Abuse of IoT is not a technical issue.
  • The problem is who controls the data.
  • Need better analysis of the was it is used that is bad.
  • "If you're not the customer, you're the product."
    • by accepting advertising.
    • by having your privacy sold.
  • Led to a conflation of IoT and Big Data.
  • Product end of life by vendors ceasing support.
  • Very little cross vendor compatibility.
  • Many devices useless if the Internet is not available.
  • Consumer grade devices often fail.
  • Weak crypto support.
  • Often due to lack of entropy, RAM, CPU.
  • Poorly thought out update cycles.

Turning Complaints into Requirements:

We need:

  • Internet independence.
  • Generic interfaces.
  • Simplified Cryptography.
  • Easier Development.

Some Solutions:

  • Peer to peer services.
  • Standards based hardware description language.
  • Shared secrets, initialised by QR code.
  • Simpler development with MicroPython.

PyConAu 2016

Planet Linux AustraliaCraige McWhirter: OpenBMC - Boot your server with Python - Joel Stanley - PyConAu 2016

Joel Stanley.

  • OpenBMC is a Free Software BMC
  • Running embedded Linux.
  • Developed an API before developing other interfaces.

Goals:

  • A modern kernel.
  • Up to date userspace.
  • Security patches.
  • Better interfaces.
  • Reliable performance.
    • REST interface.
    • SSH instead of strange tools.

The Future:

  • Support more home devices.
  • Add a web interface.
  • Secure boot, trusted boot, more security features.
  • Upstream all of the things.
  • Support more hardware.

PyConAu 2016

Planet Linux AustraliaCraige McWhirter: Teaching Python with Minecraft - Digital K - PyConAu 2016

by Digital K.

The video of the talk is here.

  • Recommended for ages 10 - 16
  • Why Minecraft?
    • Kids familiarity is highly engaging.
    • Relatively low cost.
    • Code their own creations.
    • Kids already use the command line in Minecraft
  • Use the Minecraft API to receive commands from Python.
    • Place blocks
    • Move players
    • Build faster
    • Build larger structures and shapes
    • Easy duplication
    • Animate blocks (ie: colour change)
    • Create games

Option 1:

How it works:

  • Import Minecraft API libraries to your code.
  • Push it to the server.
  • Run the Minecraft client.

What you can Teach:

  • Co-ordinates
  • Time
  • Multiplications
  • Data
  • Art works with maths
  • Trigonometry
  • Geo fencing
  • Design
  • Geography

Connect to External Devices:

  • Connect to Raspberry Pi or Arduino.
  • Connect the game to events in the real world.

Other Resources:

PyConAu 2016

Planet Linux AustraliaCraige McWhirter: Scripting the Internet of Things - Damien George - PyConAu 2016

Damien George

Damien gave an excellent overview of using MicroPython with microcontrollers, particularly the ESP8266 board.

Damien's talk was excellent and covered a broad and interesting history of the project and it's current efforts.

PyConAu 2016

Planet Linux AustraliaCraige McWhirter: ESP8266 and MicroPython - Nick Moore - PyConAu 2016

Nick Moore

Slides.

  • Price and feature set are a game changer for hobbyists.
  • Makes for a more playful platform.
  • Uses serial programming mode to flash memory
  • Strict power requirements
  • The easy way to use them is with a NodeMCU for only a little more.
  • Tool kits:
  • Lua: (Node Lua).
  • Javascript: Espruino.
  • Forth, Lisp, Basic(?!).
  • Mircopython works on the ESP8266:
    • Drives micro controllers.
    • The onboard Wifi.
    • Can run a small webserver to view and control devices.
    • WebRepl can be used to copy files, as can mpy-utils.
    • Lacks:
      • an operating system.
      • Lacks multiprocessing.
      • Debugger / profiler.
  • Flobot:
    • Compiles via MicroPython.
    • A visual dataflow language for robots.

ES8266 and MicroPython provide an accessible entry into working with micro-crontrollers.

PyConAu 2016

,

CryptogramHacking Your Computer Monitor

Here's an interesting hack against a computer's monitor:

A group of researchers has found a way to hack directly into the tiny computer that controls your monitor without getting into your actual computer, and both see the pixels displayed on the monitor -- effectively spying on you -- and also manipulate the pixels to display different images.

I've written a lot about the Internet of Things, and how everything is now a computer. But while it's true for cars and refrigerators and thermostats, it's also true for all the parts of your computer. Your keyboard, hard drives, and monitor are all individual computers, and what you think of as your computer is actually a collection of computers working together. So just as the NSA directly attacks the computer that is the hard drive, this attack targets the computer that is your monitor.

TEDGiant Olympic athletes take flight over Rio in JR’s latest work

In JR's latest work, a Sudanese high jumper curves around a building in Rio's Flamengo district. Photo: Courtesy of JR

Sudanese high jumper Mohamed Younes Idriss had to miss the Olympics this year because of an injury. But he towers over Rio de Janeiro, his back curving atop a high-rise building, in JR’s latest large-scale work. Photo: Courtesy of JR

A Sudanese high jumper towers over Rio de Janeiro, arching over a 25-story building in the Flamengo district. A triathlete plows through the waters of Botafogo Bay, mid-stroke, her wingspan as wide as a city bus, while a giant diver shows us the soles of his feet as he leaps from the stone jetty in Barra da Tijuca. Meanwhile, a truck disguised as a camera is circling the city, and a fat silver moon is taking shape atop a favela cultural center. It looks like JR is back in town.

Artist JR, winner of the 2011 TED Prize, created these three massive athletes — he calls them the “giants” — for the Rio Olympics, along with a city-wide Inside Out photo campaign that will shoot street portraits throughout the Games.

JR is known for his large-scale black-and-white wall pastings, but the “giants” represent a new technique for him — they’re suspended in the air on scaffolding, in vastly ambitious site-specific works that took almost a year to plan.

To create the gargantuan image of French triathlete Léonie Périault powering her way through Rio’s Botafogo Bay, JR wrote on Instagram, his team fought like an athlete with the navy so that this piece could be in the water. Photo: Courtesy of JR

To create the gargantuan image of French triathlete Léonie Périault powering her way through Rio’s Botafogo Bay, JR wrote on Instagram, his team “fought like an athlete … so that this piece could be in the water.” Note the tiny figures in the boat in foreground for scale. Photo: Courtesy of JR

JR feels strong ties to Rio; his classic work “Women Are Heroes” speckled the city’s hillside favelas with photographs of women’s eyes. Watching the Olympic Games here is especially meaningful to him, he wrote on Instagram before the Opening Ceremony: “Eighty years ago the Olympics happened in Berlin. Hitler wanted to use them to demonstrate the supremacy of the Aryan race. Today they will open in Rio de Janeiro, Brazil, a ‘mixed race’ country. Even though Brazil is going through political and economic turmoil and the necessity of the Games at this moment can spark controversy, the Olympic spirit will joyfully be welcomed.”

JR has also brought his TED Prize wish to the games. The Inside Out photobooth truck is parked at Praça Maua through 14 August, and will then spend a week inside the Olympic Village, right up until the August 21 close of the Games. Passersby line up to have their portrait taken, and then paste it on the ground, creating a patchwork of images representing people from all parts of the world.

Seeming to leap from the quebra mar (jetty) in Barra da Tijuca, here's the back view of diver Cleuson Lima do Rosario, a Brazilian athlete who now lives and works in France. Photo: Courtesy of JR

Seeming to leap from the quebra mar (jetty) in Barra da Tijuca, here’s the back view of diver Cleuson Lima do Rosario, a Brazilian athlete who now lives and works in France. Photo: Courtesy of JR

JR brings Inside Out to Olympic Boulevard. Photo: Courtesy of JR

JR brought the traveling Inside Out photobooth truck to Olympic Boulevard, pasting the faces of global passersby on the street for all to see. Next week it moves to the athletes’ home base in Olympic Village. Photo: Courtesy of JR

JR stands atop an unusual art space, soon to open in Rio. Photo: Courtesy of JR

JR’s team is also busy building a silver structure shaped like a fat crescent moon over Casa Amarela, a favela cultural center the artist helped open nine years ago. He hopes that artists will hold workshops in this unusual space. Photo: Courtesy of JR


Krebs on SecurityRoad Warriors: Beware of ‘Video Jacking’

A little-known feature of many modern smartphones is their ability to duplicate video on the device’s screen so that it also shows up on a much larger display — like a TV. However, new research shows that this feature may quietly expose users to a simple and cheap new form of digital eavesdropping.

Dubbed “video jacking” by its masterminds, the attack uses custom electronics hidden inside what appears to be a USB charging station. As soon as you connect a vulnerable phone to the appropriate USB charging cord, the spy machine splits the phone’s video display and records a video of everything you tap, type or view on it as long as it’s plugged in — including PINs, passwords, account numbers, emails, texts, pictures and videos.

The part of the "video jacking" demonstration at the DEF CON security conference last week in Las Vegas.

Some of the equipment used in the “video jacking” demonstration at the DEF CON security conference last week in Las Vegas. Source: Brian Markus.

[Click here if you’re the TL;DR type and just want to know if your phone is at risk from this attack.]

Demonstrations of this simple but effective mobile spying technique were on full display at the DEF CON security conference in Las Vegas last week. I was busy chasing a story at DEF CON unrelated to the conference this year, so I missed many people and talks that I wanted to see. But I’m glad I caught up with the team behind DEF CON’s annual and infamous “Wall of Sheep,” a public shaming exercise aimed at educating people about the dangers of sending email and other plain text online communications over open wireless networks.

Brian Markus, co-founder and chief executive officer for Aries Security, said he and fellow researchers Joseph Mlodzianowski and Robert Rowley came up with the idea for video jacking when they were brainstorming about ways to expand on their “juice jacking” experiments at DEF CON in 2011.

“Juice jacking” refers to the ability to hijack stored data when the user unwittingly plugs his phone into a custom USB charging station filled with computers that are ready to suck down and record said data (both Android and iOS phones now ask users whether they trust the computer before allowing data transfers).

In contrast, video jacking lets the attacker record every key and finger stroke the user makes on the phone, so that the owner of the evil charging station can later replay the videos and see any numbers or keys pressed on the smart phone.

That’s because those numbers or keys will be raised briefly on the victim’s screen with each key press. Here’s an example: While the user may have enabled a special PIN that needs to be entered before the phone unlocks to the home screen, this method captures even that PIN as long as the device is vulnerable and plugged in before the phone is unlocked.

GREAT. IS MY PHONE VULNERABLE?

Most of the phones vulnerable to video jacking are Android or other HDMI-ready smartphones from Asus, Blackberry, HTC, LG, Samsung, and ZTE. This page of HDMI enabled smartphones at phonerated.com should not be considered all-inclusive. Here’s another list. When in doubt, search online for your phone’s make and model to find out if it is HDMI or MHL ready.

Video jacking is a problem for users of HDMI-ready phones mainly because it’s very difficult to tell a USB cord that merely charges the phone versus one that also taps the phone’s video-out capability. Also, there’s generally no warning on the phone to alert the user that the device’s video is being piped to another source, Markus said.

“All of those phones have an HDMI access feature that is turned on by default,” he said. “A few HDMI-ready phones will briefly flash something like ‘HDMI Connected’ whenever they’re plugged into a power connection that is also drawing on the HDMI feature, but most will display no warning at all. This worked on all the phones we tested with no prompting.”

Both Markus and Rowley said they did not test the attack against Apple iPhones prior to DEF CON, but today Markus said he tested it at an Apple store and the video of the iPhone 6’s home screen popped up on the display in the store without any prompt. Getting it to work on the display required a special lightning digital AV adapter from Apple, which could easily be hidden inside an evil charging station and fed an extension adapter and then a regular lightning cable in front of that.

WHAT’S A FAKE CHARGING STATION?

Markus had to explain to curious DEF CON attendees who wandered near the Wall of Sheep this year exactly what would happen if they plugged their phone into his phony charging station. As you can imagine, not a ton of people volunteered but there were enough to prove a point, Markus said.

The demonstration unit that Markus and his team showed at DEF CON (pictured above) was fairly crude. Behind a $40 monitor purchased at a local Vegas pawn shop is a simple device that takes HDMI output from a video splitter. That splitter is connected to two micro USB to HDMI cables that are cheaply available in electronics stores.

Those two cords were connected to standard USB charging cables for mobiles — including the universal micro USB to HDMI adapter (a.k.a. Mobile High Definition Link or MHL connector), and a slimport HDMI adapter. Both look very similar to standard USB charging cables. The raw video files are recorded by a simple inline recording device to a small USB storage device taped to the back of the monitor.

Markus said the entire rig (minus the TV monitor) cost about $220, and that the parts could be bought at hundreds of places online.

Although it's hard to tell the difference at this angle, the USB connector on the left has a set of six extra pins that enable it to read HDMI video and whatever is being viewed on the user's screen. Both cords will charge the same phone.

Although it may be difficult to tell the difference at this angle, the Mobile High Definition Link (MHL) USB connector on the left has a set of six extra pins that enable it to read HDMI video and whatever is being viewed on the user’s screen. Both cords will charge the same phone.

SHOULD YOU CARE?

My take on video jacking? It’s an interesting and very real threat — particularly if you own an HDMI ready phone and are in the habit of connecting it to any old USB port. Do I consider it likely that any of us will have to worry about this in real life? The answer may have a lot to do with what line of work you’re in and how paranoid you are, but it doesn’t strike me as very likely that most mere mortals would have reason to worry about video jacking.

On the other hand, it would be a fairly cheap and reasonably effective (if random) way to gather secrets from a group of otherwise unsuspecting people in a specific location, such as a hotel, airport, pub, or even a workplace.

An evil mobile charging station would be far more powerful when paired with a camera (hidden or not) trained on the charger. Imagine how much data one could hoover up with a fake charging station used to gather intellectual property or trade secrets from, say….attendees of a niche trade show or convention.

Now that I think about it, since access to electric power is not a constraint with these fake charging stations, there’s no reason it couldn’t just beam all of its video wirelessly. That way, the people who planted the spying equipment could retrieve or record the victim videos in real time and never have to return to the scene of the crime to collect any of it. Okay, I’ll stop now.

What can vulnerable users do to protect themselves from video jacking?

Hopefully, your phone came with a 2-prong charging cord that plugs straight into a standard wall jack. If not, look into using a USB phone charger adapter that has a regular AC/DC power plug on one end and a female USB port on the other (just make sure you don’t buy this keystroke logger disguised as a USB phone charger). Carry an extra charging dock for your mobile device when you travel.

Also, check the settings of your mobile and see if it allows you to disable screen mirroring. Note that even if you do this, the mirroring capability might not actually turn off.

What should mobile device makers do to minimize the threat from video jacking? 

“The problem here is that device manufacturers continue to add features and not give us prompting,” Markus said. “With this feature, it automatically connects no matter what. HDMI-out should be off by default, and if turned on it should require prompting the user.”

Update: 4:52 p.m. ET: Updated paragraph about Apple iPhones to clarify that this same attack works against the latest iPhone 6.

CryptogramHackers Stealing Cars

We're seeing car thefts in the wild accomplished through hacking:

Houston police have arrested two men for a string of high-tech thefts of trucks and SUVs in the Houston area. The Houston Chronicle reports that Michael Armando Arce and Jesse Irvin Zelaya were charged on August 4th, and are believed to be responsible for more than 100 auto thefts. Police said Arce and Zelaya were shuttling the stolen vehicles across the Mexican border.

[...]

The July video shows the thief connecting a laptop to the Jeep before driving away in it. A Fiat-Chrysler spokesman told ABC News that the thieves used software intended to be used by dealers and locksmiths to reprogram the vehicle's keyless entry and ignition system.

Worse Than FailureCodeSOD: Constantly Extended

Imagine you’re a financial institution. You’ve built an application that processes financial transactions, and there are a number of flags that need to be set as constants to determine application behavior.

You might choose to write code like this:

  public static final boolean CreateNewReferenceFolderAndFiles = true;

But then your boss would say, “Bro, do you even object-orient?” before firing you, because that isn’t enterprise-y.

Skyfyre” sent us this excerpt, from a four-million line application, which is definitely very enterprise-y.

First, you need to define several classes, and since this is Java, each of them must be in its own file:

  public abstract class ArchiveCreateNewReferenceFolderAndFiles
  {
  }

  public class ArchiveCreateNewReferenceFolderAndFilesTrue extends ArchiveCreateNewReferenceFolderAndFilesAbstract
  {
  }

  public class ArchiveCreateNewReferenceFolderAndFilesFalse extends ArchiveCreateNewReferenceFolderAndFilesAbstract
  {
  }

Then, you’ve got to create your actual constants:

public static final ArchiveCreateNewReferenceFolderAndFilesAbstract ARCHIVE_CreateNewReferenceFolderAndFiles_TRUE = new ArchiveCreateNewReferenceFolderAndFilesTrue();
public static final ArchiveCreateNewReferenceFolderAndFilesAbstract ARCHIVE_CreateNewReferenceFolderAndFiles_FALSE = new ArchiveCreateNewReferenceFolderAndFilesFalse();
/// later…
public static final ArchiveCreateNewReferenceFolderAndFilesAbstract archiveCreateNewReferenceFile = ARCHIVE_CreateNewReferenceFolderAndFiles_TRUE;

Now, you’re ready to write some real conditionals based on flags:

  if(archiveCreateNewReferenceFile instanceof ArchiveCreateNewReferenceFolderAndFilesTrue) {
    //do stuff…
  }

It’s simplicity itself, and just think about how maintainable it is. When you finally need to support FileNotFound, you simply need to inherit from the ArchiveCreateNewReferenceFolderAndFiles class to produce ArchiveCreateNewReferenceFolderAndFilesFileNotFound. Think of the extensibility! Think of the readability!

[Advertisement] Otter, ProGet, BuildMaster – robust, powerful, scalable, and reliable additions to your existing DevOps toolchain.

,

Google AdsenseUse #hashtags to create trending content

It’s official, #hashtags have taken over the internet. Much like memes, gifs, and audio-less fast motion cooking instructional videos, #hashtags fill up social media news feeds. However, unlike the other popular content types, what’s unique to #hashtags is that they organize conversations across the web. Even Jimmy Fallon and Justin Timberlake had something to say about #hashtags.

#Hashtags started around 2007 on Twitter, and have rapidly grown into a common medium for users to express their feelings or interests primarily on social networks. As the summer of sport kicks off, it’s a good idea for you to consider incorporating #hashtags into your content strategy as a key ingredient to #drawthecrowds.

#Hashtags are quite simple to use and can attract new users to your content when you understand how they work. Essentially, when the pound/hash sign is used in front of a group of words it automatically turns that group of words into a searchable link. This transforms those keywords into a conversation that the entire web can participate in and follow.

The use of #hashtags can be boiled down into two main use cases:
  1. Create your own, unique #hashtag to organize your content and start a conversation. This could be tricky because there are millions of #hashtags online, so don’t be afraid to repurpose one that exists. AdSense uses original #hashtags like #AdSenseGuide to promote our downloadable content or #AskAdSense for our Q&A sessions. We’re also using #drawthecrowds during the summer season to help AdSense publishers draw crowds to your content during big events. 
  2. Use an existing #hashtag and join in on a conversation. Use social network search options to find trending #hashtags that are relevant to your audience and join the conversation. For example, #BurningMan is a popular #hashtag used in the summer months to find news and updates about the annual event. Everyone from news publications to the thousands of people in the Black Rock Desert of Nevada will be using #BurningMan to share their perspective of the Burning Man experience. Using existing and popular #hashtags presents an opportunity for you to contribute your unique perspective to the digital conversation.
To get the most out of #hashtags, here’s a “do and don’t” list to reference as you build out your content strategy:
DO
  • Use one to three #hashtags per post, any more is generally overdoing it. 
  • Use #hashtags that are relevant to your audience or ones that your industry is using. If you’re writing an article on the food to try, you could use #hashtags like #Foodie and #Yummy so users will find you when they search for those always trending keywords.
  • It’s ok to be specific. In most cases, the more specific the #hashtag, the better. If you’re going to talk about do it yourself (DIY) summer projects, you’d want to use #hashtags like #diyprojects, #diyideas or #diyweddings instead of general keywords like #DIY or #DoItYourSelf. Using specific #hashtags helps users pinpoint the exact content they’re looking for. 
  • Letters and numbers are OK to use in #hashtags.
  • Keep #hashtags short.

DON’T
  • Don’t string too many words together. #itbecomesreallyreallyhardtoread and it can take up most of your Twitter character count.
  • Don’t use punctuation marks or spaces, they will break the searchable link.
  • Don’t use the same hashtag twice in the same social post. It’s just #weird.
Now that you understand how to use #hashtags and how they can help you #drawthecrowds this summer, share with us how you’re going to incorporate them into your content strategy – we’d love to follow along. 


Posted by Jay Castro, AdSense Content Marketing Specialist



Sociological ImagesHumor Theorist Explains Trump’s “Joke” about Killing Hillary Clinton

Yesterday Donald Trump appeared to suggest that defenders of the 2nd Amendment should assassinate Hillary Clinton if she is elected. Or maybe any judges she appoints to the Supreme Court. It wasn’t very clear.

Supporters rushed to his defense, suggesting he was joking. Here’s what a humor scholar, Jason P. Steed, had to say about that via Twitter:


You can follow Jason P. Steed on Twitter here.

(View original at https://thesocietypages.org/socimages)

Planet Linux AustraliaChris Smart: Command line password management with pass

Why use a password manager in the first place? Well, they make it easy to have strong, unique passwords for each of your accounts on every system you use (and that’s a good thing).

For years I’ve stored my passwords in Firefox, because it’s convenient, and I never bothered with all those other fancy password managers. The problem is, that it locked me into Firefox and I found myself still needing to remember passwords for servers and things.

So a few months ago I decided to give command line tool Pass a try. It’s essentially a shell script wrapper for GnuPG and stores your passwords (with any notes) in individually encrypted files.

I love it.

Pass is less convenient in terms of web browsing, but it’s more convenient for everything else that I do (which is often on the command line). For example, I have painlessly integrated Pass into Mutt (my email client) so that passwords are not stored in the configuration files.

As a side-note, I installed the Password Exporter Firefox Add-on and exported my passwords. I then added this whole file to Pass so that I can start copying old passwords as needed (I didn’t want them all).

About Pass

Pass uses public-key cryptography to encrypt each password that you want to store as an individual file. To access the password you need the private key and passphrase.

So, some nice things about it are:

  • Short and simple shell script
  • Uses standard GnuPG to encrypt each password into individual files
  • Password files are stored on disk in a hierarchy of own choosing
  • Stored in Git repo (if desired)
  • Can also store notes
  • Can copy the password temporarily to copy/paste buffer
  • Can show, edit, or copy password
  • Can also generate a password
  • Integrates with anything that can call it
  • Tab completion!

So it’s nothing super fancy, “just” a great little wrapper for good old GnuPG and text files, backed by git. Perfect!

Install Pass

Installation of Pass (and Git) is easy:
sudo dnf -y install git pass

Prepare keys

You’ll need a pair of keys, so generate these if you haven’t already (this creates the keys under ~/.gnupg). I’d probably recommend RSA and RSA, 4096 bits long, using a decent passphrase and setting a valid email address (you can also separately use these keys to send signed emails and receive encrypted emails).
gpg2 --full-gen-key

We will need the key’s fingerprint to give to pass. It should be a string of 40 characters, something like 16CA211ACF6DC8586D6747417407C4045DF7E9A2.
gpg2 --list-keys

Note: Your fingerprint (and public keys) can be public, but please make sure that you keep your private keys secure! For example, don’t copy the ~/.gnupg directory to a public place (even though they are protected by a nice long passphrase, right? Right?).

Initialise pass

Before we can use Pass, we need to initialise it.
pass init

This creates the basic directory structure in the .password-store directory in your home directory. At this point it just has a plain text file with the fingerprint of the public key that it should use.

Adding git backing

If you haven’t already, you’ll need to tell Git who you are. Using the email address that you used when creating the GPG key is probably good.
git config --global user.email "you@example.com"
git config --global user.name "Your Name"

Now, go into the password-store directory and initialise it as a Git repository.
cd ~/.password-store
git init
git add .
git commit -m "intial commit"
cd -

Pass will now automatically commit changes for you!

Hierarchy

As mentioned, you can create any hierarchy you like. I quite like to use subdirectories and sort by function first (like mail, web, server), then domains (like gmail.com, twitter.com) and then server or username. This seems to work quite nicely with tab completion, too.

You can rearrange this at any time, so don’t worry too much!

Storing a password

Adding a password is simple and you can create any hierarchy that you want; you just tell pass to add a new password and where to store it. Pass will prompt you to enter the password.

For example, you might want to store your password for a machine at server1.example.com – you could do that like so:
pass add servers/example.com/server1

This creates the directory structure on disk and your first encrypted file!
~/.password-store/
└── servers
    └── example.com
        └── server1.gpg
 
2 directories, 1 file

Run the file command on that file and it should tell you that it’s encrypted.
file ~/.password-store/servers/example.com/server1.gpg

But is it really? Go ahead, cat that gpg file, you’ll see it’s encrypted (your terminal will probably go crazy – you can blindly enter the reset command to get it back).
cat ~/.password-store/servers/example.com/server1.gpg

So this file is encrypted – you can safely copy it anywhere (again, please just keep your private key secure).

Git history

Browse to the .password-store dir and run some git commands, you’ll see your history and showing will prompt for your GPG passphrase to decrypt the files stored in Git.

cd ~/.password-store
git log
git show
cd -

If you wanted to, you could push this to another computer as a backup (perhaps even via a git-hook!).

Storing a password, with notes

By default Pass just prompts for the password, but if you want to add notes at the same time you can do that also. Note that the password should still be on its own on the first line, however.
pass add -m mail/gmail.com/username

If you use two-factor authentication (which you should be), this is useful for also storing the account password and recovery codes.

Generating and storing a password

As I mentioned, one of the benefits of using a password manager is to have strong, unique passwords. Pass makes this easy by including the ability to generate one for you and store it in the hierarchy of your choosing. For example, you could generate a 32 character password (without special characters) for a website you often log into, like so:
pass generate -n web/twitter.com/username 32

Getting a password out

Getting a password out is easy; just tell Pass which one you want. It will prompt you for your passphrase, decrypt the file for you, read the first line and print it to the screen. This can be useful for scripting (more on that below).

pass web/twitter.com/username

Most of the time though, you’ll probably want to copy the password to the copy/paste buffer; this is also easy, just add the -c option. Passwords are automatically cleared from the buffer after 45 seconds.
pass -c web/twitter.com/username

Now you can log into Twitter by entering your username and pasting the password.

Editing a password

Similarly you can edit an existing password to change it, or add as many notes as you like. Just tell Pass which password to edit!
pass edit web/twitter.com/username

Copying and moving a password

It’s easy to copy an existing password to a new one, just specify both the original and new file.
pass copy servers/example.com/server1 servers/example.com/server2

If the hierarchy you created is not to your liking, it’s easy to move passwords around.
pass mv servers/example.com/server1 computers/server1.example.com

Of course, you could script this!

Listing all passwords

Pass will list all your passwords in a tree nicely for you.
pass list

Interacting with Pass

As pass is a nice standard shell program, you can interact with it easily. For example, to get a password from a script you could do something like this.
#!/usr/bin/env bash
 
echo "Getting password.."
PASSWORD="$(pass servers/testing.com/server2)"
if [[ $? -ne 0 ]]; then
    echo "Sorry, failed to get the password"
    exit 1
fi
echo "..and we got it, ${PASSWORD}"

Try it!

There’s lots more you can do with Pass, why not check it out yourself!

Worse Than FailureAged Like Vinegar

It was Brian’s first day at AutoDetective, a website for comparing listed car prices vs. blue book values. His work inbox was overflowing with style guides, best practices, and notes from the dozen or so other developers he would be working with. His interviewer, Douglas, had mentioned that the site ran on a substantial chunk of legacy code, but Brian had experience with plenty of old code.

He spent most of the day digging through the source, getting a feel for the in-house development style. It didn’t take long before he noticed how … off the code was.

An old car mouldering in a rotting garage

It wasn’t just legacy code. It was obtuse legacy code.

Just Don’t Touch It

Douglas came to debrief Brian at the end of the day. Brian explained how he spent the afternoon going through the codebase, looking for a project to get his hands wet. “There’s a lot of inefficient code,” he said. “I figured I’d fix something small.” He pointed to a bit of code:

if (isset($_SESSION['relogin_data'])) {
 $a_relogin_data = $_SESSION['relogin_data'];
 $arr_keys = array_keys($_SESSION);
 for ($i=0; $i < sizeof($arr_keys); $i++) {
 unset($_SESSION[$arr_keys[$i]]);  
 }  
 $_SESSION['relogin_data'] = $a_relogin_data;
}

“Why are we unsetting the entire array like this? I’ve pared this down in my local branch. I was about to send a pull request–”

“No,” Douglas said. “That’s our legacy code. We can’t change a single line in that.”

“But it’s inefficient–”

“It doesn’t matter. None of us who still work here understand this code. It’s like a house of cards: if we change something little, the whole thing could come crashing down.”

“Oh-kay.” Brian quietly deleted his local branch.

Stovepipes Attached to Stovepipes

Every day, when he finished his assigned trouble tickets, Brian would attempt to make sense of the huge lump of legacy code that no one in the company could ever touch. For instance, this ultra-robust array building method, which skips simple assignation for laborious key/value reassignments:

while ($row = mysqli_fetch_assoc($result))
{
 $array_keys = array_keys($row);
 $array_values = array_values($row);

 for ($i=0; $i < sizeof($array_keys); $i++)
 $dataset[$count][$array_keys[$i]] = $array_values[$i];
 $count ++;
}

The code below checks if there’s a connection already open with a given name. All database errors in the legacy codebase return false, so there’s no way to tell if it’s a bad password, a server outage, or trying to reuse the same connection name twice:

if (isset(DBClass::$open_connections_[$conn_name])) {
 // connection is already open
 return false;
}

Everything, and one means everything, was stored in $_SESSION. Dumping session data was almost useless with how much was stored in it on a regular basis:

public function getUserInfo()
{
 return $_SESSION['logged_in'];
}
$user_info = $login->getUserInfo();
$_SESSION['user_info'] = $user_info;

// ...

$_SESSION['logged_in'] = $this->a_logged_in_;
$_SESSION['view_mode'] = $_SESSION['logged_in']['view_mode'];
$_SESSION['user_info'] = $_SESSION['logged_in'];

Then there was the utterly useless, such as a defined destructor that only calls its parent (unnecessary in PHP):

 public function __destruct()
 {
 parent::__destruct();
 }

A Unit Test Too Far

One afternoon, Brian willed himself to Douglas’s office. When his boss asked what the trouble was, Brian pointed him to a bit of code he had been staring at for hours. “I was working on #75693, and trying to figure out where $this->SERIAL was getting set. It wasn’t in the class definitions, which would be natural. Instead, I found it in this out-of-the-way include.”

<?php
class MyClass {
 private $SERIAL = "";
 public __construct() {
 include(Loader::get("configuration file"));
// ... more code ...
 $this->db_loadData($this->SERIAL);
// ... more code ...
 }
}

// the included file:
<?php
switch (get_class($this)) {
 case "ClassName":
 $this->SERIAL= "H57-451";
 case "SecondClassName":
 $this->SERIAL= "H57-452"
// ... hundreds of case statements ...
}

Douglas started, “You know we can’t touch–”

“Right, I know, and I’m not. But how can we keep maintaining the codebase like this? Let me document this code. I’ll write some function comments, put together some unit tests based on existing behavior. That way–”

“We can’t, Brian. None of us have the time. I work with eleven other developers. None of them have the time to maintain unit tests, and none of them want to. And commenting the code might cause it to break.”

Dejected, Brian returned to his desk. The years of inefficient code he had witnessed ate at him. Like Sisyphus pushing his rock up a hill, he knew he’d never get a chance to fix all of it. But if he waited long enough … found the time to document it … he might get his chance. One day.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

Krebs on SecurityGot Microsoft? Time to Patch Your Windows

Microsoft churned out a bunch of software updates today fix some serious security problems with Windows and other Microsoft products like Internet Explorer (IE), Edge and Office. If you use Microsoft, here are some details about what needs fixing.

brokenwindowsAs usual, patches for IE and for Edge address the largest number of “critical” vulnerabilities. Critical bugs refer to flaws Microsoft deems serious enough that crooks can exploit them to remotely compromise a vulnerable computer without any help from the user, save for the user visiting some hacked but otherwise legitimate site.

Another bundle of critical bugs targets at least three issues with the way Windows, Office and Skype handle certain types of fonts. Microsoft said attackers could exploit this flaw to take over computers just by getting the victim to view files with specially crafted fonts — either in an Office file like Word or Excel (including via the preview pane), or visiting a hacked/malicious Web site.

Microsoft Office got its own critical patch that fixed at least seven vulnerabilities — including another one exploitable through the preview pane. Microsoft PDF also received a critical patch thanks to a bug that’s exploitable just by getting Edge users to view specially-crafted PDF content in the browser.

For the record, Adobe says it has no plans to issue a Flash Player update today (as per usual) or anytime this month. As always, if you experience any issues downloading or installing any of the Microsoft updates from this month, please don’t hesitate to leave a comment below.

For more information on these and other Microsoft security updates released today, check out the blogs at security vendors Qualys and Shavlik.

,

TEDWorld Lion Day: A visit to big-cat filmmakers Beverly & Dereck Joubert

Wildlife filmmakers Beverly and Dereck Joubert speak at the pioneering TEDWomen 2010.

Wildlife filmmakers Beverly and Dereck Joubert spoke at TEDWomen 2010 about their commitment to saving Africa’s big cats from extinction. The biggest factor that threatens these majestic animals: trophy hunters.

Dereck and Beverly Joubert have been living in the bush in Botswana, making wildlife and conservation films together, for more than 30 years. Their films have shaped an intimate and profound narrative about the interconnected relationship among people, animals and the land, adding layers of understanding based on years of close and constant observation of animal behavior. (Their latest film, The Soul of the Elephant, was just nominated for an Emmy.)

This summer, TEDWomen host Pat Mitchell visited the Jouberts in one of the Great Plains safari camps and preserves they founded: Great Plains Conservation, launched a few years ago in Botswana and Kenya. Mitchell sends this timely report — Wednesday, August 10, it turns out, is World Lion Day.

Out on a game drive, from left: Dereck Joubert, TEDWomen host Pat Mitchell, Mitchell's husband Scott Seydel, and Beverly Joubert.

Out on a game drive at Great Plains Conservation in Botswana: From left, filmmaker Dereck Joubert, TEDWomen host Pat Mitchell, Mitchell’s husband Scott Seydel, and filmmaker Beverly Joubert.

On this visit, we talked about how much has happened since their 2010 TEDWomen talk about their Big Cats Initiative. I well remember how they stunned the TEDWomen audience, describing the shocking decline in big cat populations in Africa. They told us that the number of lions had gone from about 450,000 when they were growing up to less than 45,000 in 2010 – a literal decimation – with similar declines in cheetah and leopard populations. Sadly, they told me now, the number has declined even further since then, and is approaching 20,000.

The Jouberts told me they still receive hundreds of messages a week about their TED Talk. This talk, along with their films about the lions of Duba Plains and the leopards they’ve tracked over many years, have raised public awareness about the threats to the big cats: habitat encroachment; community pressure, where conflicts arise between animals and people; and, of course, the biggest single factor, hunting. The Jouberts helped lead the fight to ban hunting in Botswana, and as a result the animal population, including big cats, is increasing here.

But in many other countries in Africa – where big cats are an important attraction in the safari experiences that bring more than $27 billion a year into local economies – at least five lions are lost per day. Working with the National Geographic Society on the Big Cats Initiative, the Jouberts are committed to changing that.

Meanwhile, they told me, they have a new cat film in production for Nat Geo Wild — not about big cats this time, but the smaller ones, the ones we call “domesticated.” The film will explore behavioral links between the cats we pet and love in our homes and the cats we admire from a safe distance.

TEDWomen host Pat Mitchell shares this epic selfie along with a lion spotted at Great Plains.

TEDWomen host Pat Mitchell shares this epic selfie with a lion spotted at Great Plains.

At this year’s TEDWomen conference in October, I’ll be sharing updates, ideas and perspectives from the front lines of conservation, in Africa and in many other places. These battles to sustain our natural environments are being fought by champions like the Jouberts, who are seeking a better balance between us and the world we inhabit.

Main theater passes are still available for TEDWomen 2016, to be held in San Francisco October 26-28. Find out more about TEDWomen 2016: It’s About Time >>


Rondam RamblingsJust in case you're still not convinced...

... that Donald Trump must not win this election, you should read this LA Times op-ed by a Minuteman III nuclear launch officer (who also happens to be a Republican): [C]onsider Trump’s words in a town hall event during the primaries: “Somebody hits us within ISIS, you wouldn’t fight back with a nuke?” Or the words of Trump’s spokeswoman, Katrina Pierson, who also asked the unaskable on Fox

LongNowBreakthrough Listen Initiative Wants to Hear From You

BTL-request4ieas

We have received an email from Jill Tarter, former director of the Center for SETI research, on a new outreach on behalf of the Breakthrough Listen Initiative. They want to hear from the general public on their ideas for new approaches for finding evidence of extraterrestrial technological civilizations. They are looking for 1 page descriptions, with specific attention paid to:

  • New parameter space to be explored;
  • Hardware and/or software required;
  • Current status of any prototyping or trial runs;
  • Any technology barriers at this time;
  • Scale of the effort – estimates of resources, time to completion, and costs;
  • Any other scientific opportunities enabled by this new approach.

Descriptions that reach Jill Tarter by 15 August, 2016 will be incorporated into the subcommittee’s deliberations later that week. Please send your approach to newideas4seti@seti.org.

CryptogramMalware from Kazakhstan

EFF has the story of malware from the Kazakhstan government against "journalists and political activists critical of Kazakhstan's authoritarian government, along with their family members, lawyers, and associates."

Worse Than FailureRepresentative Line: Compatibly Backward

I took my first official programming class circa 1997, and that year was notable, only because it was the last years that class was taught using Turbo Pascal. In future years, it was taught in C++. For the teacher, this was quite the transition. To help her make the transition, at the end of the course, she spent a few days teaching us basic C++, so she’d be more ready for the following class, and we got a little bonus education.

As far as I know, future runs of the class went just fine. I bring this up, because Frank had some co-workers who needed to make the exact same transition, from Turbo Pascal to C++. They may have done it a bit less gracefully. When reviewing some of the C++ they wrote, Frank spotted lots of code like:

if (counter > 0) then
{
  // do something in here
}

It was the then that caught his eye. It took some hunting around, but buried deep in a header file included in every other file, everywhere, he found this:

#define then

Yes, they defined an empty, do-nothing macro just so they could still type then with their if statements.

[Advertisement] Universal Package Manager – store all your Maven, NuGet, Chocolatey, npm, Bower, TFS, TeamCity, Jenkins packages in one central location. Learn more today!

CryptogramHow the Iranian Government Hacks Dissidents

Citizen Lab has a new report on an Iranian government hacking program that targets dissidents. From a Washington Post op-ed by Ron Deibert:

Al-Ameer is a net savvy activist, and so when she received a legitimate looking email containing a PowerPoint attachment addressed to her and purporting to detail "Assad Crimes," she could easily have opened it. Instead, she shared it with us at the Citizen Lab.

As we detail in a new report, the attachment led our researchers to uncover an elaborate cyberespionage campaign operating out of Iran. Among the malware was a malicious spyware, including a remote access tool called "Droidjack," that allows attackers to silently control a mobile device. When Droidjack is installed, a remote user can turn on the microphone and camera, remove files, read encrypted messages, and send spoofed instant messages and emails. Had she opened it, she could have put herself, her friends, her family and her associates back in Syria in mortal danger.

Here's the report. And a news article.