Planet Javasummit

February 22, 2012

Worse Than FailureCodeSOD: The Phony

"My first impression of my predecessor was a positive one," writes Michal Ochman, "he was well-spoken, sharply dressed, and seemed to be an all-around nice guy."

"My first impression of my predecessor's code was also positive — it was sanely structured, well-commented, and all-around seemed pretty clean. And then I started actually looking through it. It was filled with a lot of... well... this."

<?php

/* Base 64 Encoding function **
** PHP does it natively but just for consistency and 
** ease of maintenance, let's declare our own function **/
function base64Encode($plain) {

  // Initialise output variable
  $output = "";

  // Do encoding
  $output = base64_encode($plain);

  // Return the result
  return $output;
}

/* Base 64 decoding function **
** PHP does it natively but just for consistency and 
** ease of maintenance, let's declare our own function **/
function base64Decode($scrambled) {

  // Initialise output variable
  $output = "";

  // Do encoding
  $output = base64_decode($scrambled);
 
  // Return the result
  return $output;
}

?>

Michal continued, "after getting to know some of my new coworkers, they described my predecessor as 'seemingly a nice guy at first, but really just a phony'. I guess I could see that."


XKCDOrion Nebula

February 21, 2012

Worse Than FailureSteve's Email

For a ramen-fueled college student, a job offer that actually involves doing the work you're in school to learn and has a paycheck that can't be beat flipping burgers is a wonderful score. For Ted, that score came from a company run by alumni from his school that gave out pretty plum gigs to top students.

They were a support company, that contracted out to run the IT infrastructure for other companies. While the support company actually cut Ted's check, they were a small company. 90% of their business came from a single customer, the PubCong publishing conglomerate. It was for PubCong that Ted would be actually working.

"It's a little bit of a mess," Chris, Ted's boss, explained as he showed Ted around the server room. It didn't look like a mess- to the contrary, it looked like a top-flight server room all the way from beneath the raised floors to the million dollar cooling system over their heads. "It's so clean in here because we're the first people in here in six months that weren't here to dust. They paid a lot of money to have this installed, but they didn't actually hire anyone to maintain it until we came along."

Chris broke down some lowlights of the Mad Maxian <script src="http://www.cornify.com/js/cornify.js" type="text/javascript"></script> wasteland of administration-by-user, but it boiled down to, "the hardware's in great shape, but everything else has been installed, configured, and maintained by secretaries, janitors, and a creepy Austrialian kid with a razor boomerang." They didn't know what was installed or where it might be deployed, or who was responsible for it. For the time being, Ted was given an SSH shell account, some instructions to help run an inventory, and the command to "poke around, but don't break anything, try not to get killed."

Chris ran off to do whatever it is bosses do, and Ted did exactly as ordered until 6PM. The service-desk phone started ringing. Ted eyed it for a ring or two. He wasn't really sure if he should take the call- he was new, hadn't really been trained, and didn't know the procedures. Still, he had done campus helldesk. It was probably just a printer jam. Ted could handle that. Before the fifth ring, he answered.

"…you useless idiotic foolish- hello? Finally. The phone's been ringing for ages. It's Steve. You need to fix mi email."

Ted's helldesk instincts kicked in. "Okay, Steve. This is Ted. What kind of trouble are you having?"

Steve spoke in slow, even tones, as if he were explaining the complexities of quantum mechanics to a hungry, brain-damaged shark. "I. Can't. Read. My. Email."

Ted forced himself to smile at the handset, and asked, "Are you seeing some kind of error?"

"'Authorization failed.' Look, can you actually fix this or not?"

Ted went down the basic email-problem checklist, but just getting Steve's username was a challenge. "What do you mean you don't know my username? Are you slow, or something? I'm Steve!" Eventually, Ted confirmed that stevem had been hammering the POP3 server for most of the day, and the password had failed until the account was locked. Ted also noticed that the user data for stevem indicated that he was a C-level executive for the company.

"Well, it does look like your password is incorrect," Ted said.

"I'm sure you think so Tim, but I have it written down right here in front of me. I know what my password is."

"Well, perhaps you mispelled something," Ted offered helpfully.

"Then tell me what my password is and I'll check."

"I can't tell you your password," Ted said.

"Excuse me, Todd? Do you know who I am? If you value your job, you're going to tell me my password right now!"

Ted wasn't going to fiddle with an account password just because someone was screaming at him, even if he could have checked what it was. These mail accounts were also shell accounts, and he wasn't going to grant some hacker access to the system just because they screamed loudly on the phone. "I can't do this over the phone," Ted said, "but if you're in your office, I can come down and reset your password."

"In the office? Are you insane? It's 6:15. Some of us are busy people, Tad. Now tell me what my password is. Now are you going to do what I tell you or…"

"Let me get Chris on the line," Ted said. Chris knew Steve, and after only a few seconds of listening to Steve screaming about the coming lawsuit, Chris confirmed that Steve was in fact, Steve M, the CIO of PubCong. Chris gave Ted permission to do anything Steve asked.

"Okay," Ted said, "I'll just reset your password…"

"I don't want it reset. Just tell me what it is you moron!"

Ted set Steve's password to "stevie123" and said, "Oh, it's 'stevie123', all lower-case."

"Hmm… that's not what's on my post-it. Mine doesn't have any numbers in it. No, it doesn't work. You've given me someone else's password."

Ted checked the logs, and saw no attempt had been made to log in. "No, that really is your password. You're trying to log in as stevem@pubcong.com, right?"

"What? No. Stevem@aol.com. Y'know, the email service. How can you work in IT and not know that!"


Tim Bray No Iran War Please

Back in 2002, this crazy idea of responding to 9/11 by attacking Iraq first started being floated. And now we’re getting stronger and stronger whiffs of Dorky Middle East War, the Sequel: Iran. Can the sensible people of the world please stand up and say ”Please, let’s not do that“.

It’s getting to the point where the New York Times has matter-of-fact articles like Iran Raid Seen as a Huge Task for Israeli Jets. On top of which, some of the particularly bloodthirsty members of the LikudAmerica fringe are pushing like crazy on that Overton Window, for example see Does AIPAC want war?

Puzzling Evidence

It seems self-evident to me that an Israeli coup de main against Iranian nuclear facilities would be unlikely to have any effect that is positive for Israel or negative for Iran. Evidence suggests (if you read the Israeli media) that this also seems self-evident to the Israeli military leadership.

So why the continued sabre-rattling? I’ve read two theories:

  • The Israelis are trying to convince the US Administration that they’re crazy enough to do this in order to extract concessions, bribes for them not to. I don’t believe this, because it seems to me that Netanyahu’s administration has got pretty well everything they want from Obama’s: no real pressure on settlements or human-rights issues.

  • Some week, the Israelis are going to tell Washington they’re attacking next week, and the US knows it would be such a disaster that they conclude that they’ll have to do the attack on Israel’s behalf. But this is the one thing I can’t see the US administration, as Netanyahu-friendly as it is, even contemplating. Obama got where he is in some part on the basis of having been right about Iraq.

So I don’t know what the hell is actually going on. But recall the run-up to the Iraq war very vividly, and it isn’t making me happy.

Why It’s a Dumb Idea

I asserted above that an Israeli attack on Iran would be stupid and counterproductive. I ought to provide some arguments to back up that claim.

  • As that New York Times article makes clear, the risk of failure is very, very high.

  • It would strengthen Iran’s dictators. One of the few things that could cause the population to rally around the much-despised mullahs and Revolutionary Guard would be an actual foreign attack.

  • Chances are that a few minutes after the attack started, Hezbollah missiles would start flying out of South Lebanon. The conventional wisdom is that they have more and better missiles than in the 2006 war, and could potentially drop a few right into Tel Aviv.

    Now, there’s no doubt that the IDF would dearly love a chance to have another run at Hezbollah. It wouldn’t be a surprise if they did better than last time. But it also wouldn’t be a surprise if they didn’t; which feels like a deeply stupid bet for an Israeli politician to make.

Nuclear Iran?

Obviously, it’s perfectly sensible to worry about a nuclear-armed Iran; it’s fought recent wars with its neighbors. But it seems to me that it’s the neighbors who should be worrying.

I just can’t see any scenario in which Iran, if it did get a usable bomb that could be delivered at that distance, would try to nuke Israel. Here’s why: The mullahs and Revolutionary Guards who are oppressing Iran are doing just fine, thank you. They’ve managed to fend off the opposition, the oil business is good, and it’s easy for a ruling elite to ignore sanctions pain, as Saddam Hussein showed us all.

If they were so silly as to make Israel feel existentially threatened on Tuesday, Tehran would be a smoking radioactive crater on Wednesday, and that would be bad for business.

American Politics

Someone is fomenting a war. It’s hard to figure out who, and why, exactly. But this is scary as hell.

Right now, US politics is at a fever pitch. Seems to me that this campaign slogan might be worth trying: “Israel shouldn’t attack Iran, and if they do they’re on their own. As President, I won’t start a war with Iran unless they threaten America.”

February 20, 2012

Worse Than FailureCodeSOD: The Sentimental Query

After weeks, months, or even years of setting up and preparing to sell, it’s always exciting to get those few customers. Depending on the venture, it may not be worth popping open a bottle of Champagne, but no matter how small the sale, one can’t help but take a few moments to pause and smile before struggling to get that next batch of sales.

By the time the 13,853rd customer rolls around, no one blinks an eye. It’s just business as usual. That is, unless you work at Ruben’s company. When a new customer record is created, their server not only takes a moment to celebrate, but it nearly brings down the entire server with this unique way of finding the next available customer number:

SELECT * FROM customer
WHERE id_customer  != '4' AND id_customer !='10' AND id_customer !='11'
  AND id_customer !='13' AND id_customer !='14' AND id_customer !='15' 
  AND id_customer !='16' AND id_customer !='17' AND id_customer !='18' 
  AND id_customer !='19' AND id_customer !='20' AND id_customer !='21' 
  AND id_customer !='22' AND id_customer !='23' AND id_customer !='24' 
  AND id_customer !='25' AND id_customer !='26' AND id_customer !='27' 
  AND id_customer !='28' AND id_customer !='29' AND id_customer !='30' 
  AND id_customer !='31' AND id_customer !='32' AND id_customer !='33' 
  AND id_customer !='34' AND id_customer !='35' AND id_customer !='36' 
  AND id_customer !='37' AND id_customer !='38' AND id_customer !='39' 
  AND id_customer !='40' AND id_customer !='41' AND id_customer !='42' 
  AND id_customer !='43' AND id_customer !='44' AND id_customer !='45' 
  AND id_customer !='46' AND id_customer !='47' AND id_customer !='48' 
  AND id_customer !='49' AND id_customer !='50' AND id_customer !='51' 
  AND id_customer !='52' AND id_customer !='53' AND id_customer !='54' 
  AND id_customer !='55' AND id_customer !='56' AND id_customer !='57' 
  AND id_customer !='58' AND id_customer !='59' AND id_customer !='60' 
  AND id_customer !='61' AND id_customer !='62' AND id_customer !='63' 

  ... snip several thousand lines ...

  AND id_customer !='13789' AND id_customer !='13790' AND id_customer !='13792' 
  AND id_customer !='13793' AND id_customer !='13794' AND id_customer !='13795' 
  AND id_customer !='13796' AND id_customer !='13797' AND id_customer !='13798' 
  AND id_customer !='13799' AND id_customer !='13800' AND id_customer !='13801' 
  AND id_customer !='13802' AND id_customer !='13803' AND id_customer !='13804' 
  AND id_customer !='13805' AND id_customer !='13806' AND id_customer !='13807' 
  AND id_customer !='13808' AND id_customer !='13809' AND id_customer !='13810' 
  AND id_customer !='13811' AND id_customer !='13812' AND id_customer !='13813' 
  AND id_customer !='13814' AND id_customer !='13815' AND id_customer !='13816' 
  AND id_customer !='13817' AND id_customer !='13818' AND id_customer !='13819' 
  AND id_customer !='13820' AND id_customer !='13821' AND id_customer !='13822' 
  AND id_customer !='13823' AND id_customer !='13824' AND id_customer !='13825' 
  AND id_customer !='13826' AND id_customer !='13827' AND id_customer !='13828' 
  AND id_customer !='13829' AND id_customer !='13830' AND id_customer !='13831' 
  AND id_customer !='13832' AND id_customer !='13833' AND id_customer !='13834' 
  AND id_customer !='13835' AND id_customer !='13836' AND id_customer !='13837' 
  AND id_customer !='13838' AND id_customer !='13839' AND id_customer !='13840' 
  AND id_customer !='13841' AND id_customer !='13842' AND id_customer !='13843' 
  AND id_customer !='13844' AND id_customer !='13845' AND id_customer !='13846' 
  AND id_customer !='13847' AND id_customer !='13848' AND id_customer !='13849' 
  AND id_customer !='13850' AND id_customer !='13851' AND id_customer !='13852' 
;


XKCDFirst Post

Tim Bray Square Pictures

I like square photographs and wish my camera shot that way. Recently there’s been a flurry of good online talk about picture shapes.

Back in January, Mike Johnson (AKA The Online Photographer) asked: Why Not Square Sensors? This struck a chord with me, and I echoed it on Twitter. Alex Waterhouse-Hayward emailed me an eloquent little essay, mostly in opposition. I asked him to blog it and now there’s The Perfect Square.

Mike Johnson brought in guest blogger Kirk Tuck to write They stole our choice of aspect ratios. Now we’re getting them back. If you’re arguing about pictures, I think you ought to argue with pictures; so I offer Jon Ellis’ Tokyo Square.

Everybody knows that a square format works well for isolating the subject of a portrait.

Lauren Wood

But this one, happily square, is about as much unlike a portrait as can be imagined.

Red and silver truck

Ideally, I’d like the subject matter of a picture to speak as directly as possible to the person looking at it, with as little presentational clutter as possible. I can make an argument that a square presentation represents the minimum possible decoration at the level of shape, and that any departure from it is a choice that you’re imposing on the contents.

I don’t actually believe that. But I do recommend the following practice, which I apply to nearly every photo I edit: Early in the editing process, I jam it into a 1:1 aspect ratio and see if that’s going to work. I don’t always leave it there, but sometimes I do, and some other times I find myself making just a few little adjustments, ending up almost square.

Anyhow, I’d totally love to shoot with a square-sensor camera.

February 18, 2012

Tim Bray On Boats

It was Cottage Life that drove us to it. We like being there but haven’t enjoyed getting there. The water taxis are friendly and efficient, but they run on strict schedules, and leave from places that require fighting through rush-hour traffic. So we bought a boat.

Boating isn’t really a geek thing, and I’m struggling a bit trying to find words that are both interesting and nautical. But it’s taken a lot of our time and attention; “write what you know” they say, and I know a whole lot more about boats than I did a few months back.

Key Findings

What you might want to know if thinking about a boat:

  1. If you walk around a marina, you’ll notice a lot of the boats look neglected. In fact, pleasurecraft are like exercise machines; more than half languish untouched from one year to the next. So think twice first.

  2. Almost nobody buys new. The hulls wear out very slowly, and the things that go inside them (motor, electronics, pumps, you name it) can be replaced. The cost of new boats is shocking, while used boats are surprisingly reasonable.

  3. You can find a ton of boats on Craigslist or wherever, but you probably shouldn’t try to buy one yourself. There are “boat brokers” for the same reasons there are real-estate agents. You need to hire experts to do what they call “surveys” and “mechanicals”, and you don’t know who to call. Also, the broker can find lots of interesting boats for sale that aren’t on Craigslist. See the Credits below for a recommendation.

  4. Most cities with a lot of boats have not-quite-as-many places to moor them. So either you have room to store it on a trailer, plus mad reversing-with-a-heavy-hitch skillz, or you’d better start the parking-spot hunt before the boat hunt.

  5. You just gotta learn the lingo. If you don’t know what cuddy and transom and head and freeboard and kicker (and many more) mean, you’re not going to make it through.

  6. It really is a thrill to head out across part of the Pacific ocean, even if that part is entirely surrounded by a big city.

The Facts of the Case

Here it is, up “on the hard” as we sea-dogs say, being surveyed. It’s a 25-year-old Limestone 24-foot Express Cruiser. They’re still making it, with the design essentially unchanged, but the site is an all-Flash horrorshow; can’t even link to this particular model.

Boat
  1. It’s got the original engine, but that’s been rebuilt a couple of times and it tested out brilliantly.

  2. Basically nothing else is original; the electronics and furniture and galley and head and kicker and pumps and so on have been lovingly upgraded.

  3. The electronics are new and first-rate. We’re talking serious geek joy; the panel has a paged display that supports overlays or side-by-side for radar and/or maps and/or GPS and/or depth-gauge. I haven’t mastered it yet. I wonder if there are hackers’ builds of the software?

  4. Cruises comfortably at around 20kt, which should get us to the cabin in under 90 minutes.

  5. It constitutes a comfy guest bedroom for the cabin.

  6. It cost less than either of the last two automobiles I purchased.

  7. I had a stroke of unreasonably good luck and got a nice moorage at Burrard Civic Marina. As the name suggests, it’s operated by civil servants and ain’t luxurious, but the location couldn’t be better: close to home and right on English Bay.

  8. It will be named Bodoni after an old friend of a similar color.

Less Important Things

For amusement’s sake.

  1. There is nothing remotely hip about powerboats. The new ones, if they’re big, are all about the 1% rubbing it in your face, with furnishings representing the nadir of suburban vacuousness. Smaller craft tend to muscle, and a NASCAR flavor. Used ones express mostly the bad taste of past decades. The one we got is appealingly unadorned outside and comfortable inside, but looking at it doesn’t warm my heart.

    Sailboats, by contrast, are lovely and express more contemporary values. I’ve sailed some and liked it lots. Unfortunately, it’s hugely time-consuming and not really a good choice for getting your family from here to there.

  2. There aren’t many places near town you can tie up your boat and get a nice lunch with a good beer. This seems like a market failure.

  3. “BOAT” stands for “bring out another thousand”. There are many ancillary expenses.

  4. We’re naming the boat after a font. While there are boat-lettering specialists out there, the ones we’ve talked to have displayed a shocking lack of typographic sensibility. This problem remains unsolved.

  5. To drive a boat legally around here, you need something called a Pleasure Craft Operator Card. It’s not exactly hard to get. Confession: I haven’t been able to resist, with nearly every woman I know who’s a good enough friend for this sort of thing, pulling it out, leering, and saying “May I operate your pleasure craft?”. Not one has failed to dissolve in helpless giggles.

Credits

We so totally couldn’t have done this ourselves.

  1. Our boat broker was Paul Shield of Allied/Tri-Shore, and he took outstanding care of us. I’d recommend him unhesitatingly.

  2. Orientation and education by Cooper Boating. If we manage to get there and back with reasonable comfort and without loss of life or limb, it will be thanks to their classroom and on-the-water training.

  3. Boat survey and repair by Stem To Stern, who as a side-effect raised my consciousness about things maritime and mechanical.

February 17, 2012

Worse Than FailureError'd: Departing in 9999 Minutes

Tim Bray Brick Wall

This one is in the bad end of Gastown, across the street from the Alibi Room where we’d been for Valentine’s-day festivities.

Brick Wall in Gastown

Yeah, the light really was that yellow.

XKCDGood Cop, Dadaist Cop

February 15, 2012

etbeLinks February 2012

Sociological Images has an interesting article about the attempts to apply the word “Camping” to OWS and framing the issues [1].

Lester Macgurdy wrote an insightful article about “the snake”, a new technique for OWS protesters to beat riot police [2].

Ron Barassi suggests that “Australia Day” be celebrated on the 27th of May to commemorate the day in 1967 when the Australian constitution was amended to not be racist [3]. The current “Australia Day” is often referred to as “Invasion Day”. IMHO Ron deserves another “Best and Fairest” award.

Stefon Harris gave an entertaining TED talk about improv Jazz music titled “There Are No Mistakes on the Bandstand” [4]. It seems that his concepts can apply to some extent to many collaborative projects.

John Robb wrote an interesting article about the future of drone (UAV) warfare [5]. He suggests that having one person control each drone is a temporary thing and that the future is to have a cloud of cheap autonomous drones taking strategic control from one person. His comparison of Starcraft players to future drone fighters is interesting.

The OWS movement is branching out into other related areas, OccupyYourHomes.org is one of the latest ones [6]. When banks try to forclose on homes without good cause the OWS people are protesting.

Cory Doctorow wrote an important article for The Guardian about corporations using the Youtube ContentID system to pirate works that other people have uploaded [7].

Matt Taibbi’s description of Goldman Sachs as “a great vampire squid wrapped around the face of humanity, relentlessly jamming its blood funnel into anything that smells like money” will never die [8]. It has spawned many other creative descriptions of the evil and greed of Goldman Sachs and even Lloyd Blankfein of Goldman Sachs describes his company as having “burned down the Reichstag, shot the Archduke Ferdinand and fired on Fort Sumter” – he was trying to use satire, but I don’t think that Goldman Sachs people would act differently to Fritz Thyssen.

Keith Packard wrote an interesting article about the Calypso CalDAV system which he uses with Android [9]. He makes lots of good points about how to improve calendaring and contacts on Android, unfortunately I lack time to fiddle with such things at the moment so I’ll stick with Google in spite of the risks.

Asheesh Laroia wrote a great article about the problems with short (32bit) GPG keys [10]. It seems that creating keys with matching ID numbers isn’t particularly difficult and that GPG doesn’t handle them as well as we would like giving the possibility of at best annoying DoS attacks and at worse security problems due to using the wrong key.

Sociological Images has an interesting article about when game show audiences are trustworthy [11]. It seems that French people don’t want an undeserving person to win so they will intentionally advocate the wrong answer if the contestant should know it.

Paul Wayper gave a great lecture titled “SE Linux for Everyone” [12]. He covers the basics of SE Linux in a user-friendly way and explains some simple solutions to common problems which don’t involve compromising system security.

Paul Tassi wrote an insightful article for Forbes about piracy [13]. His conclusion is that the media companies should make it cheaper and easier to be a customer and not spend insane amounts of money on low quality products.

The Reid Report has an interesting article about Ron Paul’s racism [14]. Ron Paul is generally well regarded outside the US because he wants the US government to stop meddling in the affairs of other countries, but while he’s less bad than other US politicians in terms of foreign policy that doesn’t make him a good person.

Anonymous hacked some mailboxes belonging to a neo-Nazi group and found links to Ron Paul [15]. I’ve always been suspicious of the way Ron Paul wanted to avoid anti-racism legislation on supposed Libertarian principles.

The Reid Report has an interesting summary of Ron Paul news plus some criticism of Glenn Greenwald and others who associate with him [16].

Related posts:

  1. Links February 2011 Australia’s Department of Finance has mandated that the MS-Office document...
  2. Links January 2012 Cops in Tennessee routinely steal cash from citizens [1]. They...
  3. Links February 2009 Michael Anissimov writes about the theft of computers from the...

February 14, 2012

XKCDBackward in Time

February 13, 2012

etbeCooling a Thinkpad

Late last year I wrote about the way that modern laptops suck [1]. One of the problems that inspired that post was the excessive heat generated by my Thinkpad T61.

There is a partial solution to this, Fool Control explains how the kernel option pcie_aspm=force can be used on kernels from 2.6.38 onwards to solve a heat regression problem [2]. I applied this to my Thinkpad T61 and the result was that on a cool evening (ambient temperature about 24C) the temperature changed from 85C to 66C on the NVidia video card, and for the “virtual devices” it changed from 80C and 78C to 60C and 61C. I’m not sure exactly what each of those measurements refers to, but it seems that the change was somewhere between 17C and 20C.

This changes the system from being almost unbearable to use to being merely annoyingly warm.

I’m not going to make my laptop be my primary computing device again though, the combination of a desktop system with a 27″ monitor and an Android phone is working quite well for me [3]. But I haven’t yet got version control systems working for all my software. Also Wouter suggested using NBD which is something I haven’t got working yet and probably won’t until I can swap on it and therefore have a diskless workstation. Finally I still haven’t got the “Chrome to Phone” browser extension working such that a page I’m viewing at home can be loaded on my phone.

Related posts:

  1. Taking my Thinkpad Apart and Cooling Problems I’ve been having some cooling problems with my Thinkpad recently....
  2. thinkpad back from repair On Tuesday my Thinkpad was taken for service to fix...
  3. I Just Bought a new Thinkpad and the Lenovo Web Site Sucks I’ve just bought a Thinkpad T61 at auction for $AU796....

February 11, 2012

etbeMagic entries for BTRFS and Software RAID

I’ve just discovered that the magic database for the file(1) command in Debian/Unstable has no support for Linux Software RAID and that it’s support for BTRFS is lacking (no reporting of space used, number of devices, or the UUID). Below is my first draft of a change to fix these problems. I would appreciate it if someone with a big-endian system could test these out and let me know how they go, I suspect that I will have to change the “lelong” types to “long” but I’m not sure.

4096 lelong 0xa92b4efc Linux Software RAID
>4100 lelong x version 1.2 (%d)
>4112 belong x UUID=%8x:
>4116 belong x \b%8x:
>4120 belong x \b%8x:
>4124 belong x \b%8x
>4128 string x name=%s
>4168 lelong x level=%d
>4188 lelong x disks=%d

0 lelong 0xa92b4efc Linux Software RAID
>4 lelong x version 1.1 (%d)
>16 belong x UUID=%8x:
>20 belong x \b%8x:
>24 belong x \b%8x:
>28 belong x \b%8x
>32 string x name=%s
>72 lelong x level=%d
>92 lelong x disks=%d

# BTRFS
0×10040 string _BHRfS_M BTRFS Filesystem
>0x1012b string >\0 label "%s",
>0×10090 lelong x sectorsize %d,
>0×10094 lelong x nodesize %d,
>0×10098 lelong x leafsize %d,
>0×10020 belong x UUID=%8x-
>0×10024 beshort x \b%4x-
>0×10026 beshort x \b%4x-
>0×10028 beshort x \b%4x-
>0x1002a beshort x \b%4x
>0x1002c belong x \b%8x,
>0×10078 lequad x %lld/
>0×10070 lequad x \b%lld bytes used,
>0×10088 lequad x %lld devices

Related posts:

  1. Label vs UUID vs Device Someone asked on a mailing list about the issues related...
  2. Starting with BTRFS Based on my investigation of RAID reliability [1] I have...
  3. Software vs Hardware RAID Should you use software or hardware RAID? Many people claim...

February 10, 2012

etbeStarting with BTRFS

Based on my investigation of RAID reliability [1] I have determined that BTRFS [2] is the Linux storage technology that has the best potential to increase data integrity without costing a lot of money. Basically a BTRFS internal RAID-1 should offer equal or greater data protection than RAID-6.

As BTRFS is so important and so very different to any prior technology for Linux it’s not something that can be easily deployed in the same way as other filesystems. It is possible to easily switch between filesystems such as Ext4 and XFS because they work in much the same way, you have a single block device which the filesystem uses to create a single mount-point. While BTRFS supports internal RAID so it may have multiple block devices and it may offer multiple mountable filesystems and snapshots. Much of the functionality of Linux Software RAID and LVM is covered by BTRFS. So the sensible way to deploy BTRFS is to give it all your storage and not make use of any other RAID or LVM.

So I decided to do a test installation. I started with a Debian install CD that was made shortly before the release of Squeeze (it was first to hand) and installed with BTRFS for the root filesystem, I then upgraded to Debian/Unstable to get the latest kernel as BTRFS is developing rapidly. The system failed on the first boot after upgrading to Unstable because the /etc/fstab entry for the root filesystem had the FSCK pass number set to 1 – which wasn’t going to work as no FSCK program has been written. I changed that number to 0 and it then worked.

The initial install was on a desktop system that had a single IDE drive and a CD-ROM drive. For /boot I used a degraded RAID-1 and then after completing the installation I removed the CD-ROM drive and installed a second hard drive, after that it was easy to add the other device to the RAID-1. Then I tried to add a new device to the BTRFS group with the command “btrfs device add /dev/sdb2 /dev/sda2” and was informed that it can’t do that to a mounted filesystem! That will decrease the possibilities for using BTRFS on systems with hot-swap drives, I hope that the developers regard it as a bug.

Then I booted with an ext3 filesystem for root and tried the “btrfs device add /dev/sdb2 /dev/sda2” again but got the error message “btrfs: sending ioctl 5000940a to a partition!” which is not even found by Google.

The next thing that I wanted to do was to put a swap file on BTRFS, the benefits for having redundancy and checksums on swap space seem obvious – and other BTRFS features such as compression might give a benefit too. So I created a file by using dd to take take from /dev/zero, ran mkswap on it and then tried to run swapon. But I was told that the file has holes and can’t be used. Automatically making zero blocks into holes is a useful feature in many situations, but not in this case.

So far my experience with BTRFS is that all the basic things work (IE storing files, directories, etc). But the advanced functions I wanted from BTRFS (mirroring and making a reliable swap space) failed. This is a bit disappointing, but BTRFS isn’t described as being ready for production yet.

Related posts:

  1. Discovering OS Bugs and Using Snapshots I’m running Debian/Unstable on an EeePC 701, I’ve got an...
  2. Reliability of RAID ZDNet has an insightful article by Robin Harris predicting the...
  3. How I Partition Disks Having had a number of hard drives fail over the...

February 08, 2012

Paul FremantleCarbon's 3rd Birthday

Three years ago I blogged about WSO2 Carbon for the first time. I enthused about a composable server architecture and why it was important for a SOA platform.

At that point there were just 4 Carbon products. Today there are 13 products, the core framework, Stratos, and Carbon Studio all based around the Carbon Architecture.

There are two really important things I think have worked really well:

  • The composability is obviously important and we now have a set of customers doing exactly that - using p2 to combine the correct components and effectively "build your own server" with the right function for their specific needs and requirements.

    But even more interesting for those customers has been the consistency and completeness of the platform that has arisen out of Carbon. The fact that there is a "menu" or palette of low-level components (features) and high level components (products) that all interoperate, behave the same, use the same identity, clustering, registry, key management, etc has really offered customers the opportunity to build out compelling architectures.
  • The kernelization is almost just a corollary of componentization: as you build the components and identify which ones to re-use, we found a common core across those 13 products. The result is that when we came to address cloud - starting later in 2009 - we very quickly realized that there were just a few core places in which to address multi-tenancy, elasticity and metering. I have to admit this was something I for one hadn't thought of, but has been probably the most powerful driver behind the success of StratosLive and Stratos. 
There were certainly people then who doubted that OSGi was a stable basis for an ESB or Server and I hope that the 1bn+ transactions a day that eBay are doing through WSO2 ESB are enough to disprove that.

All in all, it is amazing to see how far Carbon has come and what it has enabled in 3 years.


October 25, 2011

Kelvin Lawrence - personalFour years post surgery, four years in remission!

Today is the fourth anniversary of my colon cancer surgery. As my oncologist once explained to me, the remission clock starts ticking from the date of the "curative procedure" (surgery). Therefore as of today I have now been in remission for a full four years. Much of the time since the surgery and the subsequent months of chemotherapy remains a bit of a blur and loking back, the time seems to have flown by.. I am so thankful to all of the wonderful doctors and nurses that have cared for me so well and that I am still here to enjoy life and watch my kids grow up.

As always I am incredibly grateful to all of my family and friends around the World who have been there for me the past four years. I could not have made it this far without you. The recent sad news about the death of Steve Jobs reminds us that no one, no matter how rich or poor is immune from cancer's evil reach. I encourage all of you to support cancer research in any way that you can. Likewise, to anyone reading this who has also been touched by cancer, I would just like to say keep up the fight and that you are in my thoughts and prayers.

PS - If you have been putting off getting a colonoscopy, get one. It could save your life!

August 23, 2011

Kelvin Lawrence - personalThe last (official) day of Summer

By the school calendar, which around here is the one most people live by, as of today, Summer is officially over. This is despite the fact that this week we will most likely break the old record of 69 one hundred degree days in a calendar year. The high yesterday was around 105F and it is forecast to be about the same again today. I forget when we last had serious rain but it has been months. Unless we get a tropical storm come ashore soon it may be many more weeks until we do get some rain. I love the hot weather but in my fifteen years here I have never seen a drought like this one. We usually get some good storms, especially during the Spring to top up the lakes. This year that did not happen.


Robyn and Jack leaving for school
Robyn and Jack about to leave for school

Anyway, my kids happily (yes happily) went back to school today. They even got up at 6:20am and woke me up! Robyn is going into eighth grade and Jack fourth. They have grown so quickly. Robyn is thirteen now and Jack will be ten in December. Where does the time go? Anyway, above is a photo, which has become a bit of a tradition now, of the kids going out the front door on the first day of a new school year.

August 05, 2011

Kelvin Lawrence - personal25 Years at IBM

In 1986, August the 4th was a Monday and it was my first day as an IBM employee (at Hursley Park in the UK). Twenty five years later, I'm still working at IBM (these days in Austin, Texas).  It's been an incredible journey both emotionally , technologically and geographically (from Hursley to Austin with seven years in Boca Raton, Florida in between). I have worked with some fantastic people on some very cool projects and been lucky enough to make great friends all over the World.

The 25 years have flown by!

July 19, 2011

Kelvin Lawrence - personalWhere have I been hiding?

Those of you that check-in here from time to time, and the site statistics tell me that there are still a lot of you, for which I am grateful, may be wondering what I have been up to and why I have not been posting of late. In short I have been busy at work, very busy with the kids and also had to have a spot of unexpected (and very painful) surgery from which I have been recovering since mid June.

Anyway, here is a quick update on the Summer so far...

Things medical

At the end of May I met with my colo-rectal surgeon and he performed my (now semi-annual) internal exam. Everything looked good. However, while he was checking me out he told me that I had a cyst that needed to be dealt with surgically (not cancerous). More on that in a minute!

On June 9th I met with my oncologist and he ran all of the usual tests. While I always hope for the best, this is always a nervous time for me. None the less, I am happy to report that all of my tests came back clean. Still no evidence of further cancer. I can now go twelve months before he wants to check me again. This is significant progress.

On June 10th I had surgery to take care of the cyst. The procedure ended up being a bit more involved that I had hoped it would be. Without going into details it was in an awkward place and for several weeks it was really hard to sit down and I was in a lot of pain. The wound is still healing but is in much better shape now. Hopefully in a another week or so it will have finished healing. The surgery really knocked me out of my stride for a few weeks and even now I do not feel that I fully have my energy back.

On top of a few other stresses in my life at the moment I really feel like a cannot catch a break. I hope to take a couple of weeks off soon and find a beach to lay on somewhere or something equally restful!

The Kids

Despite the intense drought and soaring temperatures we have been enduring for well over six weeks now, the Kids are having a great Summer and spending a lot of time in the pool. Robyn was able to escape the heat mid June, when she left along with 40 other kids and four adult leaders on a People to People expedition to the UK and Ireland. They visited an amazing set of places all over England, Scotland, Wales and Ireland. She got to do many things that even I, growing up there, have never done. She got to hit golf balls at St Andrews, ride the London Eye, watch a Trebuchet catapult launching a fire ball at Warwick Castle, she got to abseil down the walls of another castle in Bangor (Wales) and so many other amazing things. She took over 900 photos! I think this trip will leave her with fantastic memories and hopefully a few new friends as well!

Work

Work is very busy which is a good thing and has also helped keep me distracted a bit from other things.

Finally, less blogging more tweeting....

Finally, I will observe that while I have not been posting here much lately, that I have been making regular posts to both Twitter and Facebook and more recently Google+ as well. If you have not already done so, feel free to find me at one or more of those venues.

That's the update for now. Take care all!

April 26, 2011

Paul FremantleWSO2 Workshops in Europe

If any of you are interested in Identity, Security and Entitlement and how these fit into a SOA, Cloud and Enterprise Architectures, then I'd recommend you taking a look at our workshop that we are running in London, Paris, Zurich and Frankfurt.

PS I think I'm meant to be the one on the left with St. George's flag!



April 21, 2011

Paul FremantlePaul

April 05, 2011

Paul FremantleIntroduction to WSO2 Message Broker

<style></style>
Introduction to WSO2 Message Broker<o:p></o:p>

Introduction<o:p></o:p>
WSO2 Message Broker (MB) is a new Open Source project and product from WSO2 that provides messaging functionality within the WSO2 Carbon platform and to other clients in various languages. It works either standalone or in conjunction with products and components such as the WSO2 ESB and WSO2 Complex Event Processing Server. <o:p></o:p>

MB is based on the Apache Qpid/Java project (http://qpid.apache.org). From Apache Qpid, MB gets core support for the AMQP protocol and JMS API. On top of that WSO2 has added support for Amazon SQS APIs and WS-Eventing support.<o:p></o:p>

Understanding how the MB broker fits into Enterprise Architecture<o:p></o:p>
The Message Broker provides three main capabilities into an overall Enterprise Architecture:<o:p></o:p>
·               A queueing/persistent message facility<o:p></o:p>
·               An event distribution (pub/sub) model<o:p></o:p>
·               An intermediary where multiple systems can connect irrespective of the direction of messages.<o:p></o:p>

To give some concrete examples of these benefits, here are some scenarios:<o:p></o:p>
1)   In the WSO2 ESB, a common pattern is to persist the message from an incoming HTTP request into a persistent message queue, and then process onbound from there. MB can provide the persistent queue.<o:p></o:p>
2)   The WSO2 ESB already has an event distribution model and eventing support, but the QPid-based broker provides higher performance as well as supporting the JMS API.<o:p></o:p>
3)   For example, you may wish to send messages from outside a firewall to a server inside. You could connect an ESB or Service Host within the firewall to a Message Broker running outside the firewall (for example on Amazon EC2). This model is used by the WSO2 Cloud Services Gateway.<o:p></o:p>

Where does AMQP fit?<o:p></o:p>

AMQP (www.amqp.org) is an open protocol for messaging. Whilst the AMQP protocol is still under development, it has released three stable releases (0-8, 0-9-1, and 0-10), with a 1.0 due during 2011. There are a number of implementations of the AMQP standard in production, including Apache Qpid (both Java and C++ versions), RabbitMQ, OpenAMQ and others.<o:p></o:p>

WSO2 has been a member of the AMQP working group for several years, and we strongly support AMQP as the way to introduce interoperability and greater openness into the messaging space.<o:p></o:p>

The Qpid broker supports a variety of clients on top of the AMQP protocol. The most useful of these for Carbon is the Java JMS 1.1 API, which provides a portable API as well as the main interface with the WSO2 ESB. In addition there are C# and other APIs. WSO2 MB also extends these with WS-Eventing and Amazon SQS APIs for interoperability using HTTP, REST and SOAP.<o:p></o:p>

Installing the WSO2 MB<o:p></o:p>

You can download the WSO2 MB Beta from:<o:p></o:p>

Once you have downloaded and unzipped, simply switch to the install directory<o:p></o:p>

            cd wso2mb-1.0.0-SNAPSHOT<o:p></o:p>
            bin\wso2server.bat  [ON WINDOWS]<o:p></o:p>
            bin/wso2server.sh [ON LINUX/MACOSX]<o:p></o:p>

Let’s refer to the install directory as <wso2mb> from now on.<o:p></o:p></wso2mb>

You should see the server startup:<o:p></o:p>
[2011-03-16 14:00:12,471]  INFO {org.wso2.carbon.server.Main} -  Initializing system...<o:p></o:p>
[2011-03-16 14:00:12,840]  INFO {org.wso2.carbon.server.TomcatCarbonWebappDeployer} -  Deployed Carbon webapp: StandardEngine[Tomcat].StandardHost[defaulthost].StandardContext[/]<o:p></o:p>
[2011-03-16 14:00:14,147]  INFO {org.wso2.carbon.atomikos.TransactionFactory} -  Starting Atomikos Transaction Manager 3.7.0<o:p></o:p>
[2011-03-16 14:00:19,952]  INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} -  Starting WSO2 Carbon...<o:p></o:p>
[2011-03-16 14:00:19,983]  INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} -  Operating System : Mac OS X 10.6.6, x86_64<o:p></o:p>
[2011-03-16 14:00:19,984]  INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} -  Java Home        : /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home<o:p></o:p>
[2011-03-16 14:00:19,984]  INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} -  Java Version     : 1.6.0_24<o:p></o:p>
[2011-03-16 14:00:19,985]  INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} -  Java VM          : Java HotSpot(TM) 64-Bit Server VM 19.1-b02-334,Apple Inc.<o:p></o:p>
[2011-03-16 14:00:19,985]  INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} -  Carbon Home      : /Users/paul/wso2/wso2mb-1.0.0-SNAPSHOT<o:p></o:p>
[2011-03-16 14:00:19,985]  INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} -  Java Temp Dir    : /Users/paul/wso2/wso2mb-1.0.0-SNAPSHOT/tmp<o:p></o:p>
[2011-03-16 14:00:19,986]  INFO {org.wso2.carbon.core.internal.CarbonCoreActivator} -  User             : paul, en-US, Europe/London<o:p></o:p>
2011-03-16 14:00:12,471]  INFO {org.wso2.carbon.server.Main} -  Initializing system...<o:p></o:p>

some logs deleted<o:p></o:p>

[2011-03-16 14:00:41,691]  INFO {org.wso2.carbon.core.transports.http.HttpsTransportListener} -  HTTPS port       : 9443<o:p></o:p>
[2011-03-16 14:00:41,691]  INFO {org.wso2.carbon.core.transports.http.HttpTransportListener} -  HTTP port        : 9763<o:p></o:p>
[2011-03-16 14:00:42,422]  INFO {org.wso2.carbon.ui.internal.CarbonUIServiceComponent} -  Mgt Console URL  : https://192.168.1.100:9443/carbon/<o:p></o:p>
[2011-03-16 14:00:42,499]  INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} -  Started Transport Listener Manager<o:p></o:p>
[2011-03-16 14:00:42,500]  INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} -  Server           :  WSO2 MB -1.0.0-SNAPSHOT<o:p></o:p>
[2011-03-16 14:00:42,506]  INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} -  WSO2 Carbon started in 27 sec<o:p></o:p>
2011-03-16 14:00:12,471]  INFO {org.wso2.carbon.server.Main} -  Initializing system...<o:p></o:p>

WSO2 Message Broker is installable in more ways for production systems. Typically it is either registered as a Linux Daemon or as a Windows Service – but for now we will stick with the command-line version for simplicity.<o:p></o:p>

Once the server is running you can access the management console. Point your browser at:<o:p></o:p>
            https://localhost:9443 <o:p></o:p>

Initially you will see a browser screen warning you about the certificates. Please tell your browser to continue (For a production server you would normally install a proper SSL/TLS certificate, but for initial install we generate a self-signed certificate that you need to agree to use).<o:p></o:p>

Once you have accepted the certificate, you should see a screen like:<o:p></o:p>


<o:p></o:p>
You can login using the default user/password which is admin/admin.<o:p></o:p>

Once you login you should see the following screen:<o:p></o:p>


<o:p></o:p>
Before we examine the admin console, lets first create a simple JMS client that will communicate with the server via AMQP on TCP/IP. <o:p></o:p>

Getting Started with JMS<o:p></o:p>
The Java Message Service (JMS) specification - http://www.oracle.com/technetwork/java/index-jsp-142945.html - is a specification for talking to message brokers. It is unfortunately poorly named: the word “service” implies this is an implementation, but JMS does not define an actual messaging service, instead just the API which is used to access JMS providers. “Java Messaging API” would more accurately express what JMS is. The result is that there are a variety of JMS providers, and they often have quite different approaches to their core model. <o:p></o:p>

The WSO2 Message Broker is based on the Apache Qpid project (http://qpid.apache.org) and is a compliant implementation of the JMS specification, as well as various levels of the AMQP specification (0-8, 0-9-1, 0-10).  <o:p></o:p>

To write completely standard portable JMS code, you need to use a JNDI provider to gain access to the JMS connection, queues, etc. In this example we will use a Qpid JNDI provider backed by a simple set of properties. This makes the overall system simple and highly portable. <o:p></o:p>

Here is a sample JMS application that can be used to test access to the Message Broker.  You can find this code here: <o:p></o:p>



First are some required imports. <o:p></o:p>

import javax.jms.*;<o:p></o:p>
import javax.naming.Context;<o:p></o:p>
import javax.naming.InitialContext;<o:p></o:p>
import java.util.Properties;<o:p></o:p>

Next is a simple “main” class definition:<o:p></o:p>

public class JMSExample {<o:p></o:p>

public static void main(String[] args) {<o:p></o:p>
       JMSExample producer = new JMSExample();<o:p></o:p>
       producer.runTest();<o:p></o:p>
}<o:p></o:p>

       <o:p></o:p>
private void runTest() {<o:p></o:p>

Since this is just an example, we will place the complete logic in a try/catch block.<o:p></o:p>

try {<o:p></o:p>

Normally the JNDI is configured by a properties file, but you can also do it from an in-memory set of properties. To see a similar setup with a properties file, take a look at the ESB example below.  Here is a properties object to store the properties:<o:p></o:p>

Properties properties = new Properties();<o:p></o:p>

In order to bootstrap the JNDI entries for the connection factory and queue, we set name/value pairs into the simple properties object:                    <o:p></o:p>

properties.put("connectionfactory.cf", <o:p></o:p>
       "amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672'");<o:p></o:p>

The property name “connectionfactory.cf” denotes that we are creating an object of type ConnectionFactory with name “cf”. The value is a URL that is used to bootstrap the ConnectionFactory: this URL points to the AMQP broker. The syntax is broken up as follows:<o:p></o:p>
       amqp://               Indicates this is an AMQP URL<o:p></o:p>
       admin:admin@  This is the username/password <o:p></o:p>
       carbon/carbon The client ID and virtual host<o:p></o:p>
       ?                     separator for options<o:p></o:p>
       brokerlist=’tcp://localhost:5672’          A list of broker URLs to use<o:p></o:p>

For more information on this URL syntax please see:<o:p></o:p>

The virtual host name is part of the definition in:<o:p></o:p>
<wso2mb>/repository/conf/qpid/etc/virtualhosts.xml <o:p></o:p></wso2mb>

This file also defines aspects such as the maximum number of messages in a queue and the queue depth (maximum size in bytes of the queue).<o:p></o:p>

Now we need to create a JNDI entry for the queue we are going to talk to:<o:p></o:p>

properties.put("destination.samplequeue", "samplequeue; {create:always}");<o:p></o:p>

The property name “destination.samplequeue” indicates creating a destination with a JNDI name of “samplequeue”. The property value “samplequeue; {create:always}” indicates a queue named “samplequeue” with an attribute which tells the broker to create the queue if it doesn’t exist.<o:p></o:p>

These properties are specific to the particular JNDI implementation we are using, which is the Qpid “PropertiesFileInitialContextFactory”. So now we need to configure JNDI to use this implementation:<o:p></o:p>
                      <o:p></o:p>
                      <o:p></o:p>
properties.put("java.naming.factory.initial", "org.apache.qpid.jndi.PropertiesFileInitialContextFactory");<o:p></o:p>

Now we can do our JNDI lookups:<o:p></o:p>

Context context = new InitialContext(properties);<o:p></o:p>

ConnectionFactory connectionFactory = <o:p></o:p>
(ConnectionFactory) context.lookup("cf");<o:p></o:p>

Having “found” a JMS Connection Factory in the JNDI, we can now create a connection to the broker:<o:p></o:p>

Connection connection = connectionFactory.createConnection();<o:p></o:p>
connection.start();<o:p></o:p>

And now we can create a JMS Session:<o:p></o:p>

Session session = connection.createSession(false,<o:p></o:p>
                                    Session.AUTO_ACKNOWLEDGE);<o:p></o:p>

One more lookup from JNDI will lookup our queue:<o:p></o:p>

Destination destination = (Destination) context<o:p></o:p>
                                    .lookup("samplequeue");<o:p></o:p>


Now we can create a Producer, and send a message:<o:p></o:p>
                      <o:p></o:p>
MessageProducer producer = session.createProducer(destination);<o:p></o:p>
TextMessage outMessage = session.createTextMessage();<o:p></o:p>
outMessage.setText("Hello World!");<o:p></o:p>
producer.send(outMessage);<o:p></o:p>

Of course, in real life you would most likely NOT now retrieve that same message from the same application, but for this example we will now retrieve the message:                       <o:p></o:p>
MessageConsumer consumer = session.createConsumer(destination);<o:p></o:p>
Message inMessage = consumer.receive();<o:p></o:p>
System.out.println(((TextMessage)inMessage).getText());<o:p></o:p>

And close up the connection and the initial context:<o:p></o:p>
                      <o:p></o:p>
connection.close();<o:p></o:p>
context.close();<o:p></o:p>
} catch (Exception exp) {<o:p></o:p>
       exp.printStackTrace();<o:p></o:p>
}<o:p></o:p>

To try out this client you need the correct client JARs. <o:p></o:p>

In the beta release you will find:<o:p></o:p>
<wso2mb>/jms-client-lib/geronimo-jms_1.1_spec-1.1.0.wso2v1.jar<o:p></o:p></wso2mb>
<wso2mb>/jms-client-lib/qpid-client-0.9.wso2v2.jar<o:p></o:p></wso2mb>

You also need to reference<o:p></o:p>
<wso2mb>/lib/log4j-1.2.13.jar<o:p></o:p></wso2mb>





Once you have those in your classpath you can run the program. You should see some simple output:<o:p></o:p>

log4j:WARN No appenders could be found for logger (org.apache.qpid.jndi.PropertiesFileInitialContextFactory).<o:p></o:p>
log4j:WARN Please initialize the log4j system properly.<o:p></o:p>
Hello World!<o:p></o:p>

If you got that far, congratulations!<o:p></o:p>

In the next section we are going to look at using the ESB with the Message Broker.<o:p></o:p>

There are two approaches for this:<o:p></o:p>
1) If you are using the existing WSO2 ESB 3.0.1 or similar, you can deploy the MB client libraries and communicate using the network.<o:p></o:p>

2) As of the next WSO2 ESB release (3.1.0) it will include the Qpid/MB features as part of the release and you can utilize the Message Broker/JMS runtime locally in the same JVM.<o:p></o:p>

WSO2 MB and WSO2 ESB together<o:p></o:p>
In this first instance we are going to get the WSO2 ESB and MB to work together. <o:p></o:p>
Assuming that you already have the MB installed and running, you will first need to install the ESB and change the ports of the admin console so that they don’t clash. You can download WSO2 ESB 3.0.1 from:<o:p></o:p>
            http://wso2.org/downloads/esb <o:p></o:p>

The install procedure is similar: unzip the ESB, but don’t start it up yet. Let’s name (for this guide) the directory where you installed the ESB as <wso2esb>.<o:p></o:p></wso2esb>

First let’s edit the ports on which the ESB listens. (Alternatively you could do the same to the MB instead).<o:p></o:p>

Edit the <wso2esb>\repository\conf\mgt-transports.xml<o:p></o:p></wso2esb>

This file defines which ports the management console runs (HTTP and HTTPS). <o:p></o:p>

Please change:<o:p></o:p>
<transport class="org.wso2.carbon.server.transports.http.HttpTransport" name="http"><o:p></o:p></transport>

<transport name="http" class="org.wso2.carbon.server.transports.http.HttpTransport">
        <parameter name="port">9763</parameter>

to read:
<transport name="http" class="org.wso2.carbon.server.transports.http.HttpTransport">
        <parameter name="port">9764</parameter>

Similarly change the HTTPS port to be 9444.

Now the next step is to ensure that the ESB has the right drivers to talk to the MB. Copy the following JARs into the <wso2esb>\repository\components\lib directory:<o:p></o:p></wso2esb>
<wso2mb>/jms-client-lib/geronimo-jms_1.1_spec-1.1.0.wso2v1.jar<o:p></o:p></wso2mb>
<wso2mb>/jms-client-lib/qpid-client-0.9.wso2v2.jar<o:p></o:p></wso2mb>


We also need to configure the JMS transport correctly. To do this we edit the axis2.xml file:<o:p></o:p>
 <wso2esb>\repository\conf\axis2.xml<o:p></o:p></wso2esb>

This file has the JMS transport commented out. It also needs the settings updated to use the Qpid libraries. Change the file so that the JMS receiver and sender sections look like this:<o:p></o:p>
  <transportreceiver class="org.apache.axis2.transport.jms.JMSListener" name="jms"><o:p></o:p></transportreceiver>
       <transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">
        <parameter name="myTopicConnectionFactory" locked="false">
         <parameter name="java.naming.factory.initial" locked="false">org.apache.qpid.jndi.PropertiesFileInitialContextFactory</parameter>
         <parameter name="java.naming.provider.url" locked="false">resources/jndi.properties</parameter>
         <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">TopicConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>
        </parameter>


        <parameter name="myQueueConnectionFactory" locked="false">
         <parameter name="java.naming.factory.initial" locked="false">org.apache.qpid.jndi.PropertiesFileInitialContextFactory</parameter>
         <parameter name="java.naming.provider.url" locked="false">resources/jndi.properties</parameter>
         <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
        </parameter>


        <parameter name="default" locked="false">
         <parameter name="java.naming.factory.initial" locked="false">org.apache.qpid.jndi.PropertiesFileInitialContextFactory</parameter>
         <parameter name="java.naming.provider.url" locked="false">resources/jndi.properties</parameter>
         <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
        </parameter>
    </transportReceiver>
You can find my copy of the edited axis2.xml here:<o:p></o:p>

If you have looked through the JMS config you will notice it references a JNDI resource: resources/jndi.properties.<o:p></o:p>

This is used to do the same thing the hard-coded properties we used above do – configure the local JNDI that the JMS client inside the ESB will use. In a future release of the ESB we expect to automatically configure this JNDI, but in the meantime, we can simply create a file in the <wso2esb>/resources directory.<o:p></o:p></wso2esb>

Please create <wso2esb>/resources/jndi.properties to look like this:<o:p></o:p></wso2esb>

connectionfactory.TopicConnectionFactory = \<o:p></o:p>
amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672'<o:p></o:p>
connectionfactory.QueueConnectionFactory = \<o:p></o:p>
amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672'<o:p></o:p>
destination.dynamicQueues/myqueue=jmsdestinationqueue; {create:always}<o:p></o:p>
destination.myqueue=jmsdestinationqueue; {create:always}<o:p></o:p>


Please note that the lines ending \ are actually split for formatting and should be one continuous line.<o:p></o:p>

You can find this file here:<o:p></o:p>

Now we should be able to start the ESB. Of course it won’t actually do anything yet.<o:p></o:p>


Just for interest, you can try starting the WSO2 ESB with the MB stopped. Now that the JMS transport is enabled, you should see connection errors:<o:p></o:p>

javax.jms.JMSException: Error creating connection: Connection refused<o:p></o:p>
       at org.apache.qpid.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:286)<o:p></o:p>
       at org.apache.axis2.transport.jms.JMSUtils.createConnection(JMSUtils.java:579)<o:p></o:p>
       at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.createConnection(ServiceTaskManager.java:803)<o:p></o:p>
       at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.getConnection(ServiceTaskManager.java:688)<o:p></o:p>
       at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.receiveMessage(ServiceTaskManager.java:487)<o:p></o:p>
       at org.apache.axis2.transport.jms.ServiceTaskManager$MessageListenerTask.run(ServiceTaskManager.java:412)<o:p></o:p>
       at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:58)<o:p></o:p>


The ESB will still start but the JMS transport will be disabled. <o:p></o:p>

If you start the MB, then the ESB should start fine. You will however see some warning lines:<o:p></o:p>
[2011-04-01 09:14:05,320]  WARN - JMSUtils Cannot locate destination : WSDLValidatorService<o:p></o:p>

This is because the ESB is binding internal services to the JMS transport. In the most recent builds of the ESB this has been changed so that the ESB only binds internal services to HTTP/S transports to avoid this. <o:p></o:p>




If you go to the Message Broker web console, you can now see the queues that have been created to support the ESB. Simply click on the left-hand menu item QueuesàList.<o:p></o:p>

<o:p></o:p>

Now we can create a simple proxy service that will test the JMS connectivity. This is a slight variation on one of the standard ESB Samples.<o:p></o:p>

This proxy service expects a SOAP or XML message via HTTP POST from a client and simply puts the body of this message into a JMS queue. The server then responds with an HTTP 202 Accepted to the client. This is a great test, because we can use something as simple as curl to post messages into the ESB.<o:p></o:p>

Here is the proxy definition for the ESB:<o:p></o:p>

<o:p></o:p>
<proxy name="testJMS" startonload="true" trace="disable" transports="https jms http" xmlns="http://ws.apache.org/ns/synapse"><o:p></o:p></proxy>

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse" name="testJMS" transports="https jms http" startOnLoad="true" trace="disable">
    <target>
        <endpoint name="jmsqueue">
            <address uri="jms:/myqueue?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&amp;java.naming.factory.initial=org.apache.qpid.jndi.PropertiesFileInitialContextFactory&amp;java.naming.provider.url=resources/jndi.properties"/>
        </endpoint>
        <inSequence>
        <property action="set" name="OUT_ONLY" value="true"/>
        <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>    
        </inSequence>
        <outSequence>
            <send/>
        </outSequence>
    </target>
</proxy>
<o:p></o:p>

This file is available at http://people.apache.org/~pzf/MB/testJMS.xml <o:p></o:p>

You need to place this file here:<o:p></o:p>
<wso2esb style="font-style: normal;">/repository/conf/synapse-config/proxy-services/testJMS.xml<o:p></o:p></wso2esb>

This is a nice feature of the ESB. Effectively you can configure independent proxy services, each with their own config file or registry entry, and the ESB amalgamates them at runtime to create a single consistent ESB. This is great for doing incremental changes. You can even change this file at runtime and have the proxy hot-deployed.<o:p></o:p>

The proxy service is really simple. Basically it just sets the destination to send the message on to the JMS queue, which is defined using a combination of JNDI and the JMS URL. <o:p></o:p>

The JMS URL is made up of:<o:p></o:p>
jms:/myqueue            <o:p></o:p>
Look for a JNDI entry “myqueue” <o:p></o:p>
(see jndi properties above)<o:p></o:p>
?    <o:p></o:p>
            Separator indicating extra attributes<o:p></o:p>

transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory<o:p></o:p>
            Look up ConnectionFactory in JNDI with name<o:p></o:p>
 QueueConnectionFactory                   <o:p></o:p>
& <o:p></o:p>
            Separator (this will convert to ‘&’)<o:p></o:p>
java.naming.factory.initial=<o:p></o:p>
org.apache.qpid.jndi.PropertiesFileInitialContextFactory<o:p></o:p>
Use the Qpid properties-based JNDI we saw earlier<o:p></o:p>
&<o:p></o:p>
                        Another separator      <o:p></o:p>
java.naming.provider.url=resources/jndi.properties<o:p></o:p>
                        Look in resources/jndi.properties for the JNDI properties file<o:p></o:p>

The next part of the proxy configuration simply tells the ESB this is a one-way flow and not to expect a response:<o:p></o:p>
<property action="set" name="OUT_ONLY" style="font-style: normal;" value="true"><o:p></o:p></property>

The next line ensures the ESB sends back an HTTP 202 Accepted response to the client:<o:p></o:p>
<property name="FORCE_SC_ACCEPTED" scope="axis2" style="font-style: normal;" value="true">    <o:p></o:p></property>
        <o:p></o:p>
All the rest of the config is completely default.<o:p></o:p>

In order to try it out, there is a simple XML test file which you can send to the ESB using curl:<o:p></o:p>

samplexml.xml<o:p></o:p>
<test style="font-style: normal;" xmlns="http://fremantle.org"><o:p></o:p></test>

<test xmlns="http://fremantle.org">
   <sample>data</sample>
</test>
<o:p></o:p>

Once again you can find this file here:<o:p></o:p>

For the next step, please ensure you have a copy of curl installed. If you are on Linux or Mac you will have it by default. On Windows you can find a free version on the web. Let’s try the request against the ESB:<o:p></o:p>
curl http://localhost:8280/services/testJMS/a -X POST -H 'Content-type: text/plain' --data @samplexml.xml<o:p></o:p>

Run this a few times just for fun. You won’t see much. If all is going well, you won’t see any errors on the WSO2 ESB console either. If you add ‘–v’ to the curl command line you will see a lot more information about the HTTP section of the flow and you should see a nice sign that things are going well:<o:p></o:p>
< HTTP/1.1 202 Accepted<o:p></o:p>

Now go back to the MB console and look at the QueueàList page. You should now see some messages in the queue:<o:p></o:p>

Queue Name<o:p></o:p>
Queue Depth<o:p></o:p>
Message Count<o:p></o:p>
Created Time<o:p></o:p>
Updated Time<o:p></o:p>
Type<o:p></o:p>
echo<o:p></o:p>
0(b)<o:p></o:p>
0<o:p></o:p>
Fri Apr 01 09:14:05 BST 2011<o:p></o:p>
Fri Apr 01 09:14:05 BST 2011<o:p></o:p>
390(b)<o:p></o:p>
6<o:p></o:p>
Fri Apr 01 10:04:31 BST 2011<o:p></o:p>
Fri Apr 01 10:04:31 BST 2011<o:p></o:p>

As you can see in my example I sent 6 messages. <o:p></o:p>

As an exercise, why not try modifying the simple JMS code to pick up those messages from the JMS queue. If you get stuck there is a sample in the same place as the other code.<o:p></o:p>


Conclusion<o:p></o:p>
There is a lot more we can do with MB. In future articles I hope to cover using a C# client to interact, using the SQS support, and how the MB code can be embedded directly into the ESB to provide in-process queueing and eventing. In the meantime, I hope this has provided a simple introduction to get you started with WSO2 MB.<o:p></o:p>








February 04, 2010

Kelvin Lawrence - workLotus Symphony 3 Beta 2 posted

The latest Beta release of the new release of IBM Lotus Symphony (version 3) has been posted and is available for free download. Versions are available for Windows, Mac and Linux.

I have installed the Mac version and am now using it as my main office application program. You will find it is chock full of nice new features and it also brings the ODF support in the product into line with the one used by Open Office which will enhance document interoperability.

Here is the link to the site where you can download the Beta and also read more about the new features.

September 16, 2009

Kelvin Lawrence - workMy Symphony Experiment Week 3

So I am mid way through my third week of using Symphony as my primary office tool for working with spreadsheets, documents and slides. This week I have been on many conference calls reviewing slides with people. I noticed that I am getting sent a lot more Open Document Format (ODF)files from many of my colleagues. As a standards guy it's nice to see we are using the same standards ourselves that we have worked so hard to embrace and help define. I have also been sent several Power Point (PPT) files this week. So far, I have been able to work with them all fine in Symphony. As a UI guy, I do keep spotting things in the UI that I think could be better and the development team has been very gracious in agreeing to look through all of my feedback. That's it for now, I have some slides to go work on!!

September 03, 2009

Kelvin Lawrence - workRe: My Symphony Experiment - week 2

In response to: My Symphony Experiment - week 2

So far there is nothing I have really needed to do that I have not been able to. That said while I am a self confessed "geek" I try not to be a super power user in any of the Office Suites as I find myself having to share my work with a lot of different audiences (customers, colleagues, conferences etc) and I have learned over the years that avoiding the advanced features seems to be a good general rule. It's the old 80-20 principle yet again I suppose. That said I view the evolution of all the ODF based tools in the industry to be just that, an evolution. I think we will see incremental improvement each release.

September 02, 2009

Kelvin Lawrence - workRe: Mac OS X 10.6 (a.k.a Snow Leopard)

In response to: Mac OS X 10.6 (a.k.a Snow Leopard)

I have discovered this evening that some of the plug ins I use with Safari on the Mac had stopped working under 10.6 I was able to get them going again by running Safari in 32-bit mode. To start Safari in 32-bit mode, locate it using the Finder, click on the icon and select Get Info. There is an option to launch the app in 32-bit mode. The plug in I most wanted to work that this fixed was the Delicious plugin that I use frequently to help make bookmarking sites on Delicious.com easier.

June 03, 2009

Software Summit June 3, 2009: The Finale of Colorado Software Summit

To Our Friends and Supporters,

In these challenging economic times, business has slowed, many companies have had to resort to layoffs and/or closures, and everyone has been tightening their belts. Unfortunately, Colorado Software Summit has not been immune to this downturn. As have so many companies and individuals, we too have experienced a severe decline in our business, and as a result we are not able to continue producing this annual conference.

This year would have been our 18th conference, and we had planned to continue through our 20th in 2011, but instead we must end it now.

Producing this conference has been a wonderful experience for us, truly a labor of love, and we have been extremely privileged to have been able to do well by doing good.  We are very proud of the many people whose careers flourished through what they learned here, of the extensive community we built via the conference, and of the several businesses that were begun through friendships made here. We treasure the friends we made, and we consider them to be part of our extended family. Just as in any family, we celebrated with them through joyous life events and grieved with them through tragic ones.

This is a sad time for us, of course, but not overwhelmingly so. It's sort of the feeling you have when your son leaves for college, or your daughter gets married. You knew it was coming someday, but it is here much sooner than you imagined, and the sadness is sweetened with the joy you had in all that has come before.

We have been privileged to have created a thriving community of friends who met for the first time at the conference, and we want that community to continue. We hope that all of you will stay in touch with us and with each other, and that the Colorado Software Summit community will continue as a source of wisdom and friendship to all of you. If you have ever attended one of our conference, we hope you will consider joining the Colorado Software Summit LinkedIn group as one means to keep in touch.

With our very best wishes for your future, and with unbounded gratitude for your support,

- Wayne and Peggy Kovsky -

All presentations from Colorado Software Summit 2008 have been posted.

May 21, 2009

Software Summit May 21, 2009: Additions to Preliminary Agenda for Colorado Software Summit 2009

We have posted additions to the preliminary agenda for Colorado Software Summit 2009, in two formats:

We will continue to post additions to this agenda during the coming weeks. Please check back here from time to time for additions and/or changes to the agenda, or subscribe to our RSS feed to receive notifications of updates automatically.

Presentations from the 2008 Conference

We have posted presentations, and notes on the presentations, for this speaker from Colorado Software Summit 2008:

Presentations from Colorado Software Summit 2008 will be posted periodically throughout the year.

May 18, 2009

Software Summit May 17, 2009: Additions to Preliminary Agenda for Colorado Software Summit 2009

We have posted additions to the preliminary agenda for Colorado Software Summit 2009, in two formats:

We will continue to post additions to this agenda during the coming weeks. Please check back here from time to time for additions and/or changes to the agenda, or subscribe to our RSS feed to receive notifications of updates automatically.

Presentations from the 2008 Conference

We have posted presentations for these speakers from Colorado Software Summit 2008:

Presentations from Colorado Software Summit 2008 will be posted periodically throughout the year.

May 03, 2009

Software Summit May 3, 2009: Additions to Preliminary Agenda for Colorado Software Summit 2009

We have posted additions to the preliminary agenda for Colorado Software Summit 2009, in two formats:

We will continue to post additions to this agenda during the coming weeks. Please check back here from time to time for additions and/or changes to the agenda, or subscribe to our RSS feed to receive notifications of updates automatically.

Presentations from the 2008 Conference

We have posted presentations for these speakers from Colorado Software Summit 2008:

Presentations from Colorado Software Summit 2008 will be posted periodically throughout the year.