Flushing out some of the many tabs I’ve collected in Chrome:

• RQ, a simple Python task queue, based upon Redis
• How to get Pyglet working on Mac OS X Lion, needed for…
• pyprocessing, which I’m highly impressed by, mostly because it feels pretty speedy, even on the order of processing itself
• The ScipySuperPack which installs a collection or prebuilt scientific computing packages in the Mac OS X default python. Convenient for getting pandas installed since it’s been a bit finicky for me.
• The IPython HTML Notebook which, after watching the PyCon presentation on it, is stupefyingly cool. Actually given the HTML notebook, the Qt console, and its parallel computing facilities, IPython itself is just damn nice.
• It’s also cool that pyvideo exists, especially since they collected all of the PyCon 2012 presentations.
• d3py: “This is d3py: a plotting library for python based on d3. The aim of d3py is to provide a simple way to plot data from the command line or simple scripts into a browser window.”

I’ve actually been on Twitter since 2007, but really haven’t had much use for it. I was connected to a few friends, but wasn’t tweeting much useful or seeing many useful tweets.

Recently I added a few data science folks, and began slowly expanding my follows. Now I’m getting a lot of interesting links, even hitting Tweetbot multiple times in a day. Props to Ben Lorica, @bigdata, Jimmy Lin, @lintool and Joe Hellerstein, @joe_hellerstein for bringing the good bits. Hilary Mason @hmason and Andy Hickl @andyhickl provide a little personality and I’ve got an emerging cluster of Python folks including David Beazley, @dabeaz, Wes McKinney, @wesmckinn, and Adam Klein, @atomklein.

Well, I got all excited about the AC Milan v FC Barcelona Champions league match yesterday for nothing. A 0-0 draw makes for a dud. Even worse, the big players just missed plays in the final third. And I’ve never seen topflight defenders diddle around in the box so much. Maybe it‘s a strategic approach I‘m not sophisticated enough to get, but seemed like they were constantly playing with fire.

Hopefully the other half of the tie, next week at the Camp Nou, will be a lot better. Winner take all, bet on Barca.

I‘ve noodled around with turning the Retrosheet data into SQL for MySQL and PostgreSQL, but never did anything further. Now that I’ve dug into Tastypie a bit, it might be fun and easy to wrap such a database with a RESTful Web API. Desktop visualizations could be easily created, but even better you could do neat browser based renderings given the direction of new toolkits like D3.

Marco Arment, who stays way more up to date on this stuff than I do, is making some reasoned projections on changes to Apple’s MacBook lineup. The dream of the 15” Screen/8GB RAM/250GB SSD seems a bit more promising. I’m actually seeing more and more reports of the current line of MacBook Air processors being quite comfortable for computationally intensive activities, which is where I would have been willing to compromise.

Here’s another project, slightly more complicated than remixing AbandonedArt (although getting pyprocessing working on OS X Lion is a bit challenging). This is sparked by the fact that my long time March Madness bracket league collapsed under the weight of its own success. Last year the amount of prize money in the pool raised the ire of PayPal and the organizers barely managed to get the purse out of them. This year the organizers came to their senses, realized they have better things to do with their lives, and shut the whole thing down. I wasn’t too put out, since I never won any money anyway, and picking brackets is starting to feel stale.

But there’s still a little competitive juice flowing during March Madness.

Back in my late days of undergrad, early years of grad school, there were a couple of fun contests run on USENET, (yes USENET), around college basketball. The first was bbfun, which was essentially a confidence pool over the week of Big 10 men’s basketball games. You picked the winners at the beginning of the week, ranked them in order of confidence, and the scoring was weighted by your rankings.

The second game, Matthew Merzbacher’s point total pool, had you select 8 NCAA tournament teams. You collected contest points for each real-world point your teams scored throughout the tournament. Later on Merzbacher spiced it up by adding bonus points for having lower seeded teams in your slate. In addition to the serious attempts, which could actually involve a fair bit of analysis, plenty of people entered fun theme or joke slates.

Both of these games seem eminently implementable using modern Web frameworks and toolkits. I actually wouldn’t be surprised if either or both had already been done but it would be a fun “reinventing the wheel” project. If executed smartly, probably wouldn’t be too taxing in terms of compute resources, and a small fee from a decent sized participant pool would cover your infrastructure costs. If one could navigate the purse vice gambling issue, prize money could even be incorporated.

At the other end of the spectrum, I can’t believe I’m watching one of the worst NBA matchups this season. The Detroit Pistons against the hometown Washington Wizards. Both teams have lost at least 2 out of every 3 games this season. The Wizards have the second worst record in the league. The Pistons are in a bad bunch with the New Jersey Nets, the Toronto Raptors, and the Cleveland Cavaliers.

I know why the Wizards are awful. What I don’t understand is how the Pistons can be so bad. When you look at their roster, they’ve got two guys with championship rings in Tayshaun Prince and Ben Wallace. Ben Gordon is a proven scorer and has playoff experience. So there’s veteran experience. Greg Monroe and Brandon Knight look like promising young players. Will Bynum, Jason Maxiell, Rodney Stuckey, and Charlie Villanueva have proven NBA talent.

It’s a shame a once proud franchise has fallen so far with no relief in site.

As for the Wizards, with Andray Blatche on the sideline for “conditioning”, that’s 4 out of 5 that need to be gone. Although the way Jordan Crawford’s shot selection is going, he might make the list soon. Dude ease up on the pound the dribble for 10 seconds then shoot possessions. The Wizards show flashes of quality but not well strung together and the team really doesn’t know how to finish. But it beats the crap they had on the floor before.

So somehow Manchester United, Aresenal, and Manchester City are out of the UEFA Champions League and Chelsea is still alive in the quarterfinals. Something’s unjust in the world. Can’t say as I’m too excited about the Blues v Benfica tie, but might be worth watching given what’s at stake. Maybe Chelsea’s vets will get up off the deck and show some pride after AVB get summarily dismissed.

Now FC Barcelona versus AC Milan? That’s a battle of titans to be admired. Messi against Ibrahimovic is just the starting point. Might have to fire up the DVR for those two matches.

The modern update of the Green Lantern logo has to be at the top of the list of superhero iconography. I’ve seen the logo in a number of pop culture appearances. I might argue that it’s surpassed the Batman and Superman logos for social currency.Makes for a damn nice t-shirt. And as my OS X logon image, the operating system automatically adds some nice shiny highlights that make it look even better.

Too bad it’s tied to one of the dopiest characters in the DC universe. I mean c’mon, a guy who creates randomly convenient physical manifestations sheerly by willpower? You’ve got to be joking. Oh yeah, doesn’t work against the color yellow.

While somewhat opaque, The Open Bastion, looks like it’s running an interesting series of Python conferences. DjangoCon US, Sep 3-8 in Washington, DC is a no-brainer for me, although I’m a bit tempted by Open Django, June 8-9, in Chicago.

Shannon -jj Behrens is doing a great job summarizing a number of the PycCon talks.

I especially like his summary of Ned Batchelder’s Pragmatic Unicode talk. I agree with him that it was on of the best talks of the conference. Despite my limited attendance, we had quite a bit of overlap in session attendance. Maybe that’s an indicator I have good Python taste ;-/

I’m actually really liking the notion of remixing AbandonedArt (AbandonedRemixed?) and taking it on as a project. But I had to catch myself prematurely optimizing and fiending for new hardware.

New project? Of course I need a new MacBook Air so I can work it on it during all my idle moments. Like that worked out so well before.

Okay, it would be nice to have a new low end, or used laptop, just for this project. Ugh. Can’t get much for a measly $250. Besides, I have more important things to spend money on at the moment.

Hey! Let’s get an inexpensive desktop box just for the sake of this effort. Wait, we’re trying to cut back on stuff this year, not add more.

Howabout this. Get started using The Trusty Ole‘ MacBook. IF performance really is an issue then deal with it. If not and the mythical 15”/8 Gb RAM surfaces, reward yourself.

Right answer.

Now this might be a feasible 100 Hours project for yours truly. Download the processing sketches at AbandonedArt. Upload sketches to github. Clone each sketch, see how well it comes out in pyprocessing, fixup as needed.

Can be worked on in small discrete chunks. Public visibility would be nice and maybe others would join. And it would provide good feedback for the pyprocessing project, maybe leading to contributions.

Processing logo retrieved from Wikimedia Commons, through a Creative Commons License

N.b. Adjusted publish date to real release date. WP just doesn’t handle dates on draft posts correctly to my mind

Link parkin‘: “Circus is a program that will let you run and watch multiple processes.”

Might be a little bleeding edge for what I’m doing at work, but looks very attractive.

PyCon, the conference experience that keeps on giving the gift of blogging material.

So I actually did take advantage of the promotion that moo did with PyCon, providing a 50 business card sample for the cost of shipping. I picked out the whizzy design you see attached, placed the order, chose the eco-friendly card stock, and waited to see the results.

The pickup experience couldn’t have been easier. Just asked at the registration desk and boom, right in my hands. The unboxing was pretty cool too. moo has nice wrapping. And ultimately the cards were pretty sweet as well. I think the design I chose probably needs stock that can support a bit higher ink resolution and some gloss for best effect. But for what I paid, $4, I can’t really complain.

Also should have sprung for a cardholder.

So consider me a happy new customer of moo. The PyCon set was mostly whimsical, although I now think Mad Data Scientist LLC (or Inc) might make for a pretty good company name. And having “Mad Data Scientist” on my badge seemed to tickle a few folks. Have to hand out a few more cards and check out some more reactions.

The third Health Data Initiative Forum is holding The Health Datapalooza right here in Washington, D. C. June 5-6 2012, right downtown. I do some healthcare technology related thinking for work, so this seems like a no-brainer. Not sure I like the pure app contest + keynotes format, but I’ll give it a shot.

Link parkin’: Hilary Mason presenting a techtalk at Dropbox.

I’ve often found Mason’s talks a little fluffy, but this one looks like it has a little more meat about bit.ly engineering behind the scenes.

Link Parkin’: Adam Klein on pandas upcoming timeseries capabilities.

pandas got a lot of love at PyCon 2012. Looks like it might be the next center of gravity for Python and high performance numerical computing.

There have been upsets in the NCAA Men’s Tournament before, but nothing like Lehigh (15) beating Duke (2).

Two observations. First, Lehigh outplayed Duke in both halves of the game. Duke was a bit lucky in getting a first half lead.

Second, Lehigh looked as good as, if not better, than any ACC team that has played Duke this year (besides North Carolina who basically pwned the Blue Devils twice). Lehigh’s defense was eminently capable of guarding Duke for extended periods of time and Duke never looked to have any sort of physical advantage. I mean C. J. McCollum got ‘em for 30 points! Used to be a time when only folks at UNC, Maryland, UNLV, Michigan, Kentucky, et. al. could do that.

The margin between the rest of the country and the just-below-top tier of the big conferences is non-existent.

Yeehaw! It is a wonderful day here in Wizardsville!

Ernie Grunfeld started to evoke my worst fears with NBA trade deadline moves. Thankfully, he actually achieved two of my higher priority desires for the Wizards. Javale McGee and Nick Young are no longer with the team.

Let me say that again. NICK YOUNG AND JAVALE MCGEE ARE NO LONGER WASHINGTON WIZARDS!! With Flip Saunders fired, that makes for 3 of my 5 wishes. Even so, Young pulled a dick move, rejecting a trade to Denver and forcing a landing in Los Angeles. Good luck getting signed next year bub.

Now Ernie did saddle the franchise with the contract of NeNe, which has 4 years at $13 million per left to go, a guy who has difficulty staying healthy. Still just getting rid of two clowns who constantly wind up on viral YouTube videos for the wrong reason, and get openly mocked and laughed at on SportsCenter is a good thing.

Also, I didn’t know Ernie’s contract expires this summer. Depending on whether that’s before or after the NBA draft, Ted Leonsis can just wait him out and gracefully bring in someone new. While I thank Ernie for today’s service, he’s still a central part of the problem. His removal is part 4 of my wishes.

Too bad he couldn’t move Aundray Blatche, the fifth and final piece of the puzzle, but hey, baby steps. And (mouth watering) there’s always that veteran amnesty slot.

Unless I find something else interesting to report.

I pretty much went with the notion that the talks were going to be the centerpiece of my PyCon. But it’s pretty clear, that while the talks are solid, they should not be what one goes to the conference for. It should really be about interacting with the larger community of Python enthusiasts. Let me give you an example.

Despite my miserable condition, I did make it to a few talks. One, by Frank Wiles, was on the uses of plpython, a Python embedded in the PostgreSQL database. I have a little knowledge of the subject from a project at work.

Frank did a great job, but I thought he left out one key potential use case. PostgreSQL supports the notion of “table functions”, which allows a procedural function to return multiple, multi-attribute values. Said results are treated much like an SQL table, and can be used anywhere a table expression can. This means you can do fun things like joins against the results of a procedure call. The utility is awesome.

After Frank’s done, I feel it’s my public duty to let others know about this awesomeness. So I shamble up to a mic, grunt out a few hoarse sentences that include the phrase “table functions” and nod my head at the response.

After that, a couple of other seasoned PyConners, Charlie Clark (all the way over from Europe!) and Catherine Devlin, spend some time chatting me up. If I was feeling better I would have tried to arrange to meet one or both for a drink later. Charlie was kind enough to almost instantaneously send me a couple of interesting papers on sophisticated PostgreSQL usage.

There was a lot of stuff like that at PyCon. A lot of opportunity to do stuff (besides watch a presentation) with others: tutorials, parties, expo hall, open spaces, poster sessions, programming sprints, programming contests, a 5K run, the hallway track.

To maximize your time at PyCon all you have to do is make the effort to speak up a little. It can even be as simple as saying “Hi!” to someone you’ve never met before.

Even though I sang its praises quite early, I never really got into the Posterous product in a big way. Something about self-hosting my blogging space is just a better fit. However, I did manage to use Posterous for an extended period of narrating my 100Hours project.

Well, Posterous has recently been acquired by Twitter. Looks like the service will run for a while, but these events always are uncertainty generators. Probably a good idea to export my data for “posterity”, yuk, yuk. Congrats to the team for a successful exit.

Before bowing out to illness, I did manage to catch Paul Graham’s PyCon keynote. For someone with such a flat affect, he was surprisingly funny, probably because flat means an effective deadpan for delivering zingers.

In any event, he basically read one of his latest essays Frighteningly Ambitious Startup Ideas. They’re all thought provoking, but I found idea number 7, Ongoing Diagnosis intriguing:

“One of my tricks for generating startup ideas is to imagine the ways in which we’ll seem backward to future generations. And I’m pretty sure that to people 50 or 100 years in the future, it will seem barbaric that people in our era waited till they had symptoms to be diagnosed with conditions like heart disease and cancer.”

I find the “Quantified Self” movement somewhat pretentious because the community of people who want to continually look at charts about themselves is vanishingly small. But if there were positive outcomes of the kind Graham describes, and you didn’t actually have to look at the data, then I could see it happening.

…you won. Whatever induced you to unleash some triple threat virus attacking my eyes, throat, and stomach I’ll never know. But I gave up my last day of PyCon to lay low. And I’ll be heading back to the East Coast with my tail between my legs and an urgent need to schedule a doctor’s appointment.

Ned Batchelder’s PyCon 2012 talk on Pragmatic Unicode made the trip really worthwhile. First, it was excellently delivered with good humor. Second, it was very usefully pragmatic. I really learned something.

So I’m authoring this during one of the session breaks at PyCon. Good to be here but I seem to have caught some bug that has me running at half speed. Probably really need to spend a day or two in bed, but we shall muddle through.

Friday keynotes, Stormy Peters and Paul Graham, were entertaining, especially Graham. Talks I’ve attended have been good if not spectacular.

Definitely worth the trip so far!

Flew Virgin Airways on Tuesday. Have to say the experience didn’t match the hype. The plane had lots of shiny and rounded accents, but other than that, just about like every other airline. Maybe a little more legroom in Coach.

Was amused when the in-flight entertainment and food ordering system crashed. Then it went through a Linux reboot. I might have been the only person who enjoyed reading the boot log as it flew by.

Good on PandoDaily for generating an executive summary of the announcements at the latest Apple event. As an Apple customer, I’m intrigued by the new iPad, although it’s not a game changer. (Didn’t really expect it to be). I’ll wait to see what the reviews are on battery life versus 4G LTE are. LTE seems to me the only real discriminator.

I’m still tempted by the Apple TV, especially since it’s fairly cheap (1st world talking here). 1080p was one of my prior showstoppers, but that’s been fixed.

Still waiting for my 15 inch display, 8 GB RAM, 500 GB SSD MacBook Air-alike.

Link parkin’: Always enjoy Ted Leung’s conference trip reports. He just attended the O’Reilly Strata Conference (West Coast). I’m considering the East Coast version if it happens and Leung’s review of the event is promising.

Sergeant Phil Esterhaus: Hey, let’s be careful out there. —Hill Street Blues

Encountered a moment of intense fear but good old fashioned human kindness today.

Part of my commute involves the Rosslyn Metro. For those, not familiar with the subway system in our nation’s capital, a number of the stations are deeply underground. They have exceedingly tall escalators. We’re talking 3, 4, 5 stories deep. Rosslyn must be pretty close to the worst of the bunch. It’s a minor miracle that, given the number of daily commuters and how many rush headlong down those stairs, there aren’t many spectacular injuries (that I know of).

Well today a poor lady in front of me was the victim of the anti-miracle. We were both walking down at a reasonable pace, I was about two steps behind, and we were halfway down the escalator. She must have hit a stair edge, or slipped a heel or something.

And she started tumbling down the escalator. As I watched in horror she must have flipped over one or two times, before stopping, flat on her face, about 1/4 of the way down. Keep in mind the escalator is still moving, there are still people coming down, and she’s not really moving.

Along with a few other good samaritans, we catch up with the poor lady, get her to her feet before the end of the escalator, and help her move a few steps beyond the end. One person went back up to get the station manager and another called out to 911. As I’m giving her support, she was clearly dazed, confused and probably in shock. After a few minutes we got her over to a wall where she could lean for a spell and finally sit down. It wasn’t until then that she pulled up her pants leg and showed the horrific cuts from the escalator, and bruise that was already exploding. I would not be surprised if she hadn’t broken her leg.

Three or four of us stayed with her until station officials arrived. We gave our names and contact info and waited to see if any further assistance was needed. Luckily the EMT pros showed up in a short manner, with a stretcher, started immobilizing her leg, and took over the situation. We went our separate ways but it was an odd way to start the day.

Like Sgt. Esterhaus says “Hey, let’s be careful out there.” And let’s look out for one another.

As a newbie to following the Premiership, I realized that the stakes were high. But man, Andre Villas-Boas didn’t get much leash in managing Chelsea. AVB was sacked this weekend meaning he barely made it 3/4 of a season.

Surprising contrast with Arsenal, where Arsene Wenger’s head was on the chopping block after their early season beatdown by ManU. Now the Gunners are challenging for third position in the tables.

I still remember vividly sitting in the bedroom of our little Chicago condo and going “Holy shit!”, “Oh my god!”, and “Wow!” as I watched the worst incident in NBA history erupt. Woke up my wife from her regular evening nap on the couch.

Oft referred to as The Malice at the Palace, on a mundane Friday evening, the middle of the road Indiana Pacers and Detroit Pacers got into a brawl and then ignited a riot. In the immediate post-game reactions and dissection, the most poignant moments came from Bill Walton, a guy who loves basketball and the NBA. The sadness and dejection in his voice told the whole story. Thankfully, there were no serious injuries, but it was definitely a disturbing event.

I’m not yet into Grantland’s long form sports journalism, but I’m planning to make some time for Jonathan Abrams The Malice at the Place: An oral history of the scariest moment in NBA history

Hat tip Dan Fogarty at SportsGrid

As a graduate from a PLDI (Programming Language Design and Implementation) research group, I’m always suspicious when a brand spanking new language gets a lot of buzz. There are so many barriers against, nay even ways to measure, success that most of the newcomers wind up being also rans.

But the new language julia has some nice features, yet seems eminently pragmatic. Plus people who have good taste seem enthusiastic about its arrival.

Then again, Go was the bee’s knees not too long ago and it hasn’t taken over the world.

Given that I started and abandoned a transliteration of processing to Python, I’m interested in how others might approach the challenge. I just found out about the pyprocessing project. They’ve taken the path of using OpenGL, through pyglet, which I thought about but couldn’t figure out how the bit blitting capabilities of processing would work. That’s not saying much, since I have exceedingly minimal knowledge of OpenGL.

I’ll have to take pyprocessing out for a test drive. I hope the project succeeds. If it works well the toolkit would be a nice complement to Python’s emerging heavy duty data processing capabilities: cf. Numpy, SciPy, and pandas.

Last night, Randy Wittman, interim coach of the Washington Wizards, benched the execrable JaVale McGee and Nick Young for the second half of a game against Milwaukee. While I hope Wittman is only an interim coach, I have to give him props for the move.

“I’m done with young guys (as an excuse), Wittman added. “ If they don’t want to play the right way, young guys aren’t going to play. It does us no good. They’re not going to learn anything when they want to play the way they want to play. All they’re developing are bad habits.”

Hat tip Comcast Sports Net Washington

Even better, both players were benched for the start of tonight’s game against the Orlando Magic.

Luv, Luv, LUV, Walter Jon Williams near history science fiction work (prior MPR mentions). So all I really needed to get out of Williams’ Big Idea entry for John Scalzi’s writer self-promotion series, was that Williams’ new book The Fourth Wall is on store shelves. Even better this is a continuation of the Dagmar Shaw series. Our intrepid Alternative Reality Game designer is making a return appearance in what looks like an exploration of the future of entertainment and media.

Check out WJW’s pitch if you need more convincing. Me? I’m going straight to Amazon after I finish this post.

Luke Wroblewski does a great job with Data Monday pieces. His most recent was: Data Monday: Amazon Prime. While I don’t think I increased my spending (rate and/or magnitude) at the levels highlighted by his data links, I definitely will buy “higher priced” items. Often for me, when you add in the alternative‘s shipping costs Amazon comes out ahead.

And 2 day shipping just soothes that instant gratification itch so well.

Interesting though that Prime might not actually be a big win for Amazon.

As a self-proclaimed Mad Data Scientist (Apprentice 3rd Class), I’ve learned at least two lessons, maybe more. First, you will wait. When you get upwards of 32 Gb of data, it’s almost impossible to just read through the data in a reasonable amount of time on commodity hardware. You’ve left the land of interactivity son, live with it.

As a result of that first lesson, one gets in the habit of timing everything. If it takes a while for a task to run, you want to know how long, just in case you have to run it again. You want to know how that runtime is growing with data growth. Better be at least linear, preferably sub-linear. And if not, well you’ve got a problem.

Of course, Nathan Marz captures the issues much better than me, backed by much more real-world experience. At the end of the day, it all comes down to creating data views that go as fast as possible.

Consider the source, an author with a forthcoming book on Bell Labs, but Jon Gertner has an interesting opinion piece on innovation in today’s New York Times:

“So how can we explain how one relatively small group of scientists and engineers, working at Bell Labs in New Jersey over a relatively short span of time, came out with such an astonishing cluster of new technologies and ideas? They invented the future, which is what we now happen to call the present. And it was not by chance or serendipity. They knew something. But what?”

Looking back at the iconic computing industrial R&D labs (Bell Labs, IBM Watson Labs, Xerox PARC), I’ve become more interested in how researchers weathered the bumps, rather than the frequent organizational hagiographies. Having seen the insides of industrial R&D, it couldn’t have been all milk and honey at the core. For every great breakthrough, were there two, three, five battles with management to keep the project going? How many promising junior folks just didn’t pan out or got chewed up and spit out? Along with the great collaborations, were there backstabbing rivalries?

Mostly, I’m interested in how these organizations developed and sustained a culture of innovation, no matter how brief. I forget who it was but a senior researcher in computing floated the notion that the field can only have one or two of these organizations extant and thriving at any given time. Outside of the raw numbers game of high quality researcher, I’m wondering if there isn’t some other fundamental limitation.

PostgreSQL’s table functions are a pretty handy feature:

Table functions are functions that produce a set of rows, made up of either base (scalar) data types, or composite (multi-column) data types. They are used like a table, view, or subselect in the FROM clause of a query. Columns returned by table functions may be included in SELECT, JOIN, or WHERE clauses in the same manner as a table, view, or subselect column.

Especially when combined with the embedded procedural languages one can fold a lot of external functionality into typical SQL idioms. Of course I’m sort of partial to the fact that PostgreSQL supports Python as a procedural language straight out of the box.