My Big PyCon 2010 Review

Hello, world! Oh, wait a minute. My apologies. I've seen a good half-dozen example applications over week; I think it's starting to rub off.

In all seriousness, I've just wrapped up my first PyCon event. Yup, that's right. I've been tossing Python code around for almost six years now and this was my first convention. First impression? What an excellent experience.

I do need to throw in a caveat here. I'm the sole Python+Linux developer on a team of six. The other five individuals spend their days firmly implanted in a seat driving Visual Studio. It's not very often I get to talk about and get into "my stuff." This was a refreshing experience for that alone.

Before I step into each talk that I attended, I wanted to mention three non-technical things I picked up while there.

1. The term 'dunder methods.' Whenever I explain __XXX__ to any one, I'm usually at a loss as to what to call them. It usually comes out as "magic double underscore name double underscore method." I've now firmly picked up on this nomenclature.

2. The majority of the attendees live in a different world than I do. It seemed like many were on larger teams of Python coders. As this is not my case, I found it a little difficult to discuss some topics. Not because I didn't understand them; quite the contrary. I simply never have the opportunity to vocalize.

So, that brings me the final point.

3. I want to get more involved in the Python community (commit rights, please?). More specifically, the 3.x efforts. It's an opportunity to get into a niche that will eventually become the standard. At the same time, porting and lending coding expertise will help the adoption rate. It's a win-win.

Here's a talk-by-talk rundown of the sessions I attended.

Keynote: Building the Python Community
Mr. Steve Holden

I enjoyed this as it was (surprise!) community based. Steve introduced the concept of a premium associate PSF membership. I like the idea. I think. Generally speaking, anything that can be done to improve the quality of a professional community is a good thing.

I do wonder if it will create a bit of an unintended dichotomy between members and non-members, though. You know, "fix my bug first, I pay each year!" Not to worry, I'm confident any issues like that will be worked out.

Keynote
Mr. Guido van Rossum

Well, I had one of my questions answered, so I'm happy. I was curious as to whether we need some type of 2-to-3 resource available. I'm fairly certain a phrased my question in the wrong context so it was interpreted as "we need a porting guide."

What I really meant, was:

Should we put up some sort of community page such that people with 3.0 porting experience can look to lend a hand to the projects that need it? People seem to be waiting for others to do some of the porting work.

Optimizations and Micro-Optimizations in CPython
Mr. Larry Hastings

What a way to start the morning. As I told my employer, they earned their money back with this one alone. For starters, I had no idea that GCC supported goto references. On top of that, doing so could yield a very large performance increase.

The big take away here was that there are a lot of optimizations under the hood. Writing good, clean code is paramount. Sure, it makes sense to worry about O(n) vs O(1) vs O(n^2), but that's a good place to stop and avoid obfuscation.

Python in the Browser
Mr. Jimmy Schementi

I attended this one as I work with a gaggle of .Net'ers. I figured some of the Silverlight exposure and Microsoft stuffs would help me understand some of the stuff they're working on these days. You know, help with team dynamics and all that.

You know what? You can run Python within Silverlight! Did you know that? Really? I didn't! Had no idea. That's pretty awesome. I intend to install whateveritisineedtoinstall in order to try this out one of these days.

The talk was rather informative, especially for a Linuxy guy like myself. There were a few technical issues, but that's fine by me. I took a lot of cross-platform knowledge out of this.

Import This, That, and the Other Thing: Custom Importers
Mr. Brett Cannon

Ah. Nice and advanced. Nothing like some brain pain in the morning. The talk covered how to write importers and what needs to be done in order to do it correctly. Handling dotted-names, package 'dunder init' files (see, I told you I'd use that term from here on out), scattered locations, sys.meta_path, and so on.

This is one of those things that I'm not likely to ever do. If I decide I have an need, Brett here has already put in the time to create a nice little library which ought to handle all of the details for me.

This talk was valuable as it gave an introduction into how this is done. While I may not do it, others will. The benefit here is that when a 3rd party custom importer fails, I'll understand what's going on a bit more.

Python 3: The Next Generation
Mr. Wesley J. Chun

A nice, well-rounded overview of Py3K. I knew most of this already, but I've yet to utilize it from a practical standpoint as I've yet to have a reason to. We're running 2.4 on most machines and 2.6 on the remainder.

I would like to note that Mr. Chun is a pretty good speaker.

Cross Platform Application Development and Distribution
Mr. Stani Michiels & Ms. Nadia Alramli

I need to make a confession here. I initially started with Deconstruction of an Object. However, that talk turned out to be geared to a developer with less experience so I ducked out. Mr Rush did a great job detailing the class structures, but I felt I would absorb more in a different talk.

This was my least favorite talk. The presenters did a wonderful job and their slide deck was very well done, but the material really wasn't relevant to what I'm doing each day. I was hoping for more focus on the development cycle. For example, Build Bot slaves on both Windows & Linux, handing dependancies, and so on. This focused more on UI elements. Great material, just not my area of focus.

Powerful Pythonic Patterns
Mr. Alex Martelli

My hope going into this talk was that it would come from a practical standpoint. I wanted to see some hard and fast code examples. I don't think in terms of object diagrams and pattern components for the most part and I figured that this would be a nice introduction to implementation from that standpoint. I left the talk looking for patterns in everyday life.

When us software guys hear the word "pattern", I assume that most of us think Gang of Four. Specifics such as Factory, Abstract Factory, or Facade spring to mind. While he touched on those, he also touched on more of the generic patterns and idioms that naturally develop.

So, while I still can't tell you exactly what a Memento implemented in Python would look like without Googling the term "Memento", I've started looking at my reproducible units of work for extractable patterns. Hopefully, I can up and refractor them to get slightly better performance.

How to Write Cross-Interpreter Python Programs
Mr. Maciej Fijalkowski

I fit into the bucket that most Python developers probably fit into. We write Python code for CPython. Perhaps IronPython. We don't switch between interpreters very much at all, really. I thought that hearing a different viewpoint would be a bit of a professional helper. I have two direct action items on my plate now due to this little talk.

First, I rely on reference counting in a lot of places. Mostly in my unit test code. What do I mean by that? Here's an example:

open('record.txt', 'w').write(record)
   

Now, we initialize a new file object with the call to open, write to it, and then drop the reference count immediately when the write call returns. In CPython, the file object is closed and cleaned up as the count drops to zero. This apparently isn't so in Jython and PyPy. The open file object can hang around for quite a while as they're not reference counting systems. I'm going to fix this.

Next, if it's practical, I'm going to look into setting up a Jython or a PyPy Build Bot slave. I'm not sure where version compatibilities are at this point in time.

Oh, and don't do crazy shit in your __del__ methods.

Keynote: Cadence, Quality, and Design
Mr. Mark Shuttleworth

Well, I live here in Atlanta. I use my iPhone alarm and it's set to go off every weekday. Well, guess what? Saturday isn't a weekday. I woke up far too late and walked in just as this talk got under way. I missed the earlier talks.

Mark really reiterated a lot of things we're already doing. Testing. Predictable cycles. Fall into a cadence or a rhythm. All good points and all very true. However, the speech wasn't overly earth shattering.

There was one comment that sticks out. Mark said something (and I'm paraphrasing here) like "The Good Enough Culture is Bullshit." I spent a good part of the day trying to keep that in mind. Under many circumstances, we do things to minimize work or to shave a few minutes. Productivity is important, but so is ensuring we're putting forth the highest quality possible.

Demystifying Non-Bocking and Async. I/O
Mr. Peter A. Portante

I use Twisted internally for an account provisioning API. As all of our internal code is synchronous, I just wrapped old calls with DeferToThread and move out of the async space. This is another example of one of those technologies I've used in a limited fashion. I understand it, I've just never really implemented anything with it. This helped to solidify it through discussion.

Learning Hosting Best-Practices From WebFaction
Mr. Brandon Craig Rhodes

Due to my line of work, I had a lot of hope for this one, and it held up. Brandon is a rather good speaker and the subject matter was right up my alley. I work in the same space, but not as a direct competition. It was very interesting to see how WebFaction does it.

This really solidified a theory I've had for a while. There are two distinct types of "webmaster." First, the technical user we're all familiar with. They want the bits and the bytes, the apps and the ports. However, there's also a class of very non-technical users that simply wants a basic site and they want to do it themselves. I believe the two stomp on each other quite a bit; finding the right middle ground is difficult.

I understand the plumbing of the Web Hosting industry inside and out. I learned a few things here that I can apply to my own work. Nicely done.

Understanding the Python GIL
Mr. David Beazley

Hands down, this was the best talk of the weekend. I understand the GIL and have worked around it in the past. I use Python threading quite a bit, but only for IO driven applications. For example, I'm parsing web logs for 80,000 web sites using a Python thread pool. It works great.

I'm not going to try to paraphrase as I won't do it justice, go read the documentation yourself, it's at http://www.dabeaz.com/talks.html.

I attended the Open Space talk on the GIL after this concluded. This stuff is all theoretical to me. I don't work on schedulers or priority queues during the day (or ever, for that matter). I know more about the GIL now than I ever thought I would.

Now, Mr DictatorForLife also took part in this talk. For a split second, I felt important. But then I remembered that I didn't have a damn thing to interject, so I left to go cry. That's my attempt at mid-blog humor. Go ahead. Laugh.

To Relate or Not To Relate
Mr. Mark Ramm

This was surprisingly relevant. As an organization, our data set isn't overly large. However, our deployment methodologies are somewhat time consuming and it can take a while to get new SQL servers deployed. Standard corporation purchasing stuff.

As a way around that, I have been looking at implementing something such as MongoDB or Cassandra. As we have a somewhat low overhead, I'm weighing our ability to deploy these services using local storage on a few hundred existing servers. We hit a few birds (Spotted Owls?) with the same proverbial stone:

1.Easier access to database capacity.
2. Reuse of existing local storage.
3. I get to play with something superneatocrazycool.

I had been leaning towards Cassandra, but MongoDB may be a better choice as it's implementation story seems simpler. Once the auto-sharding is complete, it's going to be a very strong contender.

Lastly, while we don't maintain a lot of data ourselves beyond standard customer records, we do offer databases to our customers as part of a hosting package. I also wonder what the feasibility of offering these things as a service is?

Threading is Not a Model
Joe Gregorio

At the end of the talk, someone made a comment that I've had a bit of trouble decompiling. He stated that Python does support Promises via Twisted's deferred object. Now, I'm not a guru in this space, but I think I disagree with that statement.

A promise runs a long running task in the background and then blocks when a function is called or an attribute is accessed if the requested variable is not yet around. A deferred, on the other hand, is loaded with a series of callbacks (errbacks) that fire when an event has completed. Are the concepts not different?

Sure, it's probably possible to implement a promise via the deferred mechanism, but it's probably much easier just to do it via a standard thread.

Python's Dusty Corners
Jack Diederich

Generally an overview of 'dunder methods' (see, there it is again!). I already had most of this in my brain, with the exception of the following:

If you have two objects, A and B, with B being a subclass of A, the comparison order is reversed if B appears after A in an object comparison.

Huh?

Given that C is of a different type, A == C will fire A.__eq__(self, C). However, in the A == B case, Python executes it as B.__eq__(self, A). The answer is simple. B is a more specific implementation and therefore should know more about the classes it's comparing.

Here's a code example.

class A(object):
    def __eq__(self, other):
        print 'Compared by A'

class B(A):
    def __eq__(self, other):
        print 'Compared by B'

class C(object): pass

A() == C()
A() == B()

When we run it, we get the following output:

jmcneil$ python comp.py 
Compared by A
Compared by B

I did attend an Open Spaces talk chartered by Doug Hellmann. We covered writing about Python. Initially I had believed it would be more of a talk regarding blogging & community documentation. It turned out to be focused more on authoring books. That's honestly something I've always wanted to do so I just sat back and soaked it all in. In the interim, I may make an effort to do some technical reviews.

That wraps it up PyCon 2010 for me as I wasn't able to attend Sunday's sessions. What a wonderful experience. This year was covered by my employer. I've already decided that if they do not cover it next year, I'll be paying for it out of pocket. It's time to start working on that already.

If you're an IT manager and you have the power to let your development staff attend, do so. It's in your best interest. Sure, a lot of recruiting goes down, but I promise that it's eclipsed only by the amount of learning.

I'll leave you with a final thought. PyCon indeed does have a little something for you, no matter what your tastes: