I've moved my blog to jmcneil.net. This is no longer being updated!

Saturday, January 16, 2010

Unit Tests with App Engine

Posted mostly for my own benefit (I know where to look later). 

I've been working around the socket limitations on App Engine this morning trying to get outbound XML-RPC requests to work properly.   I rigged up my unit test structure and designed the following little tests:

def test_xmlrpc_call(self):
    """Run XMLRPC Call"""

    class T(threading.Thread):
        def run(self):

            def rpc_method(x):
                return x

            server = SimpleXMLRPCServer(("localhost", 5432))
            server.register_function(rpc_method)
            server.handle_request()

    # Start XMLRPC Thread
    T().start()
    time.sleep(0.25)
    proxy = rpc.xmlrpc_proxy(LIVE_URL)
    self.assertTrue(proxy.rpc_method(True))

Fairly simple. It just creates a background XMLRPC thread and calls an identity function. As long as that returns True, I consider the test passing. This exercises all of my custom urlfetch transport code.

I ran the test suite and it bombed with the following error:

ProtocolError: protocolerror for localhost:5432/RPC2: \
   500 failed to fetch http://localhost:5432/RPC2: No \
      api proxy found for service "urlfetch"

I apologize for the formatting. I found this confusing as I defined a method above in an 'rpc' module named 'xmlrpc_proxy.' After about 20 minutes of digging, I discovered that if you elect to run App Engine code from the command line, then you need to manually rig up some of the Google API stubs. I must have missed that little detail.

So, in the root __init__.py of tests/ directory, I now have this:

"""
Special setup is required for certain App Engine stuff, namely
urlfetch proxies and data store.
"""

from google.appengine.api import apiproxy_stub_map
from google.appengine.api import urlfetch_stub

# Create a stub map so we can build App Engine mock stubs.
apiproxy_stub_map.apiproxy = apiproxy_stub_map.APIProxyStubMap()

# Register App Engine mock stubs.
apiproxy_stub_map.apiproxy.RegisterStub(
    'urlfetch', urlfetch_stub.URLFetchServiceStub()) 

This wires up the stubs correctly and my App Engine unit tests pass as they should.

It's going to take a while to get used to coding for App Engine.

Friday, January 15, 2010

Help? Anyone?

So, my day job involves building, scaling, managing, tuning, tweaking, coding, breaking, fixing, and generally running the high availability clusters for Web.com.

Over the past few weeks, I've been working on finally getting FastCGI running on our cluster.  As part of that, I'm just about to deploy virtualenv & setuptools across the web and customer access SSH farms.  Assuming I know what I'm doing, end-users should now be able to run utilities such as Django, Pylons, and TurboGears via FastCGI/WSGI on our distributed Linux-based hosting platform.

I could really use a volunteer or three to test it out once it goes live.  Nothing big, just spend 30 minutes or so and throw up a quick example site and drop me a line if you see something goofy.  Anyone interested?