Month: July 2009

RESTful Django Powered Web Services

Posted on by David
What is REST

REST is an alternative to RPC based web services such as JSONRPC, XMLRPC and SOAP. Instead of simply using HTTP POST for all of its requests (with JSONRPC’s proposed GET implementation excepted) like RPC services do, it uses all the HTTP methods. It usually includes GET, POST, DELETE, PUT and other methods to achieve different results and thereby uses relatively few URIs.

Some people think any web service that makes various services available at URIs is REST. It isn’t. Some people make a service at one URI for getting an object, another for saving the object, another for getting a list of ojects, another for a list of objects matching a certain criteria. This is just RPC outisde of the realm of a specific protocol (like XMLRPC). If people are going to use simple HTTP RPC requests to get all their data but not follow any specific pattern, they’d be better off with a real RPC implementation.

How is it Better (or Worse)

REST has a lot going for it. Because it is a little more “native” to the HTTP protocol, caching can work very efficiently. Depending on language support, it may be be easier to work with a REST interface than working with a more complex RPC specific protocol. Its simplicity can be very beautiful. The RESTful idea of making your data available as a “resource” that links via hypertext to more resources can make REST very powerful.

Instead of

GET http://example.com/testcase/56

<testcase>
<results>
<result>1</result>
<result>2</result>
</results>
</testcase>

You have

GET http://example.com/testcase/56

<testcase>
<results>
<result>http://example.com/result/1</result>
<result>http://example.com/result/2</result>
</results>
</testcase>

In the 2nd method a full test case object can be generated by off of a request to the testcase object and later requests for the result objects. The client would not need to know anything special about testcases or the specific domain as it would in the first example.

RPC also has a lot going for it and there are some cases where I would pick it over REST. Caching is not always very important and when it isn’t, the benefits REST are not as apparent. Most RPC protocols already have the capability out of the box to construct objects (for SOAP — very complex objects) from web service calls. They also usually have introspection methods or WSDL to figure out what services are available. These would need to be built by a crafty REST service developer. RPC, however, doesn’t take much advantage of the HTTP protocol in that most requests are just POST requests with an RPC payload. At the same time, every HTTP implementation supports POST and not all of them support PUT or DELETE.

Next Steps

Django has a few libraries to help with REST interfaces, but nothing I’ve seen is that great. I am going to look into creating one or contributing to an existing project. Here are some things I’d like to see in a REST API:

  • In the Django 1.1 development version, PUT, DELETE, OPTIONS and HEAD are available in django.test.client.Client. A REST interface should use them by default and have another mechanism for clients that do not support these lesser used HTTP methods.
  • caching and ETags
  • different output formats (eg. XML and JSON)
  • service/resource discovery or introspection (similar to WSDL or system.listMethods)
  • a client library than can generate complex native python objects given a URI
  • models and other sources of data as REST resources
  • integration at some point with the Django trunk!
What’s Already Out There

There’s a few Django projects for making data available via REST. These efforts seem to have stalled or be in infant stages.

  • Django model views — A GSoC project to make Django model data available via REST.This project never seemed to get far off of the ground. I don’t think it has been updated much since 2007.
  • Django REST interface — Another GSoC project to create RESTful interfaces.There seem to be some active users of this one and it seems to be more fully featured than the above model views project. However, it has stalled and there hasn’t been much work on it in the past few years.
  • Django RESTAPI — Another project to make models available via REST.This project seems to have been more recently updated and it seems ok, but it still isn’t ready for prime time or in Pypi.
  • RESTinPy — A sourceforge project that makes data available in REST.
    This project seems somewhat advanced but it hasn’t been updated since the first cut was put onto sourceforge and Pypi.
  • DAPI — Another model to REST mapping module
Reading

Edit (July 15, 2010): I wrote an update involving Piston, a popular REST framework for Django.
Update (September 7, 2011): There were some updates on what folks in the community were using at Djangocon.

Brewing Update 07-22-09

Posted on by David

I just finished cleaning up after another brewing night.

Malt:
1.5 lbs Mashed
6 lbs Coopers Plain Light
2 lbs Liquid Extract

Hops:
1.5 oz Centennial
0.5 oz Amarillo @ 12 min
0.5 oz Amarillo @ 24 min
0.5 oz Centennial @ 36 min
0.5 oz Amarillo @ 48 min
0.5 oz Amarillo @ last min
2 oz Cascade dry hopped

Water:
1 gallon San Diego tap
4 gallons of Arrowhead Mountain Spring

Yeast:
11.5g Safale US-05 (hydrated)

Gravity:
1.066 @ 84F => 1.069

This should make for a very hoppy IPA provided everything goes well. The alcohol content should be pretty strong considering there’s a lot more malt in this one than there is in the pale ale. The fermentor is sitting in my bathtub where I’m trying to keep it cool. It’s currently 84F in my apartment and the water in the bathtub is at 75 even after I added 7 lbs of ice. Hopefully it can survive tomorrow while I’m at work and happy hour for 14 hours.

Lessons Learned: Releasing A Django Application

Posted on by David

As you may be aware, I released a Django application, RPC4Django, publicly a little while ago. This is what I learned.

Make It Easy

You efforts should be targeted at making adoption of your software as easy as possible. When developers look for a package that accomplishes a goal, they aren’t going to spend much time looking at any individual package. Usually a quick google search or a search in pypi and then they might spend a few minutes glancing at a package before moving on to the next one. There are a couple keys to making it easy:

  • Have easy documentation on installation, configuration and the license.
    This should be on both your webpage as well as on pypi. More on this later.
  • Give demo code or better yet, an actual demo.
    Quite often the one sentence summary on pypi (if you don’t provide a long description) is not sufficient to tell people why they should use your package.
  • List the package’s compatiblity.
    Considering very few packages are compatible between 2.5 and 3.0 or even 2.3 and 2.5, I’m very surprised by this. Nothing is more frustrating than thinking you’ve found what you need just to have installation (or worse, execution) fail.
Test Early and Test Often

When I released RPC4Django, it had a reasonable, but not great, unit tests with it. With the future releases, those improved significantly and a few more bugs were caught. Being able to quickly test python 2.4, 2.5 and 2.6 by having a fully built unit test suite was awesome. However, unit tests don’t catch everything. One bug that I did not catch for a while was a bug relating to how Django worked with mod_wsgi on Apache. No amount of unit testing or Django views testing (which is sort of a hybrid of unit and integration testing) would have caught this. It only got caught when the code was pushed to a production server.

Documentation Conundrums

Documentation usually requires keeping data in a number of places. Any project of considerable size has READMEs, a license file, HTML documentation, tutorials, a setup.py long description, class and module documentation, and more. There should be as few authoritative places for package information as possible. In the first version of RPC4Django, I rolled my own HTML documentation and and README basically told the user to read the real documentation. This solved the whole problem of duplicated documentation, but in a rather unenlightened way.

After reading more exhaustively about reST and looking at how other packages solve this problem, I found a better way. With reST, it is possible to include other documents like so:

What this allows for and what RPC4Django does now is to enable one document to be built of many documents. In this way, I keep the license in one file, the installation in another, the changelog in another and they all are included into my README which can be used to generate my HTML documentation.

If my project was even larger, I might do what both the Python project and the Django project do. They include reST documents in their source tree. These include walkthroughs, tutorials, introductions to classes and more. Sphinx, a python package built to document the python library itself, can create attractive documentation hierarchies directly from simple text documents (which are themselves human readable!). Python uses it for all of their documentation and the Django project uses it for basically the entire documentation section of their webpage — including tutorials. You wouldn’t know it since the pages are pretty attractive but they appear to be generated directly out of subversion on a nightly basis. This keeps all the documentation together and makes sure your website, HTML documentation and source documentation stay in sync.

Don’t Get Discouraged

Just because your package is now on pypi doesn’t mean people are going to flock to it and download it. I think I’ve put together a pretty solid and useful package but I have only a few downloaders and I haven’t gotten much feedback. I intend to power through and start a new project.

Brewing Update 07-19-09

Posted on by David

I bottled the beer yesterday and it tasted like an uncarbonated pale ale. I’m very happy with it so far. With a final gravity of 1.017, the finished beer should be about 5.5% alcohol. Two weeks and it should be drinkable.

RPC4Django 0.1.3 is Available

Posted on by David

Go get it!

Demo Server

I setup a demo server to allow people to preview the functionality of RPC4Django. Creating this caused me to notice an interesting aspect of Django’s request.META: the attribute CONTENT-TYPE may not be set! I had done most of my testing in the development server where it is always set, however, when deploying with mod_wsgi it was not. This was causing RPC4Django to crash. The demo server is using the Django trunk as of today.

Changes
  • Fixed a serious bug where RPC4Django relied on request.META[‘CONTENT_TYPE’] to be set by the web server
  • Added generator tag to the template with RPC4Django and the version
  • Renamed RPC4DJANGO_RESTRICT_DOCUMENTATION to RPC4DJANGO_RESTRICT_METHOD_SUMMARY for clarity
  • Built out the example better so that it could be deployed as a demo without modification
  • Improved JSON output formatting to be consistent with SimpleXMLRPCDispatcher’s XML output
Brewing Update

The beer has stopped bubbling and I’m going to bottle on Saturday. I have purchased ingredients for my next brew, an India Pale Ale.