RPC4Django v0.1.7 is Available
Go get it!
Thanks to Ale for reporting the issue with the CSRF framework.
Changes
- Fixed a bug relating to CSRF
- Added feature to allow recursive imports of RPC methods
RPC4Django v0.1.6 is Available
Go get it!
Thanks to everyone who tested out this release before it went live.
Changes
- Changed the XMLRPC dispatcher to allow sending nil which translates to the Python
None. This was already allowed with JSONRPC - Cross site request forgery framework support
- Added cross domain access control
- Added unit tests for base64 encoded binary
- Added access to HttpRequest object form inside a called RPC method and associated unit tests
RPC4Django v0.1.6 is Almost Done!
I have checked the latest changes into subversion. Here is a list of the features supported:
- Changed the XMLRPC dispatcher to allow sending nil which translates to the Python
None. This was already allowed with JSONRPC - Cross site request forgery framework support
- Added cross domain access control
- Added unit tests for base64 encoded binary
- Added access to HttpRequest object form inside a called RPC method and associated unit tests
Some of these features were fairly major and I am lucky that I have pretty good unit tests in place. However, since most of these features were user requested, I really hope that I can get some of you guys to run this version in your environment and let me know how it works.
In addition, I’m sorry that I am so late in making this release. As of a week ago, I began a new job (although with the same company). The transition was somewhat taxing, but I should have a little more time freed up for open source work now that everything has settled down.
RPC4Django v0.1.6 Feature Requests
A number of feature requests have come in and I am going to outline what I’m going to try to get done. I hope to put out a release next week (I’ve said that before), but that greatly depends on what features go into it.
Access to HttpRequest
One user submitted feature that seems particularly useful is the ability to access variables and data in the HttpRequest that dispatches the RPC method. For example:
@rpcmethod(permission='myproject.mypermission')
def testmethod(a, b, c):
# Do something
This RPC method will be authenticated, but it cannot tell who the user is for example. It does not have the ability to read the REMOTE_USER variable. This would be particularly useful if the RPC method was submitting data and needed to know who the submitter was. To rectify this situation, I am proposing sending the HttpRequest object as a keyword argument:
@rpcmethod(permission='myproject.mypermission')
def testmethod(a, b, c, **kwargs):
if 'REMOTE_USER' in kwargs['request'].META:
user = request.kwargs.META['REMOTE_USER']
# Do something
Methods that need access can accept all the additional keyword arguments and methods that don’t care can be written as normal and do not need to change.
Work with the cross site request forgery framework
Django has added a new module to protect against cross site request forgery (CSRF). Unfortunately, if a user adds the django.middleware.csrf.CsrfViewMiddleware to their project, it will break RPC4Django. CSRF protection works by ensuring that all GET requests are side-effect free and then adding a little bit of random data (the CSRF token) to every form that submits via POST. The server can then verify that the correct random data was POSTed and this ensures that the form was submitted from your site and not some remote site.
Unfortunately, RPC is usually intended to be used in a cross site fashion. The fix is relatively painless, but requires some adversarial testing to make it work under Django 1.0, Django 1.1 and Django 1.2. The code (views.py) will look something like this:
try:
# Django 1.2
from django.views.decorators.csrf import csrf_exempt
except ImportError:
try:
# Django 1.1
from django.contrib.csrf.middleware import csrf_exempt
except ImportError:
# Django 1.0
csrf_exempt = None
if csrf_exempt is not None:
csrf_exempt(serve_rpc_request)
I need to test it out very thoroughly and on a number of setups, but I think this code should do it.
HTTP access control
Usually the Javascript XmlHttpRequest object is subject to the same origin policy. This is a security measure that ensures that Javascript will only access data from the same domain, protocol and port as the initial request. There have always been hacks to get around this and the browser vendors are finally providing a uniform, secure way around.
Basically, RPC4Django will need to respond to an HTTP OPTIONS request with the header Access-Control-Allow-Origin specifying what domains are allowed to send requests. Both allowing HTTP access control and which domains are allowed will be options in settings.py.
Patches and feature requests
As always, I am happy to accept patches, but this is what I’ll be working on next week. Please let me know if there are any other important features that you would like to see in RPC4Django.
Good Documentation Makes a Good Project
Perhaps it is a platitude but it’s true that good documentation makes a project successful. Recently, Jacob Kaplan-Moss of Django fame started a series of articles on documentation — perhaps one of the most underrated aspects of a software. Although I may not agree with everything (technical documentation following MLA?! Just couldn’t get away from that lit degree huh Jacob?), the series as a whole is truly readable and he puts words to some of my own vague thoughts on documentation. With all that in mind, I want to focus on what led to Django’s documentation and some concrete examples of successful and not so successful documentation.
The evolution of software documentation
Back in the dark ages, all documentation was done by hand. Considering that developers are pretty bad about documentation now, I don’t even want to imagine what it was like back then. When I first got on the programming scene, Javadoc was a pretty new development. Javadoc was great in that it allowed you to write documentation only once and at the same time you wrote the code. Then, you used your code files to generate pretty HTML. This was amazing for API reference documentation and I still think that Java has the most well documented standard library API of any programming language. After seeing that you could auto-generate documentation, a host of clones like POD and Doxygen followed. Unfortunately, too many people decided that having Javadoc means that your project is documented (I’m looking at you SNMP4J!).
The next stage in the evolution came from PHP’s excellent documentation of all places. The PHP folks started including formatted examples, suggestions and caveats with their reference API. They also started putting together guides about particular aspects of the language such as security or working with Oracle. On top of this, they had the seemingly ingenious idea of allowing users to post comments on the documentation. At first, this worked well. Years ago, I posted a couple steps I took to get PHP working with Oracle which is still a pain to this day. However, over time, the comments got clogged with snippets of useless code, cries for help and other drivel. It’s no small wonder that Django removed the ability to comment on their documentation.
Django embodies the current incarnation of documentation and although I think it is the best documented project I’ve worked with, I’m not convinced it is ideal. Django combines the PHP-style guides with tutorials. This works exceptionally well for getting new users off the ground quickly and bringing users familiar with the project up to speed in a new area. However, I feel that Django lacks somewhat in the reference API area and part of this has to do with the fact that they are writing all of the documentation by hand instead of generating it from docstrings. The method references are usually an afterthought (because they don’t happen at the same time the code is written) and don’t contain the level of detail that the PHP or Java documentation does. Without looking at the actual code, how else would you figure out that the markdown filter can accept extra parameters.
A success story
Despite the trash I just talked about Django (blasphemy!), it is a success and the documentation is superb. It gets you up and running very quickly and has detailed documentation on virtually every aspect of the project. I attempt to model both my open source projects’ and my work projects’ documentation after Django’s — imitation is definitely a form of flattery. Django has maintained its documentation by enforcing some rules and good practices. The Django project maintains all the reST formatted documentation files in the code line and requires that patches include updates to the relevant documentation. This ensures that the docs are up-to-date with the code — a problem lots of projects suffer from. Django uses Sphinx to generate their documentation periodically from the code line — I don’t know how often, but fairly often — and make it available as the official Django documentation. Inaccuracies and problems are caught quickly.
What not to do
Early Linux suffered from some serious documentation fail. I remember being familiar with installing Windows and Mac system 8-9 from scratch and figured that I could install Linux. Perhaps I made the mistake of trying Slackware, but I can remember even after having a mostly successful install having to compile all sorts of packages from scratch. It was a great hobby for CS student back in 2001, but hardly a well documented and easy to use project. The fact that the code is great and the system is stable doesn’t help if you can’t get the thing running (insert car analogy about a supercharged engine with no diagrams for putting it into an actual car).
In the office, we suffer from documentation issues because nobody wants to write it themselves and nobody can agree on a uniform way of doing it. We ended up with multiple wikis all over the place none of which are complete. Some small teams have design documents sitting in source control or in eRoom. We have requirements sitting in various Word documents and designs in Powerpoint presentations. There’s not even a tutorial telling new employees where we store our bugs, requirements, designs, or wikis or how new bugs should be filed, new requirements introduced or how wikis should be updated. In general, we have fragmented, tribal knowledge where nobody knows the whole story.
Making it better
At some point, somebody needs to lay down the law and start creating tutorials, walkthroughs and other documentation for a project. I am only one person in a huge division of an even larger corporation, but I already have a reputation for writing documentation. Django has their benevolent dictators declaring that all patches shall come with documentation. The Ubuntu community (and the Redhat and Mandrake guys before them) has taken Linux from having an arcane install process to being as easy as Windows — and it really shows.