Thursday 6 August 2009

WSGI Architecture for NERC DataGrid Security

Over the past year I've been porting the Python based security system for NERC DataGrid to a WSGI based architecture. This is paying huge dividends in terms of the modularity of the code and ability to apply flexible deployment configurations especially when used together with Paste Deploy.

Key parts to the architecture are the authentication and authorisation handlers triggered from the respective 401 and 403 HTTP response codes together with a URI based access control policy. I've used the Python security middleware package AuthKit to help me put this together. One thing I've been meaning to do is to lay this out in a simple example. This first snippet gives an overview:


app = AuthorisationPolicyMiddleware(myApp)

app = MultiHandler(app)
app.add_method("checkerID", AuthenticationHandlerMiddleware)
app.add_checker("checkerID", AuthenticationHandlerMiddleware.trigger)

app = MultiHandler(app)
app.add_method("checkerID", AuthorisationHandlerMiddleware)
app.add_checker("checkerID", AuthorisationHandlerMiddleware.trigger)

from paste.httpserver import serve
from paste.deploy import loadapp

serve(app, host='0.0.0.0', port=9080)

The application to be protected is defined in a WSGI elsewhere. This is wrapped in a number of pieces of middleware chained together to form a pipeline to intercept requests to the application. On the last line it is served using Paste.

The first middleware component listed, AuthorisationPolicyMiddleware, checks the requested URI against a policy. If the user is not authorised, it sets a HTTP "403 Forbidden" response bypassing myApp.

Following this, there are two pieces of middleware making use of AuthKit's authkit.authenticate.multi.Multihandler. The MultiHandler accepts two key inputs: a checker function which determines the criteria for intercepting a request, and a method, a WSGI middleware to determine what action to take once an intercept has been made.

In the first case, a class method AuthenticationHandlerMiddleware.trigger has been defined to intercept HTTP "401 Unauthorized" status codes. The AuthenticationHandlerMiddleware itself determines the action taken:

class AuthenticationHandlerMiddleware(object):
"""Handler for HTTP 401 Unauthorized responses"""

triggerStatus = "401 Unauthorized"

def __init__(self, global_conf, **app_conf):
pass

def __call__(self, environ, start_response):
log.info("AuthenticationHandlerMiddleware access denied response ...")
response = "HTTP 401 Unauthorised response intercepted"
start_response('200 OK', [('Content-type', 'text/plain'),
('Content-length', str(len(response)))])
return [response]

@classmethod
def trigger(cls, environ, status, headers):
if status == cls.triggerStatus:
log.info("Authentication Trigger caught status [%s]",
cls.triggerStatus)
return True
else:
return False


In the above, the middleware simply outputs a message but it effectively provides a hook to trigger a login or other authentication interface.

A second Multihandler is in place to handle HTTP "403 Forbidden" responses. This follows a similar pattern:


class AuthorisationHandlerMiddleware(object):
"""Handler for HTTP 403 Forbidden responses"""

triggerStatus = "403 Forbidden"

def __init__(self, global_conf, **app_conf):
pass

def __call__(self, environ, start_response):
log.info("AuthorisationHandlerMiddleware access denied response ...")
response = "HTTP 403 Forbidden response intercepted"
start_response('200 OK', [('Content-type', 'text/plain'),
('Content-length', str(len(response)))])
return [response]

@classmethod
def trigger(cls, environ, status, headers):
if status == cls.triggerStatus:
log.info("Authorisation Trigger caught status [%s]",
cls.triggerStatus)
return True
else:
return False


The trigger method sets a True response to signal to the Multihandler to intercept the request and invoke AuthorizationMiddleware to deliver an access denied message.

This next snippet shows myApp effectively a test harness to demonstrate the middleware behaviour:

def myApp(environ, start_response):
"""Test application to be secured"""

if environ['PATH_INFO'] == "/test_401":
status = "401 Unauthorized"
response = status

elif environ['PATH_INFO'] == "/test_403":
status = "403 Forbidden"
response = status

elif environ['PATH_INFO'] == "/secured":
status = "200 OK"
response = "Secured URI"

else:
status = "404 Not Found"
response = status

log.info("Application is setting [%s] response..." % status)
start_response(status,
[('Content-type', 'text/plain'),
('Content-length', str(len(response)))])

return [response]


As set-up above,
  1. http://localhost:9080/test_401 will trigger the authentication middleware and
  2. http://localhost:9080/test_403 the authorisation middleware.
  3. The last, http://localhost:9080/test_secured, demonstrates the access control policy implemented in AuthorisationPolicyMiddleware:
class AuthorisationPolicyMiddleware(object):
"""Apply a security policy based on the URI requested"""

def __init__(self, app):
self.securedURIs = ['/test_secured']
self.app = app

def __call__(self, environ, start_response):
if environ['PATH_INFO'] in self.securedURIs:
log.info("Path [%s] is restricted by the Authorisation policy" %
environ['PATH_INFO'])
status = "403 Forbidden"
response = status
start_response(status, [('Content-type', 'text/plain'),
('Content-length', str(len(response)))])
return [response]
else:
return self.app(environ, start_response)


The middleware has a policy consisting of a list of URIs to be secured in the securedURIs attribute. In practice this could link to a policy file, database link or some other interface. The __call__ method intercepts request URIs which match the policy and invokes a HTTP 403 response. This in turn brings into play the AuthorisationMiddleware handler triggering it to return an access denied response.

The complete example is in the NERC DataGrid SubVersion repository.

Monday 3 August 2009

Python List Utility Classes

I've been adapting some Java code to Python recently and wanted some tighter control over list elements than the default list type. I've extended list with two custom classes. The first, TypedList restricts list elements to a given type or types e.g.

>>> t=TypedList(float)
>>> t+=[9]
Traceback (most recent call last):
File "", line 1, in
File "", line 34, in __iadd__
TypeError: List items must be of type float
The existing array type gives similar capability but with this you can put in any type ...


class TypedList(list):
"""Extend list type to enabled only items of a given type. Supports
any type where the array type in the Standard Library is restricted to
only limited set of primitive types
"""

def __init__(self, elementType, *arg, **kw):
"""@type elementType: type/tuple
@param elementType: object type or types which the list is allowed to
contain. If more than one type, pass as a tuple
"""
self.__elementType = elementType
super(TypedList, self).__init__(*arg, **kw)

def _getElementType(self):
return self.__elementType

elementType = property(fget=_getElementType,
doc="The allowed type or types for list elements")

def extend(self, iter):
for i in iter:
if not isinstance(i, self.__elementType):
raise TypeError("List items must be of type %s" %
(self.__elementType,))

return super(TypedList, self).extend(iter)

def __iadd__(self, iter):
for i in iter:
if not isinstance(i, self.__elementType):
raise TypeError("List items must be of type %s" %
(self.__elementType,))

return super(TypedList, self).__iadd__(iter)

def append(self, item):
if not isinstance(item, self.__elementType):
raise TypeError("List items must be of type %s" %
(self.__elementType,))

return super(TypedList, self).append(item)

For the second class I wanted a way of avoiding the addition of duplicate elements to a list:


>>> u=UniqList()
>>> u.append('a')
>>> u
['a']
>>> u.append('a')
>>> u
['a']

It silently ignores the duplicate element. It would be straightforward to alter to raise an exception if this was the preferred behaviour. Here's the class:


class UniqList(list):
"""Extended version of list type to enable a list with unique items.
If an item is added that is already present then it is silently omitted
from the list
"""
def extend(self, iter):
return super(UniqList, self).extend([i for i in iter if i not in self])

def __iadd__(self, iter):
return super(UniqList, self).__iadd__([i for i in iter
if i not in self])

def append(self, item):
for i in self:
if i == item:
return None

return super(UniqList, self).append(item)