Friday, 22 January 2010

MyProxy logon via OpenID

The access control architecture for CMIP5 makes use of OpenID and the MyProxy Certificate Authority for single sign on. Recently work has been underway to provide filters to secure data server applications with both OpenID and SSL client based authentication front ends. In my case helped along considerably by the WSGI based architecture adopted.

An important requirement has been to enable the user community to access data via a browser but also via scripts such as wget and dedicated client software. The issue of delegation will have to be tackled at some point and one example scenario is Live Access Server with accessing TDS instances on behalf of a user. Also in the future perhaps workflows with OGC services. There has been a lot of work done in this area already.

Given we have PKI based authentication incoporated a 'classic' grid based solution with proxy certificates would be possible. Maybe OAuth might be another avenue to look into.

One possibility with OpenID, would be to exploit Attribute Exchange (AX) extension to perform an MyProxy logon over this interface. A Relying Party could obtain a user certificate returned across the AX interface from the user's OpenID Provider (OP). MyProxy logon normally involves,
  1. creation of a key pair at the client
  2. creation of a certificate request containing the public key
  3. authentication of the client against a MyProxy Server
  4. submission of certificate request to the server
  5. certificate issued by the server and returned to the client
Once obtained the client can use the PKI credentials to authenticate against other services. In this case an OpenID Relying Party (RP) represents a service which wants to access other services on behalf of the user. It could perform steps 1 and 2 but then pass the certificate request to the Provider during the sign in process. Fortunately the AX spec has a store message feature which could make this possible.

When the user signs in at the Provider, they provide username/password as usual, but the Provider links with its own MyProxy Server and makes a logon client call passing username, password and the certificate request obtained from the RP. If all is well, MyProxy returns a new certificate and the Provider can then pass this back to the RP over the AX interface. The RP now has PKI credentials for the user delegated to it so that it can make calls to other services on the user's behalf. Of course this method only provides one level of delegation but that may be sufficient for many use cases.

The sequence diagram below illustrates the steps:


Thursday, 6 August 2009

WSGI Architecture for NERC DataGrid Security

Over the past year I've been porting the Python based security system for NERC DataGrid to a WSGI based architecture. This is paying huge dividends in terms of the modularity of the code and ability to apply flexible deployment configurations especially when used together with Paste Deploy.

Key parts to the architecture are the authentication and authorisation handlers triggered from the respective 401 and 403 HTTP response codes together with a URI based access control policy. I've used the Python security middleware package AuthKit to help me put this together. One thing I've been meaning to do is to lay this out in a simple example. This first snippet gives an overview:


app = AuthorisationPolicyMiddleware(myApp)

app = MultiHandler(app)
app.add_method("checkerID", AuthenticationHandlerMiddleware)
app.add_checker("checkerID", AuthenticationHandlerMiddleware.trigger)

app = MultiHandler(app)
app.add_method("checkerID", AuthorisationHandlerMiddleware)
app.add_checker("checkerID", AuthorisationHandlerMiddleware.trigger)

from paste.httpserver import serve
from paste.deploy import loadapp

serve(app, host='0.0.0.0', port=9080)

The application to be protected is defined in a WSGI elsewhere. This is wrapped in a number of pieces of middleware chained together to form a pipeline to intercept requests to the application. On the last line it is served using Paste.

The first middleware component listed, AuthorisationPolicyMiddleware, checks the requested URI against a policy. If the user is not authorised, it sets a HTTP "403 Forbidden" response bypassing myApp.

Following this, there are two pieces of middleware making use of AuthKit's authkit.authenticate.multi.Multihandler. The MultiHandler accepts two key inputs: a checker function which determines the criteria for intercepting a request, and a method, a WSGI middleware to determine what action to take once an intercept has been made.

In the first case, a class method AuthenticationHandlerMiddleware.trigger has been defined to intercept HTTP "401 Unauthorized" status codes. The AuthenticationHandlerMiddleware itself determines the action taken:

class AuthenticationHandlerMiddleware(object):
"""Handler for HTTP 401 Unauthorized responses"""

triggerStatus = "401 Unauthorized"

def __init__(self, global_conf, **app_conf):
pass

def __call__(self, environ, start_response):
log.info("AuthenticationHandlerMiddleware access denied response ...")
response = "HTTP 401 Unauthorised response intercepted"
start_response('200 OK', [('Content-type', 'text/plain'),
('Content-length', str(len(response)))])
return [response]

@classmethod
def trigger(cls, environ, status, headers):
if status == cls.triggerStatus:
log.info("Authentication Trigger caught status [%s]",
cls.triggerStatus)
return True
else:
return False


In the above, the middleware simply outputs a message but it effectively provides a hook to trigger a login or other authentication interface.

A second Multihandler is in place to handle HTTP "403 Forbidden" responses. This follows a similar pattern:


class AuthorisationHandlerMiddleware(object):
"""Handler for HTTP 403 Forbidden responses"""

triggerStatus = "403 Forbidden"

def __init__(self, global_conf, **app_conf):
pass

def __call__(self, environ, start_response):
log.info("AuthorisationHandlerMiddleware access denied response ...")
response = "HTTP 403 Forbidden response intercepted"
start_response('200 OK', [('Content-type', 'text/plain'),
('Content-length', str(len(response)))])
return [response]

@classmethod
def trigger(cls, environ, status, headers):
if status == cls.triggerStatus:
log.info("Authorisation Trigger caught status [%s]",
cls.triggerStatus)
return True
else:
return False


The trigger method sets a True response to signal to the Multihandler to intercept the request and invoke AuthorizationMiddleware to deliver an access denied message.

This next snippet shows myApp effectively a test harness to demonstrate the middleware behaviour:

def myApp(environ, start_response):
"""Test application to be secured"""

if environ['PATH_INFO'] == "/test_401":
status = "401 Unauthorized"
response = status

elif environ['PATH_INFO'] == "/test_403":
status = "403 Forbidden"
response = status

elif environ['PATH_INFO'] == "/secured":
status = "200 OK"
response = "Secured URI"

else:
status = "404 Not Found"
response = status

log.info("Application is setting [%s] response..." % status)
start_response(status,
[('Content-type', 'text/plain'),
('Content-length', str(len(response)))])

return [response]


As set-up above,
  1. http://localhost:9080/test_401 will trigger the authentication middleware and
  2. http://localhost:9080/test_403 the authorisation middleware.
  3. The last, http://localhost:9080/test_secured, demonstrates the access control policy implemented in AuthorisationPolicyMiddleware:
class AuthorisationPolicyMiddleware(object):
"""Apply a security policy based on the URI requested"""

def __init__(self, app):
self.securedURIs = ['/test_secured']
self.app = app

def __call__(self, environ, start_response):
if environ['PATH_INFO'] in self.securedURIs:
log.info("Path [%s] is restricted by the Authorisation policy" %
environ['PATH_INFO'])
status = "403 Forbidden"
response = status
start_response(status, [('Content-type', 'text/plain'),
('Content-length', str(len(response)))])
return [response]
else:
return self.app(environ, start_response)


The middleware has a policy consisting of a list of URIs to be secured in the securedURIs attribute. In practice this could link to a policy file, database link or some other interface. The __call__ method intercepts request URIs which match the policy and invokes a HTTP 403 response. This in turn brings into play the AuthorisationMiddleware handler triggering it to return an access denied response.

The complete example is in the NERC DataGrid SubVersion repository.

Monday, 3 August 2009

Python List Utility Classes

I've been adapting some Java code to Python recently and wanted some tighter control over list elements than the default list type. I've extended list with two custom classes. The first, TypedList restricts list elements to a given type or types e.g.

>>> t=TypedList(float)
>>> t+=[9]
Traceback (most recent call last):
File "", line 1, in
File "", line 34, in __iadd__
TypeError: List items must be of type float
The existing array type gives similar capability but with this you can put in any type ...


class TypedList(list):
"""Extend list type to enabled only items of a given type. Supports
any type where the array type in the Standard Library is restricted to
only limited set of primitive types
"""

def __init__(self, elementType, *arg, **kw):
"""@type elementType: type/tuple
@param elementType: object type or types which the list is allowed to
contain. If more than one type, pass as a tuple
"""
self.__elementType = elementType
super(TypedList, self).__init__(*arg, **kw)

def _getElementType(self):
return self.__elementType

elementType = property(fget=_getElementType,
doc="The allowed type or types for list elements")

def extend(self, iter):
for i in iter:
if not isinstance(i, self.__elementType):
raise TypeError("List items must be of type %s" %
(self.__elementType,))

return super(TypedList, self).extend(iter)

def __iadd__(self, iter):
for i in iter:
if not isinstance(i, self.__elementType):
raise TypeError("List items must be of type %s" %
(self.__elementType,))

return super(TypedList, self).__iadd__(iter)

def append(self, item):
if not isinstance(item, self.__elementType):
raise TypeError("List items must be of type %s" %
(self.__elementType,))

return super(TypedList, self).append(item)

For the second class I wanted a way of avoiding the addition of duplicate elements to a list:


>>> u=UniqList()
>>> u.append('a')
>>> u
['a']
>>> u.append('a')
>>> u
['a']

It silently ignores the duplicate element. It would be straightforward to alter to raise an exception if this was the preferred behaviour. Here's the class:


class UniqList(list):
"""Extended version of list type to enable a list with unique items.
If an item is added that is already present then it is silently omitted
from the list
"""
def extend(self, iter):
return super(UniqList, self).extend([i for i in iter if i not in self])

def __iadd__(self, iter):
return super(UniqList, self).__iadd__([i for i in iter
if i not in self])

def append(self, item):
for i in self:
if i == item:
return None

return super(UniqList, self).append(item)

Thursday, 23 July 2009

PyDev, PyLint and Refactoring

I've been using PyDev the Eclipse plugin for Python for some time now and it's certainly improving with each new release (I'm currently on 1.4.2). There are a couple of features I've been getting some benefit from recently.

With PyLint, I actually broke my Eclipse set-up for this a while ago but when I got it reinstated it's reminded me of how useful it is. One thing I hate about about interpreted languages is the huge scope for runtime errors. The PyLint plugin gives immediate feedback in the margin of the editor window with error information and hints. This is saving me a lot of time down the line. What's the metric for time spent fixing a bug immediately vs. at the other end of the scale trying to pick through code on a production deployment? :) Picking up references to undeclared variables is particularly useful. Especially in error blocks or places where test coverage might miss out. There's no doubt some messages can be annoying and getting a mark out of 10 for your code might not appeal to all. Fortunately, you can edit the settings for the Pylint command line that's executed and explicitly filter out warnings you don't like. The second feature I've revisited is the PyDev refactoring options. I have had a low expectation of these perhaps unfairly but there a couple of potential time savers. Take this simple class:


class MyTest(object):
def doSomething(self):
self.__a = None
self.__b = None
self.__c = []
Left like this PyLint will warn me that I haven't set up these attributes in an __init__ method. ;) PyDev refactoring to the rescue :) ... If I now right click in the editor and pick "Refactoring" -> "Generate Constructor using Fields ...". I'm taken through a series of steps in a dialog. I can get,

class MyTest(object):
def __init__(self, a, b, c):
self.__a = a
self.__b = b
self.__c = c

def doSomething(self):
self.__a = None
self.__b = None
self.__c = []


Not that exciting but could save some typing. More useful for me is the "Generate Properties..." to generate properties for new style classes together with their getters and setters:


class MyTest(object):
def __init__(self, a, b, c):
self.__a = a
self.__b = b
self.__c = c

def getA(self):
return self.__a

def getB(self):
return self.__b

def getC(self):
return self.__c

def setA(self, value):
self.__a = value

def setB(self, value):
self.__b = value

def setC(self, value):
self.__c = value

def delA(self):
del self.__a

def delB(self):
del self.__b

def delC(self):
del self.__c

def doSomething(self):
self.__a = None
self.__b = None
self.__c = []

a = property(getA, setA, delA, "A's Docstring")

b = property(getB, setB, delB, "B's Docstring")

c = property(getC, setC, delC, "C's Docstring")

It could be improved on - there's no explicit keywords inputs for property() - but it could save some time with the boiler plate. :)

Wednesday, 2 April 2008

Calling Python from Perl

After a long silence this is definitely worth an entry - the Perl Inline module for calling code in various other languages from Perl...

I've been looking into integrating NERC DataGrid Security (all written in Python) into the existing BADC web site Perl CGI code - quick dirty if necessary.

The NDG Security Single Sign On service has a http interface so this should be straightforward - Perl on one side of the interface and Python the other. However the various SOAP interfaces with WS-Security added into the mix doesn't make it very attractive. I've not looked into Perl SOAP support - and this may be great - but if we are migrating all our Perl code to Python it would seem a wasted effort.
Inline is really impressive. I had a python client call to the NDG Attribute Authority working in minutes - Perl calling Python calling a SOAP Service and returning the response back to Perl as a local variable:

Perl <-> Python <-> SOAP + WS-Security <-> Python Attribute Authority Web service

Installation was fairly straightforward on Ubuntu - libinline-perl package is available but Inline-Python required downloaded and install from CPAN.

I was expecting all kinds of shared library link errors but all works perfectly. It's nice when things work simply easily and quickly for a change :)


#!/usr/bin/env perl
use Inline Python => <<'END';
def getAAHostsInfo():
    from ndg.security.common.AttAuthority import AttAuthorityClient
    import os

    aaClnt = AttAuthorityClient(uri="http://localhost:5000/AttributeAuthority",
                                cfgFilePath='wssecurity.cfg')

    allHostsInfo = aaClnt.getAllHostsInfo()
    return "All hosts info = %s" % allHostsInfo
END

print getAAHostsInfo(), "\n";

Adapting Python setuptools to Execute Post-installation Steps

I've been experimenting with the Python setuptools package for sometime now to enable customisations for the NERC DataGrid Security installation (eggifying the Python OpenSSL package M2Crypto).  For NDG Security, a wrapper script ndg-security-install.py to easy_install allows for a custom M2Crypto build and for the fact that at the time the Python Twisted package was not available as an egg.

I now want to make the installation easier still. There's a config directory for the server egg that needs to be set-up post egg install. This a manual process but I've now incorporated this step into ndg-security-install.py. A copy is made of the ndg.security.server.conf package from the egg to a separate location - default is /etc/ndg/security/conf

To do this I need to know how to get a handle on the directory path ndg.security.server.conf contained in the latest egg. Digging into the setuptools pkg_resources package this turns out to be easy:


eggConfigDir = pkg_resources.resource_filename('ndg.security.server', 
'conf')



... but I then wanted some capability to avoid trashing a pre-existing conf/ from a previous installation. I've done this by installing the new conf/ with a suffix corresponding to the latest NDG Security server egg version. To get the version:


# Get distribution version info
serverDistro = pkg_resources.get_distribution('ndg-security-server')
configDirVers = "%s.%s" % (self.opt.configDir, serverDistro.version)


but when tested I noticed that it was picking up the wrong version - the version of the LAST install not the NEW one from the update. Moving my pkg_resources import to immediately BEFORE the version check fixed this!! ...


import pkg_resources
eggConfigDir = pkg_resources.resource_filename('ndg.security.server', 
'conf')
# Get distribution version info
serverDistro = pkg_resources.get_distribution('ndg-security-server')
configDirVers = "%s.%s" % (self.opt.configDir, serverDistro.version)
# Get distribution version info
serverDistro = pkg_resources.get_distribution('ndg-security-server')
configDirVers = "%s.%s" % (self.opt.configDir, serverDistro.version)


Previously I had my import at the top of the file. pkg_resources must check for the current egg version at import. The code to install the updated packages occurs of course at a later point in the execution.

Thursday, 14 June 2007

NDG Security, trust and role mapping

The NERC Data Grid security system uses a model of bilateral agreements between organisations to establish trust and so share access to data. This was agreed from early on in the project after hammer out the differing requirements of the participating organisations.

Trust agreements are made by means of role mapping. This assumes that each organisation has a pre-existing system of role based access control to data or at least that, they are prepared to create one. This was in fact a major driver for this solution: organisations already had established roles for data access. They didn't wish make major changes to their infrastructures to support new ones.

Another factor was that access to some datasets is bound by legal agreement with some third party. Organisations did not want to have to re-negotiate these agreements in order to support new roles for cross-organisational access.

To establish a role mapping agreement, an organisation agrees a number of its roles that it wishes to map to roles which the trusted organisation supports. For example, Organisation A has a dataset whose access is restricted by role datasetN. It wishes to make this data available to post graduate researchers at Organisation B. Organisation B has a role allocated to these people called postgrad. Organisation makes a mapping from role postgrad -> datasetN. Currently the system use an XML based role mapping configuration. Each organisation uses this file to establish role mappings. For the example given, Organisation A's map configuration file might look like this:


<?xml version="1.0"; encoding="UTF-8"?>
<aamap>
    <trusted name="Organisation B">
        <role local="datasetN" remote="postgrad"/>
            .
            .
            .
        <role ... />
    <trusted/>
</aamap>

Mediating access

Roles are allocated to users in Attribute Certificates, XML documents with time limited validity, digitally signed by Attribute Authority that issued them. The Attribute Authority must authenticate the request against a valid user ID in order to issue a certificate. In this case, the user must be known to the Attribute Authority. Access control decisions are made by Gatekeepers. An Attribute Certificate is presented to a gatekeeper along with user ID credentials. These are checked and if a role in the certificate matches the role controlling access to the resource, access is granted.

Attribute Authorities are also responsible for role mapping. In the earlier example, the user of Organisation B is not known to A's Attribute Authority. Therefore a request for an Attribute Certificate would be rejected. However, in this case user B can present their Organisation B Attribute Certificate. Attribute Authority A recognises this as issued from a trusted organisation and applies the mapping to the roles contained. The mapped roles are issued in a new mapped Attribute Certificate issued by a Attribute Authority B. In the example, the original certificate might have contained the role postgrad and the mapped certificate would contain datasetN.

The mapped certificate can now be presented to the appropriate gatekeeper at A in order to gain access to data.

The current system restricts role mapping to one level: it is not possible to map roles from a mapped certificate.

Sequence

The example here is taken from the DEWS project where NDG Security is employed. The steps are broken down into two diagrams, login and access at a remote site. The Portal organisation is trusted by the Health Data Server.



Critique

I want to examine the benefits and drawbacks of role mapping as a mechanism to establish trust...

benefits

  • Decentralised: trust on an organisation to organisation basis
  • Selective: organisation chooses who it wants to trust on a case by case basis
  • Scalable: organisations can easily join the system without submitting their entire user base to a central repository
  • Flexible: new trust relationships can easily be made or existing ones broken
  • Control over the Level of trust: A can set the mapping so that gold user role in B may only map to bronze in A
  • Supports organisations' existing security infrastructures with minimal change or disruption. Organisations within NDG had pre-existing authorisation systems and roles. Using role mapping their existing native roles can be maintained without change to user entitlements or associations of roles to datasets.

drawbacks

  • Role mapping adds a layer of complexity in the transaction to gain access to a resource. The client software acting on behalf of the user must obtain an original certificate and then present this for role mapping to get a mapped certificate. David Chadwick has made the case that role mapping is better applied at the gatekeeper giving the analogy of airline loyalty cards. Going to airline A's lounge you don't swap your B airline card for an A one to get in. You present your B card knowing that A will accept it. Thus, a mapped Attribute Certificate would never exist. Its roles would be mapped at the point where the access control decision is made. In favour of mapped certificates it could be argued that it enables more primitive gatekeepers. They have no need of role mapping information since it is held at a central point for the organisation ie. its Attribute Authority
  • Role mappings between a large number of organisations could become difficult to maintain
  • Bilateral agreements must be kept up to date and maintained. If one party changes the mean of a role or deletes a role used in mapping then the mapping itself is broken and this potentially without the knowledge of the other party.
  • Role mapping as an approach could be rejected entirely in favour of centralised service set up jointly between two or more organisations to issue roles.


    • Common roles are agreed and users from participating organisations are registered with the service.
    • Organisations' site administrators are delegated the responsiblity to allocate roles appropriately on a case by case basis.
    • The process of authorisation is simplified since no role mapping is involved. A token issued from the central service is recognised by individual organisation's gatekeepers.
    • Users would need to register with the new service and agree to the terms of the virtual organisation. Legally, they need to give their consent for their ID information to be held by the service.