Functional testing

web2py comes with a module gluon.contrib.webclient which allows functional testing of local and remote web2py applications. Actually, this module is not web2py specific and it can be used for testing and interacting programmatically with any web application, yet it is designed to understand web2py session and web2py postbacks.

Here is an example of usage. The program below creates a client, connects to the “index” action in order to establish a session, registers a new user, then logouts, and logins again using the newly created credentials:

from gluon.contrib.webclient import WebClient
client = WebClient('http://127.0.0.1:8000/welcome/default/',
                   postbacks=True)
client.get('index')
# register
data = dict(first_name='Homer',
            last_name='Simpson',
            email='homer@web2py.com',
            password='test',
            password_two='test',
            _formname='register')
client.post('user/register', data=data)
# logout
client.get('user/logout')
# login again
data = dict(email='homer@web2py.com',
            password='test',
            _formname='login')
client.post('user/login', data=data)
# check registration and login were successful
client.get('index')
assert('Welcome Homer' in client.text)

The WebClient constructor takes a URL prefix as argument. In the example that is “http://127.0.0.1:8000/welcome/default/“. It does not perform any network IO. The postbacks argument defaults to True and tells the client how to handle web2py postbacks.

The WebClient object, client, has only two methods: get and post. The first argument is always a URL postfix. The full URL for the GET of POST request is constructed simply by concatenating the prefix and the postfix. The purpose of this is imply making the syntax less verbose for long conversations between client and server.

data is a parameter specific of POST request and contains a dictionary of the data to be posted. Web2py forms have a hidden _formname field and its value must be provided unless there is a single form in the page. Web2py forms also contain a hidden _formkey field which is designed to prevent CSRF attacked. It is handled automatically by WebClient.

Both client.get and client.post accept the following extra arguments:

• headers: a dictionary of optional HTTP headers.
• cookies: a dictionary of optional HTTP cookies.
• auth: a dictionary of parameters to be passed to urllib2.HTTPBasicAuthHandler().add_password(**auth) in order to perform basic authentication. For more information about this we refer to the Python documentation for the urllib2 module.

The client object in the example carries on a conversation with the server specified in the constructor by making GET and POST requests. It automatically handles all cookies and sends them back to maintain sessions. If it detects that a new session cookie is issued while an existing one is already present, it interprets it as a broken session and raises an exception. If the server returns an HTTP error, it raises an exception. If the server returns an HTTP error which contains a web2py ticket, it returns a RuntimeError exception containing the ticket code.

The client object maintains a log of requests in client.history and a state associated with its last successful request. The state consists of:

• client.status: the returned status code
• client.text: the content of the page
• client.headers: a dictionary of parsed headers
• client.cookies: a dictionary of parsed cookies
• client.sessions: a dictionary of web2py sessions in the form {appname: session_id}.
• client.forms: a dictionary of web2py forms detected in the client.text. The dictionary has the form {_formname, _formkey}.

The WebClient object does not perform any parsing of the client.text returned by the server but this can easily be accomplished with many third-party modules such as BeautifulSoup. For example here is an example code that finds all links in a page downloaded by the client and checks all of them:

from BeautifulSoup import BeautifulSoup
dom = BeautifulSoup(client.text)
for link in dom.findAll('a'):
    new_client = WebClient()
    new_client.get(a.href)
    print new_client.status