Contextual/Thread-local Sessions

Recall from the section When do I construct a Session, when do I commit it, and when do I close it?, the concept of“session scopes” was introduced, with an emphasis on web applicationsand the practice of linking the scope of a Session with thatof a web request. Most modern web frameworks include integration toolsso that the scope of the Session can be managed automatically,and these tools should be used as they are available.

SQLAlchemy includes its own helper object, which helps with the establishmentof user-defined Session scopes. It is also used by third-partyintegration systems to help construct their integration schemes.

The object is the scoped_session object, and it represents aregistry of Session objects. If you’re not familiar with theregistry pattern, a good introduction can be found in Patterns of EnterpriseArchitecture.

Note

The scoped_session object is a very popular and useful objectused by many SQLAlchemy applications. However, it is important to notethat it presents only one approach to the issue of Sessionmanagement. If you’re new to SQLAlchemy, and especially if theterm “thread-local variable” seems strange to you, we recommend thatif possible you familiarize first with an off-the-shelf integrationsystem such as Flask-SQLAlchemyor zope.sqlalchemy.

A scoped_session is constructed by calling it, passing it afactory which can create new Session objects. A factoryis just something that produces a new object when called, and in thecase of Session, the most common factory is the sessionmaker,introduced earlier in this section. Below we illustrate this usage:

  1. >>> from sqlalchemy.orm import scoped_session
  2. >>> from sqlalchemy.orm import sessionmaker
  3.  
  4. >>> session_factory = sessionmaker(bind=some_engine)
  5. >>> Session = scoped_session(session_factory)

The scoped_session object we’ve created will now call upon thesessionmaker when we “call” the registry:

  1. >>> some_session = Session()

Above, some_session is an instance of Session, which wecan now use to talk to the database. This same Session is alsopresent within the scoped_session registry we’ve created. Ifwe call upon the registry a second time, we get back the sameSession:

  1. >>> some_other_session = Session()
  2. >>> some_session is some_other_session
  3. True

This pattern allows disparate sections of the application to call upon a globalscoped_session, so that all those areas may share the same sessionwithout the need to pass it explicitly. The Session we’ve establishedin our registry will remain, until we explicitly tell our registry to dispose of it,by calling scoped_session.remove():

  1. >>> Session.remove()

The scoped_session.remove() method first calls Session.close() onthe current Session, which has the effect of releasing any connection/transactionalresources owned by the Session first, then discarding the Sessionitself. “Releasing” here means that connections are returned to their connection pool and any transactional state is rolled back, ultimately using the rollback() method of the underlying DBAPI connection.

At this point, the scoped_session object is “empty”, and will createa newSession when called again. As illustrated below, thisis not the same Session we had before:

  1. >>> new_session = Session()
  2. >>> new_session is some_session
  3. False

The above series of steps illustrates the idea of the “registry” pattern in anutshell. With that basic idea in hand, we can discuss some of the detailsof how this pattern proceeds.

Implicit Method Access

The job of the scoped_session is simple; hold onto a Sessionfor all who ask for it. As a means of producing more transparent access to thisSession, the scoped_session also includes proxy behavior,meaning that the registry itself can be treated just like a Sessiondirectly; when methods are called on this object, they are proxied to theunderlying Session being maintained by the registry:

  1. Session = scoped_session(some_factory)
  2.  
  3. # equivalent to:
  4. #
  5. # session = Session()
  6. # print(session.query(MyClass).all())
  7. #
  8. print(Session.query(MyClass).all())

The above code accomplishes the same task as that of acquiring the currentSession by calling upon the registry, then using that Session.

Thread-Local Scope

Users who are familiar with multithreaded programming will note that representinganything as a global variable is usually a bad idea, as it implies that theglobal object will be accessed by many threads concurrently. The Sessionobject is entirely designed to be used in a non-concurrent fashion, whichin terms of multithreading means “only in one thread at a time”. So ourabove example of scoped_session usage, where the same Sessionobject is maintained across multiple calls, suggests that some process needsto be in place such that multiple calls across many threads don’t actually geta handle to the same session. We call this notion thread local storage,which means, a special object is used that will maintain a distinct objectper each application thread. Python provides this via thethreading.local()construct. The scoped_session object by default uses this objectas storage, so that a single Session is maintained for all who callupon the scoped_session registry, but only within the scope of a singlethread. Callers who call upon the registry in a different thread get aSession instance that is local to that other thread.

Using this technique, the scoped_session provides a quick and relativelysimple (if one is familiar with thread-local storage) way of providinga single, global object in an application that is safe to be called uponfrom multiple threads.

The scoped_session.remove() method, as always, removes the currentSession associated with the thread, if any. However, one advantage of thethreading.local() object is that if the application thread itself ends, the“storage” for that thread is also garbage collected. So it is in fact “safe” touse thread local scope with an application that spawns and tears down threads,without the need to call scoped_session.remove(). However, the scopeof transactions themselves, i.e. ending them via Session.commit() orSession.rollback(), will usually still be something that must be explicitlyarranged for at the appropriate time, unless the application actually ties thelifespan of a thread to the lifespan of a transaction.

Using Thread-Local Scope with Web Applications

As discussed in the section When do I construct a Session, when do I commit it, and when do I close it?, a web applicationis architected around the concept of a web request, and integratingsuch an application with the Session usually implies that the Sessionwill be associated with that request. As it turns out, most Python web frameworks,with notable exceptions such as the asynchronous frameworks Twisted andTornado, use threads in a simple way, such that a particular web request is received,processed, and completed within the scope of a single worker thread. Whenthe request ends, the worker thread is released to a pool of workers where itis available to handle another request.

This simple correspondence of web request and thread means that to associate aSession with a thread implies it is also associated with the web requestrunning within that thread, and vice versa, provided that the Session iscreated only after the web request begins and torn down just before the web request ends.So it is a common practice to use scoped_session as a quick wayto integrate the Session with a web application. The sequencediagram below illustrates this flow:

  1. Web Server          Web Framework        SQLAlchemy ORM Code
  2. --------------      --------------       ------------------------------
  3. startup        ->   Web framework        # Session registry is established
  4.                     initializes          Session = scoped_session(sessionmaker())
  5.  
  6. incoming
  7. web request    ->   web request     ->   # The registry is *optionally*
  8.                     starts               # called upon explicitly to create
  9.                                          # a Session local to the thread and/or request
  10.                                          Session()
  11.  
  12.                                          # the Session registry can otherwise
  13.                                          # be used at any time, creating the
  14.                                          # request-local Session() if not present,
  15.                                          # or returning the existing one
  16.                                          Session.query(MyClass) # ...
  17.  
  18.                                          Session.add(some_object) # ...
  19.  
  20.                                          # if data was modified, commit the
  21.                                          # transaction
  22.                                          Session.commit()
  23.  
  24.                     web request ends  -> # the registry is instructed to
  25.                                          # remove the Session
  26.                                          Session.remove()
  27.  
  28.                     sends output      <-
  29. outgoing web    <-
  30. response

Using the above flow, the process of integrating the Session with theweb application has exactly two requirements:

  • Create a single scoped_session registry when the web applicationfirst starts, ensuring that this object is accessible by the rest of theapplication.

  • Ensure that scoped_session.remove() is called when the web request ends,usually by integrating with the web framework’s event system to establishan “on request end” event.

As noted earlier, the above pattern is just one potential way to integrate a Sessionwith a web framework, one which in particular makes the significant assumptionthat the web framework associates web requests with application threads. It ishowever strongly recommended that the integration tools provided with the web frameworkitself be used, if available, instead of scoped_session.

In particular, while using a thread local can be convenient, it is preferable that the Session beassociated directly with the request, rather than withthe current thread. The next section on custom scopes details a more advanced configurationwhich can combine the usage of scoped_session with direct request based scope, orany kind of scope.

Using Custom Created Scopes

The scoped_session object’s default behavior of “thread local” scope is onlyone of many options on how to “scope” a Session. A custom scope can be definedbased on any existing system of getting at “the current thing we are working with”.

Suppose a web framework defines a library function get_current_request(). An applicationbuilt using this framework can call this function at any time, and the result will besome kind of Request object that represents the current request being processed.If the Request object is hashable, then this function can be easily integrated withscoped_session to associate the Session with the request. Below we illustratethis in conjunction with a hypothetical event marker provided by the web frameworkon_request_end, which allows code to be invoked whenever a request ends:

  1. from my_web_framework import get_current_request, on_request_end
  2. from sqlalchemy.orm import scoped_session, sessionmaker
  3.  
  4. Session = scoped_session(sessionmaker(bind=some_engine), scopefunc=get_current_request)
  5.  
  6. @on_request_end
  7. def remove_session(req):
  8.     Session.remove()

Above, we instantiate scoped_session in the usual way, except that we passour request-returning function as the “scopefunc”. This instructs scoped_sessionto use this function to generate a dictionary key whenever the registry is called uponto return the current Session. In this case it is particularly importantthat we ensure a reliable “remove” system is implemented, as this dictionary is nototherwise self-managed.

Contextual Session API

  • class sqlalchemy.orm.scoping.scopedsession(_session_factory, scopefunc=None)
  • Provides scoped management of Session objects.

See Contextual/Thread-local Sessions for a tutorial.

  • call(**kw)

  • Return the current Session, creating itusing the scoped_session.session_factory if not present.

  • init(session_factory, scopefunc=None)

  • Construct a new scoped_session.

    • Parameters

      • session_factory – a factory to create new Sessioninstances. This is usually, but not necessarily, an instanceof sessionmaker.

      • scopefunc – optional function which definesthe current scope. If not passed, the scoped_sessionobject assumes “thread-local” scope, and will usea Python threading.local() in order to maintain the currentSession. If passed, the function should returna hashable token; this token will be used as the key in adictionary in order to store and retrieve the currentSession.

  • configure(**kwargs)

  • reconfigure the sessionmaker used by thisscoped_session.

See sessionmaker.configure().

  • queryproperty(_query_cls=None)
  • return a class property which produces a Query objectagainst the class and the current Session when called.

e.g.:

  1. Session = scoped_session(sessionmaker())
  2.  
  3. class MyClass(object):
  4.     query = Session.query_property()
  5.  
  6. # after mappers are defined
  7. result = MyClass.query.filter(MyClass.name=='foo').all()

Produces instances of the session’s configured query class bydefault. To override and use a custom implementation, providea query_cls callable. The callable will be invoked withthe class’s mapper as a positional argument and a sessionkeyword argument.

There is no limit to the number of query properties placed ona class.

  • remove()
  • Dispose of the current Session, if present.

This will first call Session.close() methodon the current Session, which releases any existingtransactional/connection resources still being held; transactionsspecifically are rolled back. The Session is thendiscarded. Upon next usage within the same scope,the scoped_session will produce a newSession object.

  • sessionfactory = None_
  • The session_factory provided to init is stored in thisattribute and may be accessed at a later time. This can be useful whena new non-scoped Session or Connection to thedatabase is needed.
  • class sqlalchemy.util.ScopedRegistry(createfunc, scopefunc)
  • A Registry that can store one or multiple instances of a singleclass on the basis of a “scope” function.

The object implements call as the “getter”, so bycalling myregistry() the contained object is returnedfor the current scope.

  • Parameters

    • createfunc – a callable that returns a new object to be placed in the registry

    • scopefunc – a callable that will return a key to store/retrieve an object.

  • init(createfunc, scopefunc)

  • Construct a new ScopedRegistry.

    • Parameters

      • createfunc – A creation function that will generatea new value for the current scope, if none is present.

      • scopefunc – A function that returns a hashabletoken representing the current scope (such as, currentthread identifier).

  • clear()

  • Clear the current scope, if any.

  • has()

  • Return True if an object is present in the current scope.

  • set(obj)

  • Set the value for the current scope.
  • class sqlalchemy.util.ThreadLocalRegistry(createfunc)
  • Bases: sqlalchemy.util._collections.ScopedRegistry

A ScopedRegistry that uses a threading.local()variable for storage.