Scaling SSL connections (uWSGI 1.9)

Distributing SSL servers in a cluster is a hard topic. The biggest problem is sharing SSL sessions between different nodes.

The problem is amplified in non-blocking servers due to OpenSSL’s limits in the way sessions are managed.

For example, you cannot share sessions in Memcached servers and access them in a non-blocking way.

A common solution (well, a compromise, maybe) until now has been to use a single SSL terminator balancing requests to multiple non-encrypted backends. This solution kinda works, but obviously it does not scale.

Starting from uWSGI 1.9-dev an implementation (based on the stud project) of distributed caching has been added.

Setup 1: using the uWSGI cache for storing SSL sessions

You can configure the SSL subsystem of uWSGI to use the shared cache. The SSL sessions will time out according to the expiry value of the cache item. This way the cache sweeper thread (managed by the master) will destroy sessions in the cache.

Important

The order of the options is important. cache options must be specified BEFORE ssl-sessions-use-cache and https options.

  1. [uwsgi]
  2. ; spawn the master process (it will run the cache sweeper thread)
  3. master = true
  4. ; store up to 20k sessions
  5. cache = 20000
  6. ; 4k per object is enough for SSL sessions
  7. cache-blocksize = 4096
  8. ; force the SSL subsystem to use the uWSGI cache as session storage
  9. ssl-sessions-use-cache = true
  10. ; set SSL session timeout (in seconds)
  11. ssl-sessions-timeout = 300
  12. ; set the session context string (see later)
  13. https-session-context = foobar
  14. ; spawn an HTTPS router
  15. https = 192.168.173.1:8443,foobar.crt,foobar.key
  16. ; spawn 8 processes for the HTTPS router (all sharing the same session cache)
  17. http-processes = 8
  18. ; add a bunch of uwsgi nodes to relay traffic to
  19. http-to = 192.168.173.10:3031
  20. http-to = 192.168.173.11:3031
  21. http-to = 192.168.173.12:3031
  22. ; add stats
  23. stats = 127.0.0.1:5001

Now start blasting your HTTPS router and then telnet to port 5001. Under the “cache” object of the JSONoutput you should see the values “items” and “hits” increasing. The value “miss” is increased every time a session is not foundin the cache. It is a good metric of the SSL performance users can expect.

Setup 2: synchronize caches of different HTTPS routers

The objective is to synchronize each new session in each distributed cache. To accomplish that you have to spawn a special thread(cache-udp-server) in each instance and list all of the remote servers that should be synchronized.

A pure-TCP load balancer (like HAProxy or uWSGI’s Rawrouter) can be used to load balance between the various HTTPS routers.

Here’s a possible Rawrouter config.

  1. [uwsgi]
  2. master = true
  3. rawrouter = 192.168.173.99:443
  4. rawrouter-to = 192.168.173.1:8443
  5. rawrouter-to = 192.168.173.2:8443
  6. rawrouter-to = 192.168.173.3:8443

Now you can configure the first node (the new options are at the end of the .ini config)

  1. [uwsgi]
  2. ; spawn the master process (it will run the cache sweeper thread)
  3. master = true
  4. ; store up to 20k sessions
  5. cache = 20000
  6. ; 4k per object is enough for SSL sessions
  7. cache-blocksize = 4096
  8. ; force the SSL subsystem to use the uWSGI cache as session storage
  9. ssl-sessions-use-cache = true
  10. ; set SSL session timeout (in seconds)
  11. ssl-sessions-timeout = 300
  12. ; set the session context string (see later)
  13. https-session-context = foobar
  14. ; spawn an HTTPS router
  15. https = 192.168.173.1:8443,foobar.crt,foobar.key
  16. ; spawn 8 processes for the HTTPS router (all sharing the same session cache)
  17. http-processes = 8
  18. ; add a bunch of uwsgi nodes to relay traffic to
  19. http-to = 192.168.173.10:3031
  20. http-to = 192.168.173.11:3031
  21. http-to = 192.168.173.12:3031
  22. ; add stats
  23. stats = 127.0.0.1:5001
  24.  
  25. ; spawn the cache-udp-server
  26. cache-udp-server = 192.168.173.1:7171
  27. ; propagate updates to the other nodes
  28. cache-udp-node = 192.168.173.2:7171
  29. cache-udp-node = 192.168.173.3:7171

and the other two…

  1. [uwsgi]
  2. ; spawn the master process (it will run the cache sweeper thread)
  3. master = true
  4. ; store up to 20k sessions
  5. cache = 20000
  6. ; 4k per object is enough for SSL sessions
  7. cache-blocksize = 4096
  8. ; force the SSL subsystem to use the uWSGI cache as session storage
  9. ssl-sessions-use-cache = true
  10. ; set SSL session timeout (in seconds)
  11. ssl-sessions-timeout = 300
  12. ; set the session context string (see later)
  13. https-session-context = foobar
  14. ; spawn an HTTPS router
  15. https = 192.168.173.1:8443,foobar.crt,foobar.key
  16. ; spawn 8 processes for the HTTPS router (all sharing the same session cache)
  17. http-processes = 8
  18. ; add a bunch of uwsgi nodes to relay traffic to
  19. http-to = 192.168.173.10:3031
  20. http-to = 192.168.173.11:3031
  21. http-to = 192.168.173.12:3031
  22. ; add stats
  23. stats = 127.0.0.1:5001
  24.  
  25. ; spawn the cache-udp-server
  26. cache-udp-server = 192.168.173.2:7171
  27. ; propagate updates to the other nodes
  28. cache-udp-node = 192.168.173.1:7171
  29. cache-udp-node = 192.168.173.3:7171
  1. [uwsgi]
  2. ; spawn the master process (it will run the cache sweeper thread)
  3. master = true
  4. ; store up to 20k sessions
  5. cache = 20000
  6. ; 4k per object is enough for SSL sessions
  7. cache-blocksize = 4096
  8. ; force the SSL subsystem to use the uWSGI cache as session storage
  9. ssl-sessions-use-cache = true
  10. ; set SSL session timeout (in seconds)
  11. ssl-sessions-timeout = 300
  12. ; set the session context string (see later)
  13. https-session-context = foobar
  14. ; spawn an HTTPS router
  15. https = 192.168.173.1:8443,foobar.crt,foobar.key
  16. ; spawn 8 processes for the HTTPS router (all sharing the same session cache)
  17. http-processes = 8
  18. ; add a bunch of uwsgi nodes to relay traffic to
  19. http-to = 192.168.173.10:3031
  20. http-to = 192.168.173.11:3031
  21. http-to = 192.168.173.12:3031
  22. ; add stats
  23. stats = 127.0.0.1:5001
  24.  
  25. ; spawn the cache-udp-server
  26. cache-udp-server = 192.168.173.3:7171
  27. ; propagate updates to the other nodes
  28. cache-udp-node = 192.168.173.1:7171
  29. cache-udp-node = 192.168.173.2:7171

Start hammering the Rawrouter (remember to use a client supporting persistent SSL sessions, like your browser) and get cache statisticsfrom the stats server of each HTTPS terminator node. If the count of “hits” is a lot higher than the “miss” value the system is working welland your load is distributed and in awesome hyper high performance mode.

So, what is https-session-context, you ask? Basically each SSL session before being used is checked against a fixed string (the session context). If the session does not match that string, it is rejected. By default the session context is initialized to a value built from the HTTP server address. Forcing it to a shared value will avoid a session created in a node being rejected in another one.

Using named caches

Starting from uWSGI 1.9 you can have multiple caches. This is a setup with 2 nodes using a new generation cache named “ssl”.

The cache2 option allows also to set a custom key size. Since SSL session keys are not very long, we can use it to optimize memory usage. In this example we use 128 byte key size limit, which should be enough for session IDs.

  1. [uwsgi]
  2. ; spawn the master process (it will run the cache sweeper thread)
  3. master = true
  4. ; store up to 20k sessions
  5. cache2 = name=ssl,items=20000,keysize=128,blocksize=4096,node=127.0.0.1:4242,udp=127.0.0.1:4141
  6. ; force the SSL subsystem to use the uWSGI cache as session storage
  7. ssl-sessions-use-cache = ssl
  8. ; set sessions timeout (in seconds)
  9. ssl-sessions-timeout = 300
  10. ; set the session context string
  11. https-session-context = foobar
  12. ; spawn an HTTPS router
  13. https = :8443,foobar.crt,foobar.key
  14. ; spawn 8 processes for the HTTPS router (all sharing the same session cache)
  15. http-processes = 8
  16. module = werkzeug.testapp:test_app
  17. ; add stats
  18. stats = :5001

and the second node…

  1. [uwsgi]
  2. ; spawn the master process (it will run the cache sweeper thread)
  3. master = true
  4. ; store up to 20k sessions
  5. cache2 = name=ssl,items=20000,blocksize=4096,node=127.0.0.1:4141,udp=127.0.0.1:4242
  6. ; force the SSL subsystem to use the uWSGI cache as session storage
  7. ssl-sessions-use-cache = ssl
  8. ; set session timeout
  9. ssl-sessions-timeout = 300
  10. ; set the session context string
  11. https-session-context = foobar
  12. ; spawn an HTTPS router
  13. https = :8444,foobar.crt,foobar.key
  14. ; spawn 8 processes for the HTTPS router (all sharing the same sessions cache)
  15. http-processes = 8
  16. module = werkzeug.testapp:test_app
  17. ; add stats
  18. stats = :5002

Notes

If you do not want to manually configure the cache UDP nodes and your network configuration supports it, you can use UDP multicast.

  1. [uwsgi]
  2. ...
  3. cache-udp-server = 225.1.1.1:7171
  4. cache-udp-node = 225.1.1.1:7171
  • A new gateway server is in development, named “udprepeater”. It will basically forward all of UDP packets it receives to the subscribed back-end nodes. It will allow you to maintain the zero-config style of the subscription system (basically you only need to configure a single cache UDP node pointing to the repeater).
  • Currently there is no security between the cache nodes. For some users this may be a huge problem, so a security mode (encrypting the packets) is in development.