PyMongo 3 Migration Guide

PyMongo 3 is a partial rewrite bringing a large number of improvements. Italso brings a number of backward breaking changes. This guide provides aroadmap for migrating an existing application from PyMongo 2.x to 3.x orwriting libraries that will work with both PyMongo 2.x and 3.x.

PyMongo 2.9

The first step in any successful migration involves upgrading to, orrequiring, at least PyMongo 2.9. If your project has arequirements.txt file, add the line “pymongo >= 2.9, < 3.0” until you havecompletely migrated to PyMongo 3. Most of the key newmethods and options from PyMongo 3.0 are backported in PyMongo 2.9 makingmigration much easier.

Enable Deprecation Warnings

Starting with PyMongo 2.9, DeprecationWarning is raised by most methodsremoved in PyMongo 3.0. Make sure you enable runtime warnings to seewhere deprecated functions and methods are being used in your application:

  1. python -Wd <your application>

Warnings can also be changed to errors:

  1. python -Wd -Werror <your application>

Note

Not all deprecated features raise DeprecationWarning whenused. For example, the find() optionsrenamed in PyMongo 3.0 do not raise DeprecationWarning when used inPyMongo 2.x. See also Removed features with no migration path.

CRUD API

Changes to find() and find_one()

“spec” renamed “filter”

The spec option has been renamed to filter. Code like this:

  1. >>> cursor = collection.find(spec={"a": 1})

can be changed to this with PyMongo 2.9 or later:

  1. >>> cursor = collection.find(filter={"a": 1})

or this with any version of PyMongo:

  1. >>> cursor = collection.find({"a": 1})

“fields” renamed “projection”

The fields option has been renamed to projection. Code like this:

  1. >>> cursor = collection.find({"a": 1}, fields={"_id": False})

can be changed to this with PyMongo 2.9 or later:

  1. >>> cursor = collection.find({"a": 1}, projection={"_id": False})

or this with any version of PyMongo:

  1. >>> cursor = collection.find({"a": 1}, {"_id": False})

“partial” renamed “allow_partial_results”

The partial option has been renamed to allow_partial_results. Code likethis:

  1. >>> cursor = collection.find({"a": 1}, partial=True)

can be changed to this with PyMongo 2.9 or later:

  1. >>> cursor = collection.find({"a": 1}, allow_partial_results=True)

“timeout” replaced by “no_cursor_timeout”

The timeout option has been replaced by no_cursor_timeout. Code like this:

  1. >>> cursor = collection.find({"a": 1}, timeout=False)

can be changed to this with PyMongo 2.9 or later:

  1. >>> cursor = collection.find({"a": 1}, no_cursor_timeout=True)

“network_timeout” is removed

The network_timeout option has been removed. This option was always thewrong solution for timing out long running queries and should never be usedin production. Starting with MongoDB 2.6 you can use the $maxTimeMS querymodifier. Code like this:

  1. # Set a 5 second select() timeout.
  2. >>> cursor = collection.find({"a": 1}, network_timeout=5)

can be changed to this with PyMongo 2.9 or later:

  1. # Set a 5 second (5000 millisecond) server side query timeout.
  2. >>> cursor = collection.find({"a": 1}, modifiers={"$maxTimeMS": 5000})

or with PyMongo 3.5 or later:

  1. >>> cursor = collection.find({"a": 1}, max_time_ms=5000)

or with any version of PyMongo:

  1. >>> cursor = collection.find({"$query": {"a": 1}, "$maxTimeMS": 5000})

See also

$maxTimeMS

Tailable cursors

The tailable and await_data options have been replaced by cursor_type.Code like this:

  1. >>> cursor = collection.find({"a": 1}, tailable=True)
  2. >>> cursor = collection.find({"a": 1}, tailable=True, await_data=True)

can be changed to this with PyMongo 2.9 or later:

  1. >>> from pymongo import CursorType
  2. >>> cursor = collection.find({"a": 1}, cursor_type=CursorType.TAILABLE)
  3. >>> cursor = collection.find({"a": 1}, cursor_type=CursorType.TAILABLE_AWAIT)

Other removed options

The slave_okay, read_preference, tag_sets,and secondary_acceptable_latency_ms options have been removed. See the ReadPreferences section for solutions.

The aggregate method always returns a cursor

PyMongo 2.6 added an option to return an iterable cursor fromaggregate(). In PyMongo 3aggregate() always returns a cursor. Usethe cursor option for consistent behavior with PyMongo 2.9 and later:

  1. >>> for result in collection.aggregate([], cursor={}):
  2. ...     pass

Read Preferences

The “slave_okay” option is removed

The slave_okay option is removed from PyMongo’s API. ThesecondaryPreferred read preference provides the same behavior.Code like this:

  1. >>> client = MongoClient(slave_okay=True)

can be changed to this with PyMongo 2.9 or newer:

  1. >>> client = MongoClient(readPreference="secondaryPreferred")

The “read_preference” attribute is immutable

Code like this:

  1. >>> from pymongo import ReadPreference
  2. >>> db = client.my_database
  3. >>> db.read_preference = ReadPreference.SECONDARY

can be changed to this with PyMongo 2.9 or later:

  1. >>> db = client.get_database("my_database",
  2. ...                          read_preference=ReadPreference.SECONDARY)

Code like this:

  1. >>> cursor = collection.find({"a": 1},
  2. ...                          read_preference=ReadPreference.SECONDARY)

can be changed to this with PyMongo 2.9 or later:

  1. >>> coll2 = collection.with_options(read_preference=ReadPreference.SECONDARY)
  2. >>> cursor = coll2.find({"a": 1})

See also

get_collection()

The “tag_sets” option and attribute are removed

The tag_sets MongoClient option is removed. The _read_preference_option can be used instead. Code like this:

  1. >>> client = MongoClient(
  2. ...     read_preference=ReadPreference.SECONDARY,
  3. ...     tag_sets=[{"dc": "ny"}, {"dc": "sf"}])

can be changed to this with PyMongo 2.9 or later:

  1. >>> from pymongo.read_preferences import Secondary
  2. >>> client = MongoClient(read_preference=Secondary([{"dc": "ny"}]))

To change the tags sets for a Database or Collection, code like this:

  1. >>> db = client.my_database
  2. >>> db.read_preference = ReadPreference.SECONDARY
  3. >>> db.tag_sets = [{"dc": "ny"}]

can be changed to this with PyMongo 2.9 or later:

  1. >>> db = client.get_database("my_database",
  2. ...                          read_preference=Secondary([{"dc": "ny"}]))

Code like this:

  1. >>> cursor = collection.find(
  2. ...     {"a": 1},
  3. ...     read_preference=ReadPreference.SECONDARY,
  4. ...     tag_sets=[{"dc": "ny"}])

can be changed to this with PyMongo 2.9 or later:

  1. >>> from pymongo.read_preferences import Secondary
  2. >>> coll2 = collection.with_options(
  3. ...     read_preference=Secondary([{"dc": "ny"}]))
  4. >>> cursor = coll2.find({"a": 1})

See also

get_collection()

The “secondary_acceptable_latency_ms” option and attribute are removed

PyMongo 2.x supports secondary_acceptable_latency_ms as an option to methodsthroughout the driver, but mongos only supports a global latency option.PyMongo 3.x has changed to match the behavior of mongos, allowing migrationfrom a single server, to a replica set, to a sharded cluster without asurprising change in server selection behavior. A new option,localThresholdMS, is available through MongoClient and should be used inplace of secondaryAcceptableLatencyMS. Code like this:

  1. >>> client = MongoClient(readPreference="nearest",
  2. ...                      secondaryAcceptableLatencyMS=100)

can be changed to this with PyMongo 2.9 or later:

  1. >>> client = MongoClient(readPreference="nearest",
  2. ...                      localThresholdMS=100)

Write Concern

The “safe” option is removed

In PyMongo 3 the safe option is removed from the entire API.MongoClient has always defaulted to acknowledgedwrite operations and continues to do so in PyMongo 3.

The “write_concern” attribute is immutable

The write_concern attribute is immutable in PyMongo 3. Code like this:

  1. >>> client = MongoClient()
  2. >>> client.write_concern = {"w": "majority"}

can be changed to this with any version of PyMongo:

  1. >>> client = MongoClient(w="majority")

Code like this:

  1. >>> db = client.my_database
  2. >>> db.write_concern = {"w": "majority"}

can be changed to this with PyMongo 2.9 or later:

  1. >>> from pymongo import WriteConcern
  2. >>> db = client.get_database("my_database",
  3. ...                          write_concern=WriteConcern(w="majority"))

The new CRUD API write methods do not accept write concern options. Code likethis:

  1. >>> oid = collection.insert({"a": 2}, w="majority")

can be changed to this with PyMongo 2.9 or later:

  1. >>> from pymongo import WriteConcern
  2. >>> coll2 = collection.with_options(
  3. ...     write_concern=WriteConcern(w="majority"))
  4. >>> oid = coll2.insert({"a": 2})

See also

get_collection()

Codec Options

The “document_class” attribute is removed

Code like this:

  1. >>> from bson.son import SON
  2. >>> client = MongoClient()
  3. >>> client.document_class = SON

can be replaced by this in any version of PyMongo:

  1. >>> from bson.son import SON
  2. >>> client = MongoClient(document_class=SON)

or to change the document_class for a Databasewith PyMongo 2.9 or later:

  1. >>> from bson.codec_options import CodecOptions
  2. >>> from bson.son import SON
  3. >>> db = client.get_database("my_database", CodecOptions(SON))

See also

get_collection() andwith_options()

The “uuid_subtype” option and attribute are removed

Code like this:

  1. >>> from bson.binary import JAVA_LEGACY
  2. >>> db = client.my_database
  3. >>> db.uuid_subtype = JAVA_LEGACY

can be replaced by this with PyMongo 2.9 or later:

  1. >>> from bson.binary import JAVA_LEGACY
  2. >>> from bson.codec_options import CodecOptions
  3. >>> db = client.get_database("my_database",
  4. ...                          CodecOptions(uuid_representation=JAVA_LEGACY))

See also

get_collection() andwith_options()

MongoClient

MongoClient connects asynchronously

In PyMongo 3, the MongoClient constructor nolonger blocks while connecting to the server or servers, and it no longerraises ConnectionFailure if they are unavailable, norConfigurationError if the user’s credentials are wrong.Instead, the constructor returns immediately and launches the connectionprocess on background threads. The connect option is added to control whetherthese threads are started immediately, or when the client is first used.

For consistent behavior in PyMongo 2.x and PyMongo 3.x, code like this:

  1. >>> from pymongo.errors import ConnectionFailure
  2. >>> try:
  3. ...     client = MongoClient()
  4. ... except ConnectionFailure:
  5. ...     print("Server not available")
  6. >>>

can be changed to this with PyMongo 2.9 or later:

  1. >>> from pymongo.errors import ConnectionFailure
  2. >>> client = MongoClient(connect=False)
  3. >>> try:
  4. ...     result = client.admin.command("ismaster")
  5. ... except ConnectionFailure:
  6. ...     print("Server not available")
  7. >>>

Any operation can be used to determine if the server is available. We choosethe “ismaster” command here because it is cheap and does not require auth, soit is a simple way to check whether the server is available.

The max_pool_size parameter is removed

PyMongo 3 replaced the maxpool_size parameter with support for the MongoDB URI_maxPoolSize option. Code like this:

  1. >>> client = MongoClient(max_pool_size=10)

can be replaced by this with PyMongo 2.9 or later:

  1. >>> client = MongoClient(maxPoolSize=10)
  2. >>> client = MongoClient("mongodb://localhost:27017/?maxPoolSize=10")

The “disconnect” method is removed

Code like this:

  1. >>> client.disconnect()

can be replaced by this with PyMongo 2.9 or later:

  1. >>> client.close()

The host and port attributes are removed

Code like this:

  1. >>> host = client.host
  2. >>> port = client.port

can be replaced by this with PyMongo 2.9 or later:

  1. >>> address = client.address
  2. >>> host, port = address or (None, None)

BSON

“as_class”, “tz_aware”, and “uuid_subtype” are removed

The as_class, tz_aware, and uuid_subtype parameters have beenremoved from the functions provided in bson. Code like this:

  1. >>> from bson import BSON
  2. >>> from bson.son import SON
  3. >>> encoded = BSON.encode({"a": 1}, as_class=SON)

can be replaced by this in PyMongo 2.9 or later:

  1. >>> from bson import BSON
  2. >>> from bson.codec_options import CodecOptions
  3. >>> from bson.son import SON
  4. >>> encoded = BSON.encode({"a": 1}, codec_options=CodecOptions(SON))

Removed features with no migration path

MasterSlaveConnection is removed

Master slave deployments are deprecated in MongoDB. Starting with MongoDB 3.0a replica set can have up to 50 members and that limit is likely to beremoved in later releases. We recommend migrating to replica sets instead.

Requests are removed

The client methods start_request, in_request, and end_request areremoved. Requests were designed to make read-your-writes consistency morelikely with the w=0 write concern. Additionally, a thread in a request used thesame member for all secondary reads in a replica set. To ensureread-your-writes consistency in PyMongo 3.0, do not override the default writeconcern with w=0, and do not override the default read preference of PRIMARY.

The “compile_re” option is removed

In PyMongo 3 regular expressions are never compiled to Python match objects.

The “use_greenlets” option is removed

The use_greenlets option was meant to allow use of PyMongo with Geventwithout the use of gevent.monkey.patch_threads(). This option caused a lotof confusion and made it difficult to support alternative asyncio librarieslike Eventlet. Users of Gevent should use gevent.monkey.patch_all() instead.

See also

Gevent