db.collection.bulkWrite()

Definition

  • db.collection.bulkWrite()

mongo Shell Method

This page documents the mongo shell method, and doesnot refer to the MongoDB Node.js driver (or any other driver)method. For corresponding MongoDB driver API, refer to your specificMongoDB driver documentation instead.

New in version 3.2.

Performs multiple write operations with controls for order ofexecution.

db.collection.bulkWrite() has the following syntax:

  1. db.collection.bulkWrite(
  2.    [ <operation 1>, <operation 2>, ... ],
  3.    {
  4.       writeConcern : <document>,
  5.       ordered : <boolean>
  6.    }
  7. )

ParameterTypeDescriptionoperationsarrayAn array of bulkWrite() writeoperations.

Valid operations are:

See Write Operations for usage of each operation.writeConcerndocumentOptional. A document expressing the write concern. Omit to use the default write concern.

Do not explicitly set the write concern for the operation if run ina transaction. To use write concern with transactions, seeTransactions and Write Concern.orderedbooleanOptional. A boolean specifying whether the mongod instance should performan ordered or unordered operation execution. Defaults to true.

See Execution of Operations

Returns:

  • A boolean acknowledged as true if the operation ran withwrite concern or false if write concern was disabled.
  • A count for each write operation.
  • An array containing an _id for each successfully inserted orupserted documents.

Behavior

bulkWrite() takes an array of write operations andexecutes each of them. By default operations are executed in order.See Execution of Operations for controllingthe order of write operation execution.

Write Operations

insertOne

Inserts a single document into the collection.

  1. db.collection.bulkWrite( [
  2.    { insertOne : { "document" : <document> } }
  3. ] )

See db.collection.insertOne().

updateOne and updateMany

  • updateOne
  • updateMany

updateOne updates a single document in the collection that matches thefilter. If multiple documents match, updateOne will update the _first_matching document only.

  1. db.collection.bulkWrite( [
  2.    { updateOne :
  3.       {
  4.          "filter": <document>,
  5.          "update": <document or pipeline>,            // Changed in 4.2
  6.          "upsert": <boolean>,
  7.          "collation": <document>,                     // Available starting in 3.4
  8.          "arrayFilters": [ <filterdocument1>, ... ],  // Available starting in 3.6
  9.          "hint": <document|string>                    // Available starting in 4.2.1
  10.       }
  11.    }
  12. ] )

updateMany updates all documents in the collectionthat match the filter.

  1. db.collection.bulkWrite( [
  2.    { updateMany :
  3.       {
  4.          "filter" : <document>,
  5.          "update" : <document or pipeline>,          // Changed in MongoDB 4.2
  6.          "upsert" : <boolean>,
  7.          "collation": <document>,                    // Available starting in 3.4
  8.          "arrayFilters": [ <filterdocument1>, ... ], // Available starting in 3.6
  9.          "hint": <document|string>                   // Available starting in 4.2.1
  10.       }
  11.    }
  12. ] )
FieldNotes
filterThe selection criteria for the update. The same queryselectors as in thedb.collection.find() method are available.
updateThe update operation to perform. Can specify either:- A document that only contains update operator expressions.- An aggregation pipeline [<stage1>, <stage2>, … ] that specifies the modifications toperform.
upsertOptional. A boolean to indicate whether to perform an upsert.By default, upsert is false.
arrayFiltersOptional. An array of filter documents that determine whicharray elements to modify for an update operation on an arrayfield.
collationOptional. Specifies the collation to use forthe operation.
hintOptional. The index to use to support theupdate filter. If you specify an index that does not exist,the operation errors.New in version 4.2.1.

For details, see db.collection.updateOne() anddb.collection.updateMany().

replaceOne

replaceOne replaces a single document in the collection that matches thefilter. If multiple documents match, replaceOne will replace the _first_matching document only.

  1. db.collection.bulkWrite([
  2.    { replaceOne :
  3.       {
  4.          "filter" : <document>,
  5.          "replacement" : <document>,
  6.          "upsert" : <boolean>,
  7.          "collation": <document>,                    // Available starting in 3.4
  8.          "hint": <document|string>                   // Available starting in 4.2.1
  9.       }
  10.    }
  11. ] )
FieldNotes
filterThe selection criteria for the replacement operation. The samequery selectors as in thedb.collection.find() method are available.
replacementThe replacement document. The document cannot containupdate operators.
upsertOptional. A boolean to indicate whether to perform an upsert. Bydefault, upsert is false.
collationOptional. Specifies the collation to use forthe operation.
hintOptional. The index to use to support theupdate filter. If you specify an index that does not exist,the operation errors.New in version 4.2.1.

For details, see to db.collection.replaceOne().

deleteOne and deleteMany

  • deleteOne
  • deleteMany

deleteOne deletes a single document in the collection that match thefilter. If multiple documents match, deleteOne will delete the _first_matching document only.

  1. db.collection.bulkWrite([
  2.    { deleteOne : {
  3.       "filter" : <document>,
  4.       "collation" : <document>                   // Available starting in 3.4
  5.    } }
  6. ] )

deleteMany deletes all documents in the collectionthat match the filter.

  1. db.collection.bulkWrite([
  2.    { deleteMany: {
  3.       "filter" : <document>,
  4.       "collation" : <document>                    // Available starting in 3.4
  5.    } }
  6. ] )
FieldNotes
filterThe selection criteria for the delete operation. The samequery selectors as in thedb.collection.find() method are available.
collationOptional. Specifies the collation to use forthe operation.

For details, see db.collection.deleteOne() anddb.collection.deleteMany().

_id Field

If the document does not specify an _id field, then mongodadds the _id field and assign a uniqueObjectId for the document before inserting or upserting it.Most drivers create an ObjectId and insert the _id field, but themongod will create and populate the _id if the driver orapplication does not.

If the document contains an _id field, the _id value must beunique within the collection to avoid duplicate key error.

Update or replace operations cannot specify an _id value that differsfrom the original document.

Execution of Operations

The ordered parameter specifies whetherbulkWrite() will execute operations in order or not.By default, operations are executed in order.

The following code represents a bulkWrite() withfive operations.

  1. db.collection.bulkWrite(
  2.    [
  3.       { insertOne : <document> },
  4.       { updateOne : <document> },
  5.       { updateMany : <document> },
  6.       { replaceOne : <document> },
  7.       { deleteOne : <document> },
  8.       { deleteMany : <document> }
  9.    ]
  10. )

In the default ordered : true state, each operation willbe executed in order, from the first operation insertOneto the last operation deleteMany.

If ordered is set to false, operations may be reordered bymongod to increase performance.Applications should not depend on order of operation execution.

The following code represents an unorderedbulkWrite() with six operations:

  1. db.collection.bulkWrite(
  2.    [
  3.       { insertOne : <document> },
  4.       { updateOne : <document> },
  5.       { updateMany : <document> },
  6.       { replaceOne : <document> },
  7.       { deleteOne : <document> },
  8.       { deleteMany : <document> }
  9.    ],
  10.    { ordered : false }
  11. )

With ordered : false, the results of the operation may vary. For example,the deleteOne or deleteMany may remove more or fewer documentsdepending on whether the run before or after the insertOne, updateOne,updateMany, or replaceOne operations.

The number of operations in each group cannot exceed the value ofthe maxWriteBatchSize ofthe database. As of MongoDB 3.6, this value is 100,000.This value is shown in the isMaster.maxWriteBatchSize field.

This limit prevents issues with oversized error messages. If a groupexceeds this limit,the client driver divides the group into smaller groups with countsless than or equal to the value of the limit. For example, with themaxWriteBatchSize value of 100,000, if the queue consists of200,000 operations, the driver creates 2 groups, each with100,000 operations.

Note

The driver only divides the group into smaller groups when usingthe high-level API. If usingdb.runCommand() directly(for example, when writing a driver), MongoDB throws an error whenattempting to execute a write batch which exceeds the limit.

Starting in MongoDB 3.6, once the error report for a single batch growstoo large, MongoDB truncates all remaining error messages to the emptystring. Currently, begins once there are at least 2 error messages withtotal size greater than 1MB.

The sizes and grouping mechanics are internal performance details andare subject to change in future versions.

Executing an ordered list of operations on asharded collection will generally be slower than executing anunordered listsince with an ordered list, each operation must wait for the previousoperation to finish.

Capped Collections

bulkWrite() write operations have restrictions whenused on a capped collection.

updateOne and updateMany throw a WriteError if theupdate criteria increases the size of the document being modified.

replaceOne throws a WriteError if thereplacement document has a larger size than the originaldocument.

deleteOne and deleteMany throw a WriteError if used on acapped collection.

Transactions

db.collection.bulkWrite() can be used inside multi-document transactions.

If run inside a transaction, the collection must already exist forinsert and upsert: true operations.

Do not explicitly set the write concern for the operation if run ina transaction. To use write concern with transactions, seeTransactions and Write Concern.

Important

In most cases, multi-document transaction incurs a greaterperformance cost over single document writes, and theavailability of multi-document transactions should not be areplacement for effective schema design. For many scenarios, thedenormalized data model (embedded documents and arrays) will continue to be optimal for yourdata and use cases. That is, for many scenarios, modeling your dataappropriately will minimize the need for multi-documenttransactions.

For additional transactions usage considerations(such as runtime limit and oplog size limit), see alsoProduction Considerations.

For error handling inside a transaction, see Error Handling inside Transactions.

Error Handling

db.collection.bulkWrite() throws a BulkWriteErrorexception on errors (unless the operation is part of a transaction onMongoDB 4.0). See Error Handling inside Transactions.

Excluding Write Concern errors, ordered operationsstop after an error, while unordered operations continue to process anyremaining write operations in the queue, unless when run inside atransaction. See Error Handling inside Transactions.

Write concern errors are displayed in the writeConcernErrors field, whileall other errors are displayed in the writeErrors field. If an error isencountered, the number of successful write operations are displayed insteadof the inserted _id values. Ordered operations display the single errorencountered while unordered operations display each error in an array.

Transactions

db.collection.bulkWrite() can be used inside multi-document transactions.

If run inside a transaction, the collection must already exist forinsert and upsert: true operations.

Do not explicitly set the write concern for the operation if run ina transaction. To use write concern with transactions, seeTransactions and Write Concern.

Important

In most cases, multi-document transaction incurs a greaterperformance cost over single document writes, and theavailability of multi-document transactions should not be areplacement for effective schema design. For many scenarios, thedenormalized data model (embedded documents and arrays) will continue to be optimal for yourdata and use cases. That is, for many scenarios, modeling your dataappropriately will minimize the need for multi-documenttransactions.

For additional transactions usage considerations(such as runtime limit and oplog size limit), see alsoProduction Considerations.

Error Handling inside Transactions

Starting in MongoDB 4.2, if a db.collection.bulkWrite()operation encounters an error inside a transaction, the method throws a BulkWriteException (same as outside a transaction).

In 4.0, if the bulkWrite operation encounters an error inside atransaction, the error thrown is not wrapped as aBulkWriteException.

Inside a transaction, the first error in a bulk write causes theentire bulk write to fail and aborts the transaction, even if thebulk write is unordered.

Examples

Bulk Write Operations

The characters collection in the guidebook database contains the following documents:

  1. { "_id" : 1, "char" : "Brisbane", "class" : "monk", "lvl" : 4 },
  2. { "_id" : 2, "char" : "Eldon", "class" : "alchemist", "lvl" : 3 },
  3. { "_id" : 3, "char" : "Meldane", "class" : "ranger", "lvl" : 3 }

The following bulkWrite() performs multipleoperations on the collection:

  1. try {
  2.    db.characters.bulkWrite([
  3.       { insertOne: { "document": { "_id": 4, "char": "Dithras", "class": "barbarian", "lvl": 4 } } },
  4.       { insertOne: { "document": { "_id": 5, "char": "Taeln", "class": "fighter", "lvl": 3 } } },
  5.       { updateOne : {
  6.          "filter" : { "char" : "Eldon" },
  7.          "update" : { $set : { "status" : "Critical Injury" } }
  8.       } },
  9.       { deleteOne : { "filter" : { "char" : "Brisbane"} } },
  10.       { replaceOne : {
  11.          "filter" : { "char" : "Meldane" },
  12.          "replacement" : { "char" : "Tanys", "class" : "oracle", "lvl": 4 }
  13.       } }
  14.    ]);
  15. } catch (e) {
  16.    print(e);
  17. }

The operation returns the following:

  1. {
  2.    "acknowledged" : true,
  3.    "deletedCount" : 1,
  4.    "insertedCount" : 2,
  5.    "matchedCount" : 2,
  6.    "upsertedCount" : 0,
  7.    "insertedIds" : {
  8.       "0" : 4,
  9.       "1" : 5
  10.    },
  11.    "upsertedIds" : {
  12.  
  13.    }
  14. }

If the collection had contained a document with "_id" : 5"before executing the bulk write, then when the bulk write is executed,the following duplicate key exception would be thrown for the second insertOne:

  1. BulkWriteError({
  2.    "writeErrors" : [
  3.       {
  4.          "index" : 1,
  5.          "code" : 11000,
  6.          "errmsg" : "E11000 duplicate key error collection: guidebook.characters index: _id_ dup key: { _id: 5.0 }",
  7.          "op" : {
  8.             "_id" : 5,
  9.             "char" : "Taeln",
  10.             "class" : "fighter",
  11.             "lvl" : 3
  12.          }
  13.       }
  14.    ],
  15.    "writeConcernErrors" : [ ],
  16.    "nInserted" : 1,
  17.    "nUpserted" : 0,
  18.    "nMatched" : 0,
  19.    "nModified" : 0,
  20.    "nRemoved" : 0,
  21.    "upserted" : [ ]
  22. })

Since ordered is true by default, only the first operation completessuccessfully. The rest are not executed. Running thebulkWrite() with ordered : false would allow theremaining operations to complete despite the error.

Unordered Bulk Write

The characters collection in the guidebook database contains the following documents:

  1. { "_id" : 1, "char" : "Brisbane", "class" : "monk", "lvl" : 4 },
  2. { "_id" : 2, "char" : "Eldon", "class" : "alchemist", "lvl" : 3 },
  3. { "_id" : 3, "char" : "Meldane", "class" : "ranger", "lvl" : 3 }

The following bulkWrite() performs multipleunordered operations on the characters collection. Note that one ofthe insertOne stages has a duplicate _id value:

  1. try {
  2.    db.characters.bulkWrite([
  3.       { insertOne: { "document": { "_id": 4, "char": "Dithras", "class": "barbarian", "lvl": 4 } } },
  4.       { insertOne: { "document": { "_id": 4, "char": "Taeln", "class": "fighter", "lvl": 3 } } },
  5.       { updateOne : {
  6.          "filter" : { "char" : "Eldon" },
  7.          "update" : { $set : { "status" : "Critical Injury" } }
  8.       } },
  9.       { deleteOne : { "filter" : { "char" : "Brisbane"} } },
  10.       { replaceOne : {
  11.          "filter" : { "char" : "Meldane" },
  12.          "replacement" : { "char" : "Tanys", "class" : "oracle", "lvl": 4 }
  13.       } }
  14.    ], { ordered : false } );
  15. } catch (e) {
  16.    print(e);
  17. }

The operation returns the following:

  1. BulkWriteError({
  2.    "writeErrors" : [
  3.       {
  4.          "index" : 1,
  5.          "code" : 11000,
  6.          "errmsg" : "E11000 duplicate key error collection: guidebook.characters index: _id_ dup key: { _id: 4.0 }",
  7.          "op" : {
  8.             "_id" : 4,
  9.             "char" : "Taeln",
  10.             "class" : "fighter",
  11.             "lvl" : 3
  12.          }
  13.       }
  14.    ],
  15.    "writeConcernErrors" : [ ],
  16.    "nInserted" : 1,
  17.    "nUpserted" : 0,
  18.    "nMatched" : 2,
  19.    "nModified" : 2,
  20.    "nRemoved" : 1,
  21.    "upserted" : [ ]
  22. })

Since this was an unordered operation, the writes remaining in the queuewere processed despite the exception.

Bulk Write with Write Concern

The enemies collection contains the following documents:

  1. { "_id" : 1, "char" : "goblin", "rating" : 1, "encounter" : 0.24 },
  2. { "_id" : 2, "char" : "hobgoblin", "rating" : 1.5, "encounter" : 0.30 },
  3. { "_id" : 3, "char" : "ogre", "rating" : 3, "encounter" : 0.2 },
  4. { "_id" : 4, "char" : "ogre berserker" , "rating" : 3.5, "encounter" : 0.12}

The following bulkWrite() performs multipleoperations on the collection using a write concern value of"majority" and timeout value of 100 milliseconds:

  1. try {
  2.    db.enemies.bulkWrite(
  3.       [
  4.          { updateMany :
  5.             {
  6.                "filter" : { "rating" : { $gte : 3} },
  7.                "update" : { $inc : { "encounter" : 0.1 } }
  8.             },
  9.  
  10.          },
  11.          { updateMany :
  12.             {
  13.                "filter" : { "rating" : { $lt : 2} },
  14.                "update" : { $inc : { "encounter" : -0.25 } }
  15.             },
  16.          },
  17.          { deleteMany : { "filter" : { "encounter": { $lt : 0 } } } },
  18.          { insertOne :
  19.             {
  20.                "document" :
  21.                   {
  22.                      "_id" :5, "char" : "ogrekin" , "rating" : 2, "encounter" : 0.31
  23.                   }
  24.             }
  25.          }
  26.       ],
  27.       { writeConcern : { w : "majority", wtimeout : 100 } }
  28.    );
  29. } catch (e) {
  30.    print(e);
  31. }

If the total time required for all required nodes in the replica set toacknowledge the write operation is greater than wtimeout,the following writeConcernError is displayed when the wtimeout periodhas passed.

  1. BulkWriteError({
  2.    "writeErrors" : [ ],
  3.    "writeConcernErrors" : [
  4.       {
  5.          "code" : 64,
  6.          "codeName" : "WriteConcernFailed",
  7.          "errmsg" : "waiting for replication timed out",
  8.          "errInfo" : {
  9.             "wtimeout" : true
  10.          }
  11.       },
  12.       {
  13.          "code" : 64,
  14.          "codeName" : "WriteConcernFailed",
  15.          "errmsg" : "waiting for replication timed out",
  16.          "errInfo" : {
  17.             "wtimeout" : true
  18.          }
  19.       },
  20.       {
  21.          "code" : 64,
  22.          "codeName" : "WriteConcernFailed",
  23.          "errmsg" : "waiting for replication timed out",
  24.          "errInfo" : {
  25.             "wtimeout" : true
  26.          }
  27.       }
  28.    ],
  29.    "nInserted" : 1,
  30.    "nUpserted" : 0,
  31.    "nMatched" : 4,
  32.    "nModified" : 4,
  33.    "nRemoved" : 1,
  34.    "upserted" : [ ]
  35. })

The result set shows the operations executed sincewriteConcernErrors errors are not an indicator that any writeoperations failed.