8.1. Query Server Protocol

A Query Server is an external process that communicates with CouchDB via asimple, custom JSON protocol over stdin/stdout. It is used to processes alldesign functions calls: views, shows, lists, filters, updates andvalidate_doc_update.

CouchDB communicates with the Query Server process through stdin/stdout withJSON messages that are terminated by a newline character. Messages that aresent to the Query Server are always array-typed and follow the pattern[<command>, <*arguments>]\n.

Note

In the documentation examples, we omit the trailing \n for greaterreadability. Also, examples contain formatted JSON values while real datais transferred in compact mode without formatting spaces.

8.1.1. reset

Command:
reset
Arguments:
——-
Query server state (optional)
Returns:
——-
true

This resets the state of the Query Server and makes it forget all previousinput. If applicable, this is the point to run garbage collection.

CouchDB sends:

  1. ["reset"]

The Query Server answers:

  1. true

To set up new Query Server state, the second argument is used with object data.

CouchDB sends:

  1. ["reset", {"reduce_limit": true, "timeout": 5000}]

The Query Server answers:

  1. true

8.1.2. add_lib

Command:
add_lib
Arguments:
——-
CommonJS library object by views/lib path
Returns:
——-
true

Adds CommonJS library to Query Server state for further usagein map functions.

CouchDB sends:

  1. [
  2. "add_lib",
  3. {
  4. "utils": "exports.MAGIC = 42;"
  5. }
  6. ]

The Query Server answers:

  1. true

Note

This library shouldn’t have any side effects nor track its own stateor you’ll have a lot of happy debugging time if something goes wrong.Remember that a complete index rebuild is a heavy operation and this isthe only way to fix mistakes with shared state.

8.1.3. add_fun

Command:
add_fun
Arguments:
——-
Map function source code.
Returns:
——-
true

When creating or updating a view, this is how the Query Server is sent theview function for evaluation. The Query Server should parse, compile, andevaluate the function it receives to make it callable later. If this fails, theQuery Server returns an error. CouchDB may store multiple functions beforesending any documents.

CouchDB sends:

  1. [
  2. "add_fun",
  3. "function(doc) { if(doc.score > 50) emit(null, {'player_name': doc.name}); }"
  4. ]

The Query Server answers:

  1. true

8.1.4. map_doc

Command:
map_doc
Arguments:
——-
Document object
Returns:
——-
Array of key-value pairs per applied function

When the view function is stored in the Query Server, CouchDB starts sendingall the documents in the database, one at a time. The Query Server calls thepreviously stored functions one after another with a document and stores itsresult. When all functions have been called, the result is returned as a JSONstring.

CouchDB sends:

  1. [
  2. "map_doc",
  3. {
  4. "_id": "8877AFF9789988EE",
  5. "_rev": "3-235256484",
  6. "name": "John Smith",
  7. "score": 60
  8. }
  9. ]

If the function above is the only function stored, the Query Server answers:

  1. [
  2. [
  3. [null, {"player_name": "John Smith"}]
  4. ]
  5. ]

That is, an array with the result for every function for the given document.

If a document is to be excluded from the view, the array should be empty.

CouchDB sends:

  1. [
  2. "map_doc",
  3. {
  4. "_id": "9590AEB4585637FE",
  5. "_rev": "1-674684684",
  6. "name": "Jane Parker",
  7. "score": 43
  8. }
  9. ]

The Query Server answers:

  1. [[]]

8.1.5. reduce

Command:

reduce

Arguments:
  • Reduce function source
  • Array of map function results where each item representedin format [[key, id-of-doc], value]
    |Returns:
    |——-
    |
    Array with pair values: true and another array with reduced result

If the view has a reduce function defined, CouchDB will enter into the reducephase. The Query Server will receive a list of reduce functions and some mapresults on which it can apply them.

CouchDB sends:

  1. [
  2. "reduce",
  3. [
  4. "function(k, v) { return sum(v); }"
  5. ],
  6. [
  7. [[1, "699b524273605d5d3e9d4fd0ff2cb272"], 10],
  8. [[2, "c081d0f69c13d2ce2050d684c7ba2843"], 20],
  9. [[null, "foobar"], 3]
  10. ]
  11. ]

The Query Server answers:

  1. [
  2. true,
  3. [33]
  4. ]

Note that even though the view server receives the map results in the form[[key, id-of-doc], value], the function may receive them in a differentform. For example, the JavaScript Query Server applies functions on the list ofkeys and the list of values.

8.1.6. rereduce

Command:

rereduce

Arguments:
  • Reduce function source
  • List of values

When building a view, CouchDB will apply the reduce step directly to the outputof the map step and the rereduce step to the output of a previous reduce step.

CouchDB will send a list of reduce functions and a list of values, with no keysor document ids to the rereduce step.

CouchDB sends:

  1. [
  2. "rereduce",
  3. [
  4. "function(k, v, r) { return sum(v); }"
  5. ],
  6. [
  7. 33,
  8. 55,
  9. 66
  10. ]
  11. ]

The Query Server answers:

  1. [
  2. true,
  3. [154]
  4. ]

8.1.7. ddoc

Command:

ddoc

Arguments:

Array of objects.

  • First phase (ddoc initialization):
    • "new"
    • Design document _id
    • Design document object
  • Second phase (design function execution):
    • Design document _id
    • Function path as an array of object keys
    • Array of function arguments
      |Returns:
      |——-
      |
  • First phase (ddoc initialization): true
  • Second phase (design function execution): custom object depending onexecuted function

This command acts in two phases: ddoc registration and _design function_execution.

In the first phase CouchDB sends a full design document content to the QueryServer to let it cache it by _id value for further function execution.

To do this, CouchDB sends:

  1. [
  2. "ddoc",
  3. "new",
  4. "_design/temp",
  5. {
  6. "_id": "_design/temp",
  7. "_rev": "8-d7379de23a751dc2a19e5638a7bbc5cc",
  8. "language": "javascript",
  9. "shows": {
  10. "request": "function(doc,req){ return {json: req}; }",
  11. "hello": "function(doc,req){ return {body: 'Hello, ' + (doc || {})._id + '!'}; }"
  12. }
  13. }
  14. ]

The Query Server answers:

  1. true

After this, the design document will be ready to serve subcommands in thesecond phase.

Note

Each ddoc subcommand is the root design document key, so they are notactually subcommands, but first elements of the JSON path that may be handledand processed.

The pattern for subcommand execution is common:

["ddoc", <design_doc_id>, [<subcommand>, <funcname>], [<argument1>, <argument2>, …]]

8.1.7.1. shows

Command:

ddoc

SubCommand:

shows

Arguments:
  • Document object or null if document id isn’t specified in request
  • Request object
    |Returns:
    |——-
    |
    Array with two elements:

  • "resp"

  • Response object

Executes show function.

Couchdb sends:

  1. [
  2. "ddoc",
  3. "_design/temp",
  4. [
  5. "shows",
  6. "doc"
  7. ],
  8. [
  9. null,
  10. {
  11. "info": {
  12. "db_name": "test",
  13. "doc_count": 8,
  14. "doc_del_count": 0,
  15. "update_seq": 105,
  16. "purge_seq": 0,
  17. "compact_running": false,
  18. "disk_size": 15818856,
  19. "data_size": 1535048,
  20. "instance_start_time": "1359952188595857",
  21. "disk_format_version": 6,
  22. "committed_update_seq": 105
  23. },
  24. "id": null,
  25. "uuid": "169cb4cc82427cc7322cb4463d0021bb",
  26. "method": "GET",
  27. "requested_path": [
  28. "api",
  29. "_design",
  30. "temp",
  31. "_show",
  32. "request"
  33. ],
  34. "path": [
  35. "api",
  36. "_design",
  37. "temp",
  38. "_show",
  39. "request"
  40. ],
  41. "raw_path": "/api/_design/temp/_show/request",
  42. "query": {},
  43. "headers": {
  44. "Accept": "*/*",
  45. "Host": "localhost:5984",
  46. "User-Agent": "curl/7.26.0"
  47. },
  48. "body": "undefined",
  49. "peer": "127.0.0.1",
  50. "form": {},
  51. "cookie": {},
  52. "userCtx": {
  53. "db": "api",
  54. "name": null,
  55. "roles": [
  56. "_admin"
  57. ]
  58. },
  59. "secObj": {}
  60. }
  61. ]
  62. ]

The Query Server sends:

  1. [
  2. "resp",
  3. {
  4. "body": "Hello, undefined!"
  5. }
  6. ]

8.1.7.2. lists

Command:

ddoc

SubCommand:

lists

Arguments:

Executes list function.

The communication protocol for list functions is a bit complex so let’s usean example to illustrate.

Assume we have view a function that emits id-rev pairs:

  1. function(doc) {
  2. emit(doc._id, doc._rev);
  3. }

And we’d like to emulate all_docs JSON response with list function. Our_first version of the list functions looks like this:

  1. function(head, req){
  2. start({'headers': {'Content-Type': 'application/json'}});
  3. var resp = head;
  4. var rows = [];
  5. while(row=getRow()){
  6. rows.push(row);
  7. }
  8. resp.rows = rows;
  9. return toJSON(resp);
  10. }

The whole communication session during list function execution could be dividedon three parts:

  • Initialization

The first returned object from the list function is an array with thefollowing structure:

  1. ["start", <chunks>, <headers>]

Where <chunks> is an array of text chunks that will be sent to the clientand <headers> is an object with response HTTP headers.

This message is sent from the Query Server to CouchDB on thestart() call which initializes the HTTP response to the client:

  1. [
  2. "start",
  3. [],
  4. {
  5. "headers": {
  6. "Content-Type": "application/json"
  7. }
  8. }
  9. ]

After this, the list function may start to process view rows.

  • View Processing

Since view results can be extremely large, it is not wise to pass all itsrows in a single command. Instead, CouchDB can send view rows one by oneto the Query Server allowing view processing and output generation to beprocessed as a stream.

CouchDB sends a special array that carries view row data:

  1. [
  2. "list_row",
  3. {
  4. "id": "0cb42c267fe32d4b56b3500bc503e030",
  5. "key": "0cb42c267fe32d4b56b3500bc503e030",
  6. "value": "1-967a00dff5e02add41819138abb3284d"
  7. }
  8. ]

If the Query Server has something to return on this, it returns an arraywith a "chunks" item in the head and an array of data in the tail. Forthis example it has nothing to return, so the response will be:

  1. [
  2. "chunks",
  3. []
  4. ]

When there are no more view rows to process, CouchDB sends a _list_end_message to signify there is no more data to send:

  1. ["list_end"]
  • Finalization

The last stage of the communication process is the returning list tail:the last data chunk. After this, processing of the list function will becomplete and the client will receive a complete response.

For our example the last message is:

  1. [
  2. "end",
  3. [
  4. "{\"total_rows\":2,\"offset\":0,\"rows\":[{\"id\":\"0cb42c267fe32d4b56b3500bc503e030\",\"key\":\"0cb42c267fe32d4b56b3500bc503e030\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"},{\"id\":\"431926a69504bde41851eb3c18a27b1f\",\"key\":\"431926a69504bde41851eb3c18a27b1f\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}]}"
  5. ]
  6. ]

In this example, we have returned our result in a single message from the QueryServer. This is okay for small numbers of rows, but for large data sets,perhaps with millions of documents or millions of view rows, this would not beacceptable.

Let’s fix our list function and see the changes in communication:

  1. function(head, req){
  2. start({'headers': {'Content-Type': 'application/json'}});
  3. send('{');
  4. send('"total_rows":' + toJSON(head.total_rows) + ',');
  5. send('"offset":' + toJSON(head.offset) + ',');
  6. send('"rows":[');
  7. if (row=getRow()){
  8. send(toJSON(row));
  9. }
  10. while(row=getRow()){
  11. send(',' + toJSON(row));
  12. }
  13. send(']');
  14. return '}';
  15. }

“Wait, what?” - you’d like to ask. Yes, we’d build JSON response manually bystring chunks, but let’s take a look on logs:

  1. [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["start",["{","\"total_rows\":2,","\"offset\":0,","\"rows\":["],{"headers":{"Content-Type":"application/json"}}]
  2. [Wed, 24 Jul 2013 05:45:30 GMT] [info] [<0.18963.1>] 127.0.0.1 - - GET /blog/_design/post/_list/index/all_docs 200
  3. [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_row",{"id":"0cb42c267fe32d4b56b3500bc503e030","key":"0cb42c267fe32d4b56b3500bc503e030","value":"1-967a00dff5e02add41819138abb3284d"}]
  4. [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["chunks",["{\"id\":\"0cb42c267fe32d4b56b3500bc503e030\",\"key\":\"0cb42c267fe32d4b56b3500bc503e030\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}"]]
  5. [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_row",{"id":"431926a69504bde41851eb3c18a27b1f","key":"431926a69504bde41851eb3c18a27b1f","value":"1-967a00dff5e02add41819138abb3284d"}]
  6. [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["chunks",[",{\"id\":\"431926a69504bde41851eb3c18a27b1f\",\"key\":\"431926a69504bde41851eb3c18a27b1f\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}"]]
  7. [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_end"]
  8. [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["end",["]","}"]]

Note, that now the Query Server sends response by lightweight chunks and ifour communication process was extremely slow, the client will see how responsedata appears on their screen. Chunk by chunk, without waiting for the completeresult, like they have for our previous list function.

8.1.7.3. updates

Command:

ddoc

SubCommand:

updates

Arguments:
  • Document object or null if document id wasn’t specified in request
  • Request object
    |Returns:
    |——-
    |
    Array with there elements:

  • "up"

  • Document object or null if nothing should be stored
  • Response object

Executes update function.

CouchDB sends:

  1. [
  2. "ddoc",
  3. "_design/id",
  4. [
  5. "updates",
  6. "nothing"
  7. ],
  8. [
  9. null,
  10. {
  11. "info": {
  12. "db_name": "test",
  13. "doc_count": 5,
  14. "doc_del_count": 0,
  15. "update_seq": 16,
  16. "purge_seq": 0,
  17. "compact_running": false,
  18. "disk_size": 8044648,
  19. "data_size": 7979601,
  20. "instance_start_time": "1374612186131612",
  21. "disk_format_version": 6,
  22. "committed_update_seq": 16
  23. },
  24. "id": null,
  25. "uuid": "7b695cb34a03df0316c15ab529002e69",
  26. "method": "POST",
  27. "requested_path": [
  28. "test",
  29. "_design",
  30. "1139",
  31. "_update",
  32. "nothing"
  33. ],
  34. "path": [
  35. "test",
  36. "_design",
  37. "1139",
  38. "_update",
  39. "nothing"
  40. ],
  41. "raw_path": "/test/_design/1139/_update/nothing",
  42. "query": {},
  43. "headers": {
  44. "Accept": "*/*",
  45. "Accept-Encoding": "identity, gzip, deflate, compress",
  46. "Content-Length": "0",
  47. "Host": "localhost:5984"
  48. },
  49. "body": "",
  50. "peer": "127.0.0.1",
  51. "form": {},
  52. "cookie": {},
  53. "userCtx": {
  54. "db": "test",
  55. "name": null,
  56. "roles": [
  57. "_admin"
  58. ]
  59. },
  60. "secObj": {}
  61. }
  62. ]
  63. ]

The Query Server answers:

  1. [
  2. "up",
  3. null,
  4. {"body": "document id wasn't provided"}
  5. ]

or in case of successful update:

  1. [
  2. "up",
  3. {
  4. "_id": "7b695cb34a03df0316c15ab529002e69",
  5. "hello": "world!"
  6. },
  7. {"body": "document was updated"}
  8. ]

8.1.7.4. filters

Command:

ddoc

SubCommand:

filters

Arguments:
  • Array of document objects
  • Request object
    |Returns:
    |——-
    |
    Array of two elements:

  • true

  • Array of booleans in the same order of input documents.

Executes filter function.

CouchDB sends:

  1. [
  2. "ddoc",
  3. "_design/test",
  4. [
  5. "filters",
  6. "random"
  7. ],
  8. [
  9. [
  10. {
  11. "_id": "431926a69504bde41851eb3c18a27b1f",
  12. "_rev": "1-967a00dff5e02add41819138abb3284d",
  13. "_revisions": {
  14. "start": 1,
  15. "ids": [
  16. "967a00dff5e02add41819138abb3284d"
  17. ]
  18. }
  19. },
  20. {
  21. "_id": "0cb42c267fe32d4b56b3500bc503e030",
  22. "_rev": "1-967a00dff5e02add41819138abb3284d",
  23. "_revisions": {
  24. "start": 1,
  25. "ids": [
  26. "967a00dff5e02add41819138abb3284d"
  27. ]
  28. }
  29. }
  30. ],
  31. {
  32. "info": {
  33. "db_name": "test",
  34. "doc_count": 5,
  35. "doc_del_count": 0,
  36. "update_seq": 19,
  37. "purge_seq": 0,
  38. "compact_running": false,
  39. "disk_size": 8056936,
  40. "data_size": 7979745,
  41. "instance_start_time": "1374612186131612",
  42. "disk_format_version": 6,
  43. "committed_update_seq": 19
  44. },
  45. "id": null,
  46. "uuid": "7b695cb34a03df0316c15ab529023a81",
  47. "method": "GET",
  48. "requested_path": [
  49. "test",
  50. "_changes?filter=test",
  51. "random"
  52. ],
  53. "path": [
  54. "test",
  55. "_changes"
  56. ],
  57. "raw_path": "/test/_changes?filter=test/random",
  58. "query": {
  59. "filter": "test/random"
  60. },
  61. "headers": {
  62. "Accept": "application/json",
  63. "Accept-Encoding": "identity, gzip, deflate, compress",
  64. "Content-Length": "0",
  65. "Content-Type": "application/json; charset=utf-8",
  66. "Host": "localhost:5984"
  67. },
  68. "body": "",
  69. "peer": "127.0.0.1",
  70. "form": {},
  71. "cookie": {},
  72. "userCtx": {
  73. "db": "test",
  74. "name": null,
  75. "roles": [
  76. "_admin"
  77. ]
  78. },
  79. "secObj": {}
  80. }
  81. ]
  82. ]

The Query Server answers:

  1. [
  2. true,
  3. [
  4. true,
  5. false
  6. ]
  7. ]

8.1.7.5. views

Command:

ddoc

SubCommand:

views

Arguments:

Array of document objects

Returns:

Array of two elements:

  • true
  • Array of booleans in the same order of input documents.

New in version 1.2.

Executes view function in place of the filter.

Acts in the same way as filters command.

8.1.7.6. validate_doc_update

Command:

ddoc

SubCommand:

validate_doc_update

Arguments:

Executes validation function.

CouchDB send:

  1. [
  2. "ddoc",
  3. "_design/id",
  4. ["validate_doc_update"],
  5. [
  6. {
  7. "_id": "docid",
  8. "_rev": "2-e0165f450f6c89dc6b071c075dde3c4d",
  9. "score": 10
  10. },
  11. {
  12. "_id": "docid",
  13. "_rev": "1-9f798c6ad72a406afdbf470b9eea8375",
  14. "score": 4
  15. },
  16. {
  17. "name": "Mike",
  18. "roles": ["player"]
  19. },
  20. {
  21. "admins": {},
  22. "members": []
  23. }
  24. ]
  25. ]

The Query Server answers:

  1. 1

Note

While the only valid response for this command is true, to prevent thedocument from being saved, the Query Server needs to raise an error:forbidden or unauthorized; these errors will be turned into correctHTTP 403 and HTTP 401 responses respectively.

8.1.7.7. rewrites

Command:

ddoc

SubCommand:

rewrites

Arguments:

Executes rewrite function.

CouchDB send:

  1. [
  2. "ddoc",
  3. "_design/id",
  4. ["rewrites"],
  5. [
  6. {
  7. "method": "POST",
  8. "requested_path": [
  9. "test",
  10. "_design",
  11. "1139",
  12. "_update",
  13. "nothing"
  14. ],
  15. "path": [
  16. "test",
  17. "_design",
  18. "1139",
  19. "_update",
  20. "nothing"
  21. ],
  22. "raw_path": "/test/_design/1139/_update/nothing",
  23. "query": {},
  24. "headers": {
  25. "Accept": "*/*",
  26. "Accept-Encoding": "identity, gzip, deflate, compress",
  27. "Content-Length": "0",
  28. "Host": "localhost:5984"
  29. },
  30. "body": "",
  31. "peer": "127.0.0.1",
  32. "cookie": {},
  33. "userCtx": {
  34. "db": "test",
  35. "name": null,
  36. "roles": [
  37. "_admin"
  38. ]
  39. },
  40. "secObj": {}
  41. }
  42. ]
  43. ]

The Query Server answers:

  1. [
  2. "ok",
  3. {
  4. "path": "some/path",
  5. "query": {"key1": "value1", "key2": "value2"},
  6. "method": "METHOD",
  7. "headers": {"Header1": "value1", "Header2": "value2"},
  8. "body": ""
  9. }
  10. ]

or in case of direct response:

  1. [
  2. "ok",
  3. {
  4. "headers": {"Content-Type": "text/plain"},
  5. "body": "Welcome!",
  6. "code": 200
  7. }
  8. ]

or for immediate redirect:

  1. [
  2. "ok",
  3. {
  4. "headers": {"Location": "http://example.com/path/"},
  5. "code": 302
  6. }
  7. ]

8.1.8. Returning errors

When something goes wrong, the Query Server can inform CouchDB by sending aspecial message in response to the received command.

Error messages prevent further command execution and return an error descriptionto CouchDB. Errors are logically divided into two groups:

  • Common errors. These errors only break the current Query Server command andreturn the error info to the CouchDB instance without terminating the QueryServer process.
  • Fatal errors. Fatal errors signal a condition that cannot be recovered.For instance, if your a design function is unable to import a third partymodule, it’s better to count such error as fatal and terminate whole process.

8.1.8.1. error

To raise an error, the Query Server should respond with:

  1. ["error", "error_name", "reason why"]

The "error_name" helps to classify problems by their type e.g. if it’s"value_error" to indicate improper data, "not_found" to indicate amissing resource and "type_error" to indicate an improper data type.

The "reason why" explains in human-readable terms what went wrong, andpossibly how to resolve it.

For example, calling Update Functions against a non-existent document couldproduce the error message:

  1. ["error", "not_found", "Update function requires existent document"]

8.1.8.2. forbidden

The forbidden error is widely used by Validate Document Update Functions to stop further functionprocessing and prevent storage of the new document revision. Since this is notactually an error, but an assertion against user actions, CouchDB doesn’t logit at “error” level, but returns HTTP 403 Forbidden response with errorinformation object.

To raise this error, the Query Server should respond with:

  1. {"forbidden": "reason why"}

8.1.8.3. unauthorized

The unauthorized error mostly acts like forbidden one, but withthe meaning of please authorize first. This small difference helps end usersto understand what they can do to solve the problem. Similar to forbidden,CouchDB doesn’t log it at “error” level, but returns a _HTTP 401 Unauthorized_response with an error information object.

To raise this error, the Query Server should respond with:

  1. {"unauthorized": "reason why"}

8.1.9. Logging

At any time, the Query Server may send some information that will be saved inCouchDB’s log file. This is done by sending a special log object with a singleargument, on a separate line:

  1. ["log", "some message"]

CouchDB does not respond, but writes the received message to the log file:

  1. [Sun, 13 Feb 2009 23:31:30 GMT] [info] [<0.72.0>] Query Server Log Message: some message

These messages are only logged at info level.

原文: http://docs.couchdb.org/en/stable/query-server/protocol.html